Update: Recording!

Finally, the end of year madness is over and we’re in a fresh new year. That must mean it’s time to get back into #DBHangops! Tune in on Wednesday, 1/16/13 at 5:00pm PST to get in on the fun and share your experiences.

Check back on this blog post tomorrow or check this twitter search to grab the link to join the Google Hangout.

Topics for this weeks hangout include:
* triggers: maintenance, problems, etc.
* Oracle MySQL utilities (similar to Percona toolkit)
** Slideshow of MySQL utilities
* Database caching and cache

  [Read more...]

451 Research’s 2013 Database survey is now live at http://bit.ly/451db13 investigating the current use of database technologies, including MySQL, NoSQL and NewSQL, as well as traditional relation and non-relational databases.

The aim of this survey is to identify trends in database usage, as well as changing attitudes to MySQL following its acquisition by Oracle, and the competitive dynamic between MySQL and other databases, including NoSQL and NewSQL technologies.

There are just 15 questions to answer, spread over five pages, and the entire survey should take less than ten minutes to complete.

All individual responses are of course

  [Read more...]

I discovered a feature (or bug) of MySQL timestamp fields. Maybe is documented in some place what I not read yet:

When I add a new timestamp field to a table, MySQL magically adds some features to new timestamp field like a “trigger” and a default value to CURRENT_TIMESTAMP.

There is test-case script:

-- CREATING TABLE AND INSERT SOME DUMMY DATA
mysql> CREATE TABLE t(
    -> id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
    -> val VARCHAR(50)
    -> );
Query OK, 0 ROWS affected (0.15 sec)
 
mysql> INSERT INTO t (val) VALUES ("foo") ,("var");
Query OK, 2 ROWS affected (0.08 sec)
Records: 2  Duplicates: 0  Warnings: 0
 
mysql> SELECT * FROM t;
+----+------+
| id | val  |
+----+------+
|  1 | foo  |
|  2 | var  |
+----+------+
2

  [Read more...]

I got some very good comments/questions on my previous post on MariaDB global transaction ID, from Giuseppe and Robert (of Tungsten fame). I thought a follow-up post would be appropriate to answer and further elaborate on the comments, as the points they raise are very important and interesting.

(It also gives me the opportunity to explain more deeply a lot of interesting design decisions that I left out in the first post for the sake of brevity and clarity.)

On crash-safe slave

One of the things I really wanted to improve with global transaction ID is to make the replication slaves more crash safe with respect to their current replication state. This state is mostly persistently stored information about which event(s) were last executed on the slave, so that after a server restart the slave will know from  [Read more...]

The main goal of global transaction ID is to make it easy to promote a new master and switch all slaves over to continue replication from the new master. This is currently harder than it could be, since the current replication position for a slave is specified in coordinates that are specific to the current master, and it is not trivial to translate them into the corresponding coordinates on the new master. Global transaction ID solves this by annotating each event with the global transaction id which is unique and universal across the whole replication hierarchy.

In addition, there are at least two other main goals for MariaDB global transaction ID:

Make it easy to setup global transaction ID, and easy to provision a new slave into an existing replication hierarchy.

Fully support

  [Read more...]

It's the end of the year and, before the world ends ;-) , I feel the pressure to follow up on last year's article "2011, a great year for MySQL in Review". I'll adopt a similar format and mention things in context, covering products, marketing, momentum, etc. at a high level. I may have missed something and I hope my colleagues will contribute to the list.

I really do think it's been the best year for MySQL so far... well, at least since when I signed my contract with MySQL AB at the end of 2007. We're just at the beginning of our journey and I wish to the MySQL family, to our community and ecosystem an even better 2013!

Summary





  [Read more...]

I recently worked on a Rails 3.2 project that used the sweet PLupload JavaScript/Flash upload tool to upload files to the web app. To make it easier for users to upload large and/or remote files to the app, we also wanted to let them upload via SFTP. The catch was, our users didn't have SFTP accounts on our server and we didn't want to get into the business of creating and managing SFTP accounts. Enter: ProFTPD and virtual users.

ProFTPD's virtual users concept allows you to point ProFTPD at a SQL database for your user and group authentication. This means SFTP logins don't need actual system logins (although you can mix and match if you want). Naturally, this is perfect for dynamically creating and

  [Read more...]

In my ongoing efforts to migrate my fun side projects and coding experiments from SVN to Git I’ve come across some of my favorite Python based apps – which are all available in their respective repos on BitBucket, as follows:

IRC Bot Commander

• What it does: it’s an IRC bot that takes commands and does your bidding on whichever remote server the bot is installed on.
• How it does it: the bot runs on whatever server you install it on, then it connects to the IRC server and channel you configured it to connect to and it waits for you to give it commands, then it execs the commands and returns the output to your IRC chat window.

  [Read more...]

As this topic came up a few times this week for discussion at various places, I thought of composing a post on “Data Scientist vs. Data Analytics Engineer”; even though[...]

Apart from my consulting as part of ScaleIn, I also invest to bootstrap companies with really disruptive ideas; and in the process met few database specific companies who are already[...]

Here is the typical “Big” data architecture, that covers most components involved in the data pipeline. More or less, we have the same architecture in production in number of places[...]

I was tired to get manually disk space used for all MySQL databases, I just created a stored procedure to get an overview of the database sizes in our MySQL server.

MySQL don’t have a command that allows us an overall summary of the databases, something like SHOW TABLE STATUS for databases. The “SHOW DATABASES” command lists only the current databases without any other information like how many space are using or how many tables there are, etc.

The procedure that I wrote this based on INFORMATION_SCHEMA database, which contains rich information of all existing databases.

I suggest, in a previous post, place all these procedures in a database called tools, but you can still choose different location.

So when call the procedure will have a global view of information that will be useful for maintenance tasks.

  [Read more...]

Now that the thanksgiving holiday is over, it seems like the perfect time to have another #DBHangOps. If you’re interested, check this twitter search or check back on my blog here for a link to the google hangout.

Items currently on our agenda for today:
* Data corruption!
* Monitoring — what do you monitor, why, etc.
* Query killing — do you kill queries in production regularly, or in emergencies?
* Fun or useful configurations

Looking forward to chatting about MySQL stuff today!
Fun time hitting these topics this week. Check out the recording:

This Week in Website Performance is a weekly feature of the Monitis.com blog. It summarizes recent articles about website performance. Why? Because your friends at Monitis.com care.

---

Full table scan vs full index scan performance

Author: Stephane Combaudon.

Using full index scans are not always faster than a full table scan. Stephane explores the situations in which a sequential read through of a full table performs better than using randomly-read full index.

---

  [Read more...]

If you're planning to attend UKOUG in Birmingham on Dec 3-5, here's your guide to know more about Oracle's MySQL.

There's a MySQL stream on Monday 5th and we've a great list of sessions, including:

• Rich Mason's keynote on how to Deploy, Scale and Save with Linux, Virtualization and MySQL
• Keith Hollman on stage to discuss how to Embrace Database Diversity
• Mark Swarbrick's lecture on 

  [Read more...]

Woooo, just wrapped up another night of #DBHangOps, check out the video:

For those of you who aren’t familiar with this awesome series of videos, it’s a bi-weekly meeting of MySQL DBAs via a Google Hangout to simply talk about databases (specifically MySQL). The discussions range anywhere from silly stories to awesome learnings and even little gotchas to know about when working with MySQL operationally. If you’re interested in catching when the next #DBHangops will happen, check MySQL planet for a blog post about it, or check this twitter search.

We look forward to talking with you!

I write a fair number of scripts on this site and have posted a lot of code over the years. Generally if I am not pasting the code to be viewed on the webpage then I link to a file that a user can download; which leads to a lot of mish-mash code that doesn’t have a home. I’ve always kept the code files in a private SVN repo over the years but have recently moved them all to BitBucket Git repositories. So here they are: lots of code samples and useful bits of programming to save time.

Generic Shell Scripts: https://bitbucket.org/themattreid/generic-bash-scripts/src
Generic Python Scripts: https://bitbucket.org/themattreid/generic-python-scripts/src
Generic MySQL Stored Procs:

  [Read more...]

MySQL's support for case sensitive search is explained somewhat opaquely in the aptly titled Case Sensitivity in String Searches documentation. In short, it explains that by default, MySQL won't treat strings as case sensitive when executing a statement such as:

SELECT first_name FROM contacts WHERE first_name REGEXP '^[a-z]';

This simple search to look for contacts whose first name starts with a lower case letter, will return *all* contacts because in the default character set used by MySQL (latin1), upper and lower case letters share the same "sort value".

UPDATE: After many helpful comments from readers, it would seem the term I should have used was collation, not sort value. The documentation for both

  [Read more...]