Data migrations always have a wide range of challenges. I recently took on a request to determine the difficulty of converting an ecommerce shop's MySQL 5.0 database to PostgreSQL 9.3, with the first (presumably "easier") step being just getting the schema converted and data imported before tackling the more challenging aspect of doing a full assessment of the site's query base to re-write the large number of custom queries that leverage MySQL-specific language elements into their PostgreSQL counterparts.

During the course of this first part, which had contained a number of difficulties I had anticipated, I hit one that I definitely had not anticipated:

ERROR:  value too long for type character varying(20)

Surely, the error message is absolutely clear, but how could this possibly be? The obvious answer--that the varchar definitions were different lengths between MySQL and PostgreSQL--was sadly quite wrong (which you knew, …

I have just implemented MASTER_GTID_WAIT() in MariaDB 10.0. This can be used to give a very elegant solution to the problem of stale reads in replication read-scaleout, without incuring the overheads normally associated with synchronous replication techniques. This idea came up recently in a discussion with Stephane Varoqui, and is similar to the concept of Lamport logical clock described in this Wikipedia article.

I wanted to describe this, hoping to induce people to test and maybe start using this, as it is a simple but very neat idea, actually.

A very typical use of MariaDB/MySQL …

MySQL Cluster is a highly resilient and scalable database platform designed to deliver 99.999% availability with features such as self-healing and online operations, and capable of performing over 1,00,000,000 updates per minute. The full feature set includes development and management platforms alongside monitoring and administration tools, all backed by Oracle Premier Lifetime Support.

To learn more about MySQL Cluster, consider taking the MySQL Cluster training. Events already on the schedule for this 3-day instructor-led course include:

I've completed a new release of the Connector/Arduino. The new version supports a few refinements and a new feature.

• New! disconnect() method - enables disconnect from server. Note: you must call mysql_connect() to reconnect.
• Better error handling for dropped packets. No more random reboots when bad packet appears.
• Library can recover from short-term loss of connectivity. Along with bad packets is a check to make sure what is received is valid making the connector ignore garbage packets associated with a dropped connection.
• Detection of Out of Memory condition. Should there not be enough memory to allocate the buffer for the Connector, you will see an OOM error (enable the serial monitor to see the errors). This reduces random reboots when memory gets too low.

I made this release because a number of people were running into problems with noisy, tenuous, or just plain …

I believe in automation as much as possible, and I'm always working to make the day to day tasks of operations as smooth as possible.  Also I try not to be afraid to take good tools and make them better.

Here in Database Ops at Box, we use pt-kill running as a service to constantly monitor our servers and help protect against long running queries.  But our thresholds are pretty generous, and in some cases it's possible for unforeseen circumstances to cause enough queries to storm the database such that we can have problems before any of them hit the threshold for "busy time."  Ditto for idle connections.

The response is that someone has to be available to manually run another copy of pt-kill with much lower thresholds to clear out these thundering herds.  But what if we could let pt-kill handle both the "normal" mode and still protect us from herds?

That's what we've done by adding a …

In my previous post I wrote about how I achived a >40% speedup on sysbench read-only using profile-guided optimisation (PGO). While this is a preliminary result, I though it was so interesting that it deserved early mention. The fact that any benchmark can be improved that much shows clearly that PGO is something worth looking into. Even if we will probably not improve all workloads by 40%, it seems highly likely that we can obtain significant gains also for many real workloads.

I had one or two interesting comments on the post that raise valid concerns, so I wanted to write a follow-up here, explaining some of the points in more details and going deeper into the performance counter measurements. As I wrote before, actual observations and measurements are crucial to fully understand performance of complex code on modern CPUs. Intuition and …

Continuing my investigation of single-threaded performance in the MariaDB server, I managed to increase throughput of single-threaded read-only sysbench by more than 40% so far:

I use read-only sysbench 0.4.12 run like this:

    sysbench --num-threads=1 --test=oltp --oltp-test-mode=simple --oltp-read-only --oltp-skip-trx run

And mysqld is run with minimal options:

    sql/mysqld --no-defaults --basedir=X --datadir=Y --innodb-buffer-pool-size=128M

With modern high-performance CPUs, it is necessary to do detailed measurements using the built-in performance counters in order to get any kind of understanding of how an application performs and what the bottlenecks are. Forget about looking at the code and counting instructions or cycles as we did in the old days. It no longer works, not even to within an order of magnitude.

Many installations of MySQL server come with a built-in database called test. It's initially empty, and you might wonder what it's for, or even if you can delete it without any problems.

What is it for? 

The test database is installed by the MySQL Server RPM as part of the mysql_install_db process, and some other package managers run that script too. If you run that script as part of a manual install of MySQL, you'll get the same effect. It creates the database by creating an empty directory called "test" in the data directory, and creates wide-open access to the database test and any database with a name beginning with test_ by inserting a couple of rows into the mysql.db table that give everyone full access to create or use those databases.

The configuration is designed to make it easy for new users to create a playground or sandbox database to work with, one that doesn't require asking the DBA …

MySQL is an excellent choice for a low cost embedded database. Over 3,000 ISVs, OEMs and VARs choose MySQL for this purpose and benefit from significant cost savings when compared to using a competing database product.

The MySQL for Developers training course teaches you how to plan, design and implement applications using MySQL. Expert Oracle University instructors teach you through realistic examples, interactive instruction and hands-on exercises.

This course brings you lots of interesting facts. For example, the 'libmysqld' library is effectively a MySQL server in its own right. it is shipped as part of the standard MySQL installation. Developers with C/C++ ability can embed this library easily within their applications and it can do most of what a …

Sequel Pro

I often use Sequel Pro when I'm getting up to speed on the data model for a project or when I just want to debug in a more visual way than with the mysql command-line client. It's a free OS X application that lets you inspect and manage MySQL databases. I also find it very useful for making small changes to the data while I develop and test web apps.

Quickly Copy Data Between Databases

I recently needed a way to copy a few dozen records from one camp to another. I tried using the "SELECT...INTO OUTFILE" method but ran into a permissions issue with that approach. Using mysqldump was another option but that seemed like overkill in …