I came across a situation recently where I was asked if it was possible to edit a binary log to remove a part of it to restore onto a slave server. Now the choice of doing something like a hexedit did not seem appealing, and the more experienced might suggest that it is simply a matter of using  mysqlbinlog with the --start-position and/or --stop-position options. However, the problem had arisen that required the binary log to played through the replication process onto the slave based on specific options in MySQL cluster, so using an SQL dump from the binary log was of no use.

Initially this may seem like a daunting task where you will have to find some specialist tool or delve into the deep recesses of the binary log format, but a much simpler solution was found. The replication process allows the slave to be started up to a specific point in the log files. The command is the START SLAVE UNTIL... statement as seen in the manual at: …

I spent some time earlier this week trying to debug a permissions problem in Drupal.

After a lot of head-scratching, it turned out that Drupal assumes that when you run INSERT queries sequentially on a table with an auto_increment integer column, the values that are assigned to this column will also be sequential, ie: 1, 2, 3, …

This might be a valid assumption when you are the only user doing inserts on a single MySQL server, but unfortunately that is not always the situation in which an application runs.

I run MySQL in a dual-master setup, which means that two sequential INSERT statements will never return sequential integers.  The value will always be determined by the  auto_increment_increment and auto_increment_offset settings in the configuration file.

In my case, one master will only assign even numbers, the other only uneven ones.

My …

They sometimes go bad in completely unpredictable ways. Here's a problem I have now seen twice in production situations. A host boots up nicely and mounts file systems from the SAN. At some point a SAN switch (e.g., through a Fibrechannel controller) fails in such a way that the SAN goes away but the file system still appears visible to applications.

This kind of problem is an example of a Byzantine fault where a system does not fail cleanly but instead starts to behave in a completely arbitrary manner. It seems that you can get into a state where the in-memory representation of the file system inodes is intact but the underlying storage is non-responsive. The non-responsive file system in turn can make operating system processes go a little crazy. They continue to operate but show bizarre failures or hang. The result is problems that may not be …

I spent far too much time this past week trying to find out why a Drupal was not  assigning authenticated user permissions to users who clicked on the validation link they were emailed. Instead, it would simply remove the temporary unverified user permission.

It's supposed to do the latter, but then also follow it up with the former. It turns out the problem was a Drupal bug that was triggered by my MySQL setup.

My Drupals all use a server that is a master in a set-up with two masters an three slaves. This has some implications for the auto_increment integer data type. In order to avoid clashes when two masters each insert a new record simultaneously, a master gets a specific instruction about which IDs it may assign.

These instructions specify how many numbers to skip - typically the number of masters in a cluster - and an offset to add, …

There is a raft of new Tungsten open source builds available for your replication and clustering pleasure. Over the last couple of days we uploaded new binary builds for Tungsten Replicator, Tungsten Connector, Tungsten Monitor, and Tungsten SQL Router. These contain the features described in my previous blog article, including even more bug fixes ( …

Articles on this blog have been pretty scanty of late for a simple reason--we have been 100% heads-down in Tungsten code since the recent MySQL Conference. The result has been a number of excellent improvements that are already in Subversion and will appear as open source builds over the next couple of weeks.

Tungsten has a simple goal: create highly available, performant database clusters using unaltered commodity databases that are simple to manage and look as close to a single database as possible for applications. Over the last two months we completed the integration of individual Tungsten components necessary to make this happen.

Full integration is a big step forward and finally gets us to the ease-of-use we were seeking. Imagine you want to add a slave database to the cluster. …

It seems that a lot of people want to try to improve MySQL performance by focusing on server status counters and configuration variables. Looking at counters, and “tuning the server,” is better than nothing, but only barely. You care first and foremost about how long it takes to execute a query, not about how many of this-and-that the server performs or about how big or small this-and-that buffer is. What you really need is timing information.

You can use the slow query log to find timing information about queries, and then you can examine those queries with SHOW PROFILES to see the timing information about the query’s execution itself.

This concept is very simple and absolutely fundamental: if you care about time (and you do!), then measure and optimize time. But it’s so often overlooked or misunderstood.

The addition of SHOW PROFILES was a major step forward in the ability to optimize server and …

Recently I needed to replicate between MySQL and another database technology. You might say, why on earth would you want to do something like that, but believe me there are reasons and definitely not (to go away from MySQL to some other DB technology like Oracle or SQL server). Unsurprisingly there are quite a few different tools to do it from any platform towards MySQL but very few which do it the other way round, just to name a couple: Golden Gate and DSCallards.

A couple of weeks ago I wrote a lua script to use with the MySQL Proxy that transforms the Proxy into a key=>value lookup dictionary.

But I couldn't just stop there. So I decided to add replication to it :).


The basic idea is that you have one proxy that acts like a master, it can handle all write/read operations. Once it receives a write query, it will send that query to the slave proxy instances, and after the last slave gets the query, the master will return a confirmation to the mysql client.
And of course, you send your read queries to the slave proxy instances.

Show me the code.
It is available on the …

This Thursday (May 28th, 14:00 UTC), Lars Thalmann will give a MySQL University session on MySQL Replication: Walk-through of the new 5.1 and 6.0 features. (This session was originally scheduled for May 7th, but had to be put off due to technical problems. Apologies.) Lars is leading the replication and backup teams at MySQL, so this is one of the best opportunities to ask whatever questions you might have about new replication features in MySQL.

For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't have to. (Dimdim is the …