There’s a change of behaviour in MySQL 5.1.31 for Row Based Replication, if you have InnoDB transactions that also write to a MyISAM (or other non-transactional engine) table. It’s a side effect of fixing Bug #40116. Take this simple example:

Transaction 1: INSERT INTO myisam_tbl (item, val) VALUES (1, 0);
Transaction 1: INSERT INTO innodb_tbl (item, val) VALUES (1, -1), (2, -1);
Transaction 1: START TRANSACTION;
Transaction 1: UPDATE myisam_tbl SET val=val+1 WHERE item=1;
Transaction 1: UPDATE innodb_tbl SET val=( SELECT val FROM myisam_tbl WHERE item=1 ) WHERE item=1;
Transaction 2: START TRANSACTION;
Transaction 2: UPDATE myisam_tbl SET val=val+1 WHERE item=1;
Transaction 2: UPDATE innodb_tbl SET val=( SELECT val FROM myisam_tbl WHERE item=1 ) WHERE item=2;
Transaction 2: COMMIT;
Transaction 1: COMMIT;

After this, the Master innodb_tbl would look like this:

 Our latest article over at Database Journal is hot off the presses.  Advanced MySQL Replication – Improving Performance discusses some of the best ways to improve the performance of your MySQL slave setup.  If the slave is constantly getting further and further behind the master database, we discuss a number of techniques which may help you bring it under control.

Last night I gave a presentation on Tungsten Replicator to the SFO MySQL MeetUp. It was really fun--lots of excellent questions and ideas from an audience that knows MySQL in and out. Here are slides from the talk from our trusty S3 document repository.

There were many things to think about on the BART ride home to Berkeley but here are a couple that really struck me, in part because I think we can do something about them.

First, database replication needs to expose as much monitoring data as possible. The kind of data you get or can infer from SHOW SLAVE STATUS is just the beginning. This came through really strongly from people like Erin who are using load-balancers and other automated techniques for distributing load to replicas using decision criteria like availability and latency of slaves. We have …

One of my favorite MySQL configurations for high availability is master-master replication, which is just like normal master-slave replication except that you can fail over in both directions. Aside from MySQL Cluster, which is more special-purpose, this is probably the best general-purpose way to get fast failover and a bunch of other benefits (non-blocking ALTER TABLE, for example).

The benefit is that you have another server with all the same data, up and running, ready to serve queries. In theory, it's a truly hot standby (stay with me -- that's not really guaranteed). You don't get this with shared storage or DRBD, although those provide stronger guarantees against data loss if mysqld crashes. And you can use the standby (passive) master for serving some SELECT queries, taking backups, etc as …

Point-in-time recovery or PITR is one of my favorite PostgreSQL features. Most database servers have a transaction log for recovery. (MySQL is one of the few exceptions; more on that shortly.) However, PostgreSQL PITR allows and even encourages users to manipulate the logs to perform some very useful tasks:

* Creating a bit-for-bit backup of a server without halting
* Restoring a server to a specific point in time
* Creating a warm standby server

The third task is especially interesting because it's so common. One of the most pronounced trends in computing is the decreasing cost of computing power through Moore's law improvements and virtualization. Standby solutions nowadays look like a good investment compared to the cost having a server down or worse …

I’ll keep this short. Let’s say you run “mysql> show slave status\G” and you see the following error:

Last_Error: Error 'Unknown or incorrect time zone: 'Etc/UTC'' on query. Default database: 'db_name'. Query: 'INSERT INTO table SET column = CONVERT_TZ('2009-01-24', '-8:00', '+00:00')'

It’s a simple fix. Exit out of MySQL and run the following command - this works on Redhat, I’ve not tried it on other flavors.

shell> mysql_tzinfo_to_sql /usr/share/zoneinfo|mysql -u root -p

Log back into MySQL and run the following.

mysql> stop slave;
mysql> start slave;
mysql> show slave status\G;

The error should be gone and the slave should be catching up to the master now. Problem solved.

This is a simple mysql circular replication implementation on a single machine (just a proof of concept) which can easily be done on a broader scale. Just be aware of the cons of circular replication which mainly gets down to: once a node freaks out or stops for one reason or the other, it’s a bitch and you need to take care of IMMEDIATELY.

Download Sandbox from https://launchpad.net/mysql-sandbox/+download
Download MySQL from http://dev.mysql.com/downloads

Copy the downloaded software onto the your *nix box onto any folder of your preference called $BASEDIR

run:

cd /$BASEDIR

gunzip mysql_sandbox_X.X.XX.tar.gz
tar -xf mysql_sandbox_X.X.XX.tar

ln -s mysql_sandbox_X.X.XX sandbox

time /$BASEDIR/sandbox/make_replication_sandbox –circular=4 –topology=circular /$BASEDIR/mysql-5.1.30-linux-x86_64-glibc23.tar.gz


user@hostname $ time …

One very helpful use of the technique Sheeri described in Remote connections without leaving the mysql shell is making sure that replication is working properly.

According to the MySQL Reference Manual’s section on SHOW SLAVE STATUS Syntax, it shows information corresponding to the slave thread in the slave server. When replication is broken, however, or not working properly due to network issues between master and slave, this information may not be accurate. This has improved over recent releases, but it’s still not perfect.

The question, then, is: how to be 100% sure (or as close as you can get to 100%) that replication is running fine? The answer, as offered by Sheeri: use CONNECT.

Example …

This Thursday (January 29th), we're continuing our series of sessions on MySQL performance measuring and improvements with David Lutz' presentation titled Scalability Challenges in an InnoDB-based Replication Environment. David works in the Performance and Applications Engineering department at Sun Microsystems, so again, expect to get some deep insights into the inner workings of the MySQL Server.

David is based in California, so note that this session will take place in the morning (America) or evening (Europe), respectively.

For MySQL University sessions, point your browser to this page. You need a browser with a working …

If you want to learn about Tungsten Replicator first hand, I'll be doing a presentation at the February 2nd MySQL Meet-up in San Francisco. With luck I'll be able to demo not only the replicator but also some cool Amazon/RightScale integration we will be introducing in February. Teaser: It's possible to set up database clusters faster in Amazon than on a laptop.

Advance thanks to Mike Tougeron and Rich Nigra for the invite. For people suffering from obsessive curiosity about MySQL replication or database clustering in general, this talk is for you. Hope to see lots of you at the meet-up.