I wrote this up a while ago and decided that I didn’t want to lose it in a shuffle of documents during my transition to a new workstation. It’s the basics of setting up Heartbeat (LVS) + DRBD (block replication between active/passive master servers) + MySQL. This should give you the basics of a H/A system without the benefits of SAN but also without the associated cost. The validity of this setup for H/A purposes is highly dependent on your workload and environment. You should know the ins and outs of your H/A solution before deciding to blame the system for not performing as expected. As with all production systems you should test, test, test and test some more before going live.

When I get around to it later I’ll post my How-To for setting up RHCS + SAN + MySQL. You can download the DRBD document PDF here: …

One more DRBD tutorial, this time I will describe howto setup MySQL with DRBD (Distributed Replicated Block Device). Purpose This document describes how to to setup a failover system with MySQL and DRBD (Distributed Replicated Block Device). Introduction In this tutorial we will setup two Debian Linux nodes with a DRBD volume. MySQL will be [...]

When I read on the internetz that Alex Davies was about the publish a Packt book on MySQL HA I pinged my contacts at Packt and suggested that I'd review the book .

I've ran into Alex at some UKUUG conferences before and he's got a solid background on MySQL Cluster and other HA alternatives so I was looking forward to reading the book.

Alex starts of with a couple of indepth chapters on MySQL Cluster, he does mention that it's not a fit for all problems, but I'd hoped he did it a bit more prominently ... an upfront chapter outlining the different approaches and when which approach is a match could have been better. The avid reader now might be 80 pages into MySQL cluster before he realizes it's not going to be a match for his problem.

I really loved …

A couple of weeks ago one of my colleagues and I worked on a data corruption case that reminded me that sometimes people make unsafe assumptions without knowing it. This one involved SAN snapshotting that was unsafe.

In a nutshell, the client used SAN block-level replication to maintain a standby/failover MySQL system, and there was a failover that didn't work; both the primary and fallback machine had identically corrupted data files. After running fsck on the replica, the InnoDB data files were entirely deleted.

When we arrived on the scene, there was a data directory with an 800+ GB data file, which we determined had been restored from a SAN snapshot. Accessing this file caused a number of errors, including warnings about accessing data outside of the partition boundaries. We were eventually able to coax the filesystem into truncating the data file back to a size that didn't contain invalid pointers and could be read without …

This is a Request for Input. Dual MySQL masters with MMM in a single datacentre are in common use, and other setups like DRBD and of course VM/SAN based failover solutions are conceptually straightforward also. Thus, achieving various forms of resilience within a single data-centre is doable and not costly.

Doing the same across multiple (let’s for simplicity sake limit it to two) datacentres is another matter. MySQL replication works well across longer links, and it can use MySQL’s in-built SSL or tools like stunnel. Of course it needs to be kept an eye on, as usual, but since it’s asynchronous the latency between the datacentres is not a big issue (apart from the fact that the second server gets up-to-date a little bit later).

But as those who have tried will know, having a client (application server) connection to a MySQL instance in a remote data-centre is a whole other matter, latency becomes a big issue and is generally …

I recently presented a webcast hosted by O’Reilly and Webex.  In it I take you on a step-by-step installation of DRBD and MySQL.  I start by using Sun’s Virtualbox to create to virtual machines running CentOS.  I then explain how to configure them with virtual external drives to use for DRBD.  I next configure the network interfaces to support routed packets into and out of the boxes.  Then I install various packages with yum, configure drbd and finally install MySQL as the last step.  You can follow along at the command line and do it yourself on a Windows, Mac or Linux box.

Funny how different experiences lead to different evaluations of tools. The MySQL HA solutions the MySQL Performanceblog list, are almost listed in the complete opposited order of what my impressions are.

Ok agreed, I should probably not put my MySQL NDB experiences from 2-3 years ago with multiple Query of deaths and more problems than you into account anymore , but back then went in the list Less stable than a single node. I've had NDB POC setups going down for much more than 05:16 minutes
Ndb comes with a lot of restrictions, there are

As for MySQL on DRBD, I've said this before , I love DRBD, but having to wait for a long InnoDB recovery after a failover just kills your uptime ,
…

I'm happy to extend a warm welcome to two new members of the Percona team.

First is Yves Trudeau, about whom I can say many things:

• One of the top MySQL Cluster (NDB Cluster) experts in the world.
• An expert on all things High Availability, including DRBD and Heartbeat.
• Many years of experience with Huge Data.
• Half of the Waffle Grid team.
• A really nice person!

Yves joins us after a tenure of several years as a senior consultant at Sun/MySQL. Together with Matt Yonkovit, he plans to work on WaffleGrid (but as a new project under a new name, to be determined), and integration with XtraDB. Yves lives in Quebec with his family.

Next is Fernando Ipar. Fernando is our first dedicated Shift Support Captain[1]. Fernando specializes in MySQL, GNU/Linux, systems administration, and high availability. Fernando has been involved in computer programming since …

It seems pretty common to find customers install DRBD for the wrong reasons. There are many pros/cons to compare DRBD to replication, but I've managed to cut down my spiel I give to customers to these two points:

• DRBD's aim (assuming replication mode C) is to provide 100% consistency, and then as much uptime as possible.
• MySQL Replication (with a manager such as MMM) aims to have 100% availability, at the potential loss of some data surrounding a failure.

So if you are installing DRBD with the aim of purely "availability", and are not worried about losing that last write on the crash to your master database that (hopefully) happens only once every few years, you may be using the wrong technology.

While the prized "1 minute failover" is possible in DRBD, it doesn't really explain the full picture. The typical crash …

Wow, check out what just came out from Linbit: The DRBD Management Console. Written in Java (so it runs anywhere), completely open source (GPLv3), and allows you to manage DRBD and Heartbeat based clusters. You can install, configure, see your systems graphically, and a lot more. I’m interested to try the beta out, as soon as I get back to my lab (sitting in the airport now). If you know how to use DRBD/Heartbeat, and use it in production for your MySQL setup, it might be a good application to test out, and improve if need be.

From the screenshots, I’m surprised this isn’t a value added extra that Linbit would like to charge for. Kudos, Linbit, for keeping it GPLv3!

…