One of the things that repeatedly seem to puzzle users about the DRBD is the question of whether to use internal or external metadata. Remember, DRBD sets aside a small area on a local disk (on every cluster node) where it keeps the Activity Log, the quick-sync bitmap, data generation UUIDs, and a few other bits and pieces for local housekeeping.
The specific aspect that is to be discussed here is the Activity Log. Without going into too much detail, let’s be satisfied with the factoid that DRBD[Read more...]
As someone may have noticed, I recently wrote a trilogy on how to dive into the MySQL Cluster source code. Unfortunately my overtures towards the MySQL Cluster source code ended up being only a look-but-don't-touch affair, as I failed to actually get to touch her internals with my text editor. Even so, in this post I'd like to tell about the background to my love affair with this beauty, by relating to some benchmarks I've been working on together with my customers.
Oh, and I'd like to apologize already, that I cannot mention where these benchmarks were done, what the schema looked like and the exact numbers. If you want that kind of real benchmarks, you should read Mikael's blog, or watch the slides from this webinar[Read more...]
This is part II of my efforts to prove myself that I can do programming. In part one I successfully created a MySQL Cluster branch for myself and compiled it.
Let's go to the public MySQL bug database and see if there are any trivial MySQL Cluster bugs I could sharpen my teeth on. Heh, sure enough #32658 looks simple enough. There is a typo in an output string - so I could fix that without even doing any C++ code! (Funnily, a MySQL internal comment to the bug says something about it being embarrassing. Guess it is a good bug for me then, as patching over embarrasments is what Sales Engineers do routinely :-)
Let me see...
My collagues Anders and even Ivan sometimes blog about the grandeur of being a Sales Engineer. And I agree, it is a great job, probably the best I ever had, so far. But let me share a secret: It's not as technical as you'd think. Sure, they call me a "pre-sales consultant" alright, but I would be ashamed of comparing my own work with those of the real consultants. I sometimes jokingly say that the most amazing technical things in my job are airplanes (they fly in the air!) and how to make a nice slideshow. (OpenOffice Impress sucks btw, and I always envy my OS X + Keynote using friends on this one[Read more...]
By popular demand, we are now offering an on-line incarnation of our DRBD Total training sessions, normally taught in a 4-day on-site course. The next such training commences on May 18 (next Monday), and we still have a few seats left — so if you’re interested, grab one while you still can!
Here is an overview of the course highlights:
Every course attendee gets a virtual[Read more...]
Both DRBD and Dolphin Express already being a fixture in the MySQL (http://www.mysql.com) universe, this webinar should be particularly interesting to those of you who want to minimize database write latency while maintaining fully redundant and transaction-safe high availability. However, it’s also a must see for virtualization and mail service DRBD users, who also typically have a need for low latency.
This is, of[Read more...]
I’ve been asked by a number of people on how to do an upgrade from DRBD version 0.7 to DRBD 8. This upgrade does necessitate some minimal service down time, but it’s really not rocket science. And no, it does not force you to sync all of your data all over again.
Here’s my quick write-up.
Do not, I repeat do not attempt your upgrade unless you have at least read this blog entry to the finish.
First, you need to make sure that you have both your DRBD 8 userland binaries and kernel module ready to install. For our support customers, this means that you simply download two RPMs (or .debs) from our support web site. Make sure you have the right packages; you want those that match your system architecture and (for the kernel module) also your running[Read more...]
Split brain, with DRBD, is much less of a disaster than in conventional cluster setups employing shared storage. But, you ask, how can I protect my DRBD cluster against split brain in the first place? Here’s how.
Let’s briefly reiterate what split brain, in the DRBD sense, really means. DRBD split brain occurs when your nodes have lost their replication link due to network failure, and you make both nodes Primary after that.
When just the replication link dies, Heartbeat as the cluster manager will still be able to “see” the peer node via an alternate communication path (which you hopefully have configured, see this post). Thus, there is nothing that would keep Heartbeat from migrating resources to that DRBD-wise disconnected node if it so decides[Read more...]
DRBD 8.2.0, released today, includes a much requested new feature, embodied in the new
data-integrity-alg configuration option: DRBD protocol level data integrity checksums.
A few months ago, some users alerted us to DRBD replication issues where DRBD supposedly “ate their data”, i.e. corrupted replicated data in transit. Eventually we traced those problems not to DRBD errors, but in fact to network drivers messing up TCP checksums or segmentation. Typically this was related to using either TCP segmentation offloading (TSO) or TCP checksum offloading. However, at the time DRBD had no way of detecting these errors — you would only find out if you switched over to your Secondary, only to find your data not having been replicated properly.
With DRBD 8.2.0, you[Read more...]
These days, we seem to be getting a lot of inquiries from new or would-be DRBD adopters, especially MySQL (http://www.mysql.com" target="_blank) and PostgreSQL DBAs wanting to add high availability to their master database servers. And these, unsurprisingly, turn out to be two of their most popular questions:
How will using DRBD affect my write performance?
… and …
What are DRBD’s most important tunables with regard to write performance?
Let’s take a look at both of these issues. Ready? Let’s go.
Basically, there’s usually one important potential bottleneck in any DRBD “Protocol C” (synchronous replication) setup, and it’s not the network connection. With ubiquitous Gigabit[Read more...]
As I’m sure you’re aware, DRBD disallows access (any access, including read-only) to a DRBD device in Secondary mode. This always raises questions like the one I’ve taken the liberty to quote here. It came up in a MySQL webinar (http://www.mysql.com/news-and-events/on-demand-webinars/" target="_blank) on replication and HA:
Because of the asynchronous nature of [MySQL] replication we end up with a dilemma when looking at using slaves as read nodes in that the only time we go to the database for information is to build a local cache file, and that local cache file is ONLY removed when information related to that cache file changes, it is NOT based on time. If we had a synchronous method of replication we would then know the cache files
Some applications require direct access to a block device, without an intermediate file system. Some Oracle and MySQL configurations are an example, as are some Xen setups, or IET. Can you do this with DRBD? Sure you can.
However, you need to fulfill two prerequisites:
Secondaryrole. So, you must make sure DRBD is
Primarybefore your application attempts using that device.
Your cluster manager, when configured properly, normally takes care of item #1 for you. #2 is a little trickier:
Normally, DRBD’s device nodes are owned by
root:disk, with permission bits set to
rw-rw----). So in order to allow your