mysql -B -N -e 'show databases'
information_schema
employees
five
four
mysql
one
performance_schema
six
test
three
two

The open-source xtrabackup tool from Percona brings much needed hot backup functionality to MySQL deployments.  In this database journal article we discuss logical, cold, and hot backups, then explain how to use xtrabackup on your MyISAM, InnoDB, and XtraDB tables to create at-the-ready backups.  We then take you through the step-by-step process to restore them, and even the process of point-in-time recovery too.

Database Journal – Hotbackups with Percona’s Xtrabackup

I fully agree with Morgan’s An argument for not using mysqldump. This post does not come to contradict it, but rather shed a positive light on mysqldump.

I usually prefer an LVM snapshot based backup, or using XtraBackup. And, with databases as large as dozens of GB and above, I consider mysqldump to be a poor alternative. Poor in runtime, poor in overhead while taking the backup.

However once in a while I get to be reminded that mysqldump just works.

As a recent example, I had a server which was killed after an ALTER TABLE statement hanged forever (table already ALTERed, but old scheme never dropped). The old table data still hanged around the file system, but was not recognized by InnoDB. Trying out DISCARD TABLESPACE

Let’s suppose that your backup process looks like this: you stop a replication slave, shut down MySQL, and copy away the data directory. Assume that the slave is perfect and has the same data as the master. Nothing is broken, nothing is wrong, everything is working fine. In most cases, this should work, right?

Under what kinds of circumstances will you not get all your data back if you restore the file copy and start MySQL?

Related posts:

This HOWTO discusses two (unrelated) issues with mylvmbackup:

• The minimal privileges required to take MySQL backups with mylvmbackup.
• Making (non compressed) file system copy of one’s data files.

Minimal privileges

Some just give mylvmbackup the root account, which is far too permissive. We now consider what the minimal requirements of mylvmbackup are.

The queries mylvmbackup issues are:

• FLUSH TABLES
• FLUSH TABLES WITH READ LOCK
• SHOW MASTER STATUS
• SHOW SLAVE STATUS
• UNLOCK TABLES

Both SHOW MASTER STATUS & SHOW SLAVE STATUS require

LVM uses copy-on-write to implement snapshots. Whenever you’re writing data to some page, LVM copies the original page (the way it looked like when the snapshot was taken) to the snapshot volume. The snapshot volume must be large enough to accommodate all pages written to for the duration of the snapshot’s lifetime. In other words, you must be able to copy the data somewhere outside (tape, NFS, rsync, etc.) in less time than it would take for the snapshot to fill up.

While LVM allows for hot backups of MySQL, it still poses an impact on the disks. An LVM snapshot backup may not go unnoticed by the MySQL users.

Some general guidelines for making life easier with LVM backups follow.

Lighter, longer snapshots

If you’re confident that you have enough space on your snapshot volume, you may take the opportunity to make for a

Actually this is (almost) all I wanted to say. This is intentionally posted with all related keywords in title, in the hope that a related search on Google will result with this post on first page.

I’m just still encountering companies who use MyISAM as their storage engine and are unaware that their nightly backup actually blocks their application, basically rendering their product unavailable for long minutes to hours on a nightly basis.

So this is posted as a warning for those who were not aware of this fact.

There is no hot (non blocking) backup for MyISAM. Closest would be file system snapshot, but even this requires flushing of tables, which may take a while to complete. If you must have a hot backup, then either use replication – and take the risk of the slave not being in complete sync with the master – or use another storage engine, i.e. InnoDB.

Few years ago I was looking at crash recovery code, and realized that InnoDB has removed all the comments from the code [this assumption is debunked by Heikki in comments section], related to replay of transaction log. Judging by high quality of comments in the remaining codebase, I realized that it was all done to obscure any efforts to build another InnoDB hot backup solution – competitor to first Innobase standalone offering.

I was enjoying the moment when Percona launched their own implementation of the tool. Since the inception, it became more and more robust and feature rich.

We have used xtrabackup in our environment a lot – just… not for backup – the major use case right now is for cloning server instances – either

USE `schema`;

The actual trick

MySQL Backup Script has been around for a long time.  I have used it on and off for years but now I’ve needed to make some  improvements.   This script is based on VER. 2.6 – http://sourceforge.net/projects/automysqlbackup/ Copyright (c) 2002-2003 wipe_out@lycos.co.uk.
I have added my own Copyright (c) 2010 mark@grennan.com – http://www.mysqlfanboy.com/Files/automysqlbackup.sh. But as the code says:  This program is distributed in the hope that it will be useful,  but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

My improvements include:

# VER 2.6 Beta 5 – MTG – (2010-04-18)

How it works

Today I ran into my first MySQL binlog race condition: The initial problem was quite simple: A typical MySQL master->slave setup with heavy load on the master and nearly no load on the slave, which only serves as a hot fallback and job machine, showed differences on the same table on both machines. The differences showed up from time to time: entries that have been deleted from the master were still on the slave.

After several investigations I started examining the MySQL binlog from the master – a file containing all queries that will be transferred to the slave (and executed there if they don’t match any ignore-db-pattern). I grepped for ids of rows that have not been deleted on the slave as I’s interested if the DELETE statement was

Following up on Ronald Bradford’s Checked your MySQL recovery process recently? post, I wish to add a prequel.

To see whether you have a clear definition of your backup requirements, ask yourself these questions:

• Is there a backup/restore plan?
• Is there a written backup/restore plan?
• How fast do you need to recover a backup? What’s the longest downtime you will allow from the point of failure to the point of restored, functional database?
• How much data are you willing to lose in case of crash? A second’s worth of data? An hour’s worth? A day’s worth? None?
• Are you willing to allow that the database becomes read-only when taking the

Oracle

Do you

Check out how Linden Labs, creators of the popular game Second Life, upgraded their MySQL database. The MySQL they use? Straight out of Debian! Of course, now, they’re running with the Percona patchset, against MySQL 5.0.84. Definitely a good read.

Its good to see Lars post about contributing to the MySQL replication & backup codebase. It sounds like the replication & backup team have decided that mentoring is the way to go – you get a “coach developer” if the idea is accepted. I like this very much, and sincerely hope it

Following Restore one table from an ALL database dump and Restore a Single Table From mysqldump, I would like to add my own thoughts and comments on the subject.

I also wish to note performance issues with the two suggested solutions, and offer improvements.

Problem relevance

While the problem is interesting, I just want to note that it is relevant in very specific database dimensions. Too small – and it doesn’t matter how you solve it (e.g. just open vi/emacs and copy+paste). Too big – and it would not be worthwhile to restore from mysqldump anyway. I would suggest that the problem is interesting in the whereabouts of a few dozen GB worth of

What If DBAs Have No sudo?