Table optimization is a necessary evil; tables sometimes need to be optimized to reclaim space or to improve query performance.  Unfortunately, MySQL blocks writes to a table while it is being optimized.  Because optimization time is proportional to the table size, writes can be blocked for a long time.  Fractal Tree indexes support online optimization; however, the MySQL metadata lock gets in the way of writing while optimizing.  We will describe a simple patch to MySQL that enables online optimization of TokuDB tables.

Why do tables need to be optimized?  Here are some reasons.

• Insertions with random keys can result in a tree with underutilized leaf blocks.  Many tree algorithms split nodes in half when they become full.  If these nodes are stored in fixed sized blocks, like many B-trees do, then there can be a lot of wasted space.  Table optimization of B-trees write blocks with less …

We just ran into a nasty InnoDB bug that only seems to exist in MySQL 5.5.25:

An InnoDB update that modifies a rows primary key can trigger some recursive behavior that creates new rows until all disk space is exceeded. This does not affect all primary key updates in general but only gets triggered when a few other conditions are also met, so you're not too likely to run into it, but if you do your mysqld server will waste a lot of IO bandwidth and storage space on this and will usually eventually run out of disk space.

read more

As a TokuDB storage engine developer, numerous times I’ve been struck by the similarities between MySQL Cluster and TokuDB. Namely, many times where I find myself thinking, “TokuDB would benefit from this feature”, I also end up thinking “MySQL Cluster would benefit from this feature” as well.

At first glance, one may wonder why. TokuDB is a storage engine designed to work well on big data, providing compression, agility, and performance, while MySQL Cluster is a distributed database solution (http://www.mysql.com/products/cluster/) that provides (among many other things) auto sharding and 99.999% availability. TokuDB’s innovation, Fractal Trees® indexes, are designed to drastically reduce the number of disk seeks performed, but TokuDB still operates on a hard disk. MySQL Cluster operates over a network. How can we be two peas in a pod?

But when I …

Last week one of my collagues came to me with the following problem. He was not able to create an innodb table. MySQL returned ERROR 1005: Can’t create table (errno: -1). Most of the times this problem is a foreign key related issue but now this wasn’t the case (Foreign key error will return errno: 150). What actually happened is that ibdata already contains the registration of the new table but the frm file wasn’t there. This way when he wanted to create the table the server responded with error. Although he wasn’t able to remove the table either. It was caused the file-by-file cloning of the database.

For better understanding you have to be familiar with the inner operations of MySQL. With the most simplistic explaination I could say that MySQL is just working on a high SQL abstraction level which can use different storage engine for storing datas. Viewing from this aspect MySQL doesn’t care how the files …

Lately I’ve received a number of SHOW ENGINE INNODB STATUS outputs with embedded “\n” characters appearing as text, but not true line breaks.

Of course this makes reading a long, INNODB STATUS nearly impossible.

With Linux, you can more easily fix this using sed or awk (or if you have those on Windows w/ gnu or cygwin or the like).

I didn’t have either handy, nor a fancy editor (just using WordPad), but I did have a hex editor handy (necessary for such cases as this).

In the hex editor (I use Freeware Hex Editor XVI32 for Windows), fixing this took about 2 seconds.

I just replaced:

5C6E <-- which is the text "\n"

with:

0D0A <-- which is the true carriage return/newline

I realize …

As I mentioned here, there is a slight change for enabling the [original] InnoDB Plugin in MariaDB 5.5 (as compared to how you would enable it in 5.1).

Remember, in MariaDB 5.5, if you do not “enable” (i.e., add anything to the config file to do so) the InnoDB Plugin in MariaDB 5.5, you’ll end up with XtraDB+ for your InnoDb plugin. However, if you do “enable” the InnoDB plugin, then you end up with the original InnoDB plugin provided by Oracle/InnoDB.

The change is that the plugin file (.dll for Windows, .so file for Linux) which was previously named “ha_innodb_plugin.dll” is now just “ha_innodb.dll”.

Thus, if you previously enabled the plugin with (would have been in a 5.1 instance):

[mysqld]
ignore_builtin_innodb
plugin-load=innodb=ha_innodb_plugin.dll;innodb_trx=ha_innodb_plugin.dll; …

I built MariaDB 5.5.24 on Windows yesterday from source, so I just wanted to share my steps.

Here is the short version:

bzr branch lp:maria/5.5 maria-5.5
cd maria-5.5
mkdir bld
cd bld
cmake .. -G "Visual Studio 10"
cmake --build . --config relwithdebinfo --target package

Done! Nice, neat zip file is created right there.

Here is the longer version with outputs for those interested:

C:\mariadb-5.5>bzr branch lp:maria/5.5 maria-5.5
Connected (version 2.0, client Twisted)
Authentication (publickey) successful!
Secsh channel 1 opened.
Branched 3418 revision(s).

C:\mariadb-5.5>cd maria-5.5

C:\mariadb-5.5\maria-5.5>mkdir bld

C:\mariadb-5.5\maria-5.5>cd bld

C:\mariadb-5.5\maria-5.5\bld>cmake .. -G "Visual Studio 10"
-- Check for working C compiler using: Visual Studio 10
-- Check for working C compiler using: Visual Studio 10 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- …

XtraBackup is a great piece of software from Percona, which allows creating (nearly) lock-less MySQL/InnoDB backups. The tool has been around for quite some time and recently even received a major version bump. I have relied on it many times over the years. As it turns out, using it in some configurations may lead to heavy swapping or prevent MySQL from running queries.

So far I only kept complaining about the wrapper script XtraBackup has been distributed with and which was taken from Oracle’s InnoDB Hot Backup. The infamous innobackupex-1.5.1 was neither well written, nor was it even fully compatible with the XtraBackup’s feature set. This sometimes led to weird problems where there should not be any.

This time the problem can appear elsewhere. Mostly when one using the tool does not understand how it works in certain circumstances.

How does XtraBackup work?

In a few short words. XtraBackup …

You could say: what could be the reason for having really big number of tables? Just design the application properly! It’s not always that easy. And this post isn’t really about arguing whether having many tables is good or not, it’s about what happens in terms of memory usage if you already reached that point.

Btw what do I mean by *many*? From my experience it’s tens of thousends or even millions rather than hundreds.

The inspiration for me to write this post was strong desire to try out the latest declared improvements in that area announced to be done in MySQL 5.6. _1

What I did was a very simple test where I loaded sql dump of databases and tables definitions only into different versions of MySQL. During the load time I was monitoring memory usage of mysqld process (RSS …

Tackling machine data on the ground to ensure successful operations for NASA in space

Issues addressed:

• Scaling MySQL to multi-terabytes
• Insertion rates as InnoDB hit a performance wall
• Schema flexibility to handle an evolving data model

The Company:  Southwest Research Institute (SwRI) is an independent, nonprofit applied research and development organization. The staff of more than 3,000 specializes in the creation and transfer of technology in engineering and the physical sciences. Currently, SwRI is part of an international team working on the NASA Magnetospheric Multiscale (MMS) mission. MMS is a Solar Terrestrial Probes mission comprising four identically …