In my last post, I learned in disappointing fashion that sometimes you need to start small and work your way up, rather than trying to put together a finished product. This go-round, I’ll talk about my investigation into disk IO.

In an effort to better understand the hardware I have and it’s capacities, I started off by just trying to get some basic info about the RAID controller and the disks. This hardware in particular is a Supermicro, with a yet unknown RAID controller and 16 4TB disks arranged in RAID 6. Finding out more disk and controller information was the first step. “hdparm -i” wasn’t able to give me much, nor was “cat /sys/class/block/sdb/device/{model,vendor}”. “dmesg” …

As many learning experiences do, this one also starts out “So I was working on a project at work and…”.  In this case, the end result is to try to run as many concurrent copies of MySQL on a single server as possible, maintaining real time replication each running differing data sets. To help with this, I sent out to do this on a server with 36 7200rpm 4GB SATA disks, giving me roughly 120TB of available space to work with.

This isn’t an abnormal type of machine for us. Sometimes you simply need a ton of disk space. There is a quirk with this particular machine that I’ve been told: the RAID controller has some issues with addressing very large virtual disks and I should create 2 60TB volumes and stitch them together with LVM. Easy enough: pvcreate both volumes, create a volume group and a logical volume out of it and viola: …

Computer science is like an enormous tool box you can rummage through whenever you have a problem to solve. Most of the tools are sturdy and practical, like algorithms for B-trees. Some are also elegant, like consistent hashing in Dynamo. Finally there are some tools that you never quite figure out even after years of reflection. That piece of steel you are looking at could be Excalibur. Or it could be a rusty knife.

The CAP theorem falls into the last category, at least for me.  It was a major topic in the blogosphere a few years ago and Google Trends shows steadily increasing interest in the term since 2010.  It's not my goal to explain CAP fully--a good informal description is …

MySQL 5.6.21 was recently released (it is the latest MySQL 5.6, is GA), and is available for download here.

For this release, there was 1 “InnoDB Notes” and 1 “Functionality Added or Changed” bug fix (and 0 “Security Fix”), so not much there, but of course they should be noted:

1. InnoDB Note: The –skip-innodb option is now deprecated and its use results in a warning. It will be removed in a future MySQL release. This also applies to its synonyms (–innodb=OFF, –disable-innodb, and so forth).
2. Functionality Added: Internally, spatial data types such as Geometry are represented as BLOB values, so when invoked with the –hex-blob option, mysqldump now displays spatial values in hex. (Bug #43544, Bug …

This week I have been spending a little bit of time creating a module for Sysbench so that it can use libAttachSQL as a database driver. The reason for doing this is twofold:

1. Brian (my boss at HP's Advanced Technology Group) said now would be a good time to benchmark libAttachSQL
2. I really needed more than a few basic queries to shake out bugs in the library, and a benchmark is a good way to shove a few million through it

On the bug front, it did find a total of 5 bugs, a couple of them serious. Which is great, the more testing and bug finding the better. These have all been fixed in GitHub and will be part of the upcoming 0.5.0 release.

The Test Setup …

HAProxy is frequently used as a load-balancer in front of a Galera cluster. While diagnosing an issue with HAProxy configuration, I realized that logging doesn’t work out of the box on CentOS 6.5. Here is a simple recipe to fix the issue.

If you look at the top of /etc/haproxy/haproxy.cfg, you will see something like:

global
    log         127.0.0.1 local2
[...]

This means that HAProxy will send its messages to rsyslog on 127.0.0.1. But by default, rsyslog doesn’t listen on any address, hence the issue.

Let’s edit /etc/rsyslog.conf and uncomment these lines:

$ModLoad imudp
$UDPServerRun 514

This will make rsyslog listen on UDP port 514 for all IP addresses. Optionally you can limit to 127.0.0.1 by adding:

$UDPServerAddress …

Generated Columns is a new feature available in the latest lab release. This work is based on a contribution by Andrey Zhakov. Thanks, Andrey! The Optimizer team modified it to follow the current MySQL design, and to lift a number of limitations.

The syntax is:

<type>   [ GENERATED ALWAYS ]   AS   ( <expression> )   [ VIRTUAL|STORED ]
[ UNIQUE [KEY] ]   [ [PRIMARY] KEY ]   [ NOT NULL ]   [ COMMENT <text> ]

There are two kinds of Generated Columns: virtual (default) and stored. Virtual means that the column will be calculated on the fly when a record is read from a table. Stored means that the column will be …

libAttachSQL Benchmarks With Sysbench

This week I have been spending a little bit of time creating a module for Sysbench so that it can use libAttachSQL as a database driver. The reason for doing this is twofold:

1. Brian (my boss at HP’s Advanced Technology Group) said now would be a good time to benchmark libAttachSQL
2. I really needed more than a few basic queries to shake out bugs in the library, and a benchmark is a good way to shove a few million through it

On the bug front, it did find a total of 5 bugs, a couple of them serious. Which is great, the more testing and bug finding the better. These have all been fixed in GitHub and will be part of the …

Now it's time to release something useful! At least I hope so. I have been going through how I came up with this idea and how I came up with the implementation in a series of blog posts:

• Part 1
• Part 2
• Part 3
• Part 4 

 But now it's time for the real deal, the software itself. This is an Alpha 1.0 release but it should work OK in the more basic setups. It's available …

The InnoDB labs release includes a snapshot of the InnoDB Native Partitioning feature.

To better understand why we implemented this, we need to start with some background on tables, storage engines, and handlers. In MySQL an open instance of a table has a handler object as an interface to the table’s storage engine. For a partitioned table there is a main table handler that implements the partitioning feature, but for storage, each partition has its own handler. This worked fairly well, but the more partitions you had the more overhead from the per partition handlers. So to remove this overhead for partitioned InnoDB tables we’re introducing Native Partitioning support! This means a new InnoDB partitioning aware handler, so that we have a single handler object for a partitioned table and not one handler object per partition.

Let us create a simple table with 8k partitions: