Even though multiple fixes have been implemented in Percona Server and MySQL 5.5, there are still workloads in which case mutex (or rw-lock) contention is a performance limiting factor, helped by ever growing number of cores available in the systems. It is interesting though the contention may manifest itself in the different form from the system monitoring standpoint. In many cases as heavy contention happens user CPU will be very high, and the context switches will be somewhere reasonable. In others you would see the CPU usage being low with a lot of CPU being idle, increased compared to normal workload portion of system CPU and high number of context switches. These correspond to different contention situations which can be handled differently.

First situation often corresponds to Innodb spending a lot of CPU time running “loops” as part of spinlock implementation. In many cases busy wait is indeed more efficient than doing …

I got a serendipitous call (thanks!) yesterday asking what would be needed to tune[1] a database for better performance. It is a question that I hear often, but I never thought about answering it in public. Here’s a consolidated version of what I explained during our conversation.

Have realistic expectations about configuration

The first thing to know is that server configuration itself really isn’t something you should expect to deliver huge wins. If MySQL is actually badly configured, you can hurt its performance significantly. Correcting these mistakes can correspondingly improve performance. But such mistakes are relatively few and/or non-obvious to make. A few of the common ones I see are not configuring the InnoDB buffer pool size or log file size, and not using InnoDB. If your server really hasn’t been configured — that is, it’s running with a default configuration — then it’s quite possible you have …

A while back, I wrote a two part post on how you can extract an amazing amount of information about a system’s performance, scalability, queueing, and more by just measuring request arrivals and completions, and the timestamps thereof.

I promised to develop this into a more complete description of how to analyze MySQL’s performance and scalability through nothing more than TCP/IP packet headers. I presented the results of that research at Percona Live in New York City, and the slides and video are online. The video editors didn’t synch the slides …

If you need to change MySQL’s innodb_log_file_size parameter (see How to calculate a good InnoDB log file size), you can’t just change the parameter in the my.cnf file and restart the server. If you do, InnoDB will refuse to start because the existing log files don’t match the configured size.

You need to shut the server down cleanly and normally, and move away (don’t delete) the log files, which are named ib_logfile0, ib_logfile1, and so on. Check the error log to ensure there was no problem shutting down. Then restart the server and watch the error log output carefully. You should see InnoDB print messages saying that the log files don’t exist. It will create new ones and then start. At this point you can verify that InnoDB is working, and then you can delete the old log files.

The typical error …

This is cross-posted from http://www.ssdperformanceblog.com/2011/07/fusionio-720gb-write-performance/

I’ve got a FusionIO card with 720GB capacity on my hands.

It came with a HP ProLiant DL380 G6 server. Interesting that this card is not listed on FusionIO’s products page, and neither I see such card in the list of available configurations on HP’s site. I guess this card comes as some customization option.

It seems to be a MLC card (I did not hear about FusionIO SLC cards with a capacity greater than 320GB) and cost is always an interesting question. On HP.com I can find a HP IO Accelerator (which is a re-branded FusionIO card) …

I wrote about Galera about 1.5 years ago: State of the art: Galera – synchronous replication for InnoDB. It was about the 0.7 release, which was more like a proof-of-concept release (though Galera’s developers may not agree with that ) with some serious limitations (like using mysqldump for node propagation). The Galera team heard my suggestions and the new 0.8 release looks very promising. Well, it took 1.5 years to fix the limitations and come up with new features, but there is nothing to complain about it – a synchronous distributed transactional system is not an easy problem to solve, trust me.

So Galera 0.8 comes with many nice features:

• Works with MySQL 5.1 and MariaDB 5.1. The latest is more interesting for us, as it is based on XtraDB. That means Galera supports the XtraDB …

In the first part of this article I have showed how I align IO, now I want to share results of the benchmark that I have been running to see how much benefit can we get from a proper IO alignment on a 4-disk RAID1+0 with 64k stripe element. I haven’t been running any benchmarks in a while so be careful with my results and forgiving to my mistakes

The environment

Here is the summary of the system I have been running this on (for brevity I have removed some irrelevant information):

# Aspersa System Summary Report ##############################
    Platform | Linux
     Release | Ubuntu 10.04.2 LTS (lucid)
      Kernel | 2.6.32-31-server
Architecture | CPU = 64-bit, OS = 64-bit
# Processor …

Now that flash storage is becoming more popular, IO alignment question keeps popping up more often than it used to when all we had were rotating hard disk drives. I think the reason is very simple – when systems only had one bearing hard disk drive (HDD) as in RAID1 or one disk drive at all, you couldn’t really have misaligned IO because HDDs operate in 512-byte sectors and that’s also the smallest amount of disk IO that systems can do. NAND flash on the other hand can have a page size of 512-bytes, 2kbytes or 4kbytes (and often you don’t know what size it is really) so the IO alignment question becomes more relevant.

It was and still is, however, relevant with HDD RAID storage – technology we have been using for many years – when there’s striping like in RAID0, 5, 6 or any variation of them (5+0, 1+0, 1+0+0 etc.). While IO inside the RAID is perfectly aligned to disk sectors (again due to the fact operations are done in …

Released on June 8, 2011 (Downloads are available here and from the Percona Software Repositories.

Percona Server 5.1.57-12.8 is now the current stable release in the 5.1 series. It is is based on MySQL 5.1.57.

Bug Fixes

• Fixed InnoDB I/O code so that the interrupted system calls are restarted if they are interrupted by a signal. InnoDB I/O code was not fully conforming to the standard on POSIX systems, causing a crash with an assertion failure when receiving a signal on pwrite(). Bug Fixed: LaunchPad: #764395 / MySQL bug #60788 (A. …

One of item I always look at SHOW ENGINE INNODB STATUS to see if there are any transactions spending very long time in ACTIVE state. In the perfect world if you’re running online system you should not see transactions spending more than couple of seconds in ACTIVE state. Especially ACTIVE transactions which do not currently run any query are suspicious. There are however cases when screaming fire about ACTIVE transactions alone would be misleading. There is a whole set of applications which run quite fine while having ACTIVE measured in hours. It is JAVA applications which often run in AUTOCOMMIT=0 mode and do not explicitly commit transactions unless there were any writes. If database is configured in READ-COMMITTED transaction mode it is actually fine from performance point of view as Innodb does not have to preserve row versions going back to start of transactions.

There is however a better metric, which in most cases will …