Recently we have helped our customer to migrate their entire application stack from one data center to another. Before we were brought on-board, customer had already placed an order for a new set of servers with the new hosting provider. All of them were suppose to be high-end systems – many CPU cores, plenty of RAM and RAID array build on top of SSD drives. As the new machines started being available to us, we began setting up the new environment. At some point it turned out that the new machines were actually slower compared to the several year old systems and their load was much higher under comparable traffic.

We examined several of the new servers and each time the conclusion was that the problems were related poor I/O performance. In the benchmarks a RAID 10 array on Intel SSD 330 Series drives was barely able to achieve 200-300 IOPS in random writes and even that at the cost of …

A while ago I had a discussion with someone about the future of server infrastructure. Among other things, we were wondering whether companies will continue to run on dedicated servers or if eventually everyone just ends up in a Cloud environment. During the discussion I raised a point that in principle Cloud is a great idea that will keep attracting more and more people, but it is missing one important piece that stops many from using it – a high performance storage. Apparently, this has just changed.

Yesterday I received an e-mail announcing a new EC2 instance type – hi1.4xlarge. It features 16 logical CPUs (35 ECUs), 60GB of RAM, and… two 1TB SSD-based disk volumes! These are great specs that should work for nearly any database. Even assuming someone has a MySQL database larger than 2TB, not all tables will require fast storage, while more disk space can be easily added by attaching regular EBS volumes.

I decided …

There are probably thousands of articles on the Internet about disk statistics in Linux, what various columns mean, how accurate the information is, and so on. I decided to attack the problem from a little bit more practical side. Hopefully this will be just the first of many future posts on identifying various I/O related performance problems on a MySQL server.

Linux exposes disk statistics through /proc/diskstats. However the contents of this file isn’t something anyone can understand quickly. It needs a tool to transform the information into something human readable. A tool that is available for any Linux distribution is called iostat and comes with sysstat package.

How to access and read I/O statistics

Usually you want to call iostat one way:

iostat -xkd <interval> <block device>

The interval should typically be one second as it is the …