EMC launches Greenplum HD. DataStax releases Brisk. And more.

# EMC launched its Greenplum HD Hadoop distribution, with the support of Jaspersoft, Pentaho, and SnapLogic, among others.

# DataStax …

DRBD and semi-sync benchmarks on a 2x8 132 GB server

I recently had the opportunity to run some benchmarks against a relatively large server, to learn how it was behaving in its specific configuration. I got some interesting results that I'll share here.

read more

This month’s Maatkit release includes a new tool that’s kind of an old tool at the same time. We wrote it a couple years ago for a client who has a very large set of tables and many queries and developers, and wants the database’s schema and queries to self-document for data-flow analysis purposes. At the time, it was called mk-table-access and was rather limited — just a few lines of code wrapped around some existing modules, with an output format that wasn’t generic enough to be broadly useful. Thus we didn’t release it with Maatkit. We recently changed the name to mk-table-usage (to match mk-index-usage), included it in the Maatkit suite of tools, and enhanced the functionality a lot.

What’s this tool good for? Well, imagine that you’re a big MySQL user and you hire a new developer. Now you need to bring the new person up to speed with your environment. Or, you want to …

I always have difficulties with complex analysis schemes, so fall back to something that is somewhat easier. Or much easier. Here I will explain the super-powerful method of database write workload analysis.

Doing any analysis on master servers is already too complicated, as instead of analyzing write costs one can be too obsessed with locking and there’s sometimes uncontrollable amount of workload hitting the server beside writes. Fortunately, slaves are much better targets, not only because writes there are single-threaded, thus exposing every costly I/O as time component, but also one can drain traffic from slaves, or send more in order to cause more natural workload.

Also, there can be multiple states of slave load:

• Healthy, always at 0-1s lag, write statements are always immediate
• Spiky, usually at 0s lag, but has jumps due to sometimes occuring slow statements
• Lagging, because of …

I decided to try the star schema benchmark on our latest 7.2 release (link). Star schema benchmark is an analytics oriented benchmark, and MySQL Cluster has not been developed to address this kind of workload. Nevertheless I couldn't resist trying...
Setup

• 2 data-nodes each running on a 4-way Xeon E7420 @ 2.13GHx (total 16 cores) 256Gb RAM
• The mysqld was co-located with one of the data-nodes
• I used memory tables
  

Results
Queries: link

Vadim has taught me that valid benchmarks are both simple and complex. Simple, because the basic principles are few; complex, because the devil is in the details and it’s a lot of work to satisfy the basic requirements. I’ll give the simple version here.

• Benchmarks must be appropriate. The workload, sample dataset, distribution of work and data, and so on must be relevant and meaningful for the intended purpose. Running the wrong benchmark rarely teaches anything.
• Benchmarks must be fully documented. Another researcher must be able to determine exactly how you ran your benchmark, on what hardware, under what workload, what operating system, kernel version, all MySQL tuning parameters, and so on.
• Benchmarks must be repeatable. Another researcher must be able to reproduce your results. Documentation is part of this, but you need to ensure that you can reproduce your own results. If you can’t, no one else …

Here’s why I do: it’s time for me to build a new master database server. Our current main slave is too underpowered to be handle our entire load in an emergency, which means that our failover situation isn’t that great. I’ll replace the master with something new and shiny, make some performance improvements while I’m at it, and the old master will work just fine in an emergency.

For IO intensive servers, I conserve space and electricity by using 1U machines with 6 or 8 2.5″ drives.

I’d normally buy 8 Seagate Savvio 15K SAS drives and set them up as a RAID 10 array. This would run me about $1850.

We’re pretty frugal when it comes to our technology budget and I can’t really stomach spending that kind of money to effectively get 550 GB of redundant, fast magnetic disk storage. SATA MLC SSDs that blow traditional drives out of the water are currently under $2 / GB.

Disclaimer

So I’ve been doing a fair number of automated load tests these past six months. Primarily with Sysbench, which is a fine, fine tool. First I started using some simple bash based loop controls to automate my overnight testing, but as usually happens with shell scripts they grew unwieldy and I rewrote them in python. Now I have some flexible and easily configurable code for sysbench based MySQL benchmarking to offer the community. I’ve always been a fan of giving back to such a helpful group of people – you’ll never hear me complain about “my time isn’t free”. So, let me know what you want in an ideal testing environment (from a load testing framework automation standpoint) and I’ll integrate it into my existing framework and then release it via the BSD license. The main goal here is to have a standardized modular framework, based on sysbench, that allows anyone to compare their server performance via repeatable tests. It’s fun to see …

I started going over the slides from talks from MySQL Conference and Expo 2011 to pick set of must see presentations and publish the list, but this is not happening due to lack of time. Instead I’m only going to recommend 1 talk, from list of tutorials I had a chance to review. If you have not check it out yet, take a look at Linux and Hardware Optimizations by Yoshinori Matsunobu. Check out the slides which are very well done with many graphs and verbose enough explanations so you can easily follow slides alone.

Were you excited by some other talk on MySQL Conference and Expo ? Feel free to leave it as a comment to this post.

Are you rebooting your MySQL server at night?

I'de like to share input send to me by a leading MySQL provider, mostly deploying ecommerce and telco applications on Linux.

One instance is up since 1414 days not bad ....

 
18:13:02 up 355 days,  4:50,  0 users,  load average: 0.00, 0.00, 0.00
18:13:03 up 594 days,  5:04,  0 users,  load average: 1.20, 1.25, 1.26
18:13:03 up 48 days, 22:10,  0 users,  load average: 0.48, 0.38, 0.29
18:13:03 up 936 days,  7:50,  0 users,  load average: 0.00, 0.00, 0.00
18:13:03 up 173 days,  2:39,  0 users,  load average: 1.90, 1.79, 1.79
18:13:04 up 244 days,  3:25,  0 users,  load average: 3.96, 3.66, 3.90
18:13:04 up 643 days, 15:50,  0 users,  load average: 0.09, 0.08, 0.07
18:13:04 up 642 days,  5:15,  0 …