From the “I wish I attended this talk” department. When I attended Dathan’s talk about Federation at Flickr, he did mention that the next day there was going to be a really interesting talk by John Allspaw, who is an Engineering Manager at Flickr. John has posted up his slides, which I might add, are quite an interesting read (look at the speaker notes, they sort of give a heads up as to what you missed).

Its a pity no one made notes of this on Planet MySQL, so if anyone did attend the talk and you did take notes, please do place them online! This talk isn’t so much teaching you about capacity planning, its really more about all the pointers you can take away, about …

So I have been working on a tool here at work that will run a query a specified number of times using a specified number of threads running concurrently. It is actually similar to the new 'mysqlslap' utility available in MySQL 5.1, but is designed to operate on a user-defined query and database. It is what I have been using in my previous posts, and though some might find it useful. You can find it here, which is my software page. It is pretty easy to use. Just edit config.php and fill in the values as appropriate. You can also specify some command-line arguments (the first being number of threads to use, the second is the total number of iterations, and the third is the query you want to run) if you prefer. Here is an example:

tim@filedawg ~/benchmarks/SQLBuster $ php bustit.php 4 10000
SQLBuster v0.81
Written By: Tim "Sweetums" Soderstrom
---------------------------------------
Running With 4 Threads, 2500 …

In the spirit of the recent MySQL Conference, I decided to have some fun and make a MySQL Storage Engine that would be of practical use. My idea is that so, so much information is available through, say, /proc on Linux and FreeBSD that it is a shame there isn't a convenient way to get to it short of ssh or (shudder) snmp. So, I made a little storage engine that turns arbitrary files into MySQL tables. It supports basic things like space separation to derive columns, etc. Pretty simple, but I'm amazed how easy it was and how useful it could be.

Enjoy!

http://code.google.com/p/mysql-filesystem-engine/

Ideas for files to watch:

• 
  
• /proc/mdstat - how is your software RAID array doing? any bad …

Over at Tecosystems, SOG was trying to explain a misquote of his in Computerworld re:MySQL and somehow drafted a great explanation of dual-license models. The short version of my explanation went something like this: in layman's terms (I know there... READ MORE

The 2007 MySQL Conference is over, and I finally made it back home. I have some notes on some of the sessions, which really aren't that great, so if you want to see what you missed, you should read Planet MySQL. But I will give some of the highlights.

There's a lot of new development around storage engines.

MySQL-5.1 has a pluggable storage engine architecture which allows you to load and unload storage engines while the server is running. Brian Aker explained that this is for cases where you have a stable server setup and only want to upgrade the storage engine. All the storage engines in 5.1 are pluggable, and there are already some third-party proprietary storage engines available.

One of the relatively new third-party storage engines is SolidDB. Solid has been around for quite awhile. In fact, I was using Solid for a project in the late 1990's …

Over at Solfo I play sysadmin in between being a programmer and DBA. Between real hardware, xen boxes and network equipment we have a few dozen devices. One of the really neat tools we use for monitoring is Munin. It's (yet another) tool to graph system stats and anything else you can write a plugin for.

One of the default plugins is memory use. In this graph you can see the memory usage for one of our application servers. Last week we changed some configuration so the application won't grow to eat a ton of memory every few hours.

Other similar tools are Cacti and Ganglia. I've played with both but they didn't hook me like Munin did. I never really liked …

Over at Solfo I play sysadmin in between being a programmer and DBA. Between real hardware, xen boxes and network equipment we have a few dozen devices. One of the really neat tools we use for monitoring is Munin. It's (yet another) tool to graph system stats and anything else you can write a plugin for.

One of the default plugins is memory use. In this graph you can see the memory usage for one of our application servers. Last week we changed some configuration so the application won't grow to eat a ton of memory every few hours.

Other similar tools are Cacti and Ganglia. I've played with both but they didn't hook me like Munin did. I never really liked …

Ever since we switched out build process from Java 1.4 to Java 5 (1.5.0_09) we have seen FindBugs crash with an OutOfMemoryError when started from ant. The whole thing is running under RedHat Enterprise Linux 4. The output is always the same:

[findbugs] Running FindBugs...
[findbugs] Exception in thread "CompilerThread0" java.lang.OutOfMemoryError: requested 134217736 bytes for Chunk::new. Out of swap space?

Our first attempts to increase the heap size with the -Xmx VM parameter did not help. We suspected the newer FindBugs release we had installed roughly at the same time, but this turned out to be wrong, because newer and older versions showed the same behaviour.

Armed with the fresh knowledge about the SAP Memory Analyzer just learned at jax.07 I …

Ever since we switched out build process from Java 1.4 to Java 5 (1.5.0_09) we have seen FindBugs crash with an OutOfMemoryError when started from ant. The whole thing is running under RedHat Enterprise Linux 4. The output is always the same:

[findbugs] Running FindBugs...
[findbugs] Exception in thread "CompilerThread0" java.lang.OutOfMemoryError: requested 134217736 bytes for Chunk::new. Out of swap space?

Our first attempts to increase the heap size with the -Xmx VM parameter did not help. We suspected the newer FindBugs release we had installed roughly at the same time, but this turned out to be wrong, because newer and older versions showed the same behaviour.

Armed with the fresh knowledge about the SAP Memory Analyzer just learned at jax.07 I …