So I’ve been doing a fair number of automated load tests these past six months. Primarily with Sysbench, which is a fine, fine tool. First I started using some simple bash based loop controls to automate my overnight testing, but as usually happens with shell scripts they grew unwieldy and I rewrote them in python. Now I have some flexible and easily configurable code for sysbench based MySQL benchmarking to offer the community. I’ve always been a fan of giving back to such a helpful group of people – you’ll never hear me complain about “my time isn’t free”. So, let me know what you want in an ideal testing environment (from a load testing framework automation standpoint) and I’ll integrate it into my existing framework and then release it via the BSD license. The main goal here is to have a standardized modular framework, based on sysbench,

I got access to our 12-core Intel server, so I was able to do some better benchmarks to test the different group commit thread scheduling methods:

This graph shows queries-per-second as a function of number of parallel connections, for three test runs:

The best part of the recent MariaDB meeting in Lisbon for me was that I got some good feedback on my group commit work. This has been waiting in the review queue for quite some time now.

One comment I got revolve around an optimisation in the implementation related to how threads are scheduled.

A crucial step in the group commit algorithm is when the transactions being committed have been written into the binary log, and we want to commit them in the storage engine(s) in the same order as they were committed in the binlog. This

There has been an increase in the discussion about MySQL replication and how to make it happen faster. I don't think Marco Tusa's blog is on Planet MySQL, so let's highlight it first: A dream on MySQL parallel replication. This is a good account of what it looks like out in the field of MySQL databases today - you are increasingly aware of replication, fearing it will be a bottleneck you cannot fix.

read more

A special extended edition of Tech Messages for 2011-02-10 through 2011-03-07:

• Nicholas Piël » Benchmark of Python Web Servers
  Nicholas has done an in-depth benchmarking of several WSGI servers in an effort to document their differences. He takes into account the type of server and which version of HTTP it supports.
• My 10 favourite Programming Quotes | Ginktage
  If debugging is the process of removing software bugs, then programming must be the process of putting them in. –  Edsger Dijkstra

As some Facebook friends already guessed from my status updates this week, my 9 month paternity leave is now over and I've survived my first week back in work life, waking up at seven in the morning! This is just a personal-life blog post to let everyone know what I'm up to, (For those asking: Ebba is doing fine, she recently started to stand up and even takes steps if I hold her hands.)

During the past months I had many interesting conversations and ideas of what to do next, but in the end Nokia was the company standing out with a very interesting offer. So as of last Tuesday I work at Nokia as Senior Performance Architect in the Mobile Solutions division, better known as the Nokia Ovi web services.

read more

This is the final part in a series of posts about the MepSQL build system known as MepSQL Bakery. MepSQL is a (yet another) fork of the MySQL database server, with the server based on the MySQLatFacebook code and the build system based on the MariaDB build system.

In this final post I wish to draw a high level picture of the complete process of building TAR, DEB and (eventually) RPM packages from the source code. There's not much more technical details to add to the previous posts, instead I'm going to make some, shall we say "archeological", observations which imho are interesting given how the build system has evolved when being passed from one project to another. Perhaps

Yesterday I wrote down the approach used in the MepSQL build system to parameterize the TAR package name produced. Today I will follow up with how the same was done for building DEBs. The motivation is to create a system that can be used flexibly to create packages of any MySQL fork, with any brand name: mepsql-server-*.deb, percona-server-server-*.deb, mariadb-server-*.deb or even just mysql-server-*.deb (which I might do some day).

While yesterday's tricks with the TAR files were rather straightforward, with the process of building DEBs this turns out to be much more challenging. But not to worry, like

As I mentioned before, it seems that thanks to Git and Bzr introducing distributed version control workflows, the open source community is now living in a phase where forking is easy and happens frequently - referred to by Brian Aker as the Cambrian Explosion of open source. We certainly see that happening in the MySQL Community.

Assuming you have the competence and know your way around a codebase, forking a proper open source project isn't that hard. You create your own project on GitHub or Launchpad and copy the source code.

This is the first part of many posts in a series of blog posts where I want to document how the MepSQL packages were built. By doing that, I will also end up covering the MariaDB build system (which this is based on), some of BuildBot, Amazon EC2 cloud and packaging DEBs and RPMs just in general, so it could be interesting from many perspectives. In this first part I'll simply scribble some notes about reviewing the OpenSuse Build System, Launchpad PPA service vs using your own servers and automating the builds with BuildBot.

Originally I just wanted to work on some new ideas on the automated build and QA system used by MariaDB. But since leaving Monty Program I didn't have access to any of those servers anymore, so as a first step I had

The slides from the talk I gave at Percona Live San Francisco yesterday are now available on slideshare (hopefully the embed works here too):

View more presentations from Jeremy Zawodny.

Overall, I enjoyed the conference–not just meeting up folks I don’t see often enough, but also getting a renewed sense of how active the MySQL ecosystem really is.

Some time ago Stephen O'Grady and Brian Aker had an interesting Blogo-dialogue about what they call the "Cambrian Explosion" of open source development. The Cambrian Explosion means that we increasingly see forks of projects being developed in different directions, where traditionally we are used to open source development happening in relatively hierarchical and easy to follow upstream and downstream relationships. This is exactly what happens in the MySQL community currently, where in total there is more progress than ever before, but that progress is divided among several competing forks, none of which is

I’ve always been a big fan of having a customized .bashrc file. The one I distribute to all of my servers has aliases for quick commands to save me time on the command line, functions that get work done when aliases are too simplistic, reporting for the server for each cli login, and of course a formatted and colored prompt (for terms that support colors). I also change certain aspects and commands based on the operating system since I’m not always on a redhat box or linux at all. Here’s my bashrc file – maybe you have some fun additions that you’d like to share. What saves you time on the command line?

This month is a special month. It’s not because of Valentines day or even the exciting day where we see groundhogs. No, this month is special because I’m have a book contest where you, the reader, get to win something free for doing absolutely nothing more than posting a comment saying that you want one of the several books I have available in the contest.

So without getting into boring details I’ll keep this short. I’ve been reviewing a lot of books lately and I think it’s time to get some books into people’s hands to enjoy themselves. This month the giveaways are all Python oriented.

So, all you have to do is take a look at the following titles and post a comment here saying that you want one of them. At the end of the month two readers will be chosen via a random list sorting python script I’ve whipped up for just this purpose. You

I’ve been playing around with some quick system automation scripts that are handy to use when you don’t want / need to setup a chef or puppet action. I like to keep all of my hostnames and login details in a MySQL database (a cmdb actually) but for this example we’ll just use a couple of nested lists. This script executes commands in parallel across the hosts you choose in the menu system via the “pdsh” command, so make sure you have that installed before running. Alternately you can change the command call to use ssh instead of pdsh for a serialized execution, but that’s not as fun or fast. With some customizations here and there you can expand this to operate parallelized jobs for simplifying daily work in database administration, usage reporting, log file parsing, or other system automation as you see fit. Here’s the code. Comments welcome as

A special extended edition of Tech Messages for 2011-01-27 through 2011-02-02:

• NetApp File Copy | A NetApp Technical Diary
  There's a lot of ways to copy files on a NetApp Filer.
• NoSQL at Netflix
• Netflix Performance on Top ISP Networks

Packt Publishing recently sent me a copy of MySQL for Python to review and after reading through the book I must say that I’m rather impressed at the variety of topics that the book covers.

It starts off with the basics of setting up MySQL for your testing/development needs by going over several of the common installation and configuration methods. After that it’s a quick intro for connection methods and simple error reporting for connections. The author gives a quick intro to CRUD and how it relates to databases and python before heading into the common tasks of simple queries. I was surprised to see some database profiling discussion; which is rather handy for a new coder or a

Every so often you need to use a queue to manage operations in an application. Python makes this very simple. Python also, as I’ve written about before, makes threading very easy to work with. So in this quick program I’ll describe via comments, how to make a simple queue where each job is processed by a thread. Integrating this code to read jobs from a mysql database would be trivial as well; simply replace the “jobs = [..." code with a database call to a row select query.

#!/usr/bin/env python
## DATE: 2011-01-20
## FILE: queue.py
## AUTHOR: Matt Reid
## WEBSITE: http://themattreid.com
from Queue import *
from threading import Thread, Lock

'''this function will process the items in the queue, in serial'''
def processor():
    if queue.empty() == True:
        print "the Queue is empty!"
        sys.exit(1)
    try:
        job = queue.get()
        print "I'm

A special extended edition of Tech Messages for 2011-01-12 through 2011-01-15:

• Hg Init: a Mercurial tutorial by Joel Spolsky
  An introduction to Mercurial (hg) by Joel Spolsky. Includes a special introductory chapter for Subversion users.
• lessfs | Open source data de-duplication
  Open source filesystem with de-duplication.
• UCS B-Series and C-Series Servers Log Memory Errors due to Intel 5600 Erratum Issue
  On a subset of Intel Xeon 5600 series processors, during packet C6 transitions, CPU circuit marginality or internal signaling noise in the may lead to an invalid memory DRAM state, system hang,
  

A special extended edition of Tech Messages for 2011-01-07 through 2011-01-08:

• The Services Used By Y Combinator Startups [Infographic] – ReadWriteCloud
  "The results show that the majority of Y Combinator startups depend on cloud services instead of self-hosted servers, traditional web hosting or co-location services."
• Tom Limoncelli on Time Management | O'Reilly Time Management for System Administrators | Sysadmins
  Video companions to to Time Management for System Administrators by Thomas A. Limoncelli.

A special extended edition of Tech Messages for 2010-12-14 through 2010-12-21:

• Cisco IOS Hints and Tricks: IOS HTTP client sets Host: field to IP address
  Still annoying…
• ACL IP Options Selective Drop – Cisco Systems
  Most networks do not make use of IP options. You can drop packets that contain options with the "ip option drop" global configuration command.
• Which loads faster?
  This tool has it's usefulness. It's especially nice to compare your site to similar sites and see what you can do to improve