Optaros, the company I am working for, released an open source catalog which lists close to 300 so called "enterprise ready" OSS applications. I guess most of you reading my blog are not exactly in the target group as to me it mostly seems to be directed about people that lack deep networking into the various OSS communities. That being said I still think that it might be worth a look, even if its just to blog about how we totally got the rating on a specific piece of software totally wrong.

In this updated and web enabled version end users can vote on projects, which is already a first step in the direction of allowing user contributions. I hope we will figure out ways on how to expand this. Again the target group are decision makers that are not so in tune with OSS communities, so we are ok with providing a little less information (if they want all the information they can always use …

Georg, Andrey and Ulf have been hard at work designing, implementing and testing a replacement for libmysql only for us PHP users. The idea is that mysqlnd can leverage all the internal PHP infrastructure for communication and memory management. It can also be much more PHP aware, removing old cruft we do not need while adding extra goodies just for us!

This means the PHP memory limit will finally be honored, PHP user streams will be able to hook into the communication with MySQL. A client side query cache promises speed improvements and the fact that data does not need to be moved from libmysql to PHP should also stream line things. They are also looking to improve support for persistent connections.

While originally developed for PHP6, mysqlnd now also runs on PHP5 and talks to ext/mysqli and ext/mysql with PDO support planned. More over the …

Using the LAMP Virtual Appliance, it is possible to create a full Django development environment in less than 15 minutes.

This article assumes that the reader is already familiar with VMware virtualization products and has one of the following products installed and functional on their host operating system: VMware Player (free download), VMware Server (free download), or VMware Workstation.

Download the LAMP VA (build 131 or newer) and configure for your environment.

Assuming a …

One interesting solution to scalability and HA is to implement a circular master-master replication setup directly on the frontend servers. Obviously there is a reasonable limit as to how many servers you can have in such a ring, since the lag with which changes propagate will increase linearly as you add more servers to the ring. However according to an article by Giuseppe a 10 server ring is reasonable. Especially if you make your sessions sticky on a per frontend server basis, you actually have an elegant solution against the old issue of replication lag, where a user does not see changes he has made in subsequent requests, since his changes were written to a remote master server which have not propagated to the slave he is reading in the subsequent request. Latency should also improve as you do not have to open up a remote connection in the …

I just wanted to point everybody at a recent blog post by Konstantin. In the post he discusses a solution for dealing with cache invalidation issues of very large caches under heavy load. He points out that cache invalidation can severely bog down the system. The general solution he proposes is to simply deactivate the query cache entirely during invalidation. I think this is an important caveat to be aware of and actually he is asking for feedback if this "solution" is acceptable. I think its awesome that MySQL engineers are giving us the opportunity to provide feedback on such changes. Maybe there should be a dedicated "pipeline" where such requests could be found?

So the other day I stumbled over the slides of the InnoDB talk at this years MySQL users conf and I noticed "gap locking" somewhere in the middle. I have never heard of "gap locking" so I was quite intrigued by what that might be. From what I understand its InnoDB's solution to implementing REPEATABLE READ (though until MySQL 5.1 it seems this feature is also enabled for READ COMMITTED). I guess its a fairly unique approach and from my current understanding to be feasible it expects short running transactions, which luckily are quite common in web applications that most of us care about.

The problem InnoDB is trying to solve here are situations where overlapping transactions cause phantoms. Now the first surprise was when I read that gap locking is implemented by locking the relevant ranges on the index or more particular …

I’ve finally gotten around to adding the last little things that I wanted to do to version 2 of Statpack - my python script for aggregating SHOW [GLOBAL] STATUS output.

Now it connects to a running instance, and allows you to gather statistics from there to generate reports.

Here are the options it supports:

[markleith@medusa:~/Development/statpack/statpack-v2] $ ./statpack.py

Error: No arguments supplied
usage: ./statpack.py [list of arguments]

Non-interactive mode (aggregate txt files containing SHOW STATUS snapshots):

-f –files List of statistics files to aggregate (–files=stat1.txt[, stat2.txt..])
Argument must be first within list of arguments, on it’s own

Interactive mode (connect to running MySQL server for SHOW STATUS snapshots):

-h –host Host for MySQL server to connect to in interactive mode
…

.. and more annoyingly, why I spend the night in Amsterdam, even though the Dutch PHP conference is not until next weekend. So there were apparently some pretty bad thunderstorms over France, Belgium and the Netherlands. As a result the plane that should take me to Amsterdam on the way to Sandjeford arrived about an hour late. Back in the day, when airlines still afforded themselves some ground time, this would not have been that big a deal, as I had no checked luggage and so ideally I could have switched planes in time as I had 1 hour stopover time in Amsterdam. Well these days it means that if the incoming plane comes one hour late, you will likely also be one hour late. Usually they can try to make up a few minutes by burning some more fuel. In this case the thunderstorms actually forced us to fly a non standard route, which took even longer.

Now when they broke the news in Zurich …

I have some warm up questions on how MySQL handles LOBs internally and then some questions on high availability. Especially with HA I feel like I have heard all/most the solutions, but they all seem to lack in one area that really makes you hurt, but I keep stumbling over questions from people asking me which way to go. Finally I have a little backup question for extra credit at the end.

So for the LOB questions, I am wondering if MySQL (or any of the storage engines) store the actual LOB data on separate data pages or not. I am also wondering if the MySQL query cache has any special handling for LOBs, like ignoring all queries that fetch LOB data. I guess there is a setting to set the maximum size for a result set to go into the query cache that could be employed to prevent a few large LOBs to fill up the query cache, but then again you …

Hej hej from Stockholm where I arrived about two weeks ago for a fun-filled two four six weeks of meetings and face-to-face contact with the MySQL Cluster developers. And the chance to check things out when it’s not 2°C and raining, all the time, as it was when I was here in November and December of 2005. (This week, it’s 12°C and kind of misty.)

The last five or six weeks have been pretty hectic, and this evening (it’s just after 10 PM local time as I write this) represents the first time in that span that I’ve actually had time that’s not been planned out for me in one way or another. At the end of April, I moved out of my place in Brisbane and stuffed all my gear into storage in Coffs Harbour, New South Wales, where I rented a post office box that now constitutes my sole legal address for the duration.

That last sentence is actually …