Its great to see that the Obama campaign in America are after folk that dig open source. They’re after folk that can write software for nation-wide voter contact and mobilisation (something like Twitter?), fund-raising, and social networking and online organising at My.BarackObama.

They’re a LAMP shop. They’re after:

• Experience scaling large LAMP applications
• Posses deep knowledge of MySQL performance and query optimisation

I once wrote:

If they can waste your ringgit on buying proprietary software licenses, when there are clearly open source alternatives, can you trust them with spending and budgeting for a country?

It may seem extremely naive to correlate the use of open source software to creating a better budget or spending wisely, but I think its a start.

Over the last few weeks I’ve been looking at several customers’ slow query logs, and I found in many of them an odd type of query. These are SELECT statements that contain an IN clause that includes dozens, sometimes hundreds of values. These statements often end in the slow query log. I’m not sure if these queries are this way by design or if they are generated by a specific database development tool.

I did some tests in one of my own databases, one with only around 10K rows in its largest table. The database corresponds to the Amarok media player. For example, I queried for songs by B. B. King (spelled “BB King”, “B.B. King”, etc. or with other artists: “B. B. King & Eric Clapton”).

The first query used a JOIN and an IN clause with all the …

Open source databases still have a very long way to catch up to Oracle. I was at Oracle Open World touring the exhibits on Tuesday. Just for fun I asked everyone I met whether they used open source databases or saw demand for them in their businesses. The answer almost universally went like this: "No."

One simple reason explains much of the Oracle dominance as well as the inertia of many companies in switching to something else. A huge number of enterprise applications like Siebel or SAP run on Oracle. MySQL and PostgreSQL applications on the other hand are either custom code or belong to an area where open source is truly dominant, such as web site content management. Even when more applications run on open source, most …

Last week there was a Chinese article about my quote concerning Amazon being of more interest then Google that I made at OSCON.

I find Amazon more interesting because they are exposing the plumbing required to do sites in mass.

Google's AppEngine is much closer to the world of "Digital Sharecropping" that Jessie has been describing. Amazon has some of this in their API's, but I would bet that the more successful pieces to Amazon Web Services are the ones furthest from the share cropping model.

S3 and EC2 have relatively little tie in to them. You can end up with a physical addiction to the services but the mental addiction to a framework does not exist. S3 is just storage and EC2 for most is just a hosted Linux image.

I would also suspect that AppEngine solves two problems for Google. Those being that …

A paginated display is one of the top optimization scenarios we see in the real world. Search results pages, leaderboards, and most-popular lists are good examples. You know the design pattern: display 20 results in some most-relevant order. Show a "next" and "previous" link. And usually, show how many items are in the whole list and how many pages of results there are.

Rendering such a display can consume more resources than the entire rest of the site!

As an example, I'm looking at slow log analysis results (with our microslow patches, set to log all queries) for one client; the slow log contains 6300 seconds' worth of queries, and the two main queries for the paginated display consumed 2850 and 380 seconds, respectively.

Why is it so expensive? I typically see queries like this:

PLAIN TEXT …

Yesterday, I completed a most significant emotional step in the integration of MySQL into Sun. As about 75 % of MySQLers had already done prior to myself, I migrated my main mail account from MySQL’s mail server to Sun’s.

Yes, I am still reachable @mysql.com. And I was already earlier reachable on @Sun.COM, in addition to my private @arno.fi. But now Sun’s email server is the one I use for both sending and receiving email. So this minor administrative step raises questions for me whether my signature should primarily feature my @Sun.COM email address, or whether I should remain @mysql.com. Luckily (strangely? sadly?) Sun allows me to follow my own judgement on this one.

A reason for happiness is that I am reachable as “kaj” on both …

Version 1.0 of CMON (gpl license) has now been released.
It is dead easy to use and gives you a lot of helpful information about the state of the cluster. Memory utilization, statistics etc.

You can also get graphical representation using a web server and the php scripts that comes with cmon.

If you use it with the new and improved config tool then you can setup a system with process management (monit) and init.d scripts etc in minutes, and cmon will then automatically coordinate the start procedures of the data nodes in the cluster!

First you have to download cmon and then you can check out the …

This morning I finally found some time to add an init script to the Debian package of mysql-proxy (0.6.1-3 which includes it was just uploaded to unstable, I will try to get a freeze exception for lenny). It is disabled by default, because mysql-proxy is quite useless without a proper configuration. It can be enabled in /etc/default/mysql-proxy, here is an example:ENABLED="true"
OPTIONS="--proxy-lua-script=/usr/share/mysql-proxy/rw-splitting.lua
         --proxy-address=192.168.1.140:3306
         --proxy-backend-addresses=192.168.1.20:3306
         --proxy-read-only-backend-addresses=192.168.1.23:3306
         --proxy-read-only-backend-addresses=192.168.1.24:3306"In this example the proxy binds to 192.168.1.140 port 3306 and does …

This morning I finally found some time to add an init script to the Debian package of mysql-proxy (0.6.1-3 which includes it was just uploaded to unstable, I will try to get a freeze exception for lenny). It is disabled by default, because mysql-proxy is quite useless without a proper configuration. It can be enabled in /etc/default/mysql-proxy, here is an example:ENABLED="true"
OPTIONS="--proxy-lua-script=/usr/share/mysql-proxy/rw-splitting.lua
         --proxy-address=192.168.1.140:3306
         --proxy-backend-addresses=192.168.1.20:3306
         --proxy-read-only-backend-addresses=192.168.1.23:3306
         --proxy-read-only-backend-addresses=192.168.1.24:3306"In this example the proxy binds to 192.168.1.140 port 3306 and does …

Last week, Infobright announced it had open sourced its data warehousing software code. This is good news for the growing number of organizations looking to use MySQL as a data warehousing platform. According to IDC, MySQL is already the third-most deployed database for data warehousing and Infobright’s move will give users yet another reason to seriously consider MySQL for this application.

For those of you not familiar with the Infobright offering, it is essentially a column-oriented data store for data warehousing. While the column-oriented approach is not exclusive to Infobright (Kickfire’s MySQL storage engine is also column-oriented, as are some other non-MySQL data warehousing solutions on the market) Infobright does have some unique technology that Lou Agosta recently described as follows in his post on Trends in Data Warehousing for the Second Half of 2008: …