The latest from the MySQL community

GigaOm’s Structure Conference took center stage today and lots of interesting stories are coming down the pipeline. Here are today’s top picks:

I just got back from 3 days of conferences - 2 days at Velocity and one at Structure '08. The Velocity Conference was billed as the 1st conference devoted exclusively to web performance and operations. And the sessions did live up to this. They had over 700 attendees which is not bad for the first time.

Being a performance person, I chose to mostly attend the performance sessions. What I found was that  the sessions were heavily geared towards the client-side. There were sessions on how to tune your javascript, images, reduce network traffic etc. - all trying to reduce the end-user response time.

Our session was of course on tuning the server-side. There was another one on squid/varnish and mysql sharding - but beyond that, client …

I just got back from 3 days of conferences - 2 days at Velocity and one at Structure '08. The Velocity Conference was billed as the 1st conference devoted exclusively to web performance and operations. And the sessions did live up to this. They had over 700 attendees which is not bad for the first time.

Being a performance person, I chose to mostly attend the performance sessions. What I found was that  the sessions were heavily geared towards the client-side. There were sessions on how to tune your javascript, images, reduce network traffic etc. - all trying to reduce the end-user response time.

Our session was of course on tuning the server-side. There was another one on squid/varnish and mysql sharding - but beyond that, client …

James Dixon has given the thumbs-up to my stretching his Bee Keeper analogy to explain open source development models (which is nice) and in doing so has suggested a new term to help quickly explain the difference between vendor- and community- dominated development projects.

The debate about the difference between the two approaches, and the language used to describe them, has been simmering for some time. For some background on it, and an explanation about why it matters, see Ted Ts’o’s …

One of the best discussions at Tuesday's CloudCamp San Francisco was "SQL or SimpleDB - Who will win?" Cloud computing is part of a fundamental shift in computer operations propelled by virtualization of hosts and disk storage. We were already starting to argue about SimpleDB as the camp started when the person sitting next me astutely jumped up and proposed it as a topic for discussion.

The argument against SQL goes something like this. Many applications handle very simple objects using only primary key look-ups. Hashtable-based datastores like SimpleDB and BigTable handle that model and also partition data automatically. This simpler data model maps better to object models in scripting languages, many of which …

I caught Facebook - Needle in a Haystack: Efficient Storage of Billions of Photos on Flowgram. First up, I’m not a big fan of Flowgrams - the format is sensible, slide and voice, is excellent, but the delivery in a web browser isn’t optimal… make downloadable videos!

The talk however, was excellent. Do watch it, and learn a bit more about Facebook’s infrastructure. Anyway, some notes I took from the talk:

• “We’re one of the largest MySQL installations in the world”
• Use memcache - “We have memcache because databases aren’t fast” (later on in the questions)
• Separate team focusing on APE (Apache, PHP and Extensions that they work on)
• 6.5 billion total images, 4-5 sizes stored for each, so 30 billion files, of about 540TB total… During peak? 475,000 images served per second, and growing by …

This is not related directly to MySQL, but alas I must rant. In this day and age I’m not sure why an application would require IE7 and ActiveX controls to run a ticketing system. If we’re in the technical world, as sysadmins or DBAs, which run Linux/Solaris/Unix on any good server in order to get work done, it only makes sense to use a unix based OS (osx,linux,solaris) as a workstation.

It’s easier to interface with servers, there are better terminal options (in which we live our daily lives), free options to just about everything that exists in Windows, and YET there are ticketing systems (RNT) that require IE7 and ActiveX controls - which require you to run WindowsXP. It doesn’t work on windows server 2003, it doesn’t work on Windows 2000 Pro.

So what are we left with? An impossible situation that requires an employee to run two OSes in order to get work done. Ridiculous! It’s a waste of time and resources. Not to …

In case you are attending Oracle Open World 2008, the biggest Oracle conference in the world, and interested in either (or both) MySQL or Oracle Enterprise Manager Extensibility — I posted a proposal for a new presentation:

Extending Oracle Enterprise Manager by Example — Creating MySQL Management Plug-In

I’ve started looking into Oracle extensibility several years ago and since then I’ve seen lots of improvements in Extensibility Guide and many new plug-ins have seen the light of the day. However, creating a new plug-in is still considered to be something special and not available to mere mortals.
In this presentation we will see how easy it is to create a new plug-in. What are the steps …

Grand, I got my personal copy from the courier today!
Many people kept asking me over recent months "when is it coming, when can I get it"... so now you can. Officially out, and available.

Click on the book pic if you want to grab your copy via Amazon (which is probably one of the cheapest sources, even if you're overseas).

I'm also looking at acquiring a bundle of them through the author arrangement with O'Reilly, to give away to Open Query training students - well, for the courses related to MySQL at least ;-)

I recently filed Bug #37359 filesort can be more efficient based on some performance work with an industry standard benchmark. Read on if the internals of how MySQL implements filesort interests you.

Filesort, as the name implies, is used to sort records when there is an ORDER BY clause in the query. The reason it has the prefix "file" is because temporary files may be used to store intermediate results. filesort() uses a per thread sort buffer of size sort_buffer_size to sort the table. filesort() is implemented in sql/filesort.cc.

filesort may not know how many records need to be sorted. It asks the storage engine for an estimate of the number of records in the table via estimate_rows_upper_bound(). If the number of records that fit in the sort buffer is less than the estimate, filesort will use temporary files to store intermediate results. The flow …