The latest from the MySQL community

I spent Monday and Tuesday this week on Velocity Conference It was
quite interesting event worth attending and it was very good to see
the problems in this are going beyond Apache, PHP, Memcache and MySQL.

A lot of talks on this conference was focusing on what is called
"FrontEnd". The meaning of Frontend is not the frontend web server
commonly used in many architectures but rather optimization on the
client side - how to make a browser to do less requests, make them
parallel, fetch less data and execute client side code faster.

Steve Souders mentioned in his talk for Alexa 10 web sites he examined
typically 80-90% of page response time comes from other things than
fetching HTML of the main page - fetching CSS, JavaScript, Images …

A good rule of thumb is that a DBA should not let the size of his/her database grow to much more than 50% of the available disk space. I’m sure everyone has a story of being burned in some way by not following this advice.

Before promoting 5.1 to some beefier hardware, I’ve got a few large tables sitting on a small SATA-based machine. Noticing that a few were growing faster than I expected, i decided to pause my loading and convert another one of them to the archive storage engine to save some space.

I certainly saved space.

By trying to do too much at one time, I had some other temp files lying around on the same filesystem, and my conversion to the archive storage engine failed at about 80% through with my disk full.

mysql> alter table w_stats engine = Archive;
ERROR 1030 (HY000): Got error -1 from storage engine

mysql 5.1 has …

The second edition of the
MySQL 5.1 Cluster Study Guide is now available through Vervante books. Those of you who have patiently emailing me to ask when you can order your copy should should get their order in ASAP.

Open source tour of Europe: Germany

To coincide with EURO 2008, I’m embarking on a virtual European tour, taking a quick look at open source policies and deployment projects in the 16 nations that are competing in the tournament.

It doesn?t matter what the competition is, or how well the team has been playing, when it comes to international football tournaments, Germany is always amongst the favourites, and the Germans are in the final once again despite a poor performance in beating Turkey 3-2.

Similarly, when it comes to open source adoption, Germany has a long tradition of leading the world. …

GigaOm’s Structure Conference took center stage today and lots of interesting stories are coming down the pipeline. Here are today’s top picks:

I just got back from 3 days of conferences - 2 days at Velocity and one at Structure '08. The Velocity Conference was billed as the 1st conference devoted exclusively to web performance and operations. And the sessions did live up to this. They had over 700 attendees which is not bad for the first time.

Being a performance person, I chose to mostly attend the performance sessions. What I found was that  the sessions were heavily geared towards the client-side. There were sessions on how to tune your javascript, images, reduce network traffic etc. - all trying to reduce the end-user response time.

Our session was of course on tuning the server-side. There was another one on squid/varnish and mysql sharding - but beyond that, client …

I just got back from 3 days of conferences - 2 days at Velocity and one at Structure '08. The Velocity Conference was billed as the 1st conference devoted exclusively to web performance and operations. And the sessions did live up to this. They had over 700 attendees which is not bad for the first time.

Being a performance person, I chose to mostly attend the performance sessions. What I found was that  the sessions were heavily geared towards the client-side. There were sessions on how to tune your javascript, images, reduce network traffic etc. - all trying to reduce the end-user response time.

Our session was of course on tuning the server-side. There was another one on squid/varnish and mysql sharding - but beyond that, client …

James Dixon has given the thumbs-up to my stretching his Bee Keeper analogy to explain open source development models (which is nice) and in doing so has suggested a new term to help quickly explain the difference between vendor- and community- dominated development projects.

The debate about the difference between the two approaches, and the language used to describe them, has been simmering for some time. For some background on it, and an explanation about why it matters, see Ted Ts’o’s …

One of the best discussions at Tuesday's CloudCamp San Francisco was "SQL or SimpleDB - Who will win?" Cloud computing is part of a fundamental shift in computer operations propelled by virtualization of hosts and disk storage. We were already starting to argue about SimpleDB as the camp started when the person sitting next me astutely jumped up and proposed it as a topic for discussion.

The argument against SQL goes something like this. Many applications handle very simple objects using only primary key look-ups. Hashtable-based datastores like SimpleDB and BigTable handle that model and also partition data automatically. This simpler data model maps better to object models in scripting languages, many of which …

I caught Facebook - Needle in a Haystack: Efficient Storage of Billions of Photos on Flowgram. First up, I’m not a big fan of Flowgrams - the format is sensible, slide and voice, is excellent, but the delivery in a web browser isn’t optimal… make downloadable videos!

The talk however, was excellent. Do watch it, and learn a bit more about Facebook’s infrastructure. Anyway, some notes I took from the talk:

• “We’re one of the largest MySQL installations in the world”
• Use memcache - “We have memcache because databases aren’t fast” (later on in the questions)
• Separate team focusing on APE (Apache, PHP and Extensions that they work on)
• 6.5 billion total images, 4-5 sizes stored for each, so 30 billion files, of about 540TB total… During peak? 475,000 images served per second, and growing by …