The latest from the MySQL community

Yesterday, I wrote about some examples of disruption.  When an industry gets disrupted, there's usually a consistent pattern that emerges:

• A new entrant joins the market
• Their offering is perceived as limited and technically inferior
• The new entrant starts to gain a toehold in a few niches
• The incumbent waits and watches
• The entrant becomes "good enough"
• The niche market starts to boom
• The incumbent responds by trying to cram disruption
• The new model wins

I think the same pattern that happened with PCs disrupting minicomputers and …

I received an email the other day from someone at MySQL AB. They are offering free Phorum booth space in the .org Pavilion at the 2007 MySQL Conference. The .org Pavilion is for Open Source projects that are somehow related to MySQL. Not only does Phorum use MySQL as its primary database (our PostgreSQL layer stays 99% done it seems) but MySQL.com also uses it for their support forums.

So, we had a problem. The Phorum team is not by any means wealthy. We all have jobs and work on Phorum in our spare time. So, getting everyone there is going to be problem number one. There are 3 primary team members. I am in the US. Thomas Seifert is in Germany and Maurcie Makaay is in the Netherlands. We decided to turn to our users for help.

I made a …

FOSDEM, the Free and Open Source Software Developers' European Meeting will again take place in Brussels, Belgium on February, 24th & 25th. I went there last year and really had a great time. This year, I submitted a proposal for a lightning talk, which has been accepted! I've just been informed that the lightning talk schedule is now published. My talk "What's new at MySQL AB" will be on Saturday, at 17:00. I look forward to being there!

Kevin Burton writes about making MySQL Replication Parallel. Many of us have been beaten by the fact MySQL Replication is single threaded so in reality it is only able to use only single CPU and single disk effectively which is getting worse and worse as computers are getting "wider" these days with multi-core CPUs.

Kevin proposes to execute queries in parallel and it is generally good idea, the problem is however implementing it right without changing MySQL Replication semantics - which is - Slave database state corresponds to master database state at certain point in time. It is delayed but threads reading from the slave never will see state of the database which never existed on master.

As I commented in Kevins blog the problem is very simple to illustrate - assume you have 2 queries modifying 2 different tables, query A and query B. On the Master …

With the current version of MySQL there are two threads used by a slave to implement replication. One reads from the binary log of the master and the other one executes the SQL.

If you have a master which is being pounded with individual inserts that are happening on parallel threads these will be serialized by the slave and executed one by one.

The master would in theory fully saturate the disks since it would be bottlenecked. Since the INSERTs are happening in parallel and on multiple threads the disk subsystem would get more effective write performance since it can take advantage of tagged command queueing and so forth.

Since the slave doesn't have the ability to execute the INSERTs in parallel and can only use one thread it will quickly fall behind replication.

In practice the only way to fix this is to make sure the that the master isn't fully loaded or that the slaves are more powerful than the master. …

Also packed in an environment where MS Access may be present, as it does [Col Name] for identifier quoting (and Access users often use spaces in identifiers).

I have been asked numerous times what does “load average” means in top. If you don’t know what top is and you have access to linux machine, go type top now and see what it shows.

load average: 2.05, 2.17, 1.93

Quick answer is: first number (2.05) is 1 minute avg, second number (2.17) is 5 minute avg, third number (1.93) is 15 min avg. Generally system admins look at these #’s to see how is their server is doing. But now you wonder, if this is the #’s you look at, why is there cpu %? Isn’t that computer load also? Ofcourse it is. BUT, meaning of cpu % shown in [ Cpu(s): 14.2% us, 1.7% sy, 0.0% ni, 80.7% id, 3.1% wa, 0.0% hi, 0.3% si, 0.0% st ] actually just means how much % of time was spent doing stuff on cpu. On the other hand, load average takes other things such as how much cpu’s were being used and how many process had to wait for their turn to use cpu, etc. Thats why sometimes you will see high % for Cpu …

Lately I've been thinking about how open source is disrupting the software industry.  In Silicon Valley, people tend to focus on a very narrow view of disruption focusing on prices or features.  But I think disruption is more than that. 

Consider these examples:

• The PC disrupted minicomputers by giving individuals power to develop and run their own applications outside of IT
• Digital cameras disrupted film by providing instant gratification
• Netflix disrupting Blockbuster by eliminating late fees
• Salesforce.com disrupted Siebel by eliminating long CRM implementation cycles

Typically new entrant disrupts the market by doing something different than the incumbent vendor.  PC's weren't just smaller, cheaper minicomputers.  They served a different …

Hola, MySQL Developers, DBAs and Users alike. I've been absent from the blogging scene for a number of weeks now (bad Jay, bad!). What have I been doing? Oh, just planning and organizing the biggest and baddest conference the world has EVER seen. Ah, you laugh. You mock. The 2007 MySQL Conference and Expo registration is now open. As program chair, I have a duty, no, an insatiable desire, to tell you all why this year's conference is worth ten times more than the price of admission. So here goes.

The Nerdliest of All Geeks So Speaketh

You want technical sessions about scaling, developing, monitoring, administering, and tuning MySQL that will blow the socks off anything you'll find at any other DB conference? Check this short list out:

Google, flickr.com, digg.com, FotoLog, Xing.com

…

Ok, so this doesn't work on all Linux systems. This because many don't ship with LinuxThreads libraries any more. While that's nice in theory, reality is that NPTL threading does not always work well, in particular MySQL server on 64-bit machines can experience trouble. Other apps don't see the problems because they don't use that many threads.

If a threading hacker with spare time were to care about looking over the NPTL code to make it work well with hundreds (up to a few thousand) threads, that would be grand.

In the meantime, you may find the below patch useful: it modifies mysqld_safe so you don't need to keep tweaking it when you upgrade your MySQL version. See the inline comments for further details.

--- mysql-5.1/scripts/mysqld_safe.sh 2007-01-29 10:26:34.000000000 +1000
+++ mysqld_safe.sh 2007-01-30 10:39:12.000000000 +1000
@@ -33,6 +33,7 @@
--ledir=DIRECTORY …