This blog entry is a bit of a written confession from the Sun Drizzle team. It tells the story of the last few weeks, and how we didn't "Do the Right Thing". It's a tale which ends, hopefully, in our redemption through acknowledgement of what happened, a written commitment that it won't happen again, and a bit of a manual on how to have our actions match our team values.

What Happened?

A few weeks ago, we received an email from Allan Packer, who works in the Performance Engineering group at Sun. The email contained the results of some benchmarks that Allan had performed comparing Drizzle and a version of MySQL 5.1.

The results for Drizzle weren't flattering. In fact, at lower concurrencies, the results were downright embarrassing, with some runs showing 5.1 outpacing Drizzle in throughput by 100%.

"How could this be?" we askedcursed. …

Opening introduction from Colin Charles got us started. Karen Tegan Padir VP MySQL & Software Infrastructure was the opening keynote.

She comes from a strong tech background and is passionate about open source, the communities and how to make a successful product.

There isn’t a person that doesn’t go a day without interacting with a website or hardware system that uses a MySQL database.

The big news was the announcement of MySQL 5.4 - Performance & Scalability. Key features include.

• InnoDb scalability 16way x86 and 64 way CMT servers
• subquery optimization
• new query algorithms
• improved stored procedures, and prepared statements
• enhanced Information Schema
• improved DTrace Support

More information at …

Here I am again, at a conference away from home, waking up to learn that my employer has been acquired. MySQL to Sun. Sun to Oracle.

This time I have fewer opinions, and more things to think about.

I have many options and opportunties, and fortunately I am in a place where I can decide without rushing.

As for MySQL and Drizzle and Gearman and MemCacheD? They are all open source, they are all GPLed, so they are all remarkably immune to the threats and risks of corporate games. For as long as people use them, there will always be support and services and fixes and features for those projects.

Sun Java and JVM are open source and GPL now, so they also are safe.

For other stuff? I have no idea. I hope for the best.

The other day I wrote about what I did inside of Drizzle to create the new Protocol plugin type. A few people have asked me to expand on this a bit more, so here is a diagram of how this will work:

As you can see, you will be able to communicate with the server not just through the traditional client library, but through anything that can implement the Protocol class. The first new class will be the new libdrizzle (which also supports MySQL protocol so Drizzle can talk to those clients), but soon we’ll also be able to handle direct HTTP, Gearman, and possibly memcached requests. This will eliminate the need for the client driver altogether in some applications (for example, an AJAX request could query Drizzle directly and ask for a JSON object to be returned by the module).

You will also be able to do various local or embeddable interfaces through this same interface. …

I’ve just successfully configured Drizzle with the PHP Extension and successfully retrieve data to present on a web page.

Qudos to Eric Day for his work. I was able to identify a problem with the current tar release, and a quick confirmation on #drizzle at IRC confirmed a fix had already been commited.

I’m looking forward to evaluating WordPress and Drupal, two popular and common LAMP stack applications that run on MySQL, and to provide any feedback to the community for future support of Drizzle.

As I mentioned in my previous entry, InnoDB has it’s own auto increment counter which it uses to generate the next value for the database kernel (as we call it in Drizzle). At Drizzle project, we came across a suspicion that InnoDB doesn’t increment it’s internal counter on row updates. So what can this mean to you as a database admin?

Well, consider this simple table:

CREATE TABLE t1 (
    a INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    val INT
) ENGINE=InnoDB;

and the following statements:

drizzle> INSERT INTO t1 (val) VALUES (1);
Query OK, 1 row affected (0.01 sec)
 
drizzle> UPDATE t1 SET a=4 WHERE a=1;
Query OK, 1 row …

We have an ongoing Drizzle milestone called low-hanging-fruit. The idea is that when there’s something that  could be done, but we don’t quite have the time to do it immediately, we’ll add a low-hanging-fruit blueprint so that people looking to get a start on the codebase and contributing code to Drizzle have a place to go to find things to do.

Some of my personal favourites are:

• Remove sprintf
• Convert …

We’re having a Drizzle Developer Day just after the MySQL Conference and Expo next week. You don’t have to be attending the conference to come to the Drizzle Developer Day. Just bring your enthusiasm for free databases, Drizzle and good software. Spaces are limited, so head on over to the signup page and fill in your name if you haven’t already

If coming from the Hyatt Regency Santa Clara (where the MySQL Conference and Expo is), at least I will be driving from there, so let me know if you want a lift.

Lately I’ve been having lots of fun going through Drizzle and InnoDB‘s sourcecode to get a grasp of how auto increment is processed internally. I think I now have a fairly good grasp of what’s going on so I’m writing this entry as a note for myself. I’m also hoping that this will be helpful to those that are interested in this topic too.

So in MySQL and Drizzle, the storage engine (in this case InnoDB) is responsible for computing the auto increment value. Here’s an abbreviated execution path for a simple INSERT statement to a table with an auto increment column:

mysql_parse() -> mysql_execute_command() -> mysql_insert() ->
write_record() -> handler::ha_write_row() -> ha_innobase::write_row() ->
handler::update_auto_increment() -> ha_innobase::get_auto_increment()

Working on Drizzle. A while ago the "log to gearman" module I wrote got incorporated into the main tree. The current task is to finish "get gearman via INFORMATION_SCHEMA", and then Drizzle will be able to push its query stream out into a chain of gearman based map reduce system.


So here is a question for the DBAs big system architects out there. If you had an array of database server instances, generating a compiled query log stream of many thousands of statements per second, and you could map, reduce, and analyze that query stream for "interesting stuff"...

What sorts of "interesting stuff" would you want to look for?


There is going to be a BOF on this at the MySQL Users Conference next week.