Something I’ve added to BlitzDB recently that was pretty high on my todo list is support for variable width tables. So what is a variable width table? it is a table that contains columns that can vary in size, namely BLOB and TEXT types.

Going back to the basics, when a new row is to be written, a storage engine is given a pointer to the row data in MySQL format that it must somehow store for later lookup/retrieval. By meaning “somehow”, the storage engine is given the freedom to do whatever it likes with the row.

Writing a row for a fixed length table (a table with columns that are always the same size) is deadly easy. A storage engine can choose to not temper with the row and simply write or copy the data to it’s storage mechanism. This is because the storage engine is given a row that contains all the data. Rows for variable width tables …

Using a write cache protected by battery (BBWC) is well known and one of the best practices in RDBMS world. But I have frequently seen situations that people do not set write cache properly. Sometimes they just forget to enable write cache. Sometimes write cache is disabled even though they say they set properly.

Make sure that BBWC is enabled. If not enabled, you will be able to easily get better performance by just enabling it. The following is a DBT-2 example.

Write cache is disabled:

# iostat -xm 10
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
        21.16    0.00    6.14   29.77    0.00   42.93

Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s
sdb               2.60   389.01 283.12 47.35     4.86
wMB/s avgrq-sz avgqu-sz   await  svctm  %util
2.19    43.67     4.89   14.76   3.02  99.83


Write cache is enabled:

# iostat -xm 10 …

I’ve decided to start a series of blog entries on not-so-obvious findings that I’ve found while working on my new project. By archiving the findings, I’m hoping that I can help those that are looking into developing a storage engine for the MySQL family in the future.

Accumulating these mini-knowledge would also be useful for me since I can refer back to it when I forget something. Also, once I write enough entries I’m planning on summarizing them and making it available on the Drizzle Wiki. If MySQL is interested in updating the engine documentation, I would be more than happy to help there too.

So to begin with, I’ll describe something trivial that I stumbled across while trying to catch an error on duplicate primary key insertion to the data table.

I will be hosting a webinar on May 7 about how companies like Wikimedia and Smugmug are using Open Storage and MySQL to deliver rich media (photos, videos) to their users. You can view the webinar live or on demand here.

I will be hosting a webinar on May 7 about how companies like Wikimedia and Smugmug are using Open Storage and MySQL to deliver rich media (photos, videos) to their users. You can view the webinar live or on demand here.

I will be hosting a webinar on May 7 about how companies like Wikimedia and Smugmug are using Open Storage and MySQL to deliver rich media (photos, videos) to their users. You can view the webinar live or on demand here.

Kazuho Oku of Cybozu Labs, Inc., a community contributor to MySQL and SCA signatory, gives a talk on Q4M, a message queue stroage engine for MySQL.

Kazuho Oku of Cybozu Labs, Inc., a community contributor to MySQL and SCA signatory, gives a talk on Q4M, a message queue stroage engine for MySQL.

Kazuho Oku of Cybozu Labs, Inc., a community contributor to MySQL and SCA signatory, gives a talk on Q4M, a message queue stroage engine for MySQL.

Lately I’ve been having lots of fun going through Drizzle and InnoDB‘s sourcecode to get a grasp of how auto increment is processed internally. I think I now have a fairly good grasp of what’s going on so I’m writing this entry as a note for myself. I’m also hoping that this will be helpful to those that are interested in this topic too.

So in MySQL and Drizzle, the storage engine (in this case InnoDB) is responsible for computing the auto increment value. Here’s an abbreviated execution path for a simple INSERT statement to a table with an auto increment column:

mysql_parse() -> mysql_execute_command() -> mysql_insert() ->
write_record() -> handler::ha_write_row() -> ha_innobase::write_row() ->
handler::update_auto_increment() -> ha_innobase::get_auto_increment()