I'm developing a demonstration site for new system we are about to unleash on the world, so far it's going great but one of the worst things about demo sites is the lack of real world data at hand. I don't want to give too much away but the site is vehicle sales based and I need to generate a bunch of data. The problem I needed to solve was that the table with the cars in didn't have associated dealer data and the pages looked very bare when there wasn't any as much of the data comes from the dealers table.

I didn't fancy going in to the table and amending 165 cars by hand, but I also didn't want to run the update in blocks as I wanted an a pretty even spread of the different cars between the 12 dealers. Take for example this data set...

mysql> select id, Make, dealer from cars_cars limit 5;

+----+------------+-----------+
| id | Make       | dealer    | …

Dear Kettle & MySQL fans!

I’m really looking forward to go to the MySQL User Conference next week, not just because I’m speaking in 2 sessions again, but perhaps also because these are “interesting” times for MySQL and Sun Microsystems.  Pivotal times it would seem.

Here are the 2 sessions I’m going to do:

• Cloud Computing with MySQL and Kettle : I’m particularly happy that MySQL accepted this session: it will demonstrate how easy it has become to do cloud computing exercises with tools like MySQL and Kettle.

• Starring …

In short, the news are:

• @@optimizer_switch support was backported into MySQL 5.1
• The switch syntax was changed from ‘no_optimization_name’ to ‘optimization_name=on|off|default’.
• Added switches for index_merge, index_merge_intersection, index_merge_union, and index_merge_sort_union optimizations.

The changes will be available in next releases, that is, MySQL 5.1.34 and 6.0.11.
Now with more details:

New switch names

Until now, the syntax mimicked an enum column or the @@sql_mode variable. One could set the value of @@optimizer_switch to a set of keywords, e.g.

SET @@optimizer_switch='no_semijoin,no_materialization';

Presence of a no_xxx keyword meant that its optimization was disabled, its absence meant it was disabled.

As of the next MySQL 5.1/6.0, @@optimizer_switch value is a …

Sure, you've heard it before: [some company's logo] has a new MySQL backup tool that promises to solve all of your data recovery needs. The good news is most of these tools work pretty well. However, they tend to suffer from a similar set of limitations. Most require sophisticated infrastructures or complex setup and maintenance and can become a resource drain for some organizations. You're probably wondering, "Why can't someone build a fully automated MySQL backup solution that you can just turn on and forget?"

I am happy to say that the MySQL Developers at Sun are doing just that. In fact, a prototype will be demonstrated at the 2009 MySQL Users' Conference that will show the feasibility of a fully automated MySQL backup and recovery tool. It's being called the MySQL Time Machine and (with all due respect to all vendors with products of similar names) it allows you to recover your data using a datetime value. How cool is that? Even MySQL …

Sure, you've heard it before: [some company's logo] has a new MySQL backup tool that promises to solve all of your data recovery needs. The good news is most of these tools work pretty well. However, they tend to suffer from a similar set of limitations. Most require sophisticated infrastructures or complex setup and maintenance and can become a resource drain for some organizations. You're probably wondering, "Why can't someone build a fully automated MySQL backup solution that you can just turn on and forget?"

I am happy to say that the MySQL Developers at Sun are doing just that. In fact, a prototype will be demonstrated at the 2009 MySQL Users' Conference that will show the feasibility of a fully automated MySQL backup and recovery tool. It's being called the MySQL Time Machine and (with all due respect to all vendors with products of similar names) it allows you to recover your data using a datetime value. How cool is that? Even MySQL …

Lately I’ve been having lots of fun going through Drizzle and InnoDB‘s sourcecode to get a grasp of how auto increment is processed internally. I think I now have a fairly good grasp of what’s going on so I’m writing this entry as a note for myself. I’m also hoping that this will be helpful to those that are interested in this topic too.

So in MySQL and Drizzle, the storage engine (in this case InnoDB) is responsible for computing the auto increment value. Here’s an abbreviated execution path for a simple INSERT statement to a table with an auto increment column:

mysql_parse() -> mysql_execute_command() -> mysql_insert() ->
write_record() -> handler::ha_write_row() -> ha_innobase::write_row() ->
handler::update_auto_increment() -> ha_innobase::get_auto_increment()

I introduced Olio a little while ago as a toolkit to help web developers and deployers as well as performance/operations engineers. Olio includes a web2.0 application as well as the necessary software required to drive load against it. Today, we are showcasing the first major deployment of Olio on Sun's newest Intel Nehalem based systems - the SunFire X2270 and the SunFire X4270. We tested 10,000 concurrent users (with a database of 1 million users) using over 1TB of storage in the unstructured object store.

The diagram below shows the configuration we tested.

The Olio/PHP web application was deployed on two X2270 systems. Since these systems …

I introduced Olio a little while ago as a toolkit to help web developers and deployers as well as performance/operations engineers. Olio includes a web2.0 application as well as the necessary software required to drive load against it. Today, we are showcasing the first major deployment of Olio on Sun's newest Intel Nehalem based systems - the SunFire X2270 and the SunFire X4270. We tested 10,000 concurrent users (with a database of 1 million users) using over 1TB of storage in the unstructured object store.

The diagram below shows the configuration we tested.

The Olio/PHP web application was deployed on two X2270 systems. Since these …

MySQL Scalability on Nehalem systems (Sun Fire X4270)

Today Sun announced multiple servers based on Intel's Nehalem Processor. I had early access to a Sun Fire X4270 server (2 socket) for a couple of weeks. I used this opportunity to test some of latest MySQL performance and scalability enhancements. For someone unfamiliar with this system, this is a 2 socket 2U server with support for a max of 144GB of memory. With hyperthreading turned on, the operating system sees 16 CPUs.

Before I share the results of my findings, lets get clear on the terminology. Socket refers to physical sockets on the motherboard. CPU refers to the number of processors seen by the operating system. Core refers to the physical processing unit. A Nehalem socket has 4 cores. Thread refers to the hyperthreading threads. One Nehalem core has 2 threads. Using this terminology, the …

MySQL Scalability on Nehalem systems (Sun Fire X4270)

Today Sun announced multiple servers based on Intel's Nehalem Processor. I had early access to a Sun Fire X4270 server (2 socket) for a couple of weeks. I used this opportunity to test some of latest MySQL performance and scalability enhancements. For someone unfamiliar with this system, this is a 2 socket 2U server with support for a max of 144GB of memory. With hyperthreading turned on, the operating system sees 16 CPUs.

Before I share the results of my findings, lets get clear on the terminology. Socket refers to physical sockets on the motherboard. CPU refers to the number of processors seen by the operating system. Core refers to the physical processing unit. A Nehalem socket has 4 cores. Thread refers to the hyperthreading threads. One Nehalem core has 2 threads. Using this terminology, the …