Everyone knows the Internet Movie Database, and I wanted to use their public data as a large data set to create some complex queries. The MySQL world database is excellent to teach SQL basics, but not sufficient to teach performance tuning or real life queries.

Since the data comes in a bunch of clumsy text files, it is not easy to parse and import them. Fortunately, this work was done by some German guys. All you need, is their JMDB software, a java environment, of course a MySQL database and... lots of time.

They claim that it's possible to load all the data within 2 hours, but it took my little laptop almost a day. After the import procedure, you end up with a database called "jmdb" and a nice number of huge tables. In total, it's more than 20 million rows in 44 tables.

It's all MyISAM tables, which is fast enough as the import doesn't run parallel queries. And for reading, MyISAM will also be the best option, …

Its been a while since I have posted on my blog. I should at least note that I will be speaking at MySQL User Conf in Santa Clara as well as php|tek in Chicago. Aside from that I am working a little bit on improving Doctrine a very promising DBAL/ORM based on PDO for PHP. Anyways lets get back to the original topic which has nothing to do with PHP or databases directly, but might also be of concern to all of you still. Since I am now in the consulting business I am spending a lot of time working at client premises using my laptop. I really miss an external monitor though. I find it highly unproductive to alt+tab between my editor and the browser or some documentation all the time. …

I read through a lot of blogs each day in a variety of technology categories. I've always fashioned myself as a jack-of-all-trades and that helps me a great deal with my current position. However, it does mean consuming a lot of feeds to try and stay up in all the areas I have a profound interest in. Here are my shared feeds on share.opml.org.

Google Reader has a nice feature where I can share items I find interesting. There are a ton of good blog posts each day, so I've started marking them to be shared. There are two ways to view these shared items: one is the web page and the …

My fellow MySQLers who I have chatted with recently on IRC, know how much the AUTO-INC table lock in MySQL has been bugging me. Whenever my server gets an increased number of concurrent INSERTs, I start experiencing thrashing.

Yesterday night I spent all night trying to find the bottleneck within my system which happened to be none other than disk bound.

Everyone knows that InnoDB is excellent for its concurrency, thanks to its row level lock granularity. What comes as a surprise, to some, is that if you happen to use an AUTO INCREMENT column, then InnoDB needs to acquire a special, table level, AUTO-INC lock.

Everything was fine until I started finding many threads in InnoDB queue waiting to acquire this lock.

iostat -xnz 5 reported disk busy at 100% most of the times.

…

Recently, I found out that I'll be speaking at the upcoming MySQL Conference in Santa Clara, California. Last year, it was a lot of fun and the conference was very successful. I can't wait to see everyone there again.

A couple of weeks ago I submitted a request to open a new project on Sourceforge for the innotop MySQL and InnoDB monitor. I want to make it easier for others to collaborate, especially package maintainers. Yesterday I got word of its approval. I have done a quick-and-dirty import of the source code into its new home, and I'm now continuing work on the next major version, which I've been working on for about six weeks. This post is about Sourceforge, what I've gotten done, and also to ask for your help.

One small fact about backup table and restore table that isn’t listed in the manual is that these commands lose the auto increment value if rows at the head of the table are deleted. For example if you have auto increment values of 1,2,3,4 in a table the auto increment value is 5. If you delete row 4 the next auto increment will still be 5. If you backup/restore the table the auto increment will be reset to 4. The auto increment value in myisam is stored in the MYI file. Since this file isn’t backed up myisam restores the auto increment value from the highest existing value in the table. This value may or may not be the actual value of auto increment. As the manual says these commands should not be used. Instead use mysqlhotcopy.

A couple of weeks ago I submitted a request to open a new project on Sourceforge for the innotop MySQL and InnoDB monitor. I want to make it easier for others to collaborate, especially package maintainers. Yesterday I got word of its approval. I have done a quick-and-dirty import of the source code into its new home, and I’m now continuing work on the next major version, which I’ve been working on for about six weeks.

A few weeks ago, nineteen MySQLers had a successful meeting in Berlin. The QA and Build teams invited some guests from other Engineering teams and from Support, and we experimented with a couple of new meeting practices. We’re not yet at Meeting Nirvana, but we felt that we had taken great strides compared to several earlier MySQL Dev Mtgs.

Plenty of literature and blogs exist on arranging good meetings. The scope of this blog article is limited to physical meetings in virtual organisations, i.e. collecting a group of people who work together every day but see each other once or at most twice a year. That special context puts the emphasis on certain decisions and practices, which I attempt at highlighting below.

The twelve items come in a chronological order.

Before, during and after the meeting 1. Set the right meeting goals.

1. Try to solve fewer problems than you …

The other day I heard that InnoDB UPDATE statements perform DELETE and INSERTs and REPLACE is therefore more efficient. Now I'm trying to confirm it by finding documentation on it.