So, I mentioned earlier that I was looking at Infobright's Brighthouse technology as a storage backend for heaps and heaps of traffic and user data from Habbo. Turns out it works fine (now that it's in V3 and supports more of the SQL semantics), and we took it into use. Been pretty happy with that, and I expect to talk more about the challenge and our solution at the next MySQL Conference in April 2009.

However, our DWH team needs extra help. If you're interested in solving business analytics problems by processing lots of data and the idea of working in a company that leads the virtual worlds industry excites you, …

A few months ago I was at dinner with Yves Trudeau discussing what all consultants discuss in the late hours after a long day of hard work… how to improve performance and scalability.  I brought up an idea to him to utilize memcached as an L2 cache for innodb.  At first he was skeptical, but as we talked he was more and more intrigued by the idea.  The idea was simple, add a set to memcached when something hit the LRU…  then issue a get from memcached when you do not find the data locally stored in the buffer pool but before you read from disk.  Starting from that point, you can work out any of the issues that would be sure to follow.   So Yves continued emailing me asking me questions…  then he sent me a note that he had made huge progress with the idea. Huge progress means that he wrote version 0.1 and had it working.    That’s when the Idea really turned into a project.

We called …

Following up on my previous post Peter asked if he could see a 24 hour run on the Intel drive…  took me a few days because I am doing some testing on a few other things, but I kicked a run off yesterday before leaving the Vegas airport…  Here ya go:

These are roughly 10 minute TPM averages.  As you can see their is a definate decline in performance the longer the drive is active.   Its realtively small% wise, only about 6% off of peak…  but the delcine is easy to spot.

What’s this mean?  It you beat the hell out of the drive over a long contious period things slow down the longer the drive is in continuous use.   Just an FYI, the drive was 78% full during these tests.

At LISA 2008, I gave a presentation entitled "How to Stop Hating MySQL: Fixing Common Mistakes and Myths".

The presentation slides can be downloaded as a PDF at:

http://technocation.org/files/doc/stophatingmysql.pdf

View the video online at http://technocation.org/node/646/play or download the 202.5 MB Flash video file (.flv) directly at http://technocation.org/node/646/download.

Here are some notes and links I referred to:

read more

Thought I would throw this out here so I remember it, and it maybe useful for someone else.  During a session here at Sun’s CEC conference one of the presenter was talking about setting CPU affinity for hardware interrupts.  This made me think of our hack ( workaround ) for innodb scalability on a 16 core machine ( set the mysqld affinity to cpu’s 0-7 or 8-15 ).   In order to get the top performance out of a 16 core machine we probably want to ensure everything else goes to the cpu’s not set to be servicing mysql.   So a quick google search came up with this article on setting the affinity for IRQ’s in the linux kernel.   So with a 16 core box, I could set the affinity for Mysqld to cpu’s 8-15, while setting the affinity of the interrupts to something in the 0-7 range.

You might be wondering what's been happening with MySQL performance since Sun arrived on the scene. The good news is that we haven't been idle. There's been general recognition that MySQL could benefit from some performance and scalability enhancements, and Sun assembled a cross-organizational team immediately after the acquisition to get started on it. We've enjoyed excellent cooperation between the engineers from both organizations.

This kind of effort is not new for Sun - we've been working with proprietary database companies on performance for years, with source code for each of the major databases on site to help the process. In this case, the fact that the MySQL engineers are working for the same company certainly simplifies a lot of things.

If you'd like to get some insight into what's been happening, a video has just …

You might be wondering what's been happening with MySQL performance since Sun arrived on the scene. The good news is that we haven't been idle. There's been general recognition that MySQL could benefit from some performance and scalability enhancements, and Sun assembled a cross-organizational team immediately after the acquisition to get started on it. We've enjoyed excellent cooperation between the engineers from both organizations.

This kind of effort is not new for Sun - we've been working with proprietary database companies on performance for years, with source code for each of the major databases on site to help the process. In this case, the fact that the MySQL engineers are working for the same company certainly simplifies a lot of things.

If you'd like to get some insight into what's been happening, a video has just …

You might be wondering what's been happening with MySQL performance since Sun arrived on the scene. The good news is that we haven't been idle. There's been general recognition that MySQL could benefit from some performance and scalability enhancements, and Sun assembled a cross-organizational team immediately after the acquisition to get started on it. We've enjoyed excellent cooperation between the engineers from both organizations.

This kind of effort is not new for Sun - we've been working with proprietary database companies on performance for years, with source code for each of the major databases on site to help the process. In this case, the fact that the MySQL engineers are working for the same company certainly simplifies a lot of things.

If you'd like to get some insight into what's been happening, a video has just …

If you are a MySQL 5.x/6.0 InnoDB replication user, right now you take a significant performance hit on the replication master simply by turning on the binlog. The good news is that we've taken a big step toward eliminating that performance gap. I'll describe the problem, how I was able to track down the root cause, and point to a patch that fixes the problem. Since the changes are in the InnoDB code, right now we're waiting on Oracle/Innobase to review the fix and formally commit it. Once that happens, you should see it show up in binary releases. In the meantime, if you build your own binaries you can test the prototype patch yourself.

One of the things I have been working on quite a bit over the past several months is scalability of the nodes within a MySQL scale-out replication environment.  The reason being that there has been a rapid …