Tokuview (http://blogs.tokutek.com/tokuview/iibench/) is building a test scripts/program for 1B record insert performance.  This is fine, but ultimately the logistics of the DB environment will dictate what ‘tricks’ can be pulled to insert records the fastest.  Can you access raw MyISAM files or not?  Are you running concurrent inserts?  Do you require ACID compliance? There are 3 basic [...]

As one of the co-founders of the Waffle Grid project, I beam with pride every time I get a stellar benchmark or every time I find a new use for the Waffle.  But as a professional I still have to be critical of all solutions I would recommend or deploy.  One of the big goals of Waffle Grid is to replace disk IO which should be slow with remote memory which should be much faster.  But what happens when the disk is no longer slow?  This leads me to ask myself, is Waffle Grid only good for servers with slower disk?  Or is this a solution that can also help systems with fast disk?  So which should you deploy SSD -vs- Waffle?  Are they competitors?  Or are they complementary technologies?

I am going to say this, in these tests latency is king.  The faster the drives can deliver data, the higher the benchmarks should be.  Basically if my interconnect can deliver faster then the drive can serve up …

Maybe I should make the sub-tag line, make your MySQL database run up to 300% faster by using Waffle Grid…   Ahhh its all about marketing.

Had an interesting holiday, snuck away several times to benchmark and test out the new Waffle Grid release… nothing says turkey day like Waffles and a really fast database  ( Hey! Yes my wife knew she married a complete and utter nerd, she just might not have understood the depths of my nerdom ).  As I mentioned before with the release of MySQL 5.1.30 I switched over all my testing over to this version, and in my tests this combination is running really well.  Additionally I had a hardware issue which compelled me to retest some of the tests I did last week. So who loves benchmarks!  I DO I DO!!!  I like to go fast, so lets get right down to it.

Continuing on with the discussion on Waffle Grid, I have put together a few interesting benchmarks and wanted to expand on some of the possible deployment scenarios using Waffle.

My home laboratory is rather small, so my datasets are limited in size for the moment. To compensate I have shrunk the data sizes and database setting to simulate as best as I can the performance on a larger system. For this first test I decided to go with a 20 Warehouse DBT2 test, which equates to about 3GB of data in the database. In order to get a better feel for the performance benefit of fitting all most of your data in local memory + remote I setup my local machine with 768M allocated to the innodb buffer pool, and setup a remote machine with 768M allocated to memcached. This should scale up at a somewhat similar rate. What I mean by this is the increase seen with a 3GB dataset with 768/768 local/remote split I should also see with a 6GB dataset with a …

So, I mentioned earlier that I was looking at Infobright's Brighthouse technology as a storage backend for heaps and heaps of traffic and user data from Habbo. Turns out it works fine (now that it's in V3 and supports more of the SQL semantics), and we took it into use. Been pretty happy with that, and I expect to talk more about the challenge and our solution at the next MySQL Conference in April 2009.

However, our DWH team needs extra help. If you're interested in solving business analytics problems by processing lots of data and the idea of working in a company that leads the virtual worlds industry excites you, …

A few months ago I was at dinner with Yves Trudeau discussing what all consultants discuss in the late hours after a long day of hard work… how to improve performance and scalability.  I brought up an idea to him to utilize memcached as an L2 cache for innodb.  At first he was skeptical, but as we talked he was more and more intrigued by the idea.  The idea was simple, add a set to memcached when something hit the LRU…  then issue a get from memcached when you do not find the data locally stored in the buffer pool but before you read from disk.  Starting from that point, you can work out any of the issues that would be sure to follow.   So Yves continued emailing me asking me questions…  then he sent me a note that he had made huge progress with the idea. Huge progress means that he wrote version 0.1 and had it working.    That’s when the Idea really turned into a project.

We called …

Following up on my previous post Peter asked if he could see a 24 hour run on the Intel drive…  took me a few days because I am doing some testing on a few other things, but I kicked a run off yesterday before leaving the Vegas airport…  Here ya go:

These are roughly 10 minute TPM averages.  As you can see their is a definate decline in performance the longer the drive is active.   Its realtively small% wise, only about 6% off of peak…  but the delcine is easy to spot.

What’s this mean?  It you beat the hell out of the drive over a long contious period things slow down the longer the drive is in continuous use.   Just an FYI, the drive was 78% full during these tests.

At LISA 2008, I gave a presentation entitled "How to Stop Hating MySQL: Fixing Common Mistakes and Myths".

The presentation slides can be downloaded as a PDF at:

http://technocation.org/files/doc/stophatingmysql.pdf

View the video online at http://technocation.org/node/646/play or download the 202.5 MB Flash video file (.flv) directly at http://technocation.org/node/646/download.

Here are some notes and links I referred to:

read more

Thought I would throw this out here so I remember it, and it maybe useful for someone else.  During a session here at Sun’s CEC conference one of the presenter was talking about setting CPU affinity for hardware interrupts.  This made me think of our hack ( workaround ) for innodb scalability on a 16 core machine ( set the mysqld affinity to cpu’s 0-7 or 8-15 ).   In order to get the top performance out of a 16 core machine we probably want to ensure everything else goes to the cpu’s not set to be servicing mysql.   So a quick google search came up with this article on setting the affinity for IRQ’s in the linux kernel.   So with a 16 core box, I could set the affinity for Mysqld to cpu’s 8-15, while setting the affinity of the interrupts to something in the 0-7 range.