In a couple of recent consulting cases, I needed a tool to analyze how a log of queries accesses indexes and tables in the database, specifically, to find out which indexes are not used. I initially hacked together something similar to Daniel Nichter’s mysqlidxchk, but using the framework provided by Maatkit, which gave me a pretty good start right out of the box. This was useful in the very tight time constraints I was under, but was not a complete solution.

John Hawkins is now the second person with a real-life WordPress tattoo. You can see it finished here, and two in-progress shots. John used the variation of the logo they did for WordCamp Las Vegas which he also organizes. Perhaps we should create a new category on Code Poet for 9seeds.

This is part three of an ongoing series about my experiences while writing the MySQL Admin Cookbook for Packt Publishing. All previous parts can be found under the mysql-admin-cookbook label.

Even though I said I would be presenting things in mostly chronological order, I think after the previous - rather dry - part, a little more technical and fun information would be nice for a change: The tools used to create the MySQL Admin Cookbook (well, at least those used by Udo and me). To give a detailed account of what software products we used during the whole experience I will split this topic up into multiple posts. Otherwise it would just become either way too …

In The Doom Of Multiple Storage Engines, PeterZ lists a number of unique technological challenges that MySQL has to face precisely because it chooses Storage Engines and the way it implements them.

There is another, nontechnical challenge that needs solving by vendors that are producing commercial closed source storage engines for MySQL - my experience is with one specific vendor, but the situation is basically the same for each and every closed source SE:

Where I work, we are a MySQL support customer. Actually, we are a very satisfied MySQL support customer, because in our opinion the MySQL support just plain rocks and has more than once saved our corporate asses, or at least greatly improved our MySQL experience.

If we were to load a closed source storage engine into our MySQL binaries, we would make life …

I perpetually see something like the following:

My server load is high and my queries are slow and my server crashes. Can you help me tune my server? Here is some information.

[random sample of SHOW GLOBAL STATUS, like the query cache counters]

my.cnf:

[mysqld]
key_buffer_size=1500M
query_cache_size= 64M
max_connections = 256
key_buffer = 8M
sort_buffer_size = 100M
read_buffer_size = 8M
delay_key_write = ALL

There are many problems in this my.cnf file, but the sort_buffer_size is a glaring one that identifies the user as someone who should not be playing with live ammunition. Therefore, I have developed an advanced process for tuning sort_buffer_size, which you can follow to get amazing performance improvements. It’s magical.

1. How expert are you?
   • I know that there is a sort buffer, and that it is related to sort_merge_passes. When …

I continually see consulting engagements like the following: My server load is high and my queries are slow and my server crashes. Can you help me tune my server? Here is the my.cnf: [mysqld] sort_buffer_size = 100M Such a large sort_buffer_size is a serious problem, but unfortunately there is a lot of cargo cult advice on the Internet, in books, and in “tuning scripts,” that perpetuates the harmful advice to increase it.

One of the big "Selling Points" of MySQL is support for Multiple Storage engines, and from the glance view it is indeed great to provide users with same top level SQL interface allowing them to store their data many different way. As nice as it sounds the in theory this benefit comes at very significant cost in performance, operational and development complexity. Let me touch just on the few complications it brings in

Conversions - Each storage engine has its own data storage format so MySQL has to do a lot of copying and conversions while accessing the data, which significantly limits performance compared to basically "zero copy" design one may have streaming data from memory when it fits there.
Optimizer and Execution Storage engines are not created equal especially if you look at In Memory storage engines vs Distributed ones. Trying to handle all different cases in Optimizer - such as in memory …

Few years ago I was looking at crash recovery code, and realized that InnoDB has removed all the comments from the code [this assumption is debunked by Heikki in comments section], related to replay of transaction log. Judging by high quality of comments in the remaining codebase, I realized that it was all done to obscure any efforts to build another InnoDB hot backup solution – competitor to first Innobase standalone offering.

I was enjoying the moment when Percona launched their own implementation of the tool. Since the inception, it became more and more robust and feature rich.

We have used xtrabackup in our environment a lot – just… not for backup – the major use case right now is for cloning server instances – either for building new replicas, shadow servers, or …

Answering questions asked on the site.

Ivo Radev asks:

I am trying to make a very simple query.

We have a log table which different machines write to. Given the machine list, I need to find the latest log timestamp.

Currently, the query looks like this:

SELECT  MAX(log_time)
FROM    log_table
WHERE   log_machine IN ($machines)

, and I pass the comma-separated list of $machines from PHP.

The weird thing is that the query is literally instant when there is only one machine (any) in the list but slow when there are multiple machines.

I’m considering doing it in separate queries and then process the results in PHP. However I’d like to know if there is a fast solution in MySQL.

Most probably, there is a composite index on (log_machine, log_time) which is being used for the query.

Usually, a query …

Our Universe is full of ironies. But some ironies are just too hard to take.

As you may have guessed (!!!), I am an avid Linux developer and user. Though once upon a time I did develop under Windows. Yes, believe it. And on one particular case, I got to be on a first-name basis with some of the Microsoft Software Engineers to resolve issues we were having with their OLE crap — what the Holy Gods of Microsoft decided to redub as “Active-X”.

But I digress. For the past 10 years, I have been solid Linux and have defenestrated Windows for the most part. But as you know, you can never really completely eliminate Windows.  Despite your best efforts, it will always be (for now, at least) the 500 pound gorilla in any room you care to be in. The installed software base there is just staggering, and most have no Linux options.

But then that’s why projects like Wine and the many wonderful hypervisors …