A few years back Deva wrote about how to use tcpdump on very busy hosts. That post sparked my interest about exploring how to measure the impact of tcpdump on very busy hosts. In this post, I wanted to highlight how much of an impact there really is and what options you have to make the query collection much more effective.

Some things you need to know:

• The test is a sysbench read-only workload, 8 tables, 8 threads, 1000000 rows each with 16G of buffer pool. Dataset fully in memory.
• sysbench is ran on the same host, on 1Gbps connection, sysbench can saturate the network and therefore affect my network test with netcat so I decided to run locally.
• There are 13 tests, 5 minutes each with 1 minute interval, varying on how the dump file is captured.
  • First one as …

Percona Toolkit has a great tool, pt-query-digest, that can use tcpdump data. Capturing raw tcp data can be taxing on a server, however, when you see the following message:

64000 packets received by filter
12000 packets dropped by kernel

When there is a significant amount of user cpu% being used, the kernel will drop packets you are trying to capture, leading to a partial picture and missing data. I’ve found that if you write it using the native tcpdump format, it’s more efficient and you drop less. There are also recommendations on Stack Overflow on how to help prevent this.

To perform a capture for a specific length of time, here’s the trick I …

You probably know that it’s possible to set configuration variables to log queries that don’t use indexes to the slow query log in MySQL. This is a good way to find tables that might need indexes.

But what if the slow query log isn’t enabled and you are using (or consulting on) MySQL 5.0 or earlier, where it can’t be enabled on the fly unless you’re using a patched server such as Percona’s enhanced builds? You can still capture these queries.

The key is knowing what it really means for a query to “not use an index.” There are two conditions that trigger this — not using an index at all, or not using a “good” index. Both of these set a bit. If either bit is set, the query is captured by the filter and logged. Both of these bits also …