Do you rely on pt-diskstats from Percona Toolkit instead of the standard iostat a lot? There appears to be a nasty bug in pt-diskstats 2.1, which makes it produce bad results.

I noticed some of the numbers I was getting weren’t right, so I tried running iostat and two different releases of pt-diskstat side by side. Here’s what I got:

I can understand the slight differences between the lines in iostat and pt-diskstats 2.0 as they probably weren’t reading /proc/diskstats contents in the same moments, so the values they were seeing could be a bit different. However both lines practically show the same thing.

On the other hand, the line based on the pt-diskstats 2.1 output looks completely different. It seems like maybe the script returns some kind of averages rather than per second deltas. I filed a bug report, so if you’re a pt-diskstats user, be sure to follow the ticket before upgrading.

For the time being, I suggest sticking to pt-diskstats 2.0, which I think was the only one that appears to be free of problems.

Update: See below for Baron’s explanation of the problem. Using non-default --group-by restores the expected default behaviour.