Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
MySQL and the Leap Second, High CPU and the Fix
+5 Vote Up -0 Vote Down

ETA: For those wanting to know the root cause, see the comments on the original post.

So, MySQL seemed to be faring just fine during and after the leap second. There were no big problems or catastrophic failures. However, we noticed that MySQL (and Java) servers* had a huge spike in CPU – which was odd, because nothing out of the ordinary was running. We were able to fix the problem quite easily. Here’s the graph, and the fix:

Note that the CPU spikes starting at 17:00 Pacific, which is midnight UTC. We fixed it around 19:35 Pacific.

The fix is quite simple – simply set the date. Alternatively, you can restart the machine, which also works. Restarting MySQL (or Java, or whatever) does NOT fix the problem. We put the following into puppet to run on all our machines:

$ cat files/bin/leap-second.sh
#!/bin/bash
# this is a quick-fix to the 6/30/12 leap second bug

if [ ! -f /tmp/leapsecond_2012_06_30 ]
then
/etc/init.d/ntpd stop; date -s "`date`" && /bin/touch /tmp/leapsecond_2012_06_30
fi

We ran this in puppet, which we set to run the command on the next puppet run, and puppet would also restart ntp (the way we have it configured; your mileage may vary).

* and a bunch of our IT guys have personal, at-home minecraft servers, which were also similarly affected.

Votes:

You must be logged in with a MySQL account to vote on Planet MySQL entries. More information on PlanetMySQL voting.

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.