Ever get called out for a MySQL issue only to realize that there was no issue? It was a false alarm from the monitor. We sure have and it’s frustrating, especially at 3:00 or 4:00 in the morning!
Many DBAs work in an environment where there is some sort of first level support that gets assigned tickets first. Unfortunately, many of the times these groups are, shall we say, less than skilled in MySQL. As a result, they quickly escalate the ticket onto the primary on-call DBA, even when there is really nothing wrong.
Much of the time, there are multiple types of MySQL topology in these environments: standalone, galera cluster, replication, etc. Writing large runbooks with detailed test cases can be a daunting process and one that will cause many first-level support engineers to give up and simply escalate the issue anyway.
In an effort to avoid undue call outs, we developed a simple bash …[Read more]