Simple auditing, i.e., knowing what changed recently, can save you tons of time while troubleshooting.
I know that, in the ideal world:
- Everything is supposed to be done through configuration management.
- Everything is documented and all changes are tracked through a VCS.
- Every DDL or set global is trapped via MySQL Proxy and logged.
But there are always ways to bypass the gatekeepers. Changes can go in unnoticed. An hour or so later, your database performance suddenly changes for the worse, and you get that phone call.
First you check if anything caused an actual error. You look around at a few log files and nothing shows up. The next thing you ask yourself is, did someone change anything in the last little while. Of course, everybody says no. After a few hours of digging, comparing schemas, diff-ing old and current config files, you actually find what has changed, …[Read more]