The alert fires at 06:37 and by the time anyone opens a laptop
the database is back. The application log holds a wall
of Lost connection to MySQL server during
query, uptime says 41 days, and the
error log records a clean shutdown: no crash, no signal, no
out-of-memory kill. The next morning it happens at 06:12, and the
morning after that not at all.
On a default Ubuntu or Debian
server, unattended-upgrades installs
security updates on a daily systemd timer, and when one touches
MySQL, dpkg stops mysqld, upgrades the data
directory to match the new binary, and starts it again while the
machine stays up. Nothing rebooted. The database was replaced
underneath a host that never stopped running.
That is deliberate, and the part nobody signed up for is that “a service” includes the database. We put a client in front of MySQL on an Amazon Elastic …
[Read more]