MySQL Adventures: How max_prepared_stmt_count can bring down
production
We recently moved an On-Prem environment to GCP for better
scalability and availability. The customer’s main database is
MySQL. Due to the nature of customer’s business, it’s a highly
transactional workload (one of the hot startups in APAC). To deal
with the scale and meet availability requirements, we have
deployed MySQL behind ProxySQL — which takes care of routing some
of the resource intensive SELECTs to chosen replicas. The setup
consists of:
- One Master
- Two slaves
- One Archive database server
Post migration to GCP, everything was nice and calm for a couple
of weeks, until MySQL decided to start misbehaving and leading to
an outage. We were able to quickly resolve and bring the system
back online and what follows are lessons from this experience.
The configuration of the …
[Read more]