At Twitter, MySQL is an important part of our storage story. We can have a typical database node with thousands of open connections. It works mostly fine because generally only a few of these connections actively working inside the server. Given the scale at which we have to operate spikes happen – mostly at unexpected times and with unexpected ferocity. The challenge is how to control concurrency within MySQL server during spikes.
Historically we have been using innodb_thread_concurrency to control number of active threads within the server. In Twitter MySQL-5.5 we introduced the idea of Query Throttling where incoming requests are throttled if the number of active threads goes beyond a configurable threshold. These solutions focus on maintaining the health of the system. However, they don’t help us to scale to the workload spikes. With …[Read more]