So, at midnight I got a call from customer service saying our site was slow. I narrowed it down to one of our auxiliary databases, that seems to have gotten wedged just about midnight. Normal queries that took less than 4 seconds started taking longer and longer, moving up to 5 seconds and past 30 seconds in the span of a minute or so.
In the moment, I thought killing off all the queries would be a good move. My kill script, which consists of:
for i in `/usr/bin/mysql -u user -pPass -e ’show full
processlist’ | grep appuser | cut -f1`
do
mysql -u user -pPass -e “kill $i”
done
This will attempt to kill any mysql connection owned by the appuser. I used it a few times, and it didn’t work. So I used a trick I learned when we bring our site down — sometimes there are straggling connections to mysql, so what I do is change the app user’s password by direct …
[Read more]