I've been talking and writing a bit lately about the scaling
problems I'm seeing on fast servers running lots of queries. As a
rough guide, I'm seeing this in servers running 20k queries per
second and higher, lots of memory, lots of CPU cores, and most
queries are running faster than one millisecond; some in the 50
to 100 microsecond range.
At such speeds, how would you know if your server froze for half
a second? This is not a hypothetical question. If your server
freezes routinely for half a second, then it's occasionally going
to freeze for much longer than that. Knowing whether it's
freezing routinely for short bursts is important to know. Add
just a bit more data, or add a bit more load, and suddenly things
can drop off a cliff.
I had a customer a while back whose server suddenly took a 50x
performance hit, overnight. He didn't know of anything that had
changed. Reducing the load by half through the load balancer did
not …
[Read more]