Queries in MySQL, Sphinx and many other database or search
engines are typically single-threaded. That is when you issue a
single query on your brand new r910 with 32 CPU cores and 16
disks, the maximum that is going to be used to process this query
at any given point is 1 CPU core and 1 disk. In fact, only one or
the other.
Seriously, if query is CPU intensive, it is only going to be
using 3% of the available CPU capacity (for the same 32-core
machine). If disk IO intensive – 6% of the available IO capacity
(for the 16-disk RAID10 or RAID0 for that matter).
Let me put it another way. If your MySQL or Sphinx query takes
10s to run on a machine with a single CPU core and single disk,
putting it on a machine with 32 such cores and 16 such disks will
not make it any better.
But you knew this already. Question is – can you do something
about it?
In case of Sphinx – indeed you can! And with very little …
[Read more]