This is part 3 of my Tyrant extra's, part 1 focused on
durability, part 2 focused on the perceived performance wall.
#3. Tokyo Cabinet Can have only a single writer thread,
bottlenecking performance
When writing an application using Tokyo Cabinet only one
connection can be opened as a “writer” while the rest are
readers. Tyrant allows for multiple “writes” to be
sent in from multiple applications but it still single threads
them when writing out to disk. If you run several
threads all just inserting into Tyrant your will see tyrant hit
100% Cpu on 1 core, and your writes will start to peter out
quickly.
In my tests when I was not disk bound (FS Cache writes) I was
able to complete 4Million inserts in a little over 91 seconds
using 8 threads. I actually averaged 43896.98 inserts per
second during my 8 thread test. Moving to 10 threads doing
the same 4Million inserts I …
[Read more]