Tokutek created the iiBench benchmark back in 2008. The point of the benchmark is to measure the performance of indexed insertions over time. It uses an extremely simple schema, one table with a sequential insertion pattern for the primary key along with three integer fields storing random values. The table maintains 3 secondary indexes, each including several of the random integer fields. The iiBench application itself is currently maintained on Launchpad.

B-tree implementations generally require maintenance operations to update leaf nodes (an insertion is one such operation). When the entire B-tree index does not fit in RAM an IO is required, and performance drops dramatically. Fractal Tree Indexes do not exhibit this performance drop as leaf node maintenance is deferred, allowing for several hundred (or thousand) leaf node maintenance operations to be performed with a single IO.

I created a new version of iiBench specifically for MongoDB to compare the performance over time of TokuMX versus MongoDB. For anyone interested the code is available on GitHub. It is a simple benchmark application written in Java (sorry Henrik) and can easily be modified for other benchmark tasks.

Benchmark Environment

• HP Proliant DL380 G6, (2) Xeon 5520, 16GB RAM, P410i (512MB, write-back), 8x10K SAS/RAID 0
• Centos 5.8 (64-bit), XFS file system
• MongoDB v2.2.3 and MongoDB v2.2.0 + Fractal Tree Indexes

Benchmark Results – Insertion Performance

As the performance graph shows, the insertion performance of MongoDB drops substantially when the indexes no longer fit in RAM while TokuMX does not. The exit throughput* of MongoDB is 169.49 inserts per second, while TokuMX is 10,609.63 inserts per second. In other words, the exit throughput of TokuMX is over 60 times faster than MongoDB.

“exit throughput” = average inserts per second over the last 5 million inserts.

Benchmark Results – IOPs

Also worth noting is a comparison of the IO with MongoDB and TokuMX. Over the entire benchmark, MongoDB averaged 1,378.94 write operations per second, whereas TokuMX averaged just 55.27. Keep in mind that the TokuMX line in this graph is so short since it finished the 100 million inserts in just over 2 hours, MongoDB required over 29 hours for the same number of insertions. Also, I suspect the write IO in MongoDB decreased over time given the reduction of document inserts per second.

Try TokuMX for Yourself

We are in the Release Candidate process for TokuMX, if you are interested in participating please drop us a line.