Showing entries 1 to 7
Displaying posts with tag: wiredtiger (reset)
Peter Zaitsev webinar January 27th: Compression In Open Source Databases

Percona invites you to attend a webinar Wednesday, January 27th, with CEO Peter Zaitsev: Compression In Open Source Databases. Register now!

Data growth has been tremendous in the last decade and shows no signs of stopping. To deal with this trend database technologies have implemented a number of approaches, and data compression is by far the most common and important. Compression in open source databases is complicated, and there are a lot of different approaches – each with their own implications.

In this talk we will perform a survey of compression in some of the most popular open source database engines including: Innodb, TokuDB, MongoDB, WiredTiger, RocksDB, and PostgreSQL.

Important information: …

[Read more]
Checkpoint strikes back

In my recent benchmarks for MongoDB, we can see that the two engines WiredTiger and TokuMX struggle from periodical drops in throughput, which is clearly related to a checkpoint interval – and therefore I correspond it to a checkpoint activity.

The funny thing is that I thought we solved checkpointing issues in InnoDB once and for good. There are bunch of posts on this issue in InnoDB, dated some 4 years ago.  We did a lot of research back then working on a fix for Percona Server

[Read more]
Using Cgroups to Limit MySQL and MongoDB memory usage

Quite often, especially for benchmarks, I am trying to limit available memory for a database server (usually for MySQL, but recently for MongoDB also). This is usually needed to test database performance in scenarios with different memory limits. I have physical servers with the usually high amount of memory (128GB or more), but I am interested to see how a database server will perform, say if only 16GB of memory is available.

And while InnoDB usually respects the setting of innodb_buffer_pool_size in O_DIRECT mode (OS cache is not being used in this case), more engines (TokuDB for MySQL, MMAP, WiredTiger, RocksDB for MongoDB) usually get benefits from OS cache, and Linux kernel by default is generous enough to allocate as much memory as available. There I should note that while TokuDB (and TokuMX for MongoDB) supports DIRECT mode (that is bypass OS cache), we found there is a performance gain if OS cache is used for compressed pages.

[Read more]
Percona Acquires Tokutek : My Thoughts #2 : TokuMX and TokuMXse

A few days ago I wrote up my thoughts about the Percona acquisition of Tokutek with respect to TokuDB. In this blog I'm going to do the same for TokuMX and TokuMXse. And in a few days I'll wrap up this trilogy by sharing my thoughts about Fractal Tree Indexes.

Again, when I'm writing up something that I was very involved with in the past I think it's important to disclose that I worked at Tokutek for 3.5 years (08/2011 - 01/2015) as VP/Engineering and I do not have any equity in Tokutek or Percona.

Since much of the MySQL crowd might be hearing about Tokutek's "other products" for the first time I'll provide a little history of both of the products before I dive in deeper.

TokuMX is a fork of MongoDB …

[Read more]
How to benchmark MongoDB

There are generally three components to any benchmark project:

  1. Create the benchmark application
  2. Execute it
  3. Publish your results

I assume many people think they want to run more benchmarks but give up since step 2 is extremely consuming as you expand the number of different configurations/scenarios.

I'm hoping that this blog post will encourage more people to dive-in and participate, as I'll be sharing the bash script I used to test the various compression options coming in the MongoDB 3.0 storage engines. It enabled me to run a few different tests against 8 different configurations, recording insertion speed and size-on-disk for each one.

If you're into this sort of thing, please read on and provide any feedback or improvements you can think of. …

[Read more]
MongoDB Storage Engine Shootout : Round 1 : Indexed Insertion

The next release of MongoDB includes the ability to select a storage engine, the goal being that different storage engines will have different capabilities/advantages, and user's can select the one most beneficial to their particular use-case. Storage engines are cool. MySQL has offered them for quite a while. One very big difference between the MySQL and MongoDB implementations is that in MySQL the user gets to select a particular storage engine for each table, whereas in MongoDB it's a choice made at server startup. You get a single storage engine for everything on the particular mongod instance. I see pros and cons to each decision, but that's a blog for another day.

In MongoDB 3.0

[Read more]
Prediction: MongoDB 2.8 storage engines and the rise of the MongoDBA

MongoDB has always been about ease of use. With nothing more than the mongod binary, starting a MongoDB server is as simple as:
./mongod --dbpath=/path/to/dataAs a long time user of Oracle and MySQL I'm extremely impressed by just how simple this is. It certainly encourages new users to try it out.

In MongoDB 2.6 and earlier there has only been a single "storage engine" available in the server. That storage engine has very few tunable parameters, so the defaults are fine for most users. If you don't like the defaults you can probably change them with a little review of the documentation.

MongoDB 2.8 adds the ability to support an unlimited number of storage engines via a storage engine API. Using the alternative WiredTiger storage engine is as simple as asking for it on the command line:
./mongod --dbpath=/path/to/data --storageEngine …

[Read more]
Showing entries 1 to 7