November 19, 2013 By Severalnines

The Severalnines team is pleased to announce the release of ClusterControl 1.2.4. This release contains key new features along with performance improvements and bug fixes.

We have outlined some of the key features below. For additional details about the release:

• 1.2.4 ChangeLog
• Documentation
• Upgrade instructions
  • Manual upgrade instructions
  • …

Attending Percona Live in London next week?

Don’t miss the chance to hear Tokutek’s Vice President of Engineering, Tim Callaghan, discuss how to use your MySQL knowledge to become an instant MongoDB Guru and the advantages of using Fractal Tree® indexes in MySQL and MongoDB. Tim will be speaking about these topics in two separate sessions at 12:00pm and 5:00pm on November 12.

For more information on these sessions and Percona Live-London, visit https://www.percona.com/live/london-2013/users/tim-callaghan.

Since our initial release last summer, TokuMX has supported fully ACID and MVCC multi-statement transactions. I’d like to take this post to explain exactly what we’ve done and what features are now available to the user.

But before beginning, an important note: we have implemented this for non-sharded clusters only. We do not support distributed transactions across different shards.

At a high level, what have we done?

We have taken MongoDB’s basic transactional behavior, and extended it. MongoDB is transactional with respect to one, and only one, document. MongoDB guarantees single document atomicity. Journaling provides durability for that document. The database …

Since introducing TokuMX, we’ve discussed benefits that TokuMX has for existing MongoDB applications that require no changes. In this post, I introduce an extension we’ve made to the indexing API: clustering indexes, a tool that can tremendously improve query performance. If I were to speak to someone about clustering indexes, I think the conversation could go something like this…

What is a Clustering Index?

A clustering index is an index that stores the entire document, not just the defined key.

A common example is …

One of our customers reported that ‘create table select from’ statements stall for a period of time equal to the TokuDB lock timeout.  This indicated a lock conflict between multiple transactions.  In addition, other MySQL clients that were opening unrelated tables were also stalled.  This indicated that some shared mutex is held too long.  We discuss details about this bug and how it was fixed.  The bug fix will be distributed in TokuDB 7.1.0.

Example
Suppose that we set the tokudb lock timeout to 60 seconds just to exaggerate the stall.

mysql> set global tokudb_lock_timeout=60000;
Query OK, 0 rows affected (0.00 sec)


We then create a simple table.

mysql> create table s (id int primary key);
Query OK, 0 rows affected (0.02 sec)


When we create new table (t) and populate it from some …

When I get to talk to people about TokuMX and how it’s an optimized MongoDB, I sometimes get follow-up questions like:

• “Is it an in-memory proxy?”
• “Write optimized? So you buffer all of the writes in memory and lose them on crash?”
• “Did you re-implement the server and match the protocol?”

None of these things describe TokuMX, but it demonstrates that there are many schools of thought on how to optimize databases, and MongoDB in particular. I’d like to elaborate more on what TokuMX really is and how we built it. First, let’s talk about what MongoDB is.

MongoDB consists of a server process that stores data and executes queries, mongod, a sharding router process, mongos, as well as a wire protocol for interacting with these servers, and clients and language drivers that implement this …

Last week I described TokuDB’s new Hot Backup feature.  This week we are going to briefly discuss the same feature, but as it was added to TokuMX, our version of MongoDB.

Since the Hot Backup library is essentially a shim between MySQL and the Linux kernel, intercepting file system calls for the life of the process, it should be easy to add this to any other system, including TokuMX.  Indeed with our addition of transactions and logging to TokuMX we can gain a consistent backup of any data set at any time.

Unlike MySQL, where system tables use the non-transactional MyISAM storage engine, TokuMX uses internal (non-explicit) transactions for all meta-data changes and regular CRUD operations on BSON data.  This …

One of our customers sometimes observed lots of simple insertions taking far longer than expected to complete. Usually these insertions completed in milliseconds, but the insertions sometimes were taking hundreds of seconds. These stalls indicated the existence of a serialization bug in the Fractal Tree index software, so the hunt was on. We found that these stalls occurred when a big transaction was committing and the Fractal Tree index software was taking a checkpoint. This problem was fixed in both TokuDB 7.0.3 and TokuMX 1.0.3. Please read on as we describe some details about this bug and how we fixed it. We describe some of the relevant Fractal Tree index algorithms first.

What is a Big Transaction?

Each transaction builds a rollback log as it performs Fractal Tree index operations. The rollback log is maintained in memory until it gets too big and is spilled to the TokuDB rollback file. We define a small …

TokuMX v1.2.1 introduces two simple new features to help you understand the performance characteristics of your database: lock diagnostics and index usage statistics. We’d like to take you through a few examples of what these features are and how to use them.

Lock Diagnostics

Since we introduced TokuMX, one of the most frequent complaints has been about “lock not granted” errors.  These arise when a long-running operation takes document-level locks, and other clients timeout while waiting to acquire the same locks.

This is a new problem in TokuMX that doesn’t exist in MongoDB, because MongoDB doesn’t have document-level locks.  It has a single lock per database, and if an operation takes a long time, it simply …

In talking to existing MongoDB users and TokuMX evaluators, I’ve often heard that the performance of MongoDB is very good as long as your working data set fits in RAM. The story continues that if your working data set grows to be larger than the RAM on your server, the built-in sharding capabilities of MongoDB allow you to scale horizontally.

As my benchmarking presentation at Percona Live 2013 pointed out, I’m never one to accept something without at least running it once myself. I decided to run my Sysbench for MongoDB benchmark on an in-memory workload, meaning that all of the data fits …