At tomorrow’s Effective MySQL Meetup, I’ll be presenting “Fractal Tree Indexes : Theory and Practice (MySQL and MongoDB).” The meetup is at 6:30pm Tuesday, May 14, 2013, and will be held at Alley NYC in New York City.

I’ll give an overview on how Fractal Tree® indexes work, and then get into specific product features that Fractal Trees enable in MySQL and MongoDB.  Some benchmarking and customer use-cases will be discussed, but my intent is for this to be a deep technical dive.  Several Tokutek Engineers will also be on hand, so bring any questions you’ve got.

I hope to see you there!

I was reviewing the Percona Live sponsors list the other day and pondering the potential success stories associated with this product or that one…. and as I was preparing to put more thought on the topic, a PlanetMySQL post caught my eye. It was penned by Mike Hogan and titled, “Thoughts on Xeround and Free!”

For some reason the author of that post makes a connection between a free account in a cloud-based service and Open Source software. I think it’s

After compiling Percona Server with TokuDB, of course I wanted to compare InnoDB performance vs TokuDB.
I have a particular workload I’m interested in testing – it is an insert-intensive workload (which is TokuDB’s strong suit) with some roll-up aggregation, which should produce updates in-place (I will use INSERT .. ON DUPLICATE KEY UPDATE statements for that), so it will produce all good amount of reads.

A few words about the hardware: I am going to use new the Dell PowerEdge R420 with two Intel(R) Xeon(R) CPU E5-2450 0 @ 2.10GHz, 48GB of RAM and SATA SSD: Kingston HyperX 3K 240 GB.

Workload: I will use two different schemas. The first schema is from sysbench, and

Last week I had to present a tutorial at Percona Live 2013, a presentation at SkySQL’s MySQL & Cloud Database Solution Day and last but not least, a presentation on a Saturday morning at Linuxfest Northwest. It wasn’t easy, but giving the presentations after our announcement early in the week about going open source was very exciting given the

With this version, the source code is now freely available under the GPL License v2. For more details, see our blog here. Open source pioneer Mozilla has been using TokuDB to manage its MySQL-driven Datazilla Data cluster, an open-source system for managing and visualizing performance data.

Date: May 2nd
Time: 2 PM EST / 11 AM PST
REGISTER TODAY

In the past TokuDB has been free for evaluation; the new TokuDB Community Edition extends free use to deployed environments. With this release Tokutek is also planning on making available a TokuDB Enterprise Edition, which includes technical support,

Since we had the pleasure to announce that TokuDB is open source on Monday, it’s been a thrilling ride. With several members of the team out west all week, back on the east coast we’ve been seeing quite a lot of questions, suggestions, and exciting results.

Here are some of the highlights of our first week of open source:

We started hearing back from the community almost immediately after the announcement with discussions in multiple forums. We even reached #2 on Hacker News for a bit.

On Tuesday,

Yesterday we posted the photos and lyrics. Now we’ve got the YouTube video (click here)!

And for those who want some behind the scenes photos, see here and here. (courtesy of @seattlegaucho).

We wanted to thank everyone for naming Tokutek the Corporate Contributor of the Year 2013 for ongoing contribution to the MySQL community.

The MySQL Community Awards are given annually to the people and companies that support the MySQL ecosystem. The MySQL Community Award for Corporate Contributor of the Year recognizes a company or other organization or entity that has made valuable contributions to the MySQL ecosystem either in terms of open source code, knowledge,

Thanks to all of those who came to the lightning talks this evening. We’ve got the audio posted here and the lyrics below.

A special thanks to Erin Grace O’Malley O’Neill for the great performance. Thanks also to @NuoDB and @geobdz (whose photo is below) for tweets.

Stay tuned for the video as well….

As I browse through  [Read more...]

Since we announced that TokuDB is now open source, there has been a lot of positive feedback (thanks!) and also some questions about the details. I want to take this opportunity to give a quick high level guide to describe what our repositories on Github are.

Here are the repositories:

• ft-index. This repository is the “magic”. It contains the Fractal Tree data structures we have been talking about for years. This is also the main piece that was previously closed source. Here are some interesting directories:
• src: This directory is a layer that implements an API that is similar to the BDB API.
• locktree: an in-memory data structure that maintains transactions’ row-level locks.

I enjoyed Stewart Smith’s MySQL storage engine blog last week. In it he noted “I cannot emphasize how much more interesting TokuDB would be if it were open source.” Well, with our open source announcement yesterday, hopefully we are getting interesting.

We wanted to thank everyone for the great feedback. Here is a sampling from some of the forums where dialogue is occurring:

Reddit:

BrianAtDTS: “With this update, this puts MySQL in

Every few months, I get the fun job of announcing what’s new in TokuDB®, but this time is special. With Version 7, TokuDB for MySQL and MariaDB is going open source.

The free Community Edition is fully functional and fully performant. It has all the compression you’ve come to expect from TokuDB. It has hot schema changes: no-down-time column insertion, deletion, renaming, etc., as well as index creation. It has clustering secondary keys. We are also announcing an Enterprise Edition (coming soon) with additional benefits, such as a support package and advanced backup and recovery tools.

Making TokuDB open source is a natural next step for Tokutek’s involvement in the MySQL community. So far, Tokutek has been involved in the community in many ways:

• We’ve

If T.S. Eliot were a MySQL DBA, I think he would have been more upbeat about April.

We are gearing up for an incredible second half of April. We will be presenting three separate sessions at the Percona Live: MySQL Conference and Expo 2013, April 22-25, in Santa Clara, CA. In addition, we will be presenting at SkySQL’s MySQL & Cloud Database Solutions Day on Friday, April 26 at the same location.

Come by to see us in Booth #114, or stop by one of our sessions:

Earlier, I talked about the transactional semantics we are introducing to MongoDB. As I hinted at the end of the post, we are actually doing more. We are introducing multi-statement transactions. That’s right, multiple queries, updates, deletes, and inserts will be able to run inside of a single transaction. We are working on the details of the semantics as we develop our beta, but at a high level, think of it as having the same semantics as TokuDB and InnoDB’s multi-statement transactions in MySQL.

So how will it work? We introduce three new commands:

db.runCommand({"beginTransaction", "isolation": "mvcc"})

This begins a transaction with the isolation level of MVCC, which means queries will use a snapshot of the system.

People claim that MongoDB is not transactional. It actually is, and that’s a good thing.

In MongoDB 2.2, individual operations are Atomic. By having per database locks control reads and writes to collections, write operations on collections are Consistent and Isolated. With journaling on, operations may be made Durable. Put these properties together, and you have basic ACID properties for transactions.

The shortcoming with MongoDB’s implementation is that these semantics apply to individual write operations, such as an individual insert or individual update. If a MongoDB statement updates 10 rows, and something goes wrong with the fifth row, then the statement

Last month my colleague Rich Prohaska covered the technical details of our “Fast Update” feature which we added to TokuDB in version 6.6.  The message based architecture of Fractal Tree Indexes allows us to defer certain operations while still maintaining the semantics that MySQL users require.

In the case of Fast Updates, TokuDB is avoiding the read-before-write requirement that the existing MySQL update statement imposes on storage engines.  We can simply inject an update message into the Fractal Tree Index, and apply that message at a later time.  The message is dynamically applied if a user selects that specific

During the second half of our CUBE discussion with Wikibon analyst Jeff Kelly at this year’s Strata Conference in Santa Clara, we talked about the tipping point for Big Data. Strata veterans could see at a glance that this year’s conference was markedly different. No longer the exclusive domain of geeks and database administrators, this year’s Strata featured some of the biggest enterprise vendors around. With heavy weight enterprise players Intel and EMC Greenplum announcing their own Hadoop distributions, big data is clearly going mainstream. Now that we know how to capture, store, access and analyze big data, what’s the next step? Listen in to hear my conversation with Jeff Kelly about taking big data

We had the opportunity to do a CUBE interview with Wikibon analyst Jeff Kelly at last week’s Strata Conference in Santa Clara. In the first part of our conversation, we discuss how our success in integrating Tokutek’s Fractal Tree® technology into MySQL has led us to another popular database, MongoDB. We explain the results of our recent benchmarking tests with MongoDB, which indicate that adding indexing can also improve performance for this popular NoSQL database with faster insertion rates, lower query latency and

One doesn’t have to look far to see that there is strong interest in MongoDB compression. MongoDB has an open ticket from 2009 titled “Option to Store Data Compressed” with Fix Version/s planned but not scheduled. The ticket has a lot of comments, mostly from MongoDB users explaining their use-cases for the feature. For example, Khalid Salomão notes that “Compression would be very good to reduce storage cost and improve IO performance” and Andy notes that “SSD is getting more and more common for

I’ve said it before, and, as is the nature of these things, I’ll almost certainly say it again: your database performance is only as good as your indexes.

That’s the grand thesis, so what does that mean? In any DB system — SQL, NoSQL, NewSQL, PostSQL, … — data gets ingested and organized. And the system answers queries. The pain point for most users is around the speed to answer queries. And the query speed (both latency and throughput, to be exact) depend on how the data is organized. In short: Good Indexes, Fast Queries; Poor Indexes, Slow Queries.

But building indexes is hard work, or at least it has been for the last several decades, because almost all indexing is done with B-trees. That’s true of

With TokuDB v6.6 out now, I’m excited to present one of my favorite enhancements: fast updates with TokuDB. Update intensive applications can have their throughput limited by the random read capacity of the storage system. The cause of the throughput limit is the read-modify-write algorithm that MySQL uses when processing update statements. MySQL reads a row from the storage engine, applies the updates to it, and then writes the new row to the storage engine. To address this throughput limit, TokuDB uses a different update algorithm that simply encodes the update expressions of the SQL statement into tiny programs that are stored in an update Fractal Tree® message. This update message is

In Part 1, we showed performance results of some of the work that’s gone in to TokuDB v6.6. In this post, we’ll take a closer look at how this happened, on the engineering side, and how to think about the performance characteristics in the new version.

Background

It’s easiest to think about our concurrency changes in terms of a Fractal Tree® index that has nodes like a B-tree index, and buffers on each node that batch changes for the subtree rooted at that node. We have materials that describe this available here, but we can proceed just knowing that:

With TokuDB v6.6 out now, I’m excited to present one of my favorite enhancements: concurrency within a single index. Previously, while there could be many SQL transactions in-flight at any given moment, operations inside a single index were fairly serialized. We’ve been working on concurrency for a few versions, and things have been getting a lot better over time. Today I’ll talk about what to expect from v6.6. Next time, we’ll see why.

Summary of Results

Running multiple iiBench clients on a single MySQL instance, we see a big improvement in the cumulative insertion speed at all concurrency levels. We see a gain of 33.9% in single-threaded performance and 51.8% at