Thanks to Tim Callaghan for speaking Tuesday night at the Effective MySQL New York meetup on Fractal Tree Indexes : Theory and Practice (MySQL and MongoDB). There was a good turnout and a full room to learn how the TokuDB storage engine from Tokutek is changing how to handle big data in MySQL.

Also interesting is how the same technology has been applied for use in MongoDB including giving MongoDB transactions; a big change for NoSQL.

Related News: Tokutek Meets Big Data Demand With Open Source TokuDB

At tomorrow’s Effective MySQL Meetup, I’ll be presenting “Fractal Tree Indexes : Theory and Practice (MySQL and MongoDB).” The meetup is at 6:30pm Tuesday, May 14, 2013, and will be held at Alley NYC in New York City.

I’ll give an overview on how Fractal Tree® indexes work, and then get into specific product features that Fractal Trees enable in MySQL and MongoDB.  Some benchmarking and customer use-cases will be discussed, but my intent is for this to be a deep technical dive.  Several Tokutek Engineers will also be on hand, so bring any questions you’ve got.

I hope to see you there!

Last week I had to present a tutorial at Percona Live 2013, a presentation at SkySQL’s MySQL & Cloud Database Solution Day and last but not least, a presentation on a Saturday morning at Linuxfest Northwest. It wasn’t easy, but giving the presentations after our announcement early in the week about going open source was very exciting given the

With this version, the source code is now freely available under the GPL License v2. For more details, see our blog here. Open source pioneer Mozilla has been using TokuDB to manage its MySQL-driven Datazilla Data cluster, an open-source system for managing and visualizing performance data.

Date: May 2nd
Time: 2 PM EST / 11 AM PST
REGISTER TODAY

In the past TokuDB has been free for evaluation; the new TokuDB Community Edition extends free use to deployed environments. With this release Tokutek is also planning on making available a TokuDB Enterprise Edition, which includes technical support,

Since we had the pleasure to announce that TokuDB is open source on Monday, it’s been a thrilling ride. With several members of the team out west all week, back on the east coast we’ve been seeing quite a lot of questions, suggestions, and exciting results.

Here are some of the highlights of our first week of open source:

We started hearing back from the community almost immediately after the announcement with discussions in multiple forums. We even reached #2 on Hacker News for a bit.

On Tuesday,

Yesterday we posted the photos and lyrics. Now we’ve got the YouTube video (click here)!

And for those who want some behind the scenes photos, see here and here. (courtesy of @seattlegaucho).

We wanted to thank everyone for naming Tokutek the Corporate Contributor of the Year 2013 for ongoing contribution to the MySQL community.

The MySQL Community Awards are given annually to the people and companies that support the MySQL ecosystem. The MySQL Community Award for Corporate Contributor of the Year recognizes a company or other organization or entity that has made valuable contributions to the MySQL ecosystem either in terms of open source code, knowledge,

Thanks to all of those who came to the lightning talks this evening. We’ve got the audio posted here and the lyrics below.

A special thanks to Erin Grace O’Malley O’Neill for the great performance. Thanks also to @NuoDB and @geobdz (whose photo is below) for tweets.

Stay tuned for the video as well….

As I browse through  [Read more...]

Since we announced that TokuDB is now open source, there has been a lot of positive feedback (thanks!) and also some questions about the details. I want to take this opportunity to give a quick high level guide to describe what our repositories on Github are.

Here are the repositories:

• ft-index. This repository is the “magic”. It contains the Fractal Tree data structures we have been talking about for years. This is also the main piece that was previously closed source. Here are some interesting directories:
• src: This directory is a layer that implements an API that is similar to the BDB API.
• locktree: an in-memory data structure that maintains transactions’ row-level locks.

I enjoyed Stewart Smith’s MySQL storage engine blog last week. In it he noted “I cannot emphasize how much more interesting TokuDB would be if it were open source.” Well, with our open source announcement yesterday, hopefully we are getting interesting.

We wanted to thank everyone for the great feedback. Here is a sampling from some of the forums where dialogue is occurring:

Reddit:

BrianAtDTS: “With this update, this puts MySQL in

Every few months, I get the fun job of announcing what’s new in TokuDB®, but this time is special. With Version 7, TokuDB for MySQL and MariaDB is going open source.

The free Community Edition is fully functional and fully performant. It has all the compression you’ve come to expect from TokuDB. It has hot schema changes: no-down-time column insertion, deletion, renaming, etc., as well as index creation. It has clustering secondary keys. We are also announcing an Enterprise Edition (coming soon) with additional benefits, such as a support package and advanced backup and recovery tools.

Making TokuDB open source is a natural next step for Tokutek’s involvement in the MySQL community. So far, Tokutek has been involved in the community in many ways:

• We’ve

If T.S. Eliot were a MySQL DBA, I think he would have been more upbeat about April.

We are gearing up for an incredible second half of April. We will be presenting three separate sessions at the Percona Live: MySQL Conference and Expo 2013, April 22-25, in Santa Clara, CA. In addition, we will be presenting at SkySQL’s MySQL & Cloud Database Solutions Day on Friday, April 26 at the same location.

Come by to see us in Booth #114, or stop by one of our sessions:

One doesn’t have to look far to see that there is strong interest in MongoDB compression. MongoDB has an open ticket from 2009 titled “Option to Store Data Compressed” with Fix Version/s planned but not scheduled. The ticket has a lot of comments, mostly from MongoDB users explaining their use-cases for the feature. For example, Khalid Salomão notes that “Compression would be very good to reduce storage cost and improve IO performance” and Andy notes that “SSD is getting more and more common for

I’ve said it before, and, as is the nature of these things, I’ll almost certainly say it again: your database performance is only as good as your indexes.

That’s the grand thesis, so what does that mean? In any DB system — SQL, NoSQL, NewSQL, PostSQL, … — data gets ingested and organized. And the system answers queries. The pain point for most users is around the speed to answer queries. And the query speed (both latency and throughput, to be exact) depend on how the data is organized. In short: Good Indexes, Fast Queries; Poor Indexes, Slow Queries.

But building indexes is hard work, or at least it has been for the last several decades, because almost all indexing is done with B-trees. That’s true of

In Part 1, we showed performance results of some of the work that’s gone in to TokuDB v6.6. In this post, we’ll take a closer look at how this happened, on the engineering side, and how to think about the performance characteristics in the new version.

Background

It’s easiest to think about our concurrency changes in terms of a Fractal Tree® index that has nodes like a B-tree index, and buffers on each node that batch changes for the subtree rooted at that node. We have materials that describe this available here, but we can proceed just knowing that:

With TokuDB v6.6 out now, I’m excited to present one of my favorite enhancements: concurrency within a single index. Previously, while there could be many SQL transactions in-flight at any given moment, operations inside a single index were fairly serialized. We’ve been working on concurrency for a few versions, and things have been getting a lot better over time. Today I’ll talk about what to expect from v6.6. Next time, we’ll see why.

Summary of Results

Running multiple iiBench clients on a single MySQL instance, we see a big improvement in the cumulative insertion speed at all concurrency levels. We see a gain of 33.9% in single-threaded performance and 51.8% at

University of Montreal Tracks Genomic Data With Tokutek’s TokuDB.

Faster insertion rates, improved scalability and agility support lab’s fast growing research database as it grows from 100s of GBs to 1 TB and beyond.

Issue addressed: MySQL database used for genomic research must be able to quickly ingest huge amounts of incoming data – hundreds of thousands of records every day. It also must be able to retrieve data quickly in response to a diverse set of research requests.

Enabling the Hunt for New Cures for Diseases by Seamlessly Processing Billions of Mutations  [Read more...]

We wanted to take a moment to say thanks to all of our customers and to the wider MySQL and MariaDB community. Today we announced a doubling of our customer base for the year ending December 31, 2012. Significant milestones over the last year included new technology and service partnerships, several awards, rapid hiring, as well as three upgrades to TokuDB®. We even dabbled in some MongoDB benchmarks. And to fuel continued growth in 2013, we secured additional venture capital funding last November.

Did You Hear? NASA Uses TokuDB for Big Data with MySQL!

To read the full press release and learn more,

TokuDB® is a proven solution that scales MySQL® and MariaDB® from GBs to TBs with unmatched insert and query speed, compression, replication performance and online schema flexibility. Tokutek’s recently launched TokuDB v6.6 delivers all of these features and more, with additional improvements in multi-client, fast SQL updates, and in-memory performance.

Date: January 15th
Time: 2 PM EST / 11 AM PST
REGISTER TODAY

Topics will include:

• Performance – With a 10x or more improvement in insertions and indexing, TokuDB delivers faster, more complex ad hoc queries in live production systems without rewriting or tuning applications. Offering high performance even when tables are too large for memory, TokuDB scales MySQL and MariaDB

We are excited to announce TokuDB® v6.6, the latest version of Tokutek’s flagship storage engine for MySQL and MariaDB.

This version offers three types of performance improvements: in-memory, multi-client and fast updates.

Although TokuDB is optimized for large tables, which are larger than memory, many workloads consist of a mix of large and small tables. TokuDB v6.6 offers improvements on in-memory performance, with a more than 100% improvement on Sysbench at many concurrency levels and more than 200% improvement on TPC-C at many concurrency levels. Details to follow.

We have also made improvements in multi-threaded performance. For example, single threaded trickle loads have always been fast in TokuDB. But now multi-threaded trickle loads are even faster. An

What a flashback this week. Staring at a text terminal trying to establish a connection with a remote server, I began to fret whether I would get my homework assignment done on time. My mind raced back to college nights years ago in the Fishbowl, hunched over an Athena workstation. Would this be another late night fueled by Jolt cola in order to get my problem set done?

Thankfully, no!

Embarking on my first software class in quite a while was relatively painless, and I have Sheeri Cabral and her detailed guidance to thank. This week I started the

Well, it’s that time of the year again for top ten lists. There have been many versions showing up on the web the last few days, including Time Magazine’s “Top 10 Everything of 2012″ list, with 55 wide ranging lists!

Last year we started using Google Analytics to see what content for blogs was most popular on Tokutek.com and generated a 2011 top ten list, ending up with a few surprises.  This year saw spikes in some interesting areas as well, including flash performance, NASA and Big Data, and MongoDB.

Without further adieu, here is the top ten list for 2012:

10. Announcing TokuDB v6.1 –

Every time I visit my family for the holidays, as the date approaches, I find myself filled with dread. It’s nothing sinister, my family’s great, and the season is nice. The reason is simple:

I hate packing.

In fact, I hate both kinds of packing: trip packing, and bit packing. Let me tell you a story about bit packing.

Bit packing

If you’ve ever browsed around the available type attributes in GCC, you may have noticed the entry for “packed”. It seems straightforward enough, and if you’re trying to cram a lot of data in a system (like we do), it can be pretty attractive.

There are plenty

Tired of the same old holiday tunes year after year? Join us as we Rock around the Fractal Tree here at Tokutek.

We’ve fiddled with the chorus; won’t you join us?

Rockin’ around the Fractal Tree
(sung to the tune of Rockin’ around the Christmas Tree)

Rockin’ around the Fractal Tree
As the bits begin to hop
Leaves are full on trees that are not B
Our indexes cannot be stopped

Rockin’ around the Fractal Tree
Let the MySQL spirit ring
Time to compress all into an SSD
And watch out for the Merciless Ming

You will get a sensational feeling when you hear voices singing
“No, not a fable; online changes done with alter table!”
Rockin’ around the Fractal Tree

We get a lot of questions about how Fractal Tree indexes work. It’s a write-optimized index with fast queries, but which write-optimized indexing structure is it?

In this ~15 minute video (which uses these slides), I give a quick overview of how they work and what they are good for.

In this webinar we will show step by step how to install, configure, and test TokuDB for a typical performance evaluation. We’ll also be flagging potential pitfalls that can ruin the eval results. It will describe the differences between installing from scratch and replacing an existing MySQL / MariaDB installation. It will also review the most common issues that may arise when running TokuDB binaries.

Date: December 11th
Time: 2 PM EST / 11 AM PST
REGISTER TODAY

Topics will include:

• Memory allocation
• Initial data load
• Indexing, including clustering indexes
• compression algorithms

We look forward to having you join

One of the great things about the MassTLC unConference is the spontaneity of the ideas. In the morning I ran into an old colleague whose startup was looking at switching databases and struggling with the options. Hence, “Scalable Databases for Startups” seemed like a great topic, so I proposed it, and then was off and running full steam after lunch.

The session brought in a wide variety of firms. While there were several vendors there – Basho, Calpont, InterSystems,

There is obviously much being written these days about Big Data. While the term has many different meanings to many different folks, our MySQL and MariaDB customers tend to find their data to be uncomfortably big when the tables become too large for memory. In this case, more storage has to be acquired, performance starts to lag, and making changes to the schema becomes a challenge.

TokuDB addresses these issues for big MySQL instances by delivering high compression rates, faster insertion and query performance, and agile