Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 67 Next 30 Older Entries

Displaying posts with tag: fractal tree indexes (reset)

Thoughts on Small Datum – Part 2
+0 Vote Up -0Vote Down

If you did not read my first blog post about Mark Callaghan’s (@markcallaghan) benchmarks as documented in his blog, Small Datum, you may want to skim through it now for a little context.

——————-

On March 11th, Mark, a former Google and now Facebook database guru, published an insertion rate benchmark comparing MySQL (http://www.mysql.com) outfitted with the InnoDB storage engine with two NoSQL alternatives — basic MongoDB and TokuMX (the Tokutek high-performance

  [Read more...]
Thoughts on Small Datum – Part 1
+0 Vote Up -0Vote Down

A little background…

When I ventured into sales and marketing (I’m an engineer by education) I learned I would often have to interpret and simply summarize the business value that is sometimes hidden in benchmarks. Simply put, the people who approve the purchase of products like TokuDB® and TokuMX™ appreciate the executive summary.

Therefore, I plan to publish a multipart series here on TokuView where I will share my simple summaries and thoughts on business value for the benchmarks Mark Callaghan (@markcallaghan), a former Google and now Facebook database guru, is publishing on his blog, Small Datum.

I’m going to start with his first benchmark post and work my way forward to

  [Read more...]
How TokuMX was Born
+2 Vote Up -0Vote Down

With TokuMX 1.4 coming out soon, with (teaser) wonderful improvements made to sharding and updates (and plenty of other goodies), I’ve recently reminisced about how we got TokuMX to this point. We (actually, really John) started dabbling with integrating Fractal Tree® indexes into MongoDB in the summer of 2012, where we (really, he) prototyped using Fractal Tree indexes only for secondary indexes. As cool as that prototype was, it

  [Read more...]
Put your MySQL Knowledge to Good Use with Tim Callaghan at Percona Live-London, November 12
+0 Vote Up -0Vote Down

Attending Percona Live in London next week?

Don’t miss the chance to hear Tokutek’s Vice President of Engineering, Tim Callaghan, discuss how to use your MySQL knowledge to become an instant MongoDB Guru and the advantages of using Fractal Tree&#174 indexes in MySQL and MongoDB. Tim will be speaking about these topics in two separate sessions at 12:00pm and 5:00pm on November 12.

For more information on these sessions and Percona Live-London, visit https://www.percona.com/live/london-2013/users/tim-callaghan.

A TokuDB Stall Caused by a Big Transaction and How It was Fixed
+1 Vote Up -0Vote Down

One of our customers sometimes observed lots of simple insertions taking far longer than expected to complete. Usually these insertions completed in milliseconds, but the insertions sometimes were taking hundreds of seconds. These stalls indicated the existence of a serialization bug in the Fractal Tree index software, so the hunt was on. We found that these stalls occurred when a big transaction was committing and the Fractal Tree index software was taking a checkpoint. This problem was fixed in both TokuDB 7.0.3 and TokuMX 1.0.3. Please read on as we describe some details about this bug and how we fixed it. We describe some of the relevant Fractal Tree index algorithms first.

What is a Big Transaction?

Each transaction builds a rollback log as it performs Fractal Tree index operations. The rollback log is maintained in

  [Read more...]
Building TokuMX and TokuDB for Production
+1 Vote Up -0Vote Down

Recently, we’ve seen a few people ask us about building TokuMX from scratch. While it’s best if you just use the binaries you can get from us (they have all the right optimizations, we’ve tested them, and we can interpret coredumps they generate), we recognize there are other reasons you might need to do a custom build.

Since we actually build six distinct products all using the Fractal Tree indexing® library (community and enterprise versions of TokuDB for MySQL, TokuDB for MariaDB, and TokuMX), our build process is pretty complicated, compared to software packages that might, for example, just involve one source repository and link against a few standard libraries. Our TokuMX builds involve four git repositories, three

  [Read more...]
Slides from Boston MongoDB User Group Meetup on 7/31/13
+0 Vote Up -0Vote Down

On Wednesday night, the Boston MongoDB User group was kind enough to have me speak about TokuMX Internals. I spoke about Fractal Tree® indexes and the technical reasons behind the benefits they provide to MongoDB applications. Although the talk mostly references TokuMX and MongoDB, all the theory applies to TokuDB and MySQL as well.

My slides are on our technology overview page, along with other great content.

Opportunities to present technical material to an engaged audience asking tough questions is rare, and much appreciated. So thank you to the Boston MongoDB User group for having me present.

Why Unique Indexes are Bad
+1 Vote Up -0Vote Down

Before creating a unique index in TokuMX or TokuDB, ask yourself, “does my application really depend on the database enforcing uniqueness of this key?” If the answer is ANYTHING other than yes, do not declare the index to be unique. Why? Because unique indexes may kill your write performance. In this post, I’ll explain why.

Unique indexes are a strange beast: they have no impact on standard databases that use B-Trees, such as MongoDB and MySQL, but may be horribly painful for databases that use write optimized data structures, like TokuMX’s Fractal Tree(R) indexes. How? They

  [Read more...]
How TokuMX Gets Great Compression for MongoDB
+0 Vote Up -0Vote Down

In my last post, I showed what a Fractal Tree® index is at a high level. Once again, the Fractal Tree index is the data structure inside TokuMX and TokuDB, our MongoDB and MySQL products. One of its strengths is the ability to get high levels of compression on the stored data. In this post, I’ll explain why that is.

At a high level, one can argue that there isn’t anything special about our compression algorithms. We basically do this: we take large chunks of data, use known compression methods (e.g. zlib,

  [Read more...]
Announcing TokuMX v1.0: Toku+Mongo = You Can Have It All
+5 Vote Up -0Vote Down

Tokutek is known for its full-featured fast-indexing technology. MongoDB is known for its great document-based data model and ease of use. TokuMX, version 1.0, combines the best of both worlds.

  • So what, exactly, is TokuMX? The simplest (but incomplete) answer is that TokuMX is MongoDB with all its storage code replaced by Tokutek’s Fractal Tree indexes.
  • How do Fractal Tree indexes improve MongoDB? The direct benefits include high-performance indexing, strong compression, and performance stability – in other words, the performance stays high, even when data is larger than RAM.
  • Are there any features in TokuMX that MongoDB doesn’t have? Yes. We have added support for transactions to TokuMX, so that TokuMX is ACID compliant and has MVCC. We have also added support for clustering indexes, which
  [Read more...]
TokuMX: Fractal Trees with MongoDB
+2 Vote Up -0Vote Down

Over several blog posts, Tim has presented performance results on large data sets of TokuMX, our MongoDB product with fractal tree indexes integrated, side by side with MongoDB. Results look good. We’ve shown improved throughput numbers on a sysbench benchmark, faster load times, and high compression.

So what is TokuMX, and how does it achieve this performance?

TokuMX has replaced ALL of the storage code in MongoDB with fractal trees. Every collection, every secondary index, every metadata collection is stored with fractal trees, the same technology that implements the TokuDB storage engine for MySQL. That is, all data is stored and managed with our transactional, ACID and MVCC-compliant, write-optimized storage library.

TokuMX achieves

  [Read more...]
Sysbench Benchmark for MongoDB – v0.1.0 Performance Update
+2 Vote Up -0Vote Down

Two months ago I posted a performance comparison running Sysbench on MongoDB versus MongoDB with Fractal Tree Indexes v0.0.2. The benchmark showed a 133% improvement in throughput. Nice, but our engineering team had an effort on our road-map for lock refinement that we believed would really boost our performance, which is now available in v0.1.0. The benchmark application itself is unchanged and available on GitHub.

For anyone curious about Sysbench itself, the details are available from the prior blog. The only change for this run was hardware. Our Sun x4150 server recently began rebooting

  [Read more...]
May 2nd Webinar: Introduction to TokuDB v7 Community & Enterprise Editions
+0 Vote Up -0Vote Down

With this version, the source code is now freely available under the GPL License v2. For more details, see our blog here. Open source pioneer Mozilla has been using TokuDB to manage its MySQL-driven Datazilla Data cluster, an open-source system for managing and visualizing performance data.

Date: May 2nd
Time: 2 PM EST / 11 AM PST
REGISTER TODAY

In the past TokuDB has been free for evaluation; the new TokuDB Community Edition extends free use to deployed environments. With this release Tokutek is also planning on making available a TokuDB Enterprise Edition, which includes technical support,



  [Read more...]
Opening Week for TokuDB
+0 Vote Up -0Vote Down

Since we had the pleasure to announce that TokuDB is open source on Monday, it’s been a thrilling ride. With several members of the team out west all week, back on the east coast we’ve been seeing quite a lot of questions, suggestions, and exciting results.

Here are some of the highlights of our first week of open source:

We started hearing back from the community almost immediately after the announcement with discussions in multiple forums. We even reached #2 on Hacker News for a bit.

On Tuesday,

  [Read more...]
MySQL Paradise: YouTube Video
+2 Vote Up -0Vote Down

Yesterday we posted the photos and lyrics. Now we’ve got the YouTube video (click here)!

And for those who want some behind the scenes photos, see here and here. (courtesy of @seattlegaucho).

Thanks to Community for Selecting Tokutek for Prestigious MySQL Award
+3 Vote Up -0Vote Down

We wanted to thank everyone for naming Tokutek the Corporate Contributor of the Year 2013 for ongoing contribution to the MySQL community.

The MySQL Community Awards are given annually to the people and companies that support the MySQL ecosystem. The MySQL Community Award for Corporate Contributor of the Year recognizes a company or other organization or entity that has made valuable contributions to the MySQL ecosystem either in terms of open source code, knowledge,

  [Read more...]
Open Source TokuDB Resources
+2 Vote Up -0Vote Down

Since we announced that TokuDB is now open source, there has been a lot of positive feedback (thanks!) and also some questions about the details. I want to take this opportunity to give a quick high level guide to describe what our repositories on Github are.

Here are the repositories:

  • ft-index. This repository is the “magic”. It contains the Fractal Tree data structures we have been talking about for years. This is also the main piece that was previously closed source. Here are some interesting directories:
    • src: This directory is a layer that implements an API that is similar to the BDB API.
    • locktree: an in-memory data structure that maintains transactions’ row-level locks.
  [Read more...]
Announcing TokuDB v7: Open Source and More
+3 Vote Up -0Vote Down

Every few months, I get the fun job of announcing what’s new in TokuDB®, but this time is special. With Version 7, TokuDB for MySQL and MariaDB is going open source.

The free Community Edition is fully functional and fully performant. It has all the compression you’ve come to expect from TokuDB. It has hot schema changes: no-down-time column insertion, deletion, renaming, etc., as well as index creation. It has clustering secondary keys. We are also announcing an Enterprise Edition (coming soon) with additional benefits, such as a support package and advanced backup and recovery tools.

Making TokuDB open source is a natural next step for Tokutek’s involvement in the MySQL community. So far, Tokutek has been involved in the community in many ways:

  • We’ve
  [Read more...]
April is the Coolest Month
+0 Vote Up -0Vote Down

If T.S. Eliot were a MySQL DBA, I think he would have been more upbeat about April.

We are gearing up for an incredible second half of April. We will be presenting three separate sessions at the Percona Live: MySQL Conference and Expo 2013, April 22-25, in Santa Clara, CA. In addition, we will be presenting at SkySQL’s MySQL & Cloud Database Solutions Day on Friday, April 26 at the same location.

Come by to see us in Booth #114, or stop by one of our sessions:

  [Read more...]
MongoDB Multi-Statement Transactions? Yes We Can!
+0 Vote Up -1Vote Down

Earlier, I talked about the transactional semantics we are introducing to MongoDB. As I hinted at the end of the post, we are actually doing more. We are introducing multi-statement transactions. That’s right, multiple queries, updates, deletes, and inserts will be able to run inside of a single transaction. We are working on the details of the semantics as we develop our beta, but at a high level, think of it as having the same semantics as TokuDB and InnoDB’s multi-statement transactions in MySQL.

So how will it work? We introduce three new commands:

db.runCommand({"beginTransaction", "isolation": "mvcc"})

This begins a transaction with the isolation level of MVCC, which means queries will use a snapshot of the system.

  [Read more...]
MongoDB Transactions? Yes
+1 Vote Up -1Vote Down

People claim that MongoDB is not transactional. It actually is, and that’s a good thing.

In MongoDB 2.2, individual operations are Atomic. By having per database locks control reads and writes to collections, write operations on collections are Consistent and Isolated. With journaling on, operations may be made Durable. Put these properties together, and you have basic ACID properties for transactions.

The shortcoming with MongoDB’s implementation is that these semantics apply to individual write operations, such as an individual insert or individual update. If a MongoDB statement updates 10 rows, and something goes wrong with the fifth row, then the statement

  [Read more...]
TokuDB Fast Update Benchmark
+1 Vote Up -0Vote Down

Last month my colleague Rich Prohaska covered the technical details of our “Fast Update” feature which we added to TokuDB in version 6.6.  The message based architecture of Fractal Tree Indexes allows us to defer certain operations while still maintaining the semantics that MySQL users require.

In the case of Fast Updates, TokuDB is avoiding the read-before-write requirement that the existing MySQL update statement imposes on storage engines.  We can simply inject an update message into the Fractal Tree Index, and apply that message at a later time.  The message is dynamically applied if a user selects that specific

  [Read more...]
Wanted: Evaluators to Try MongoDB with Fractal Tree Indexing
+0 Vote Up -0Vote Down

We recently resumed our discussion around bringing Fractal Tree indexes to MongoDB.  This effort includes Tokutek’s interview with Jeff Kelly at Strata as well as my two recent tech blogs which describe the compression achieved on a generic MongoDB data set and performance improvements we measured using on our implementation of Sysbench for MongoDB.  I have a full line-up of benchmarks and blogs planned for the next few months, as our project continues.  Many of these

  [Read more...]
Sysbench Benchmark for MongoDB
+1 Vote Up -0Vote Down

As we continue to test our Fractal Tree Indexing with MongoDB, I’ve been updating my benchmark infrastructure so I can compare performance, correctness, and resource utilization.  Sysbench has long been a standard for testing MySQL performance, so I created a version that is compatible with MongoDB.  You can grab my current version of Sysbench for MongoDB here.

So what exactly is Sysbench?  According to the Sysbench homepage, “Sysbench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS [Operating System] parameters that are important for a system running a database under intensive load.”

  • Sysbench schema
    • 16 copies of the same collection,

  [Read more...]
MySQL and MongoDB – Strata Discussion with Jeff Kelly of Wikibon (Part 1)
+1 Vote Up -0Vote Down

We had the opportunity to do a CUBE interview with Wikibon analyst Jeff Kelly at last week’s Strata Conference in Santa Clara. In the first part of our conversation, we discuss how our success in integrating Tokutek’s Fractal Tree® technology into MySQL has led us to another popular database, MongoDB. We explain the results of our recent benchmarking tests with MongoDB, which indicate that adding indexing can also improve performance for this popular NoSQL database with faster insertion rates, lower query latency and

  [Read more...]
Concurrency Improvements in TokuDB v6.6 (Part 2)
+0 Vote Up -0Vote Down

In Part 1, we showed performance results of some of the work that’s gone in to TokuDB v6.6. In this post, we’ll take a closer look at how this happened, on the engineering side, and how to think about the performance characteristics in the new version.

Background

It’s easiest to think about our concurrency changes in terms of a Fractal Tree® index that has nodes like a B-tree index, and buffers on each node that batch changes for the subtree rooted at that node. We have materials that describe this available here, but we can proceed just knowing that:

  • To inject data into the tree, you need to store a message in a buffer at the root of the tree. These
  •   [Read more...]
    Concurrency Improvements in TokuDB v6.6 (Part 1)
    +2 Vote Up -0Vote Down

    With TokuDB v6.6 out now, I’m excited to present one of my favorite enhancements: concurrency within a single index. Previously, while there could be many SQL transactions in-flight at any given moment, operations inside a single index were fairly serialized. We’ve been working on concurrency for a few versions, and things have been getting a lot better over time. Today I’ll talk about what to expect from v6.6. Next time, we’ll see why.

    Summary of Results

    Running multiple iiBench clients on a single MySQL instance, we see a big improvement in the cumulative insertion speed at all concurrency levels. We see a gain of 33.9% in single-threaded performance and 51.8% at

      [Read more...]
    Tracking 5.3 Billion Mutations: Using MySQL for Genomic Big Data
    +1 Vote Up -0Vote Down

    University of Montreal Tracks Genomic Data With Tokutek’s TokuDB.

    Faster insertion rates, improved scalability and agility support lab’s fast growing research database as it grows from 100s of GBs to 1 TB and beyond.

    Issue addressed: MySQL database used for genomic research must be able to quickly ingest huge amounts of incoming data – hundreds of thousands of records every day. It also must be able to retrieve data quickly in response to a diverse set of research requests.

    Enabling the Hunt for New Cures for Diseases by Seamlessly Processing Billions of Mutations  [Read more...]

    The Results Are In!
    +1 Vote Up -0Vote Down

    We wanted to take a moment to say thanks to all of our customers and to the wider MySQL and MariaDB community. Today we announced a doubling of our customer base for the year ending December 31, 2012. Significant milestones over the last year included new technology and service partnerships, several awards, rapid hiring, as well as three upgrades to TokuDB®. We even dabbled in some MongoDB benchmarks. And to fuel continued growth in 2013, we secured additional venture capital funding last November.

    Did You Hear? NASA Uses TokuDB for Big Data with MySQL!

    To read the full press release and learn more,

      [Read more...]
    Announcing TokuDB v6.6: Performance Improvements
    +1 Vote Up -0Vote Down

    We are excited to announce TokuDB® v6.6, the latest version of Tokutek’s flagship storage engine for MySQL and MariaDB.

    This version offers three types of performance improvements: in-memory, multi-client and fast updates.

    Although TokuDB is optimized for large tables, which are larger than memory, many workloads consist of a mix of large and small tables. TokuDB v6.6 offers improvements on in-memory performance, with a more than 100% improvement on Sysbench at many concurrency levels and more than 200% improvement on TPC-C at many concurrency levels. Details to follow.

    We have also made improvements in multi-threaded performance. For example, single threaded trickle loads have always been fast in TokuDB. But now multi-threaded trickle loads are even faster. An

      [Read more...]
    Showing entries 1 to 30 of 67 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.