Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 146 Next 30 Older Entries

Displaying posts with tag: storage engine (reset)

An Updated Description of Clustering Keys for TokuDB
+0 Vote Up -0Vote Down

Covering indexes can result in orders of magnitude performance improvements for queries. Bradley’s presentation on covering indexes describes what a covering index is, how it can effect performance, and why it works. However, the definition of a covering index can get cumbersome since MySQL limits the number of columns in a key to 16 (32 on MariaDB).

Tokutek introduced multiple clustering indexes into MySQL to address these problems. Zardosht describes the multiple clustering indexes feature and how clustering indexes differ from covering indexes. Zardosht

  [Read more...]
New Webinar on July 9th: How To Set Up SQL Load Balancing with HAProxy
+1 Vote Up -0Vote Down
June 17, 2014 By Severalnines

 

We continuously see great interest in MySQL load balancing and HAProxy, so we thought it was about time we organised a live webinar on the topic!

 

As most of your will know, database clusters and load balancing go hand in hand. 

 

Once your data is distributed and replicated across multiple database nodes, a load balancing mechanism helps distribute database requests, and gives applications a single database endpoint to connect to. 

 

Instance failures or

  [Read more...]
How to improve InnoDB performance by 55% for write-bound loads
+2 Vote Up -0Vote Down

During April’s Percona Live MySQL Conference and Expo 2014, I attended a talk on MySQL 5.7 performance an scalability given by Dimitri Kravtchuk, the Oracle MySQL benchmark specialist. He mentioned at some point that the InnoDB double write buffer was a real performance killer. For the ones that don’t know what the innodb double write buffer is, it is a disk buffer were pages are written before being written to the actual data file. Upon restart, pages in the double write buffer are rewritten to their data files if complete. This is to avoid data file corruption with half written pages. I knew it has an impact on performance, on ZFS since it is transactional I always disable it, but I never realized how important the performance

  [Read more...]
Upcoming MariaDB 10.0.7 will have more engines – mroonga, OQGRAPH
+2 Vote Up -2Vote Down

In recent time, MariaDB 10 has been getting many new storage engines. We’ve seen TokuDB, CONNECT, SEQUENCE, SPIDER, CassandraSE for various use cases. For a long time, MariaDB shipped OQGRAPH, but it was disabled in MariaDB 5.5. It will make a come back as OQGRAPH v3 has been worked on actively by Andrew McDonnell. Keep track of this via MDEV-5319.

Another engine being worked on by Kentoku Shiba & team is the mroonga engine, which allows you to do full text search. It is optimised for CJK languages, and is supposedly very fast. To track this, follow

  [Read more...]
What does the ‘Incorrect key file for table’ error mean?
+0 Vote Up -0Vote Down

What does it mean if MySQL returns the ‘Incorrect key file for table’ error for one of my queries? The answer is complicated and depends on which storage engine is returning the error. We have debugged two cases which we describe here.

File system out of space

When running the random query generator, one of the queries failed.

Query: SELECT * FROM (mysql . general_log AS table1 INNER JOIN INFORMATION_SCHEMA . INNODB_BUFFER_PAGE AS table2 
ON ( table2 . SPACE = table1 . user_host ) ) ORDER BY table1 . thread_id LIMIT 168 
failed: 126 Incorrect key file for table '/data/mysql7/performance_schema_vardir/tmp/#sql_6b8_17.MYI'; 
try to repair it

Since this query requires a sort, MySQL creates a hidden temporary table called ‘#sql_6b8_17.MYI’ to hold the intermediate results. While the query was executing, some

  [Read more...]
MariaDB 10.0.5 storage engines – check the Linux packages
+3 Vote Up -1Vote Down

Today before Ivan’s tutorial, he told me that in the 10.0.5 virtual machine images he created, he couldn’t find the Cassandra storage engine. I told him it had to be installed separately, and this is true – you have to install some engines separately!

When you do a yum install MariaDB-server MariaDB-client like the installation instructions tell you to do, you don’t get all storage engines (so running SHOW ENGINES might have you wondering what happened to a bunch of engines). This can easily be seen by doing a yum search MariaDB. On a CentOS 6.4 server with the MariaDB 10.0 repository configured, you should see the following:

MariaDB-cassandra-engine.x86_64 : MariaDB: a very fast and robust SQL database server
MariaDB-client.x86_64 :
  [Read more...]
Problems with Multiple XA Storage Engines in MySQL 5.6
+2 Vote Up -0Vote Down

While integrating TokuDB into MySQL 5.6, we found that MySQL 5.6 does not support more than one XA storage engine. For example, there is an assert in the ha_recover function that fires when the total number of XA storage engines is greater than one. After disabling this assert, we found lots of bugs in the MySQL 5.6 implementation of the TC_LOG_MMAP class, which is used when running with the binlog turned off.

There are two alternatives that we know of to fix this problem in MySQL 5.6:

  • First, we could merge code from MariaDB 5.5 into MySQL 5.6. The advantage of this approach is that we have been running this code with TokuDB in MariaDB 5.5 for a long time, so we have confidence in its correctness.
  • Second, we found that MySQL 5.7.2 has made changes to allow multiple XA storage engines. This is great news for TokuDB since we have one
  [Read more...]
October 8 Webinar: Getting Started with MySQL
+0 Vote Up -0Vote Down

Save Time & Money – Do It Right The First Time

SPEAKER: Gerry Narvaja, Tokutek
DATE: Tuesday, October 8th
TIME: 1pm ET

If you are thinking of using the leading open source database in a project, learn just how easy it is to get started with MySQL, and how important it is to do it right. A few simple decisions early in development, and knowing when and how to effectively use the TokuDB performance engine, can save significant time and money down the road. Key metrics for MySQL success include performance, scalability, database size, and agility. Register Now!

Attend this webinar to learn:

  • The basics of installing and configuring MySQL and TokuDB
  • How to maximize performance for long-term scalability


  [Read more...]
SPIDER in MariaDB
+2 Vote Up -0Vote Down

One thing you might have missed from the MariaDB 10.0.4 Alpha release announcement is that SPIDER is finally in-tree, inside MariaDB. I’ve been watching the SPIDER engine for quite some years and fondly remember the time when Kentoku Shiba dressed up as spiderman at a MySQL Conference.

There are many factors that plays to an engines success. Distribution is one of them. Having a company backing it with services is another, so it is with great pleasure that I also show you Spiral Arms Ltd by Kentoku and Goto.

Overall, a good time to celebrate and I’m very happy for Kentoku and Goto. Now to promote its use.

  [Read more...]
Building TokuMX and TokuDB for Production
+1 Vote Up -0Vote Down

Recently, we’ve seen a few people ask us about building TokuMX from scratch. While it’s best if you just use the binaries you can get from us (they have all the right optimizations, we’ve tested them, and we can interpret coredumps they generate), we recognize there are other reasons you might need to do a custom build.

Since we actually build six distinct products all using the Fractal Tree indexing® library (community and enterprise versions of TokuDB for MySQL, TokuDB for MariaDB, and TokuMX), our build process is pretty complicated, compared to software packages that might, for example, just involve one source repository and link against a few standard libraries. Our TokuMX builds involve four git repositories, three

  [Read more...]
Comparing MongoDB, MySQL, and TokuMX Data Layout
+2 Vote Up -0Vote Down

A lot is said about the differences in the data between MySQL (http://www.mysql.com/) and MongoDB. Things such as “MongoDB is document based”, “MySQL is relational”, “InnoDB has a clustering key”, etc.. Some may wonder how TokuDB, our MySQL storage engine, and TokuMX, our MongoDB product, fit in with these data layouts. I could not find anything describing the differences with a simple google search, so I figured I’d write a post explaining how things compare.

So who are the players here? With MySQL, users are likely familiar with two storage engines: MyISAM, the original default up until MySQL 5.5, and

  [Read more...]
How TokuMX Gets Great Compression for MongoDB
+0 Vote Up -0Vote Down

In my last post, I showed what a Fractal Tree® index is at a high level. Once again, the Fractal Tree index is the data structure inside TokuMX and TokuDB, our MongoDB and MySQL products. One of its strengths is the ability to get high levels of compression on the stored data. In this post, I’ll explain why that is.

At a high level, one can argue that there isn’t anything special about our compression algorithms. We basically do this: we take large chunks of data, use known compression methods (e.g. zlib,

  [Read more...]
Understanding Tokutek Fractal Tree Indexes
+1 Vote Up -0Vote Down

Download PDF Presentation

Thanks to Tim Callaghan for speaking Tuesday night at the Effective MySQL New York meetup on Fractal Tree Indexes : Theory and Practice (MySQL and MongoDB). There was a good turnout and a full room to learn how the TokuDB storage engine from Tokutek is changing how to handle big data in MySQL.

Also interesting is how the same technology has been applied for use in MongoDB including giving MongoDB transactions; a big change for NoSQL.

Related News: Tokutek Meets Big Data Demand With Open Source TokuDB

Presenting at tomorrow’s Effective MySQL Meetup (New York City)
+0 Vote Up -1Vote Down

At tomorrow’s Effective MySQL Meetup, I’ll be presenting “Fractal Tree Indexes : Theory and Practice (MySQL and MongoDB).” The meetup is at 6:30pm Tuesday, May 14, 2013, and will be held at Alley NYC in New York City.

I’ll give an overview on how Fractal Tree® indexes work, and then get into specific product features that Fractal Trees enable in MySQL and MongoDB.  Some benchmarking and customer use-cases will be discussed, but my intent is for this to be a deep technical dive.  Several Tokutek Engineers will also be on hand, so bring any questions you’ve got.

I hope to see you there!

Last Week’s Presentations Posted
+0 Vote Up -0Vote Down

Last week I had to present a tutorial at Percona Live 2013, a presentation at SkySQL’s MySQL & Cloud Database Solution Day and last but not least, a presentation on a Saturday morning at Linuxfest Northwest. It wasn’t easy, but giving the presentations after our announcement early in the week about going open source was very exciting given the

  [Read more...]
May 2nd Webinar: Introduction to TokuDB v7 Community & Enterprise Editions
+0 Vote Up -0Vote Down

With this version, the source code is now freely available under the GPL License v2. For more details, see our blog here. Open source pioneer Mozilla has been using TokuDB to manage its MySQL-driven Datazilla Data cluster, an open-source system for managing and visualizing performance data.

Date: May 2nd
Time: 2 PM EST / 11 AM PST
REGISTER TODAY

In the past TokuDB has been free for evaluation; the new TokuDB Community Edition extends free use to deployed environments. With this release Tokutek is also planning on making available a TokuDB Enterprise Edition, which includes technical support,



  [Read more...]
Opening Week for TokuDB
+0 Vote Up -0Vote Down

Since we had the pleasure to announce that TokuDB is open source on Monday, it’s been a thrilling ride. With several members of the team out west all week, back on the east coast we’ve been seeing quite a lot of questions, suggestions, and exciting results.

Here are some of the highlights of our first week of open source:

We started hearing back from the community almost immediately after the announcement with discussions in multiple forums. We even reached #2 on Hacker News for a bit.

On Tuesday,

  [Read more...]
MySQL Paradise: YouTube Video
+2 Vote Up -0Vote Down

Yesterday we posted the photos and lyrics. Now we’ve got the YouTube video (click here)!

And for those who want some behind the scenes photos, see here and here. (courtesy of @seattlegaucho).

Thanks to Community for Selecting Tokutek for Prestigious MySQL Award
+3 Vote Up -0Vote Down

We wanted to thank everyone for naming Tokutek the Corporate Contributor of the Year 2013 for ongoing contribution to the MySQL community.

The MySQL Community Awards are given annually to the people and companies that support the MySQL ecosystem. The MySQL Community Award for Corporate Contributor of the Year recognizes a company or other organization or entity that has made valuable contributions to the MySQL ecosystem either in terms of open source code, knowledge,

  [Read more...]
MySQL Paradise – Percona Live Lightning Talk
+1 Vote Up -0Vote Down

Thanks to all of those who came to the lightning talks this evening. We’ve got the audio posted here and the lyrics below.

A special thanks to Erin Grace O’Malley O’Neill for the great performance. Thanks also to @NuoDB and @geobdz (whose photo is below) for tweets.

Stay tuned for the video as well….


MySQL Paradise

As I browse through  [Read more...]

Open Source TokuDB Resources
+2 Vote Up -0Vote Down

Since we announced that TokuDB is now open source, there has been a lot of positive feedback (thanks!) and also some questions about the details. I want to take this opportunity to give a quick high level guide to describe what our repositories on Github are.

Here are the repositories:

  • ft-index. This repository is the “magic”. It contains the Fractal Tree data structures we have been talking about for years. This is also the main piece that was previously closed source. Here are some interesting directories:
    • src: This directory is a layer that implements an API that is similar to the BDB API.
    • locktree: an in-memory data structure that maintains transactions’ row-level locks.
  [Read more...]
Getting Interesting
+0 Vote Up -0Vote Down

I enjoyed Stewart Smith’s MySQL storage engine blog last week. In it he noted “I cannot emphasize how much more interesting TokuDB would be if it were open source.” Well, with our open source announcement yesterday, hopefully we are getting interesting.

We wanted to thank everyone for the great feedback. Here is a sampling from some of the forums where dialogue is occurring:


Reddit:

BrianAtDTS: “With this update, this puts MySQL in

  [Read more...]
Announcing TokuDB v7: Open Source and More
+3 Vote Up -0Vote Down

Every few months, I get the fun job of announcing what’s new in TokuDB®, but this time is special. With Version 7, TokuDB for MySQL and MariaDB is going open source.

The free Community Edition is fully functional and fully performant. It has all the compression you’ve come to expect from TokuDB. It has hot schema changes: no-down-time column insertion, deletion, renaming, etc., as well as index creation. It has clustering secondary keys. We are also announcing an Enterprise Edition (coming soon) with additional benefits, such as a support package and advanced backup and recovery tools.

Making TokuDB open source is a natural next step for Tokutek’s involvement in the MySQL community. So far, Tokutek has been involved in the community in many ways:

  • We’ve
  [Read more...]
The MEMORY storage engine
+3 Vote Up -0Vote Down

I recently wrote about Where are they now: MySQL Storage Engines and The MERGE storage engine: not dead, just resting…. or forgotten. Today, it’s the turn of the MEMORY storage engine – otherwise known as HEAP.

This is yet another piece of the MySQL server that sits largely unmaintained and unloved. The MySQL Manual even claims that it supports encryption… with the caveat of having to use the SQL functions for encryption/decryption rather than in the engine itself (so,

  [Read more...]
April is the Coolest Month
+0 Vote Up -0Vote Down

If T.S. Eliot were a MySQL DBA, I think he would have been more upbeat about April.

We are gearing up for an incredible second half of April. We will be presenting three separate sessions at the Percona Live: MySQL Conference and Expo 2013, April 22-25, in Santa Clara, CA. In addition, we will be presenting at SkySQL’s MySQL & Cloud Database Solutions Day on Friday, April 26 at the same location.

Come by to see us in Booth #114, or stop by one of our sessions:

  [Read more...]
MySQL 5.6: an inside perspective
+2 Vote Up -0Vote Down

MySQL 5.6 through the eyes of a custom storage engine MySQL plugin

MySQL is famous for its pluggable storage engine architecture which allows a DBA or an application developer to choose the right engine for the task. An application uses MySQL API and is isolated from all of the low-level implementation details at the storage level. As an example, the Cloud Storage Engine (ClouSE) enables existing MySQL applications to use cloud storage such as Amazon S3 or Google Cloud Storage to store its data. The application doesn’t need to be changed or even redeployed: with ClouSE, remote cloud storage will look like a better (ultra-scalable, durable, always-on) alternative to the local storage.

As you may already know, ClouSE now supports

  [Read more...]
MongoDB + Fractal Tree Indexes = High Compression
+2 Vote Up -0Vote Down

One doesn’t have to look far to see that there is strong interest in MongoDB compression. MongoDB has an open ticket from 2009 titled “Option to Store Data Compressed” with Fix Version/s planned but not scheduled. The ticket has a lot of comments, mostly from MongoDB users explaining their use-cases for the feature. For example, Khalid Salomão notes that “Compression would be very good to reduce storage cost and improve IO performance” and Andy notes that “SSD is getting more and more common for

  [Read more...]
NoSQL is Great, But You Still Need Indexes
+1 Vote Up -0Vote Down

I’ve said it before, and, as is the nature of these things, I’ll almost certainly say it again: your database performance is only as good as your indexes.

That’s the grand thesis, so what does that mean? In any DB system — SQL, NoSQL, NewSQL, PostSQL, … — data gets ingested and organized. And the system answers queries. The pain point for most users is around the speed to answer queries. And the query speed (both latency and throughput, to be exact) depend on how the data is organized. In short: Good Indexes, Fast Queries; Poor Indexes, Slow Queries.

But building indexes is hard work, or at least it has been for the last several decades, because almost all indexing is done with B-trees. That’s true of

  [Read more...]
Concurrency Improvements in TokuDB v6.6 (Part 2)
+0 Vote Up -0Vote Down

In Part 1, we showed performance results of some of the work that’s gone in to TokuDB v6.6. In this post, we’ll take a closer look at how this happened, on the engineering side, and how to think about the performance characteristics in the new version.

Background

It’s easiest to think about our concurrency changes in terms of a Fractal Tree® index that has nodes like a B-tree index, and buffers on each node that batch changes for the subtree rooted at that node. We have materials that describe this available here, but we can proceed just knowing that:

  • To inject data into the tree, you need to store a message in a buffer at the root of the tree. These
  •   [Read more...]
    Concurrency Improvements in TokuDB v6.6 (Part 1)
    +2 Vote Up -0Vote Down

    With TokuDB v6.6 out now, I’m excited to present one of my favorite enhancements: concurrency within a single index. Previously, while there could be many SQL transactions in-flight at any given moment, operations inside a single index were fairly serialized. We’ve been working on concurrency for a few versions, and things have been getting a lot better over time. Today I’ll talk about what to expect from v6.6. Next time, we’ll see why.

    Summary of Results

    Running multiple iiBench clients on a single MySQL instance, we see a big improvement in the cumulative insertion speed at all concurrency levels. We see a gain of 33.9% in single-threaded performance and 51.8% at

      [Read more...]
    Showing entries 1 to 30 of 146 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.