Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Previous 30 Newer Entries Showing entries 31 to 60 of 254 Next 30 Older Entries

Displaying posts with tag: TokuDB (reset)

Big trouble with zero-length character columns in TokuDB
+0 Vote Up -0Vote Down

What good is a zero-length character column in a MySQL table? A zero-length character column has type of ‘char(0)’. If it is nullable, then it can at least store one bit. If it is not nullable, then the value for this column in all rows is a null string. IMO, not very useful. However, the MySQL Reference Manual says that there are valid uses for such a column, so TokuDB should support it. Unfortunately, we recently found and fixed a bug related to zero length character columns in TokuDB.

A Random Query Generator (RQG) trial generated a table with a ‘char(0)’ column and caused TokuDB to crash when executing an alter

  [Read more...]
March 20 Webinar: How to Scale MySQL for Big Data Applications
+0 Vote Up -0Vote Down

You may think that you have to buy, install, and get up to speed on a new database if you want to work with large amounts of data, but you can do more than you think with the MySQL you already have.
Register Now!

SPEAKER: Jon Tobin, Tokutek
DATE: Thursday, March 20th
TIME: 1pm ET

Without having to change your application or do special tuning you can increase performance and save significant time and money when you need to scale.

Join Tokutek’s Jon Tobin as he demonstrates how to use MySQL or MariaDB in Big Data applications by simply upgrading the storage engine with TokuDB, and how to effectively evaluate TokuDB for increased performance, compression and agility.

During this webinar you will learn:

  • How to dramatically increase performance without having to rewrite



  [Read more...]
Announcing TokuDB v7.1.5
+0 Vote Up -0Vote Down

Yesterday we released TokuDB v7.1.5, which includes the following important features and fixes:

  • Upgraded MySQL and MariaDB to version 5.5.36.
  • Six months of performance improvements to our underlying Fractal Tree indexing.
  • Fixes for bugs, stalls, and behavioral issues reported by users.
  • Fixes for issues identified by the addition of Random Query Generator (RQG) testing.
  • Fixes for issues identified by the addition of Valgrind testing.
  • Full details on the changes in TokuDB v7.1.5 can be found in the release notes section of the TokuDB User’s Guide, available from our documentation page.

As always, you can download the Community and Enterprise

  [Read more...]
Tokutek and Percona Live 2014
+1 Vote Up -0Vote Down

I’ve been a little behind in recent blogging efforts, and realized that in less than a month we’ll be back at Percona Live: MySQL Conference and Expo 2014, aka PLMCE. Last year’s PLMCE was my first, as well as the event where Tokutek announced the open sourcing of TokuDB.

It’s hard to believe that a year has gone by, but the customer adoption in both enterprise and community users has been awesome. TokuDB is available from our website for both MySQL and MariaDB, and is also

  [Read more...]
The MySQL ARCHIVE storage engine – Alternatives
+0 Vote Up -0Vote Down

In my previous post I pointed out that the existing ARCHIVE storage engine in MySQL may not be the one that will satisfy your needs when it comes to effectively storing large and/or old data. But are there any good alternatives? As the primary purpose of this engine is to store rarely accessed data in disk space efficient way, I will focus here on data compression abilities rather then on performance.

The InnoDB engine provides compressed row format, but is it’s efficiency even close to the one from that available in archive engine? You can also compress MyISAM tables by using myisampack tool, but that also means a table will be read only after such operation.

Moreover, I don’t trust MyISAM nor Archive when it comes to data

  [Read more...]
The Effects of Database Heap Storage Choices in MongoDB
+0 Vote Up -0Vote Down

William Zola over at MongoDB gave a great talk called “The (Only) Three Reasons for Slow MongoDB Performance”. It reminded me of an interesting characteristic of updates in MongoDB. Because MongoDB’s main data store is a flat file and secondary indexes store offsets into the flat file (as I explain here), if the location of a document changes, corresponding entries in secondary indexes must also change. So, an update to an unindexed field that causes the document to move also causes modifications to every secondary index, which, as William points out, can be expensive. If a document has indexed an array, this

  [Read more...]
January 28 Webinar: Get More Out of MySQL with TokuDB
+0 Vote Up -0Vote Down

You love MySQL and MariaDB for its ease of deployment, but what if you could increase performance and save significant time and money when your application starts to scale without having to change your applications?
Register Now!

SPEAKER: Tim Callaghan, VP of Engineering at Tokutek
DATE: Tuesday, January 28th
TIME: 1pm ET

Join this interactive webinar with Tokutek’s VP of Engineering, Tim Callaghan, as he walks through the potential pitfalls when using MySQL or MariaDB for Big Data applications, and how to effectively use TokuDB to increase performance, reduce database size and achieve true schema agility.

Attend this webinar to learn:

  • How easy it is to install and configure TokuDB with MySQL or MariaDB
  • How to dramatically increase performance without having to rewrite code



  •   [Read more...]
    MariaDB world record price per row 0.0000005$ on a single DELL R710
    +0 Vote Up -0Vote Down
    Don't look at an industry benchmark here, it's a real client story.

    200 Billion records in a month and it should be transactional but not durable.

    For regular workload we use LOAD DATA INFILE into partitioned InnoDB, but here we have estimated 15TB of RAID storage. This is a lot of disks and it can't no more stay inside a single server internal storage.

    MariaDB 5.5 come with TokuDB storage engine for compression, but is it possible in the time frame impose by the workload?

    We start benchmarking 380G of raw input data files,  6 Billion rows.

    First let's check the compression with the dataset.













      [Read more...]
    What does the ‘Incorrect key file for table’ error mean?
    +0 Vote Up -0Vote Down

    What does it mean if MySQL returns the ‘Incorrect key file for table’ error for one of my queries? The answer is complicated and depends on which storage engine is returning the error. We have debugged two cases which we describe here.

    File system out of space

    When running the random query generator, one of the queries failed.

    Query: SELECT * FROM (mysql . general_log AS table1 INNER JOIN INFORMATION_SCHEMA . INNODB_BUFFER_PAGE AS table2 
    ON ( table2 . SPACE = table1 . user_host ) ) ORDER BY table1 . thread_id LIMIT 168 
    failed: 126 Incorrect key file for table '/data/mysql7/performance_schema_vardir/tmp/#sql_6b8_17.MYI'; 
    try to repair it

    Since this query requires a sort, MySQL creates a hidden temporary table called ‘#sql_6b8_17.MYI’ to hold the intermediate results. While the query was executing, some

      [Read more...]
    Put your MySQL Knowledge to Good Use with Tim Callaghan at Percona Live-London, November 12
    +0 Vote Up -0Vote Down

    Attending Percona Live in London next week?

    Don’t miss the chance to hear Tokutek’s Vice President of Engineering, Tim Callaghan, discuss how to use your MySQL knowledge to become an instant MongoDB Guru and the advantages of using Fractal Tree&#174 indexes in MySQL and MongoDB. Tim will be speaking about these topics in two separate sessions at 12:00pm and 5:00pm on November 12.

    For more information on these sessions and Percona Live-London, visit https://www.percona.com/live/london-2013/users/tim-callaghan.

    November 6 Webinar: 5 Pitfalls to Avoid with MySQL and Big Data
    +0 Vote Up -0Vote Down

    You love MySQL for its ease of deployment – but are you worried about how your application will perform when it starts to scale?

    SPEAKER: Gerry Narvaja, Tokutek
    DATE: Wednesday, November 6th
    TIME: 1pm ET
    Register Now!

    Join this interactive webinar with Gerry Narvaja of Tokutek as he walks through the potential pitfalls when using MySQL for Big Data applications, how you can avoid unnecessary tolls on time and resources and tips on how to get the most out of your MySQL applications with open source TokuDB.

    Attend this webinar to learn how to:

    • dramatically increase performance without having to rewrite code
    • reduce the total cost of your servers and



      [Read more...]
    Problems with Multiple XA Storage Engines in MySQL 5.6
    +2 Vote Up -0Vote Down

    While integrating TokuDB into MySQL 5.6, we found that MySQL 5.6 does not support more than one XA storage engine. For example, there is an assert in the ha_recover function that fires when the total number of XA storage engines is greater than one. After disabling this assert, we found lots of bugs in the MySQL 5.6 implementation of the TC_LOG_MMAP class, which is used when running with the binlog turned off.

    There are two alternatives that we know of to fix this problem in MySQL 5.6:

    • First, we could merge code from MariaDB 5.5 into MySQL 5.6. The advantage of this approach is that we have been running this code with TokuDB in MariaDB 5.5 for a long time, so we have confidence in its correctness.
    • Second, we found that MySQL 5.7.2 has made changes to allow multiple XA storage engines. This is great news for TokuDB since we have one
      [Read more...]
    TokuDB configuration variables of interest
    +1 Vote Up -0Vote Down

    During our experiments I came upon a few TokuDB variables of interest; if you are using TokuDB you might want to look into these:

    • tokudb_analyze_time

    This is a boundary on the number of seconds an ANALYZE TABLE will operate on each index on each partition on a TokuDB table.

    That is, if tokudb_analyze_time = 5, and your table has 4 indexes (including PRIMARY) and 7 partitions, then the total runtime is limited to 5*4*7 = 140 seconds.

    Default in 7.1.0: 5 seconds

    • tokudb_cache_size

    Similar to innodb_buffer_pool_size, this variable sets the amount of memory allocated by TokuDB for caching pages. Like InnoDB the table is clustered within

      [Read more...]
    New MySQL features, related technologies at Percona Live London
    +0 Vote Up -0Vote Down

    The upcoming Percona Live London conference, November 11-12, features quite a number of talks about the latest MySQL features and related technologies. There will be a lots of talks about the new MySQL 5.6 features:

      [Read more...]
    Interactive Debugging of Transaction Conflicts with TokuDB
    +0 Vote Up -0Vote Down

    I am developing a concurrent application that uses TokuDB to store its database. Sometimes, one of my SQL statements returns with a ‘lock wait timeout exceeded’ error. How do I identify the cause of this error? First, I need to understand a little bit about how TokuDB transactions use locks. Then, I need to understand how to use the MySQL information schema to look at the current state of the locks.

    Transactions and Locks

    TokuDB uses key range locks to implement serializable transactions. These locks are acquired as the transaction progresses. The locks are released when the transaction commits or aborts.

    TokuDB stores these locks in a data structure called the lock tree. The lock tree stores the set of range locks granted to each transaction. In addition, the lock tree stores the set of locks that are not granted due to a conflict

      [Read more...]
    Announcing TokuDB v7.1
    +0 Vote Up -0Vote Down

    Today we released TokuDB v7.1, which includes the following important features and fixes:

    • Added ability for users to view lock information via information_schema.tokudb_trx, information_schema.tokudb_locks, and information_schema.tokudb_lock_waits tables.
    • Changed the default compression to zlib and default basement node size to 64K.
    • Changed default analyze time to 5 seconds.
    • Added server variable to control amount of memory allocated for each bulk loader. In prior TokuDB versions each loader allocated 50% of the available TokuDB cache.
    • Changed table close behavior such that all data for the table remains in the cache (and is not flushed immediately).
    • Removed user reported stalls due to cache pressure induced by the bulk loader, lock tree escalation, and a particular open table stall.
    • Several bugs and behavioral issues
      [Read more...]
    Converting an OLAP database to TokuDB, part 3: operational stuff
    +1 Vote Up -0Vote Down

    This is the third post in a series of posts describing our experience in migrating a large DWH server to TokuDB (see 1st and 2nd parts). This post discusses operations; namely ALTER TABLE operations in TokuDB. We ran into quite a few use cases by this time that we can shed light on.

    Quick recap: we've altered one of out DWH slaves to TokuDB, with the goal of migrating most of out servers, including the master, to TokuDB.

    Adding an index

    Shortly after migrating our server to TokuDB we noticed an unreasonably disproportionate slave lag on our TokuDB slave (red line in chart below) as compared to other slaves.

      [Read more...]
    Introducing TokuMX Clustering Indexes for MongoDB
    +1 Vote Up -0Vote Down

    Since introducing TokuMX, we’ve discussed benefits that TokuMX has for existing MongoDB applications that require no changes. In this post, I introduce an extension we’ve made to the indexing API: clustering indexes, a tool that can tremendously improve query performance. If I were to speak to someone about clustering indexes, I think the conversation could go something like this…

    What is a Clustering Index?

    A clustering index is an index that stores the entire document, not just the defined key.

    A common example is

      [Read more...]
    A TokuDB Stall Caused by Conflicting Transactions When Opening a Table
    +0 Vote Up -0Vote Down

    One of our customers reported that ‘create table select from’ statements stall for a period of time equal to the TokuDB lock timeout.  This indicated a lock conflict between multiple transactions.  In addition, other MySQL clients that were opening unrelated tables were also stalled.  This indicated that some shared mutex is held too long.  We discuss details about this bug and how it was fixed.  The bug fix will be distributed in TokuDB 7.1.0.

    Example
    Suppose that we set the tokudb lock timeout to 60 seconds just to exaggerate the stall.

    mysql> set global tokudb_lock_timeout=60000;
    Query OK, 0 rows affected (0.00 sec)

    We then create a simple table.

    mysql> create table s (id int primary key);
    Query OK, 0 rows affected (0.02







      [Read more...]
    Inexpensive SSDs for Database Workloads
    +2 Vote Up -0Vote Down

    The cost of SSDs has been dropping rapidly, and at the time of this writing, 2.5-drives have reached the 1TB capacity mark.  You can actually get inexpensive drives for as little as 60 cents per GB. Even inexpensive SSDs can perform tens of thousands of IOPs and come with 1.5M – 2M hous MTBF and a 5-year warranty: check out the Intel SC S3500 specs as an example. There is however one important factor you need to take into account when considering  SSDs as opposed to conventional hard drives – Write Endurance.

    Many of us have heard about SSDs having limits in terms of how many writes SSDs can handle, many however assume this is what is already

      [Read more...]
    A TokuDB Stall Caused by a Big Transaction and How It was Fixed
    +1 Vote Up -0Vote Down

    One of our customers sometimes observed lots of simple insertions taking far longer than expected to complete. Usually these insertions completed in milliseconds, but the insertions sometimes were taking hundreds of seconds. These stalls indicated the existence of a serialization bug in the Fractal Tree index software, so the hunt was on. We found that these stalls occurred when a big transaction was committing and the Fractal Tree index software was taking a checkpoint. This problem was fixed in both TokuDB 7.0.3 and TokuMX 1.0.3. Please read on as we describe some details about this bug and how we fixed it. We describe some of the relevant Fractal Tree index algorithms first.

    What is a Big Transaction?

    Each transaction builds a rollback log as it performs Fractal Tree index operations. The rollback log is maintained in

      [Read more...]
    TokuDB Hot Backup – Part 2
    +0 Vote Up -0Vote Down

    In my last post, I discussed the existing backup solutions for MySQL.  At the end I briefly discussed why the backup solutions for InnoDB do not apply to TokuDB.  Now I’m going to outline the backup solution we created.  Our solution works for both TokuDB and InnoDB.  It also has no knowledge of the log files and does not require any changes to either storage engine.  In fact, the library could be used with almost any process; it has no knowledge of what types of files are being backed up.

    Shims

    Tokutek’s Hot Backup is essentially a shim between the mysqld process and the operating system (Linux only, at this point.)  It is a separately compiled C++ library that simply gets linked into the mysqld application at the end of the respective build process.  We ship this library

      [Read more...]
    MariaDB 5.5.33 Now Available
    +0 Vote Up -0Vote Down

    The MariaDB project is pleased to announce the immediate availability of MariaDB 5.5.33. This is a Stable (GA) release. See the Release Notes and Changelog for detailed information on this release and the What is MariaDB 5.5? page in the AskMonty Knowledgebase for general information about the MariaDB 5.5 series.

    Download MariaDB 5.5.33

    Release Notes Changelog

      [Read more...]
    TokuDB Hot Backup – Part 1
    +1 Vote Up -0Vote Down

    There are multiple ways to backup a MySQL database. Some are more painful than others. In this two part blog we are going to discuss why the new hot backup system in TokuDB is special amidst the existing solutions. First let’s look at existing backup solutions for MySQL and InnoDB.

    Let’s start with the most obvious, and possibly painful, way to make a backup of MySQL: a manual copy of the MySQL data directory.

    Coarse Copy

    The copying itself isn’t the painful part; any DBA worth their salt can copy a directory. Guaranteeing what comes out the other end, however, is difficult. In other words, what will the state of each table in each database look like when the backup is complete? It turns out, without additional help, we don’t know!

    If you think about the dynamic state of a database, and the serial copying of the same database files

      [Read more...]
    Converting an OLAP database to TokuDB, part 2: the process of migration
    +2 Vote Up -0Vote Down

    This is a second in a series of posts describing our experience in migrating a large DWH server to TokuDB. This post discusses the process of migration itself.

    As a quick recap (read part 1 here), we have a 2TB compressed InnoDB (4TB uncompressed) based DWH server. Space is running low, and we're looking at TokuDB for answers. Early experiments show that TokuDB's compression could make a good impact on disk space usage. I'm still not discussing performance -- keeping this till later post.

    Those with weak hearts can skip right to the end, where we finally have a complete conversion. You can also peek at the very end to find out how much 4TB uncompressed InnoDB data is worth in TokuDB. But

      [Read more...]
    TokuDB vs InnoDB in timeseries INSERT benchmark
    +0 Vote Up -0Vote Down

    This post is a continuation of my research of TokuDB’s  storage engine to understand if it is suitable for timeseries workloads.

    While inserting LOAD DATA INFILE into an empty table shows great results for TokuDB, what’s more interesting is seeing some realistic workloads.

    So this time let’s take a look at the INSERT benchmark.

    What I am going to do is to insert data in 16 parallel threads into the table from the previous post:

    CREATE TABLE `sensordata` (
      `ts` int(10)
      [Read more...]
    Converting an OLAP database to TokuDB, part 1
    +0 Vote Up -0Vote Down

    This is the first in a series of posts describing my impressions of converting a large OLAP server to TokuDB. There's a lot to tell, and the experiment is not yet complete, so this is an ongoing blogging. In this post I will describe the case at hand and out initial reasons for looking at TokuDB.

    Disclosure: I have no personal interests and no company interests; we did get friendly, useful and free advice from Tokutek engineers. TokuDB is open source and free to use, though commercial license is also available.

    The case at hand

    We have a large and fast growing DWH MySQL setup. This data warehouse is but one component in a larger data setup, which includes Hadoop, Cassandra and more. For online dashboards and most reports, MySQL is our service. We populate this warehouse mainly via Hive/Hadoop. Thus, we have an hourly load of data from Hive, as

      [Read more...]
    Considering TokuDB as an engine for timeseries data
    +2 Vote Up -0Vote Down

    I am working on a customer’s system where the requirement is to store a lot of timeseries data from different sensors.

    For performance reasons we are going to use SSD, and therefore there is a list of requirements for the architecture:

    • Provide high insertion rate
    • Provide a good compression rate to store more data on expensive SSDs
    • Engine should be SSD friendly (less writes per timeperiod to help with SSD wear)
    • Provide a reasonable response time (within ~50 ms) on SELECT queries on hot recently inserted data

    Looking on these requirements I actually think that TokuDB might be a good fit for this task.

      [Read more...]
    Building TokuMX and TokuDB for Production
    +1 Vote Up -0Vote Down

    Recently, we’ve seen a few people ask us about building TokuMX from scratch. While it’s best if you just use the binaries you can get from us (they have all the right optimizations, we’ve tested them, and we can interpret coredumps they generate), we recognize there are other reasons you might need to do a custom build.

    Since we actually build six distinct products all using the Fractal Tree indexing® library (community and enterprise versions of TokuDB for MySQL, TokuDB for MariaDB, and TokuMX), our build process is pretty complicated, compared to software packages that might, for example, just involve one source repository and link against a few standard libraries. Our TokuMX builds involve four git repositories, three

      [Read more...]
    Slides from Boston MongoDB User Group Meetup on 7/31/13
    +0 Vote Up -0Vote Down

    On Wednesday night, the Boston MongoDB User group was kind enough to have me speak about TokuMX Internals. I spoke about Fractal Tree® indexes and the technical reasons behind the benefits they provide to MongoDB applications. Although the talk mostly references TokuMX and MongoDB, all the theory applies to TokuDB and MySQL as well.

    My slides are on our technology overview page, along with other great content.

    Opportunities to present technical material to an engaged audience asking tough questions is rare, and much appreciated. So thank you to the Boston MongoDB User group for having me present.

    Previous 30 Newer Entries Showing entries 31 to 60 of 254 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.