Showing entries 121 to 130 of 333
« 10 Newer Entries | 10 Older Entries »
Displaying posts with tag: TokuDB (reset)
New MySQL features, related technologies at Percona Live London

The upcoming Percona Live London conference, November 11-12, features quite a number of talks about the latest MySQL features and related technologies. There will be a lots of talks about the new MySQL 5.6 features:

  • Opening keynote highlights MySQL 5.6 new features.
  • New InnoDB Compression talk will cover the new compression algorithm, implemented by Facebook and included in MySQL 5.6.
  • New  …
[Read more]
Interactive Debugging of Transaction Conflicts with TokuDB

I am developing a concurrent application that uses TokuDB to store its database. Sometimes, one of my SQL statements returns with a ‘lock wait timeout exceeded’ error. How do I identify the cause of this error? First, I need to understand a little bit about how TokuDB transactions use locks. Then, I need to understand how to use the MySQL information schema to look at the current state of the locks.

Transactions and Locks

TokuDB uses key range locks to implement serializable transactions. These locks are acquired as the transaction progresses. The locks are released when the transaction commits or aborts.

TokuDB stores these locks in a data structure called the lock tree. The lock tree stores the set of range locks granted to each transaction. In addition, the lock tree stores the set of locks that are not granted due to a conflict with locks granted to some other transaction.

TokuDB 7.1 provides access to …

[Read more]
Announcing TokuDB v7.1

Today we released TokuDB v7.1, which includes the following important features and fixes:

  • Added ability for users to view lock information via information_schema.tokudb_trx, information_schema.tokudb_locks, and information_schema.tokudb_lock_waits tables.
  • Changed the default compression to zlib and default basement node size to 64K.
  • Changed default analyze time to 5 seconds.
  • Added server variable to control amount of memory allocated for each bulk loader. In prior TokuDB versions each loader allocated 50% of the available TokuDB cache.
  • Changed table close behavior such that all data for the table remains in the cache (and is not flushed immediately).
  • Removed user reported stalls due to cache pressure induced by the bulk loader, lock tree escalation, and a particular open table stall.
  • Several bugs and behavioral issues reported by users. …
[Read more]
Converting an OLAP database to TokuDB, part 3: operational stuff

This is the third post in a series of posts describing our experience in migrating a large DWH server to TokuDB (see 1st and 2nd parts). This post discusses operations; namely ALTER TABLE operations in TokuDB. We ran into quite a few use cases by this time that we can shed light on.

Quick recap: we've altered one of out DWH slaves to TokuDB, with the goal of migrating most of out servers, including the master, to TokuDB.

Adding an index

Shortly after migrating our server to TokuDB we noticed an unreasonably disproportionate slave lag on our TokuDB slave (red line in chart below) as compared to other slaves.

[Read more]
Introducing TokuMX Clustering Indexes for MongoDB

Since introducing TokuMX, we’ve discussed benefits that TokuMX has for existing MongoDB applications that require no changes. In this post, I introduce an extension we’ve made to the indexing API: clustering indexes, a tool that can tremendously improve query performance. If I were to speak to someone about clustering indexes, I think the conversation could go something like this…

What is a Clustering Index?

A clustering index is an index that stores the entire document, not just the defined key.

A common example is …

[Read more]
A TokuDB Stall Caused by Conflicting Transactions When Opening a Table

One of our customers reported that ‘create table select from’ statements stall for a period of time equal to the TokuDB lock timeout.  This indicated a lock conflict between multiple transactions.  In addition, other MySQL clients that were opening unrelated tables were also stalled.  This indicated that some shared mutex is held too long.  We discuss details about this bug and how it was fixed.  The bug fix will be distributed in TokuDB 7.1.0.

Example
Suppose that we set the tokudb lock timeout to 60 seconds just to exaggerate the stall.

mysql> set global tokudb_lock_timeout=60000;
Query OK, 0 rows affected (0.00 sec)

We then create a simple table.

mysql> create table s (id int primary key);
Query OK, 0 rows affected (0.02 sec)

When we create new table (t) and populate it from some …

[Read more]
Inexpensive SSDs for Database Workloads

The cost of SSDs has been dropping rapidly, and at the time of this writing, 2.5-drives have reached the 1TB capacity mark.  You can actually get inexpensive drives for as little as 60 cents per GB. Even inexpensive SSDs can perform tens of thousands of IOPs and come with 1.5M – 2M hous MTBF and a 5-year warranty: check out the Intel SC S3500 specs as an example. There is however one important factor you need to take into account when considering  SSDs as opposed to conventional hard drives – Write Endurance.

Many of us have heard about SSDs having limits in terms of how many writes SSDs can handle, many however assume this is what is already accounted for in the warranty period and so if the hard drives claim to have sequential write speed of 450MB/sec and a warranty of 5 years we expect …

[Read more]
OQGraph and TokuDB

Today, while at Oracle's MySQL Connect conference in San Francisco, I had the opportunity to talk to Gerardo Narvaja and that inspired me to try using OQGraphv3 and TokuDB. Of course, the first challenge is to compile it on my MacBook which has MacOS Lion 10.7. I installed the Apple's latest XCode Command Line tools and eventually compiled a MariaDB which had both TokuDB and OQGraph available

A TokuDB Stall Caused by a Big Transaction and How It was Fixed

One of our customers sometimes observed lots of simple insertions taking far longer than expected to complete. Usually these insertions completed in milliseconds, but the insertions sometimes were taking hundreds of seconds. These stalls indicated the existence of a serialization bug in the Fractal Tree index software, so the hunt was on. We found that these stalls occurred when a big transaction was committing and the Fractal Tree index software was taking a checkpoint. This problem was fixed in both TokuDB 7.0.3 and TokuMX 1.0.3. Please read on as we describe some details about this bug and how we fixed it. We describe some of the relevant Fractal Tree index algorithms first.

What is a Big Transaction?

Each transaction builds a rollback log as it performs Fractal Tree index operations. The rollback log is maintained in memory until it gets too big and is spilled to the TokuDB rollback file. We define a small …

[Read more]
TokuDB Hot Backup – Part 2

In my last post, I discussed the existing backup solutions for MySQL.  At the end I briefly discussed why the backup solutions for InnoDB do not apply to TokuDB.  Now I’m going to outline the backup solution we created.  Our solution works for both TokuDB and InnoDB.  It also has no knowledge of the log files and does not require any changes to either storage engine.  In fact, the library could be used with almost any process; it has no knowledge of what types of files are being backed up.

Shims

Tokutek’s Hot Backup is essentially a shim between the mysqld process and the operating system (Linux only, at this point.)  It is a separately compiled C++ library that simply gets linked into the mysqld application at the end of the respective build process.  We ship this library with our own enterprise …

[Read more]
Showing entries 121 to 130 of 333
« 10 Newer Entries | 10 Older Entries »