Showing entries 121 to 130 of 329
« 10 Newer Entries | 10 Older Entries »
Displaying posts with tag: TokuDB (reset)
Introducing TokuMX Clustering Indexes for MongoDB

Since introducing TokuMX, we’ve discussed benefits that TokuMX has for existing MongoDB applications that require no changes. In this post, I introduce an extension we’ve made to the indexing API: clustering indexes, a tool that can tremendously improve query performance. If I were to speak to someone about clustering indexes, I think the conversation could go something like this…

What is a Clustering Index?

A clustering index is an index that stores the entire document, not just the defined key.

A common example is …

[Read more]
A TokuDB Stall Caused by Conflicting Transactions When Opening a Table

One of our customers reported that ‘create table select from’ statements stall for a period of time equal to the TokuDB lock timeout.  This indicated a lock conflict between multiple transactions.  In addition, other MySQL clients that were opening unrelated tables were also stalled.  This indicated that some shared mutex is held too long.  We discuss details about this bug and how it was fixed.  The bug fix will be distributed in TokuDB 7.1.0.

Example
Suppose that we set the tokudb lock timeout to 60 seconds just to exaggerate the stall.

mysql> set global tokudb_lock_timeout=60000;
Query OK, 0 rows affected (0.00 sec)

We then create a simple table.

mysql> create table s (id int primary key);
Query OK, 0 rows affected (0.02 sec)

When we create new table (t) and populate it from some …

[Read more]
Inexpensive SSDs for Database Workloads

The cost of SSDs has been dropping rapidly, and at the time of this writing, 2.5-drives have reached the 1TB capacity mark.  You can actually get inexpensive drives for as little as 60 cents per GB. Even inexpensive SSDs can perform tens of thousands of IOPs and come with 1.5M – 2M hous MTBF and a 5-year warranty: check out the Intel SC S3500 specs as an example. There is however one important factor you need to take into account when considering  SSDs as opposed to conventional hard drives – Write Endurance.

Many of us have heard about SSDs having limits in terms of how many writes SSDs can handle, many however assume this is what is already accounted for in the warranty period and so if the hard drives claim to have sequential write speed of 450MB/sec and a warranty of 5 years we expect …

[Read more]
OQGraph and TokuDB

Today, while at Oracle's MySQL Connect conference in San Francisco, I had the opportunity to talk to Gerardo Narvaja and that inspired me to try using OQGraphv3 and TokuDB. Of course, the first challenge is to compile it on my MacBook which has MacOS Lion 10.7. I installed the Apple's latest XCode Command Line tools and eventually compiled a MariaDB which had both TokuDB and OQGraph available

A TokuDB Stall Caused by a Big Transaction and How It was Fixed

One of our customers sometimes observed lots of simple insertions taking far longer than expected to complete. Usually these insertions completed in milliseconds, but the insertions sometimes were taking hundreds of seconds. These stalls indicated the existence of a serialization bug in the Fractal Tree index software, so the hunt was on. We found that these stalls occurred when a big transaction was committing and the Fractal Tree index software was taking a checkpoint. This problem was fixed in both TokuDB 7.0.3 and TokuMX 1.0.3. Please read on as we describe some details about this bug and how we fixed it. We describe some of the relevant Fractal Tree index algorithms first.

What is a Big Transaction?

Each transaction builds a rollback log as it performs Fractal Tree index operations. The rollback log is maintained in memory until it gets too big and is spilled to the TokuDB rollback file. We define a small …

[Read more]
TokuDB Hot Backup – Part 2

In my last post, I discussed the existing backup solutions for MySQL.  At the end I briefly discussed why the backup solutions for InnoDB do not apply to TokuDB.  Now I’m going to outline the backup solution we created.  Our solution works for both TokuDB and InnoDB.  It also has no knowledge of the log files and does not require any changes to either storage engine.  In fact, the library could be used with almost any process; it has no knowledge of what types of files are being backed up.

Shims

Tokutek’s Hot Backup is essentially a shim between the mysqld process and the operating system (Linux only, at this point.)  It is a separately compiled C++ library that simply gets linked into the mysqld application at the end of the respective build process.  We ship this library with our own enterprise …

[Read more]
MariaDB 5.5.33 Now Available

The MariaDB project is pleased to announce the immediate availability of MariaDB 5.5.33. This is a Stable (GA) release. See the Release Notes and Changelog for detailed information on this release and the What is MariaDB 5.5? page in the AskMonty Knowledgebase for general information about the MariaDB 5.5 series.

Download MariaDB 5.5.33

Release Notes Changelog

[Read more]
TokuDB Hot Backup – Part 1

There are multiple ways to backup a MySQL database. Some are more painful than others. In this two part blog we are going to discuss why the new hot backup system in TokuDB is special amidst the existing solutions. First let’s look at existing backup solutions for MySQL and InnoDB.

Let’s start with the most obvious, and possibly painful, way to make a backup of MySQL: a manual copy of the MySQL data directory.

Coarse Copy

The copying itself isn’t the painful part; any DBA worth their salt can copy a directory. Guaranteeing what comes out the other end, however, is difficult. In other words, what will the state of each table in each database look like when the backup is complete? It turns out, without additional help, we don’t know!

If you think about the dynamic state of a database, and the serial copying of the same database files from one space to another, some questions may start to …

[Read more]
Converting an OLAP database to TokuDB, part 2: the process of migration

This is a second in a series of posts describing our experience in migrating a large DWH server to TokuDB. This post discusses the process of migration itself.

As a quick recap (read part 1 here), we have a 2TB compressed InnoDB (4TB uncompressed) based DWH server. Space is running low, and we're looking at TokuDB for answers. Early experiments show that TokuDB's compression could make a good impact on disk space usage. I'm still not discussing performance -- keeping this till later post.

Those with weak hearts can skip right to the end, where we finally have a complete conversion. You can also peek at the very end to find out how much 4TB uncompressed InnoDB data is worth in TokuDB. But you might want to read through. The process was not smooth, and not as expected (it's …

[Read more]
TokuDB vs InnoDB in timeseries INSERT benchmark

This post is a continuation of my research of TokuDB’s  storage engine to understand if it is suitable for timeseries workloads.

While inserting LOAD DATA INFILE into an empty table shows great results for TokuDB, what’s more interesting is seeing some realistic workloads.

So this time let’s take a look at the INSERT benchmark.

What I am going to do is to insert data in 16 parallel threads into the table from the previous post:

CREATE TABLE `sensordata` (
  `ts` int(10) unsigned NOT NULL DEFAULT '0',
  `sensor_id` int(10) unsigned NOT NULL,
  `data1` double NOT NULL,
  `data2` double NOT NULL,
  `data3` double NOT NULL,
  `data4` double NOT NULL,
  `data5` double NOT …
[Read more]
Showing entries 121 to 130 of 329
« 10 Newer Entries | 10 Older Entries »