OpenSQLCamp was a huge success! Not many folks have blogged about what they learned there….if you missed it, all is not lost. We did take videos of most of the sessions (we only had 3 video cameras, and 4 rooms, and 2 sessions were not recorded).

All the videos have been processed, and I am working on uploading them to YouTube and filling in details for the video descriptions. Not all the videos are up right now….right now all the lightning talks are up.

• All the lightning talks belowin one continuous video (just over 54 minutes total)
  
• The Graph Engine (Antony Curtis)


• Cluster/J, a new set of Java APIs to

For the past two months, I have been running tests on TokuDB in my free time. TokuDB is a storage engine put out by Tokutek. TokuDB uses fractal tree indexes instead of B-tree indexes to improve performance, which is dramatically noticeable when dealing with large tables (over 100 million rows).

For those that like the information “above the fold”, here is a table with results from a test comparing InnoDB and TokuDB. All the steps are explained in the post below, if you want more details, but here’s the table:

ActionInnoDBTokuDB
Importing ~40 million rows119 min 20.596 sec69 min 1.982 sec
INSERTing again, ~80 million rows total5 hours 13 min 52.58 sec56 min 44.56 sec
INSERTing again, ~160 million rows total20 hours 10 min 32.35 sec2 hours 2 min 11.95 sec
Size of table on

Since I started doing the occasional consulting job for Open Query, I've seen a lot of MySQL servers that have been installed once and then forgotten about. This gave me the idea to do a short presentation about some basic MySQL server configuration. The first go was at DrupalCampMelbourne and I recently tried (and failed) to cram it into a three minute lightning talk slot at the LUV September meeting.

The title of the talk is (now) MySQL > YourSQL. I chose this not because I think that MySQL is better than the $other_database you use or because I may or may not run a newer version of MySQL on better hardware, but because I use InnoDB and not MyISAM as the default table

Mike Hogan, CEO of ScaleDB spoke at the Boston MySQL User Group in September 2009:

ScaleDB is a storage engine for MySQL that delivers shared-disk clustering. It has been described as the Oracle RAC of MySQL. Using ScaleDB, you can scale your cluster by simply adding nodes, without partitioning your data. Each node has full read/write capability, eliminating the need for slaves, while delivering cluster-level load balancing. ScaleDB is looking for additional beta testers, there is a sign up at http://www.scaledb.com.

Slides are online (and downloadable) at http://www.slideshare.net/Sheeri/scale-db-preso-for-boston-my-sql-meetup-92009

Watch the video online at http://www.youtube.com/watch?v=emu2WfNx4KA or directly embedded here:

I was asked this question recently “I am wanting to create a simple MySQL database consisting of 5 tables”?

While it’s easy to tell people to RTFM, the question does warrant an answer for the MySQL beginner to provide a more specific guidance as to where to start, and what to do. As a expert in MySQL it’s easy to forget how you would describe what to do. Here are my tips to getting started.

Step 1. Download the MySQL 5.1 software for your platform (e.g. Linux, Windows, Mac etc) from MySQL 5.1 Downloads. There are many different versions of MySQL, MySQL 5.1 is the current production version.

Step 2. You will need to install the MySQL software. The MySQL reference manual is the place to go, Chapter 2

This seems quite a trivial question, but developers don’t often know what a MySQL storage engine is and how to determine what storage engine is used for a table.

The first choice is to describe the table with the DESC[RIBE] command. Side Note: people often don’t realize that DESC is a short acceptable version here.

mysql> desc stats;
+---------+---------------------+------+-----+-------------------+----------------+
| Field   | Type                | Null | Key | Default           | Extra          |
+---------+---------------------+------+-----+-------------------+----------------+
| stat_id | int(10) unsigned    | NO   | PRI | NULL              | auto_increment |
| created | timestamp           | NO   |     | CURRENT_TIMESTAMP |                |
| version | tinyint(3)

This is the Sharding for the masses: Introducing the SPIDER storage engine by Giuseppe Maxia, given at OpenSQLCamp, at FrOSCon, in August 2009. These are somewhat live notes, and the slides are available too.

Why sharding? Scaling, of course. The MySQL way to solve this, is replication (even Yahoo! and Google use this).

When the master doesn’t have enough resources to cope with what you do (i.e. large data sets), replication

RethinkDB is all the rage today, as its a Y Combinator funded startup, which also launched a developer pre-alpha today. So what is RethinkDB you ask? Yet-another-MySQL-storage-engine, that’s what. But this time, its tuned for solid-state drives (SSDs), which also happen to be all the rage these days.

Anyway, check them out more, and the materials currently tell me that they’re using append-only algorithms, which allow for live schema changes and hot backups, with instantaneous recovery from power failure. Those are just some of the exciting bits.

What didn’t excite me so much was the fact that you were only getting 32-bit or 64-bit Linux binaries, built against MySQL 5.1.31 and you’ll just

This is the first post in a new series of “Kickfire Basics” blog posts by myself and others here at Kickfire.  This series will review the basics of the Kickfire appliance starting from this post describing how data is stored on disk, to future posts on topics such as loading data into the appliance and writing queries which best leverage the capabilities of the SQL chip.

The Kickfire Equation
Column store + Compression + SQL Chip = performance

The Kickfire Analytic Appliance features the new KFDB storage engine which was built from scratch to handle queries over vast amounts of data.  KFDB is a column store in contrast to most MySQL storage engines which are row stores.  What follows is a description of our column oriented storage engine and how it improves performance over typical row stores.

This post concerns itself with the first part

ScaleDB provides a pluggable storage engine for MySQL that delivers shared-disk clustering . Brian Akers once described ScaleDB as "the closest thing to Oracle RAC for MySQL." The ScaleDB storage engine turns MySQL into a clustered database, where all of the nodes share the same data. It eliminates the need to partition the data. It also allows you to add and remove nodes without interrupting the application. It will (in time) provide high-availability, but we can’t promise that for the beta version.

So we’re looking for companies with problems we can uniquely solve. We view the beta process as an investment on both sides, we invest in supporting you and making you successful and you invest your time using our software. I have found

ScaleDB provides a pluggable storage engine for MySQL that delivers shared-disk clustering . Brian Akers once described ScaleDB as "the closest thing to Oracle RAC for MySQL." The ScaleDB storage engine turns MySQL into a clustered database, where all of the nodes share the same data. It eliminates the need to partition the data. It also allows you to add and remove nodes without interrupting the application. It will (in time) provide high-availability, but we can’t promise that for the beta version.

So we’re looking for companies with problems we can uniquely solve. We view the beta process as an investment on both sides, we invest in supporting you and making you successful and you invest your time using our software. I have found

"New Shimmer is both a floor wax and a dessert topping!" - Chevy Chase, Saturday Night Live

On a Saturday Night Live comedy skit, a husband is arguing that Shimmer is a dessert topping, while his wife insists that it’s a floor wax. Then the Shimmer spokesman-Chevy Chase-explains that it is both. While this juxtaposition is humorous, in many respects this is what databases have been doing for years. Instead of specializing on one specific task, most mainstream databases provide an acceptable combination of performance and capabilities across all use cases. The pluggable storage engine architecture is now changing this, ushering in an era of on-demand specialization.

MySQL allows you to select the storage engine not just for the application, but for each table used by the application. Let me use an analogy to describe how powerful

The spirit and intent of the Free Software Foundation (FSF) and the GPL license are right on target. However, we must be careful to ensure that the GPL license is interpreted in a manner fulfills the spirit and intent behind its framing. Richard Stallman and associates set out to draft a license agreement that ensures that free software remains free. They didn’t want to see open source become corrupted with the insertion of proprietary code that would eat away at the freedoms they envisioned.

To protect the eternal purity of the open source software, they created constraints on how proprietary code can interact with the GPL code. Their one weapon in this battle is the automatic and forced expansion of their GPL license to any code that integrates with the base GPLed code. I often refer to this process as acting like a virus. I don’t use this term to infer

While Zack covered the storage engine and appliances sessions pretty well, I feel he’s missed out on a few important new engines (or engine related talks):

• DDEngine will be represented, speaking about Automated data versioning with a storage engine
• Long time MySQL Community Member Beat Vontobel, will be presenting on the SeqEngine, in Solving Common SQL Problems with the SeqEngine
• Kazuho Oku, from Cybozu Labs in Japan, will be talking about Q4M, a queuing engine for MySQL. Mixi.jp uses it for their Twitter like service,

A couple interesting things coming out of startup land.

For one, Posterous has a little writeup on Building and Scaling a Startup on Rails: 12 Things We Learned the Hard Way. Good things to take away include using Sphinx/Solr for search, but the real important takeaway for the MySQL crowd is Storage engine matters, and you should probably use InnoDB. If you’re writing an application, know your storage engines. There are also bits to tell you how to use query_viewer and New Relic to help you fix database bottlenecks, use memcached later, and more. Its a great read.

Next up, there’s How FriendFeed uses MySQL to store schema-less data. I hope Bret

I had the pleasure of addressing a crowd of over 1,000+ people yesterday, at the Sun Tech Days event in Hyderabad. I think this might as well be the biggest number of attendees at a talk that I’ve given. I spoke on MySQL: The Database for Web 2.0, and the notes for this talk are largely indexed at MySQL for Developers. Its more or less the standard deck for the Tech Days events these days.

The best part? The questions. I had intelligent questions, and they lasted well over twenty minutes, and there was even more chatter afterwards. Twenty minutes might not seem like a lot, but this is Asia, and in some audiences, you’d be hard pressed to get even a single question! MySQL is

I read Tokyo Cabinet: Beyond Key-Value Store today from one of the news sites, and it reminded me of Brian’s hack on Tokyo Cabinet == Tokyo Engine. Looking at TokyoEngine in Brian’s Mercurial repository, there have been no updates in over a year. Is anyone planning on taking up development of this? Tokyo Cabinet looks really interesting, and Brian has already started the enabling of making it a MySQL engine.

If you are using InnoDB Hot Backup and a recent version of mysqld (at least 5.0.67 or higher, including 5.1.30, though it may be later versions), your backup will run fine and output OK! at the end, as it should.

Except for one thing.

The binary log file and position do not appear in their rightful place. Here’s a snippet of the output from the backup:

innobackup: MySQL binlog position: filename 'Warning', position (Code 1287):
'TYPE=storage_engine' is deprecated; use 'ENGINE=storage_engine' instead
090210 03:55:04  innobackup: innobackup completed OK!

That’s pretty misleading — looks like the backup completed OK, but it did not show us the binary log position. What about the ibbackup_binlog_info file?

[mysql@db3:~] more ibbackup_binlog_info
Warning (Code 1287): 'TYPE=storage_engine' is

In a nutshell: What’s New in MySQL 5.1.

Release notes: Changes in release 5.1.x (Production).

And yes, very early on (at about two minutes in), I talk about my take on Monty’s controversial post at Oops, we did it again.

To play the video directly, go to http://technocation.org/node/663/play. To download the 146 Mb video to your computer for offline playback, go to http://technocation.org/node/663/download. The slides can be downloaded as a

Open source the way it ought to be. Today, Percona announced a replacement for InnoDB that improves performance and fixes bugs. The new engine is called XtraDB.

According to Vadim at Percona:

It's 100% backwards-compatible with standard InnoDB, so you can use it as a drop-in replacement in your current environment. It is designed to scale better on modern hardware, and includes a variety of other features useful in high performance environments.

The release is pure GPL (v2) and commercial support is available from Percona. If percona keeps this up, they just might become the new MySQL.

The source