Architecture of Maria: A New Storage Engine with a Transactional Design

Goals of Maria:

• 
  
• ACID compliant
  
• MVCC, transactional
  
• default non-transactional and default transactional storage engine for MySQL
  
• MyISAM replacement, including temporary table use
  
• Storage engine good for data warehousing.
  
• Allow any length transactions to take place
  
• all indexes should have equal speed (clustered indexes are not in the plan)
  
• log shipping — incremental backups just by copying the logs
  
• used as a standalone library
  
• fast count(*)
  
• allow copying of Maria tables between different Maria servers
  
• Better blob handling (than MyISAM) — no memory copying, or extra memory used for blobs on INSERT/UPDATE
  
…

The solidDB for MySQL database engine for MySQL may have lost its sponsor following IBM’s acquisition of Solid Info Tech but events at this week’s MySQL Conference and Expo prove the certified engines program is alive and well.

Not only has Oracle announced that its Innobase subsidiary has updated InnoDB transactional storage engine, but there is also a new member of the certified engines program.

Kickfire has recently …

Today Robin Schumacher, MySQL's Director of Product Management, announced that the mysql Falcon storage engine has moved into a beta release stage. Falcon, a new transactional storage engine introduced in mysql 6 (aka 5.2), has been in alpha for years. Other popular storage engines include MyISAM, InnoDB, which Falcon is supposed to challenge (successfully? :-/), and the upcoming Maria.

Falcon …

Someone asked me what applications were good/bad for MySQL Cluster. As I’ve now actually had experience with a Cluster setup and a real-life application of it, and dug through the manual, I present a few characteristics of applications that will work with Cluster, and why they are so (so that if you have an [...]

Ever since I submitted the "External Language Stored Procedures" project which Eric and myself have been working on to be listed on FreshMeat, it has gained a small number of interested users and we have just started to get some feedback. Yay! Always nice to know when people are using your code. Okay, so one of the emails contained a feature request: Support for PHP stored procedures. I think I

Brian mentioned Gemini on his blog... As many people remember, there was a stunned silence for a while when Oracle purchased InnoBase OY only 2 years ago... Very similar to the stunned silence about 6 years ago when the suits at NuSphere tried to do a SCO. Back on to the topic at hand... Gemini. About two years ago, Brian fed me a copy of the Gemini engine saying that "Hey, you remember Gemini? Here's the code. Get it working. Don't talk about it and call it 'Amira'.". Perhaps not exactly his words but close enough.

So the Amira project had a shaky beginning... I ported the code to work on MySQL 5.0 and started examining it, removing platform-specific code, and improving the performance of it. From the first working run to the last, I think performance improved by about a factor of 10 but I don't remember specifics. The Amira project was cancelled 2Q2006. I suppose …

Brian postulates the idea of a "Queue" engine and why it is impractical. I agree that a queue storage engine doesn't really work too well. What would work better would be either a special 'queue' object type which may use any user accessible storage engine or perhaps a non-standard extension to the SQL syntax:

DEQUEUE message FROM queue_table LIMIT 1;

This can be done in (perhaps) the MySQL Proxy, where it is translated into a SELECT/DELETE combination or it can be added to the server, and should not cause much parser complications.


Then any storage engine may be used for a queue, purely dependent upon the user's requirements as some people will be satisfied with a non-persistent MEMORY based queue, MYISAM queues will be adequate for many and for a few who need …

What a surprisingly busy week.

Spent quite a lot of time tracking down a linking problem on Itanium. It appears that GNU ld 2.14 has some serious deficiencies. GNU ld 2.17 (the current stable release) does fix the problem. Unfortunately, the last supported version of GNU ld on RedHat AS 3 for Itaniam is version 2.14.90.0.4.20030529 - which clearly cannot contain the fix which was written in November of 2003. I suppose what to do is now a decision to be made by someone else.

Also spent some time with Federated - Bulk insert support within the storage engine and also how to make Federated play better with transactions and network overhead. Have to put together a new WorkLog entry if I cannot find a relevant one. I need to work on writing better changeset comments. Ingo says, and I have to admit that I agree a bit, that my comments assume that the reader is familiar with the code and bug in question. I need to make them a …

The 2007 MySQL Conference is over, and I finally made it back home. I have some notes on some of the sessions, which really aren't that great, so if you want to see what you missed, you should read Planet MySQL. But I will give some of the highlights.

There's a lot of new development around storage engines.

MySQL-5.1 has a pluggable storage engine architecture which allows you to load and unload storage engines while the server is running. Brian Aker explained that this is for cases where you have a stable server setup and only want to upgrade the storage engine. All the storage engines in 5.1 are pluggable, and there are already some third-party proprietary storage engines available.

One of the relatively new third-party storage engines is SolidDB. Solid has been around for quite awhile. In fact, I was using Solid for a project in the late 1990's …

I find myself thinking about Federated and about how to improve it's connection pooling. Whatever is done should be made safe for transctions. Unfortunately, we cannot suspend and join XA transactions which is a pity as it would make it much easier.