This is the first part of many posts in a series of blog posts where I want to document how the MepSQL packages were built. By doing that, I will also end up covering the MariaDB build system (which this is based on), some of BuildBot, Amazon EC2 cloud and packaging DEBs and RPMs just in general, so it could be interesting from many perspectives. In this first part I'll simply scribble some notes about reviewing the OpenSuse Build System, Launchpad PPA service vs using your own servers and automating the builds with BuildBot.

Originally I just wanted to work on some new ideas on the automated build and QA system used by MariaDB. But since leaving Monty Program I didn't have access to any of those servers anymore, so as a first step I had to look into what alternatives there are for building binary packages for many …

After another week of hacking on MepSQL the DEB files for Ubuntu are now available.(MepSQL is my new "just a hobby" MySQL fork project.)

The Download page has instructions on how to install the packages with a simple apt-get install command. Debian packages will appear soon as they are now easy to add - I mostly just need to add new Amazon images for each.

read more

Monty Program AB have recently released a tar.gz package of MariaDB for Solaris 10 x86, something I have been waiting on for a while. Thank you guys!

Download here — Note that this is a beta release so use with caution and not in prod unless you know what you are doing or nuts. If you have any issues MPAB can be contacted here.

[mysql@dc /mysql/mariadb-5.1.53-solaris10-i386 03:15:27]$  uname -a
SunOS dc 5.10 Generic_141415-08 i86pc i386 i86pc
[mysql@dc /mysql/mariadb-5.1.53-solaris10-i386 03:15:33]$  bin/mysql -uroot
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 5
Server version: 5.1.53-MariaDB-log Source distribution

This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under …

Some time ago Stephen O'Grady and Brian Aker had an interesting Blogo-dialogue about what they call the "Cambrian Explosion" of open source development. The Cambrian Explosion means that we increasingly see forks of projects being developed in different directions, where traditionally we are used to open source development happening in relatively hierarchical and easy to follow upstream and downstream relationships. This is exactly what happens in the MySQL community currently, where in total there is more progress than ever before, but that progress is divided among several competing forks, none of which is strictly in an upstream-downstream hierarchy with …

Via SpamExperts:

SpamExperts has their own anti-spam filtering cloud which is provided as a software-as-a-service (SaaS) model. In addition they have an e-mail security product which they install, update and monitor on-site. Most of SpamExperts’ anti-spam technology has been developed in-house and makes extensive use of the MySQL database. All clients contribute in real-time to their filtering effectiveness, so they have many different data flows that are handled by MySQL. Replication is used to synchronize the data between the systems in a cluster and to push data feeds in real-time. The SaaS cloud is replicated across four countries for redundancy, whereas client installations spread the data retrieval …

On Thursday January 27th at 10am PST I will doing a webinar on Tungsten Replicator together with my colleague Giuseppe Maxia.  The title is "What MySQL Replication Cannot Do.  And How to Get Around It."  Basically it is a nuts and bolts description of Tungsten Replicator capabilities like multi-master replication, failover, parallel apply, and using replication for zero-downtime upgrade.  If you have ever wanted an in-depth look at the Tungsten Replicator this is a good opportunity. 

During 2010 we implemented an amazing number of new replication features ranging from pipelines early in the year to fast disk logs, multiple replication services per process, bi-directional replication, and parallel apply by the end.  We will be building out all of …

As Hakan mentioned previously, the full text of SQL-99 Complete, Really by Peter Gulutzan and Trudy Pelzer, is now in the AskMonty.org Knowledgebase. Importing the text and formatting it for the Knowledgebase was a major project and I’m glad that it’s done.

Having the full text of this book freely available is a great thing for anyone who uses SQL because the book is about the SQL-99 standard and not about any particular database implementation. They do talk about different implementations, but those sections are clearly marked as such, and serve as examples of  how some databases implement (or diverge from) the standard.

The question now is: What’s …

A couple of days ago I ran into a Tungsten Replicator case where several MySQL tables became corrupted on slaves and needed to be restored from the master.   We identified the tables that had problems fairly quickly using Tungsten Replicator's consistency checks.  However, that led to another problem:  how to restore the slave tables efficiently from the master.  The MySQL server in question processes around 10M tranactions per day--there is virtually no downtime.  Though the tables were not large, we could not be sure whether they were in use. 

Fortunately, you can use a simple MySQL trick to get all the rows of a table to replicate through to slaves.  The idea is to dump the table, delete the rows, then reload it again.  The delete and subsequent reload replicate out to slaves, after which everything is consistent again.  Let's say we have a table called tpcb.history that needs to be …

Following a previous post using MySQL as a NoSQL store with the native Handler Interface, some  refactoring have now been coded in MariaDB-5.3 as described in the following worklog   On concurrency,  doing less work in HANDLER READ to put more caching on HANDLER OPEN is credited with a 7% performance gain.
Source code via bzr branch lp:~maria-captains/maria/5.3-handler
More, HANDLER READ can now be used with prepared statement to save precious CPU cycles on parsing.    

handler t1 open;
prepare stmt from 'handler t1 read a=(?)';
set @a=1000;
execute stmt using @a;
deallocate prepare …

I just received a confirmation that my presentation proposal for the MySQL user conference 2011 was accepted! The title for my proposal is MQL-to-SQL: a JSON-based Query Language for RDBMS Access from AJAX Applications, and it covers pretty much everything implied by the title.

As always, the Hyatt Regency Hotel in Santa Clara, California serves as the venue. The conference will be held from April 11-14. Except for the venue and period, I think this year's conference will bear few similarities to previous editions. Let me try and explain.

This year's theme is "MySQL, the ecosystem and …