I have recently created a new library called libMariaS3 for a couple of teams at MariaDB which provides a simple C API to Amazon’s S3. This was created because we needed a library which could link to GPLv2 software and Amazon’s own library is Apache 2.0 licensed which is incompatible.

It is not a perfect code base because I had a very short amount of time to get the first release out but it was a very fun project to work on. It led me to take a quick look this morning at a couple of other things I have created in the past to see where they are at today.

libdrizzle-redux

Around 5 years ago I worked for HP’s Advanced Technology Group and I worked on several different Open Source projects during that time. One of those projects was called libAttachSQL which was a …

Several years ago there was a fork of the unreleased MySQL 6.0 called Drizzle. It was designed to be a lightweight, cloud/web/UTF8 first database server with a microkernel style core. I worked for a while as one of the core developers of Drizzle until the corporate sponsor I worked for ceased funding its development.

Fast-forward to 2016 and I start working on MariaDB ColumnStore and one of the biggest surprises to me is that it incorporated part of Drizzle! Specifically the BSD licensed MySQL/MariaDB compatible client library called libdrizzle.

ColumnStore’s MariaDB plugin gets the entire query plan tree for a query and passes it on to its internal processes to break it up into parts that can be worked on in parallel. Since this doesn’t happen inside MariaDB server it needs a way to get at data that is not part of ColumnStore (such as …

The new MySQL 8.0.0 milestone release that was recently announced brings something that has been a looooong time coming: the removal of the FRM file. I was the one who implemented this in Drizzle way back in 2009 (July 28th 2009 according to Brian)- and I may have had a flashback to removing the tentacles of the FRM when reading the MySQL 8.0.0 announcement.

As an idea for how long this has been on the cards, I’ll quote Brian from when we removed it in Drizzle:

We have been talking about getting rid of FRM since around 2003. I remember a drive up to northern Finland with Kaj Arnö, where we spent an hour …

With JSON functionality, alternate protocols (HTTP, memcache), a move towards saner defaults and crash safety, pluggable logging etc it really looks like MySQL is following what we did in Drizzle years ago, which is great!

Recently, Morgan has been writing on deprecating some MySQL features and inspired by that while working on MySQL on POWER, I wondered “What is the impact of the MySQL query cache on modern hardware?”

We’ve known for over six years (since before we started Drizzle) that the query cache hurt performance. It was for that reason that the query cache was one of the early things to be removed from Drizzle, it just didn’t scale on multi core systems that  we were targeting.

So what about modern hardware? While working on MySQL 5.6 on POWER8, I enabled the query cache and ran a benchmark. …

I’ve previously covered MySQL 5.6 on POWER (with patch), MySQL 5.6 Performance on POWER8 (spoiler: new performance record) and MySQL 5.7 on POWER.

Of course, The postings on this site are my own and don’t necessarily represent IBM’s positions, strategies or opinions. Also, these numbers should be considered preliminary, but trust me – I did get them and it’s not April 1st.

From my last post, you saw that with my preliminary patch for MySQL 5.7 to work on POWER, we could easily match the previous record for sysbench point select …

There’s a pattern I keep seeing in threaded programs (or indeed multiple processes) writing to a common log file. This is more of an antipattern than a pattern, and is often found in code that has existed for years.

Basically, it’s having a mutex to control concurrent writing to the log file. This is something you completely do not need.

The write system call takes care of it all for you. All you have to do is construct a buffer with your log entry in it (in C, malloc a char[] or have one per thread, in C++ std::string may do), open the log file with O_APPEND and then make a single write() syscall with the log entry.

This works for just about all situations you care about. If doing multi megabyte writes (a single log entry with multiple megabytes? ouch) then you may get into trouble on some systems and get partial writes (IIRC it may have been MacOS X and 8MB) and O_APPEND isn’t …

It may not be surprising that there’s been a few projects over the years that I’ve worked on where we’ve had to care about stack usage (to varying degrees).

For threaded userspace applications (e.g. MySQL, Drizzle) you get a certain amount of stack per thread – and you really don’t want to bust that. For a great many years now, there’s been both a configuration parameter in MySQL to set how much stack each thread (connection) gets as well as various checks in the source code to ensure there’s enough free stack to do a particular operation (IIRC open_table is the most hairy one of this in MySQL).

For the Linux Kernel, stack usage is a relatively (in)famous problem… although by now just about every real problem has been fixed and merely mentioning it is probably just the influence of the odd grey beard hairs I’m pretending not to notice.

In a current project I’m working on, it’s also something we have …

So, back in late 2008, rather soon after we got to start working on Drizzle full time, someone discovered unicodesnowmanforyou.com, or:

☃

Since we had decided that Drizzle was going to be UTF-8 everywhere,(after seeing for years how hard it was for people to get character sets correct in MySQL) we soon added ☃.test to the tree, which tried a few interesting things:

CREATE TABLE ☃; CREATE DATABASE ☃; etc etc

Because what better to show off UTF-8 than using odd Unicode characters for table names, database names and file names. Well… it turns out we were all good except if you attempted to check out the source tree on Solaris. It was some combination of Python, Bazaar and Solaris that meant you just got python stacktraces and no source tree. So, if you look now it’s actually …

Yesterday someone opened a Launchpad question asking "is Drizzle dead?".  I have answered that question on Launchpad but wanted to blog about it to give a bit of background context.

As I am sure most of the people who read this know, Drizzle is an Open Source database which was originally forked from the alpha version of MySQL 6.0.  At the time it was an extremely radical approach to Open Source development, many features were stripped out and re-written as plugins to turn it into a micro-kernel style architecture.  Every merge request was automatically throughly tested on several platforms for regressions, memory leaks and even positive/negative performance changes.

In fact Drizzle has influenced many Open Source projects today.  Openstack's Continuous …