Suomeksi: MySQL käyttäjätapaaminen Helsingissä 1. marraskuuta. Klikkaa allaolevaa linkkiä ilmoittautuaksesi, siellä saat myös lisätietoa suomeksi.

Finally it's here! So many of you have always asked about it. Markus and other Elisa guys. Osma and Ilkka at Habbo Hotel. And others... MySQL was born in Helsinki, InnoDB was born in Helsinki, a lesser known database / also MySQL engine called Solid was born in Helsinki, and 2 great replication companies, Continuent with multiple generations of clustering for MySQL, and Codership with Galera, are Helsinki companies. And amidst this embarrassment of riches, what did we not have?

A MySQL User Group.

read more

Just wanted to blog about some of the latest updates to dbqp.  We just merged some interesting changes into Drizzle (just in time for the impending Fremont beta).  In additional to general code cleanup / reorganization, we have the following goodies:

Randgen in the Drizzle tree

One of the biggest things is that the random query generator (aka randgen) is now part of the Drizzle tree.  While I did some of the work here, the major drivers of this happening were Brian and Stewart:

So, it has been a while since I’ve blogged.  As some of you may have read, I have a new job and Stewart and I have been busy planning all kinds of testing goodness for Percona >: ) (I’ve also been recovering from trying to keep up with Stewart!)

Rest assured, gentle readers, that I have not forgotten everyone’s favorite modular, community-driven database ; )  Not by a long-shot.  I have some major improvements to dbqp getting ready for a merge (think randgen in-tree / additional testing modes / multiple basedirs of multiple types).  Additionally, I’ve been cooking up some code to test the mighty Mr. Shrews’

Both Henrik and myself will be at Percona Live London 2011 in late October speaking on the wonderful Drizzle database server.

Other speakers at the conference will be talking about a wide range of topics surrounding the MySQL ecosystem including performance monitoring, backup, search, scaling and data recovery.

P.S. I do have a discount code – ask me in the comments for it!

Update: I won't be in Moscow after all. I was denied visa on grounds that my passport is beginning to fall apart and there wasn't time to get new passport, invitation and visa. Maybe next year - I was excited to go.

October brings 2 very interesting conferences. I will be speaking first on Oct 3rd at HighLoad++ in Moscow and a few weeks later on Oct Oct 25 at Percona Live in London. I will give a talk called Choosing a MySQL Replication / High Availability Solution which is based on my thinking developed in my recent blog post

Exciting news, Oracle just announced commercial MySQL extensions that they'll be offering paid extensions to the core MySQL free product.

To be sure, this has raised waves of concern among the community, but on the whole I suspect it will be a good thing for MySQL.  This brings more commercial addons to the table, which only increases the options for customers.  Many will continue to use the core database product only, and avoid license hassles while others will surely embark on a hybrid approach if it solves their everyday business problems.

Speaking of licensing, back in May, Amazon announced that it's RDS or Relational Database Service would now offer Oracle as an option.

Just wanted to record for the history books that:


drizzle> select js_eval('var d = new Date(); "Drizzle started running JavaScript at: " + d;')\g
+----------------------------------------------------------------------------------+
| js_eval('var d = new Date(); "Drizzle started running JavaScript at: " + d;') |
+----------------------------------------------------------------------------------+
| Drizzle started running JavaScript at: Mon Aug 29 2011 00:23:31 GMT+0300 (EEST) |
+----------------------------------------------------------------------------------+
1 row in set (0.001792 sec)


I will push this onto launchpad tomorrow, after a good nights sleep and final code cleanups.

read more

It's my turn to apologize. Andrew and I apparently really angered people by being upset about something last week, and for that, as he already has, I apologize. I don't like making people angry or upset.

I believe Henrik made an excellent point, which is that for various different reasons, there are those of us who were upset when Oracle bought MySQL and yet felt complelled to not communicate this publically. To be honest, emotions related to a business transaction ARE a little weird, so I'm not sure it's completely odd that people don't know how to appropriate express them. But as Henrik rightly pointed out, the Oracle takeover has been the elephant in the room (sorry Postgres - it's not you) and we've all been spending a good amount

There's been quite the thread on Google+ (my how technology changes quickly...) over a comment Andrew Hutchings made on an Oracle MySQL Blog Annoucment for their new "Meet The MySQL Experts" Podcast. I should have ignored it - because I honestly could not give two shits one way or the other about Oracle or any podcasts that they may or may not decide to broadcast. But to be straightfoward about it ... the title of the podcast is ludicrous. In case you were wondering, "The" in English is the definite article and implies a singular quality to the thing that it describes... effectively implying that Oracle's MySQL Experts are, in fact, the only MySQL Experts. We all know that's false- Percona and SkySQL are both full of experts as well - likely have more MySQL Experts per-capita than Oracle does, as if a per-capita

It’s notoriously hard to measure the usage of open-source software. Software that’s open-source or free can be redistributed far and wide, so the original creators have no idea how many times it’s installed, deployed, or distributed. As a proxy, we often use downloads, but that’s woefully inadequate.

I’ve recently begun trying to figure out how many job openings are mentioning various open-source projects. I think that this might be a better metric because it’s driven by the end result (usage), rather than intermediate processes (downloads, etc). I think that it’s likely that usage and demand for skilled people is somewhat realistically related.

To be more concrete, I’ve been watching RSS feeds from job posting aggregators for several alternative versions of MySQL: Percona Server, MariaDB, and Drizzle.

In December I covered the topic The state of MySQL forks: co-operating without co-operating (which was also a response to Giuseppe Maxia's take on the same topic). Since half a year has now passed, I was wondering if I should follow up with an update. (Drizzle having a GA release would be the major news in such an update.)

But I see that Peter Zaitsev covered this topic in the opening keynote of their Percona Live conference. Since I agree with Peter's view on this topic, I just recommend you watch the talk on Percona.TV. He also uses the same categorizations of the forks, and includes "community patches"

One of the long term testing goals for Drizzle is to move all of our test logic directly in-tree.  Currently, we use a system called drizzle-automation to execute a variety of tests for our staging branch.  This is the final set of tests patches must pass before being allowed to merge into Drizzle trunk and includes things like sysbench, dbt2, the randgen, etc.  With the development of dbqp, we can now move this testing logic directly into the tree (and even move some of the testing tools there as well).  Of course, I’ve rambled on about this before, but I personally think it is cool and useful ; )  However enough of the sales pitch, on to the new modes!

sysbench mode

With but a simple incantation of ./dbqp –mode=sysbench [--suite=readonly|readwrite], you too can invoke the mighty sysbench configurations that we use to ensure

Today was another example of where a correct SQL_MODE saved customer data from being corrupted. By default, MySQL does not enforce data integrity. It allows what is called silent truncations where the result of what you INSERT or UPDATE does not represent truth. NOTE: I see very few customers ever have this correctly configured, those that do have actually listened to my advice.

If you do not read any further, your production MySQL environments should be running with at the bare minimum of SQL_MODE=STRICT_ALL_TABLES however I would also advocate for additional SQL_MODE settings.

For this example, some modified undesirable code attempted to reduce a counter by 1, however because of an UNSIGNED data type and a correctly set SQL_MODE, the application produced an error and data was not corrupted.

This is what should happen with your SQL.

• Drizzle support now fully works again


• Fixes so mydumper compiles in FreeBSD (thanks to Kirill A. Korinskiy)

Today marks the release of mydumper 0.2.3.  Mydumper is a multi-threaded high-performance data dumper (and loader) for MySQL and Drizzle written in C. This is a minor bugfix release whilst I work on the upcoming 0.5 version. The end goal here will be that there is always a ‘stable’ and ‘development’ version. 0.2 will be the first stable version and will only have bug fixes. 0.5 will be the first ‘development’ version where the next planned set of new features will hit.

So, the changes since 0.2.2 are:

• Drizzle support now fully works again
• Fixes so mydumper compiles in FreeBSD (thanks to Kirill A. Korinskiy)

If you wish to try this version the source can be downloaded

• PBMS is fully integrated with MySQL 5.5:
  PBMS is now provided as a patch for MySQL 5.5 which simplifies installation and provides numerous benefits.
  
  
• All engines are "PBMS enabled":
  PBMS no longer requires that you have a "PBMS enabled" storage engine to be able to use PBMS.
  
  
• The MySQL client lib provides the PBMS client API:
  You no longer need to link your application to a separate PBMS lib to use the PBMS 'C' API.
  
  
• mysqldump understands PBMS BLOB URLS:
  When dumping tables or databases containing PBMS BLOB URLs mysqldump will dump the referenced BLOBs as binary data to a separate file. Since the BLOBs are dumped to their own file there is no need to
  

So, as LinuxJedi so eloquently noted here, Rackspace and Drizzle are parting ways.  While they were kind enough to offer other opportunities with them, my preferences were similar to Andrew’s – to remain in the MySQL/Drizzle world.

I was fortunate enough that SkySQL had need of my services and am happy to announce that today marks my first day as a Senior QA Engineer with them.  I am very honored to join such a promising and skilled group and am very excited about the opportunities ahead.

My work will have me developing QA solutions (tests, code, whatever) for a wide variety of things.  Naturally, this includes SkySQL’s Reference Architecture where I will work on tests to ensure the

Mydumper 0.2.2 has been released today with a number of fixes and new features.  The one that most people have been asking for is consistent snapshots for non-InnoDB tables (such as MyISAM).  We have been able to achieve this without locking the database for the entire backup using the following method:

I have also started work on Drizzle support this week.  The Drizzle support is not entirely complete

I love STL containers in C++. They're so much better in general to work with than their C counterparts. The bit that's missing though is an instantiation-time optimized version of map.

Take, for instance, reading a set of commands at startup from a set of dynamically loaded plugins. It's not going to change, in this hypothetical case, because we neither load nor unload plugins after startup. But it's dynamic in the sense that we do not know it at compile time - so doing something like gperf to generate a perfect hash is out of the question.

But why should we pay the lookup cost on a dynamic container for every lookup if I can determine, based on program flow, that the contents of the container are not going to change once it's instantiated.

Something like:

const perfect_map(some_dynamic_map_I_built);

Where the constructor would do a perfect hash generation once, and the

As you may have read on the MySQL Performance Blog post – I’ve recently joined Percona. This is a fairly exciting next step. I’ll be in New York for Percona Live next week, where I’ll be giving a session titled “Drizzle 7, GA and Supported: Current & Future Features”.

I’ll write more soon, there’s a lot keeping me busy already!

A week ago Baron wrote a blog post which can only be described as the final nail in the coffin for MMM. At MySQL AB we never used or recommended MMM as a High Availability solution. I never really asked about details about that, but surely one reason was that it is based on using the MySQL replication. At MySQL/Sun we recommended against asynchronous replication as a HA solution so that was the end of it as far as MMM was concerned. Instead we recommended DRBD, shared disk or MySQL Cluster based solutions. Of course, to replicate across continents (geographical redundancy) you will mostly just use asynchronous replication, also MySQL Cluster used the standard MySQL replication for that purpose.

read more

Why would one want to do this, you may ask?  Well, for starters, it makes a great ‘canary-in-the-coal-mine‘ in regards to backwards compatibility!

For Drizzle, we’ve created some tables (via the randgen’s data generator if you are curious), saved a copy of the datadir, and then created a test case that uses said datadir for the test server.  The test executes some simple SQL queries to make sure we can read the tables properly.  This way, if we ever do something to either the server or .dfe format (data format exchange – had a most enlightening conversation with the team about this

https://code.launchpad.net/~stewart/drizzle/json-interface/+merge/59859

Currently a very early version of course, but it’s there in trunk if you want to play with it. Just have libcurl and libevent installed and you can submit queries via HTTP and JSON. Of course, the next steps are getting a true non-sql interface going and seeing how people go with it.

In March I posted a series of blog posts on my paternity leave MepSQL project, which I called MepSQL. There was still one piece created in the MepSQL buildsystem that I didn't publish or blog about. Since it is generally useful, I wanted to generalize and polish it and publish it separately. I finally had that done last week, when I also found that somebody else, namely alestic.com already published a similar solution 2 years ago. So yesterday I ported my BuildBot setup to use that system instead and am happy to publish it at the Open DB Camp 2011 in Sardinia.

Ok, so let's go back a little... What is the

shell> bar -if=data.sql | mysql

A couple of years ago I wrote a patch for the MySQL command line client which shows the progress of a import as it happens (I also created a similar patch for mysqldump which later made it into Drizzle).  I don’t have the blog archives from back then but Harrison Fisk commented suggesting I use a utility called ‘bar‘ instead.

The ‘bar’ utility actually is a lot better than the patch I wrote and I highly recommend it when you are importing a large dump file.  To use it simply run:

shell> bar -if=data.sql | mysql

This will generate an output such as:

If you are using Ubuntu then it is a

I've been thinking a bunch about email and mailing lists recently... about how to deal with them, how to follow the ones that are important but not get drowned in a sea of email noise. I have a non-tech email account which is where friends mail me personally and where I handle theatre-related stuff. I do not receive a stupid amount of email on this account, and I am almost always interested in reading the messages that arrive there. So on my inaugust.com email address, I've been considering unsubscribing from all mailing lists and setting both Launchpad and github to stop emailing me most things.

For Launchpad and github, most things I care about have status pages of some sort: lists of bugs or branches I need to attend to or whatnot. I really don't need to get an email every time someone files a bug - it's noise.

Mailing lists are the trickier one, because so many open source projects are