So, what impact does enabling the slow query log have on MySQL?
I decided to run some numbers. I’m using my laptop, as we all know the currently most-deployed database servers have mulitple cores, SSDs and many GB of RAM. For the curious: Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz
The benchmark is going to be:
mysqlslap -u root test -S var/tmp/mysqld.1.sock -q 'select 1;' --number-of-queries=1000000 --concurrency=64 --create-schema=test
Which is pretty much “run a whole bunch of nothing, excluding all the overhead of storage engines, optimizer… and focus on logging”.
My first run was going to be with the slow query log on. I’ll start the server with mysql-test-run.pl as it’s just easy:
I promised to still post some general comments about the MySQL ecosystem, to conclude my outlook of State of the MySQL forks and Drizzle. I will do this now in the form of answering questions I got in the comments, twitter and some that I make up just myself.
I got several comments and questions on my previous blog "The State of the MySQL forks". One question was "Why didn't you mention Drizzle?" So I will say something about Drizzle here before concluding with other remarks.
So why didn't you mention Drizzle?
Mainly because the post was already long and also I had to wrap up and call into a meeting.
It's been some time since I last wrote an overview of the state of the MySQL forks, but the last few weeks have been eventful enough that it is a good time to again see how the competing variants are positioned against each other.
I have written on this topic 1-2 times a year. Here are links to the previous overviews:[Read more...]
Just for the pure insane fun of it, I accepted the challenge of “what can you do with the text format of the schedule?” for BarCampMel. I’m a database guy, so I wanted to load it into a database (which would be Drizzle), and I wanted it to be easy to keep it up to date (this is an unconference after all).
For Drizzle and for all of the projects we work on at Percona we use the Bazaar revision control system (largely because it’s what we were using at MySQL and it’s what MySQL still uses). We also use Jenkins.
We have a lot of jobs in our Jenkins. A lot. We build upstream MySQL 5.1, 5.5 and 5.6, Percona Server 5.1, Percona Server 5.5, XtraBackup 1.6, 2.0 and 2.1. For each of these we also have the normal trunk builds as well as parameterised ones that allow a developer to test out a tree before they ask for it to be merged. We also have each of these products across seven operating systems and for each of those both x86 32bit and 64bit. If we weren’t already in the[Read more...]
I have a friend who is fond of telling a story from way back in November 2008 at the OpenSQL camp in Charlottesville, Virgina. This was relatively shortly after we had announced to the public that we’d started something called Drizzle (we did that at OSCON) and was even closer to the date I started working on Drizzle full time (which was November 1st). Compared to what it is now, the Drizzle code base was in its infancy. One of the things we hadn’t yet sorted out was the rewrite of the replication code.
So, I had my laptop plugged into a projector, and somebody suggested opening up some random source file… so I did. It was a bit of the replication code that we’d inherited from MySQL.[Read more...]
Taxonomy of Database Tools
In the MySQL ecosphere there is an ecosystem of tools. Like real-world ecosystems, the “creatures” in the MySQL tools ecosystem can be classified and organized by a taxonomy. There are already multiple taxonomies of software bugs (e.g. A Taxonomy of Bugs), but as far as I know this is the first Taxonomy of Database Tools. A taxonomy of database tools serves useful purposes, as discussed in the previously linked page. For me, the most useful purpose is the high-level ecosystem view which I[Read more...]
A few weeks ago I blogged about the HTTP JSON api in Drizzle. (See also a small demo app using it.) In this post I want to elaborate a little on the design decisions taken. (One reason to do this is to provide a foundation for future work, especially in the form of a GSoC project.)
This is the second and final part of my notes from the MySQL conference. In this part I'll focus on the technical substance of talks I saw, and didn't see.
More than ever before I was a contributor rather than attendee at this conference. Looking back, this resulted in seeing less talks than I would have wanted to, since I was speaking or preparing to speak myself. Sometimes it was worse than speaking, for instance I spent half a day picking up pewter goblets from an egnravings shop... (congratulations to all the winners again :-) Luckily, I can make up for some of that by going back and browse their slides. This is especially important whenever 2 good talks are scheduled in the same slot, or in the same slot when I was to speak. So I have categorized topics here along various axes, but also along the "things I did see" versus "things I missed"[Read more...]
I have finally recovered from my trip to Santa Clara enough that I can scribble down some notes from this year's MySQL Conference. Writing a travel report is part of the deal where my employer covers the travel expense, so even if many people have written about the conference, I need to do it too. And judging from the many posts for instance from Pythian's direction, Nokia is perhaps not the only company with such a policy?
There has usually always been something that can be called a "soft keynote". Pirate Party founder Rick Falckvinge speaking at a database conference is a memorable example (I still keep in touch with him, having met him at the Hyatt Santa Clara). This year there was one less day, and therefore less keynotes. The soft keynote was therefore taken care of by Baron using some time out of Peter's opening keynote.[Read more...]
I decided to take a look at Drizzle today and was encouraged by what I saw. Here’s my favorite part:
There is no UNSIGNED (as per the standard). * There are no spatial data types GEOMETRY, POINT, LINESTRING & POLYGON (go use Postgres). * No YEAR field type. * There are no FULLTEXT indexes for the MyISAM storage engine (the only engine FULLTEXT was supported in). Look at either Lucene, Sphinx, or Solr. * No “dual” table. * The “LOCAL” keyword in “LOAD DATA LOCAL INFILE” is not supported
GO USE POSTGRES. Awesome.
So yesterday I introduced the newly committed HTTP JSON key-value interface in Drizzle. The next step of course is to create some simple application that would use this to store data, this serves both as an example use case as well as for myself to get the feeling for whether this makes sense as a programming paradigm.
Personally, I have been a fan of the schemaless key-value approach ever since I graduated university and started doing projects with dozens of tables and hundreds of columns in total. Especially in small projects I always found the array structures in languages like PHP and Perl and Python to be very flexible to develop with. As I was developing and realized I need a new variable or new data field somewhere, it was straightforward to just toss a new[Read more...]
Oh boy, I'm starting to feel the stress of having to prepare a little bit of this and a little bit of that for the upcoming MySQL User Conference (Santa Clara, April 10 to 13). But I wanted to also jump on this meme and list a few sessions I definitively want to attend:
I'm speaking, so I suppose I need to attend:
Now that the snow is melting and spring is in the air, the SkySQL Team is hitting the road and making the rounds of key industry events, trade shows, and meetups around the globe. Come meet the team, pick-up a few tips and tricks for using the MySQL database, network with your peers, and learn more about SkySQL’s products and services. Here are some the events we’ll be at this spring:
BIG Data, A New Horizon for Data Analysis
March 20 - 21, 2012
Cité Internationale Univeritaire de Paris, Paris, France
March 28-29, 2012
Columbia Metropolitan Convention Center, Columbia, South Carolina
A week ago I again had the pleasure to give a guest lecture at Tampere University of Technology. I've visited them the first time when I worked as MySQL pre-sales in Sun.
To be trendy, I of course had to talk about the cloud. It turns out every section has the subtitle "...and why it is more difficult for databases". I also rightfully claim to have invented the NoSQL key-value development model in 2005.
Today I'm coming out of the closet. Since I'm a professional database expert I try to be like the mainstream and use the commercial MySQL forks (including MySQL itself). But I think those close to me have already known for some time that I like community based open source projects. I cannot deny it any longer, so let me just say it: I'm a Drizzle contributor and I'm very much engaged!
I've been eyeing the Drizzle project since it started in 2008. Already then there were dozens of MySQL hackers for which this project was a refuge they instantly flocked to. Finally a real open source project based on MySQL code that they could contribute to, and they did. It was like a breath of fresh air in a culture that previously had only accepted one kind of relationships: that between an employer and an employee. Drizzle was more liberal. It accepted also forms of engagement already common in most other[Read more...]
Henrik has already posted it over on the Drizzle Blog, but I thought I’d give a shout out here too.
We’re holding a Drizzle Day right after the Percona Live MySQL Conference and Expo in April. So, since you’re all like me and don’t book your travel this far in advance, it’ll be easy to stay for the extra day and come and learn awesome things about Drizzle.
I’m also pretty glad that my employer, Percona is sponsoring the event.
Usually people do this around New Year, I will do it in February. Actually, I was inspired to do this after reviewing all the talks for this year's MySQL Conference - what a snapshot into the state of where we are! It made me realize we've made important progress in the past year, worth taking a moment to celebrate it. So here we go...
In the past few years there was a lot of fear and doubt about MySQL due to Oracle taking over the ownership. But if you ask me, I was more worried for MySQL because of MySQL itself. I've often said that if MySQL had been a healthy open source project - like the other 3 components in the LAMP stack - then most of the NoSQL technologies we've seen come about would never have been started as their own projects, because it would have[Read more...]
I’m speaking at Percona Live MySQL Conference & Expo 2012. My two talk are: Getting Started with Drizzle 7.1 and Verifying MySQL Replication Safely With pt-table-checksum 2.0. No, there’s no relationship between those topics; they’re just things I know well.
I’ve been stalking Drizzle for many years. When it went GA last year, I began hacking Drizzle, focusing on plugins which give it nearly all its functionality. Recently, I helped overhaul the configuration,[Read more...]
Selena and I gave a talk on the various issues of running databases “in the cloud” at the recent linux.conf.au in Ballarat. Video is up, embedded below:
In my previous post I explained why I believe the production of RPM and DEB packages should be more integrated with the rest of your development process. Now it's time to look into how you can put the RPM build scripts inside your main source code repository, and in particular how I did that to produce RPM packages for Drizzle.
Last weekend I released rpm files for the latest Drizzle Fremont beta (announcement). As part of that work I've also integrated the spec file and other files used by the rpmbuild into the main Drizzle bzr repository (but not yet merged into trunk). In this post I want to explain why I think this is a good thing, and in a follow up post I'll go into what I needed to do to make it work.
(And speaking of stuff you can download, phpMyAdmin 3.5.0-alpha1 now supports Drizzle!)
First of all, I wish you a happy new year.
Many things happened last year, it was really exciting to be involved in the MySQL ecosystem.
I hope this enthusiasm will be increased this year, up to you !
To start the year, I propose the MySQL[plus] Awards 2011
It will only take 5 minutes to fill out these polls.
Answer with your heart first and then with your experience with some of these tools or services.
Polls will be closed January 31, so, vote now !
For “other” answers, please, let me a comment with details.
Don’t hesitate to submit proposal for tools or services in the comments.[Read more...]