We’ve all heard the term “Big Data” thrown around a fair amount in the last several years ever since the rise of Hadoop and other distributed storage methods. But defining “Big Data” has always been a subjective term that hinges on perspective; what one engineer considers big can be vastly different than another’s.

However, there’s finally a definite description that says Big Data no matter what perspective you operate from: “That facility by my calculations that I submitted to the court for the Electronic Frontiers Foundation against NSA would hold on the order of 5 zettabytes of data. Just that current storage capacity is being advertised on the web that you can buy. And that’s not talking about what they have in the near future.” You can read more about the facility and its purpose here: …

***This post is out of date and old. I have just migrated to blogger and this post is displaying as recent. Apologies***

Whenever I talk about MySQL performance my first recommendation is normally something along the lines of “Use the InnoDB storage engine” and I always get asked the same two questions 1) Why use InnoDB over MyISAM? and 2) Isn’t MyISAM faster? The short answers to these questions are:
1) There’s rarely any reason not to.
2) No……….(pause) well sometimes… in most cases no.
In this post I will aim to explain my choice of InnoDB and try and loosely define the cases where MyISAM may be better suited for your application. I am also very aware that there are many MySQL storage engines, but I am just going to cover the big two here.
First I am going to start with the reasons that InnoDB is better suited than MyISAM:
Data safety - I cannot stress …

“Some people even think that “Hadoop” and “Big Data” are synonymous (though this is an over-characterization). Unfortunately, Hadoop was designed based on a paper by Google in 2004 which was focused on use cases involving unstructured data (e.g. extracting words and phrases from Webpages in order to create Google’s Web index). Since it was not [...]

awesome, thank for this very informative arcticle

We'll be running 3 webinars next week and hope many of you will be able to join us:

MySQL Replication: Simplifying Scaling and HA with GTIDs

Wednesday, December 12, at 15.00 Central European Time

Join the MySQL replication developers for a deep dive into the design and implementation of Global Transaction Identifiers (GTIDs) and how they enable users to simplify MySQL scaling and HA. GTIDs are one of the most significant new replication capabilities in MySQL 5.6, making it simple to track and compare replication progress between the master and slave servers.

Register Now

MySQL 5.6: Building the Next Generation of Web/Cloud/SaaS/Embedded Applications and Services …

You may have already seen the announcement MariaDB Foundation to Safeguard Leading Open Source Database. We at Open Query wholeheartedly support this (r)evolution of the MySQL ecosystem, which appears to be increasingly necessary as Oracle Corp is seriously dropping the ball with security updates and actually just general development and innovation. Oracle has actually done some very good work, I happily acknowledge that – but security issues are critical, having crashing bugs and incorrect query results in a .28 of a GA release is uncool, and not incorporating awesome development efforts by the community is just astonishing.

MariaDB is where the Sphinx fulltext …

Here are my slides from the tutorial I gave at Percona Live UK.


Scaling MySQL Strategies for Developers 

I could not find a way to upload my slides for today's talk entitled "MySQL Compatible Open Source Connectors" on the Percona Live website so the PDF can be viewed on Slideshare.  Enjoy!

For years now I have been confusing Stripe Segment Size with Stripe Size, when configuring my RAID arrays. I always thought that Stripe Size is the number of KB of a file written per disk before moving to the next disk in the array. So, if I had a stripe size of 16KB and I am writing a 20KB file-16KB would be on the first disk, 4KB would be on the second. This concept of moving to the next data bearing disk in the range is notthe stripe size. This is the stripe segment size; in lays the confusion. The stripe size is 16K * Number of spindles. So when setting the stripe size in your raid array its the stripe segment size * the number of spindles. In this example of setting 16KB stripe size the stripe segment size is 16KB/# of data bearing disks.
After reading a pdf and …

People say that to have a good vacation, you need to do something else, something you don't do every day at work.
So, instead of hacking on Tarantool, I did some good old MySQL hacking. Thanks to Alexey Rybak from Badoo I had a nice opportunity for it -- a task to improve MySQL user level locks.

GET_LOCK() function in MySQL allows a connection to hold at most one user level lock. Taking a new lock automatically releases the old lock, if any.
The limit of one lock per session existed since early versions of MySQL didn't have a deadlock detector for SQL locks. MDL patches in MySQL 5.5 added a deadlock detector, so starting from 5.5 it became possible to take multiple locks in any order -- a deadlock, should it occur, would be detected and an error returned to the client that closed the wait chain.
So, thanks to MDL, implementing user level locks seemed to be an easy task, …