In a previous post, I described how to use natural database compression on sets to reduce their footprint in the database by removing duplicates and replacing them with a single row and a count column. I also showed an easy way to detect if any subset of numbers in one set exists in another set in a very efficient way using a table encoded using that method. I also showed a special case that allowed you to find any two numbers that summed to zero. That example was a case of the P vs NP problem subset sum problem. That example was a foothold on a way to efficiently test a decision problem on a data set of virtually any size. I then set about solving the subset sum problem because it is easily understood and simple to solve once you …

The most useful feature of the relational database is that it allows us to easily process data in sets, which can be much faster than processing it serially.

When the relational database was first implemented, write-ahead-logging and other technologies did not exist. This made it difficult to implement the database in a way that matched with natural set theory. You see in, in set theory there is no possibility of duplicates in sets. This seems hard to fathom with the way we are used to dealing with data in a computer, but intuitive when you “think about how you think”.

When you say, “I have ten fingers and ten toes”, and I say, “how many is that combined?” you say 20, of course. Did you stop to add up the number one 10 times, and then then ten times more? Of course not. You did simple arithmetic to make the math faster, adding 10 + 10. …

Some time ago I wrote about MySQL 5.5 semi-synchronous replication. Since then, I have wanted to benchmark the overhead of semi-synchronous replication with a decent server. Now the occasion presented itself, thanks to some related business that I had to benchmark, and thus I did a few simple runs with and without semi-synchronous replication enabled, to see the impact of this feature on performance. If you haven't read the article on semi-synchronous replication, the bottom line is that, with this feature enabled, the master waits until at least one slave has acknowledged receipt for the data before returning a positive result to the client. This means that for each commit there are two network calls between master and slave. My gut feeling was that this feature would be costly in terms of query response time, although I was not prepared to …

In the InnoDB plugin, a new variable was added named innodb_io_capacity, which controls the maximum number of I/O operations per second that InnoDB will perform (which includes the flushing rate of dirty pages as well as the insert buffer (ibuf) batch size).

First off, let me just say this is a welcome addition (an addition provided by the Google Team, fwiw).

However, before this was configurable, the internal hard-coded value for this was 100. But when this became configurable, the default was increased to 200.

For many systems, this is not an issue (i.e., the overall system can perform 200 IOPS).

However, there are still many disks (which is often the bottleneck) out there that are …

Background Knowledge

I will explain how to convert a DATETIME (data type) value in Microsoft SQL Server to Unix timestamp and how to convert Unix timestamp to DATETIME. A Unix timestamp is a integer value of seconds since January 1, 1970 at midnight. For further explanation of Unix timestamps refer to Wikiepedia, UnixTimestamp.com or http://unixtimesta.mp/.

Note: This solution only work on dates prior to 2038-01-19 at 3:14:08 AM, where the delta in seconds exceeds the limit of the INT data type (integer is used as the result of DATEDIFF). See source for further details as I have not verified a solution to this problem.

Solutions

Convert Datetime Value to Unix Timestamp (today)

On May 17, 2006 – exactly 5 years ago – I started my new job as web developer at MySQL AB.

Since then I closed 3,049 web requests, worked with 9 colleagues in the web team, had 4 direct managers, attended the MySQL User Conference in Santa Clara, CA 4 times (2006-2009, although in 2006 I wasn’t an employee yet, but that was when I got hired, so lets count it) and one all-company meeting in Orlando, FL and went through 2 acquisitions: in 2008 to Sun Microsystems and in 2010 to Oracle.

Time to say thank you to all the great people who I had the pleasure to work with during these 5 years!

While Andrey is busy implementing partitioned replication infrastructure code for the PHP replication and load balancing plugin (PECL/mysqlnd_ms), I continued my search for ideas to steal. Mr. Robert Hodges, I’ve robbed the idea of a service level and caching.. If an application is able to function with stale data read from a MySQL replication slave, it can also deal with stale data from a local cache. The replication plugin (PECL/mysqlnd_ms) could, in certain cases, populate the query cache plugin (PECL/mysqlnd_qc) for you and read replies from it.

In the blog posting " …

While Andrey is busy implementing partitioned replication infrastructure code for the PHP replication and load balancing plugin (PECL/mysqlnd_ms), I continued my search for ideas to steal. Mr. Robert Hodges, I’ve robbed the idea of a service level and caching.. If an application is able to function with stale data read from a MySQL replication slave, it can also deal with stale data from a local cache. The replication plugin (PECL/mysqlnd_ms) could, in certain cases, populate the query cache plugin (PECL/mysqlnd_qc) for you and read replies from it.

In the blog posting " …

I love STL containers in C++. They're so much better in general to work with than their C counterparts. The bit that's missing though is an instantiation-time optimized version of map.

Take, for instance, reading a set of commands at startup from a set of dynamically loaded plugins. It's not going to change, in this hypothetical case, because we neither load nor unload plugins after startup. But it's dynamic in the sense that we do not know it at compile time - so doing something like gperf to generate a perfect hash is out of the question.

But why should we pay the lookup cost on a dynamic container for every lookup if I can determine, based on program flow, that the contents of the container are not going to change once it's instantiated.

Something like:

const perfect_map(some_dynamic_map_I_built);

Where the constructor would do a perfect hash generation once, and the map itself would be const (not …

As you may have read on the MySQL Performance Blog post – I’ve recently joined Percona. This is a fairly exciting next step. I’ll be in New York for Percona Live next week, where I’ll be giving a session titled “Drizzle 7, GA and Supported: Current & Future Features”.

I’ll write more soon, there’s a lot keeping me busy already!