Last week I was surprised to see this paper bubble back up on Planet MySQL. It describes the pros and cons of thread and event based programming for high concurrency applications (like a web server), arguing that thread-based programming is superior if you use an appropriate lightweight threading implementation. I don’t entirely disagree with this, but the problem is such a library does not exist that is standard, portable, and useful for all types of applications. We have POSIX threads in the portable Linux/Unix/BSD world, so we need to work with this. Other experimental libraries based on lightweight threads or “fibers” are really interesting as they can maintain your stack without all the normal overhead, but it is hard to get the scheduling correct for all …

--edit
Continuent says that Tungsten replication can indeed read from binary logs, including RBR logs. I need to investigate this further.
--end edit

MySQL versions prior to 5.1 lack a relatively easy way to capture the row-change information from a MySQL database. Oracle and DB2 both provide convenient interfaces for reading log data and producing a set of changes. Oracle calls this 'log miner' and I don't honestly remember the name of the tool that has similar functionality on DB2.

Prior to MySQL 5.1, the capturing of row change information could be done only through triggers. This method of change-data-capture (CDC) has some serious drawbacks, including the inability to detect transaction commit order due to the nature of two-phase locking. This means that trigger based implementations of CDC often have to on a central lock table to serialize activity and generate transaction ids. Trigger based CDC …

Today evening one of my friend asked me in the IM to look into one of his production server where a query was taking ~11 seconds to run on 20 million row table, even though the query is using the right index and the plan as shown below:

mysql> explain SELECT channel, COUNT(channel) AS visitors FROM [...]

There’s a good number of videos appearing online from the MySQL Conference and Expo that was on last week.

Here’s a short list of interesting things to look at if you weren’t able to make the sessions. Obviously, this is from my view as a Drizzle developer. There were other interesting things, but this list is more focused towards where my Drizzle brain is stimulated.

• State of Drizzle (Brian Aker)
• MySQL at Facebook (Mark Callaghan)
• High Concurrency MySQL ( …

This is not my notes about the MySQL conference that just occurred. These are my thoughts about MySQL conferences in general. Baron wrote in The History of OpenSQL Camp:

After O’Reilly/MySQL co-hosted MySQL Conference and Expo (a large commercial event) that year, there was a bit of dissatisfaction amongst a few people about the increasingly commercial and marketing-oriented nature of that conference. Some people refused to call the conference by its new name (Conference and Expo) and wanted to put pressure on MySQL to keep it a MySQL User’s Conference.

During this year’s conference, I heard a lot of concern about whether or not O’Reilly would have a MySQL conference, and whether or not Oracle would decide to sponsor. I heard all of the following (in no particular order):

* If O’Reilly does not have a …

If you’re like me, you’ve gotten tired of writing endless test cases for parsers that can understand the thousands of variations of text output by SHOW INNODB STATUS. I’ve decided to solve this issue once and for all by patching MySQL and InnoDB to output XML, the universal markup format, so tools can understand and manipulate it easily. Here’s a sample snippet: <status><![CDATA[ ===================================== 100320 15:46:24 INNODB MONITOR OUTPUT ===================================== .

EAVB_VFZUHIARHI To whom it may concern -

The mysqldump program can be used to make logical database backups. Although the vast majority of people use it to create SQL dumps, it is possible to dump both schema structure and data in XML format. There are a few bugs (#52792, #52793) in this feature, but these are not the topic of this post.XML output from mysqldumpDumping in XML format is done with the --xml or -X option. In addition, you should use the --hex-blob option otherwise the BLOB data will …

The new ndbinfo interface in 7.1 is really useful to assist in tuning MySQL Cluster. Here is an example (more will follow):

I started with one test where I inserted two blobs (1KB + 1KB) in one table.
From 16 threads (colocated with one mysqld, two data nodes, separate computers) and one application driving the load I reached about 6960TPS, and the utilization of the redo buffers (controlled by the parameter RedoBuffer in config.ini) looked like:

mysql< select * from ndbinfo.logbuffers;
+---------+----------+--------+----------+----------+--------+
| node_id | log_type | log_id | log_part | total    | used   |
+---------+----------+--------+----------+----------+--------+
|       3 | REDO     |      0 |        1 | 50331648 | 196608 |
|       3 | REDO     |      0 |        2 | 50331648 | 294912 |
|       3 | REDO     |      0 |        3 | 50331648 | 131072 |
| …

This time last year I obtained a domain called raindb.org which I was intending on using for my storage engine project. RainDB was the project name I had in mind for BlitzDB at the time. Since I now have a different project name, I no longer have any use for this domain.

So, rather than letting it go to waste I’d like to contribute this domain for yet another potential open source database project. Your project can be anything – MySQL Storage Engine, Drizzle Storage Engine, Embedded Library, Stand Alone Server, whatever. RainDB would be a good name for a highly concurrent database since the analogy is – “it can be rained on”.

If you’re interested please feel free to email, tweet, or even just leave a comment on this blog entry.

Last Friday we held the Drizzle Developer Day at the Santa Clara convention center, taking advantage of the fact that many developers and interested contributors were already there for the MySQL Conference & Expo. Minus a few small glitches like wifi and pizza consumption location, I would say it was an overall success. There were a lot of new folks interested in learning about Drizzle and getting the server up and running. The day was organized by splitting folks up into small groups with matching interests, and then switching up groups every hour or so. We had groups focused on replication, documentation, writing plugins, the optimizer, Boots (the new client tool), and a “getting started” group.

The first group I participated in was about Boots, the new command line tool …