• compressed backup patch
  • actually works rather well… and restoring from compressed backup too.
  • need to modify the rate-limiting code though… may as well rate limit the writing of the *compressed* data stream… otherwise the option isn’t nearly as useful
• compressed LCP patch
  • well… the *restoring* of compressed LCPs…. can write them
• working out exactly what more information I want out of the linux memory manager to find out what kswapd is really doing (and the patch that exports the right info)
• re-jigging my procmail filters for commits@lists.mysql.com
• fixing up my offlineimap patch and getting it in upstream
• disk pre-allocation for MythTV recordings
• buying workstation
• unpacking the last few boxes around the house
• finishing this …

Over the past N weeks/couple of months, we’ve been making a number of improvements to how backups are done in MySQL Cluster.

Once you get to large data sets, you start to really care about how long a backup takes.

Traditionally, MySQL Cluster has been in-memory only. The way to back this up is to just write from memory to disk (rate limited) and synchronised across the cluster.  Since memory is really fast (compared to the rate we’re writing out to disk) - never had a problem.

In MySQL 5.1 (and Cluster Carrier Grade Edition- CGE), disk based attributes are supported. This means that a row has both in memory and disk based parts.  As we all (should) know, disk seeks take a very long time. We don’t want to seek.

So, at some point recently we changed the scanning order from in-memory order (which previously made perfect sense) to on disk order. Randomly seeking through RAM is much cheaper than all the …

I just posted version 3 of mysqlbackup to MySQL Forge.

Small bugfix: Added option --add-drop-table to the default options for mysqldump. This was causing a failure in restoring views.

I would like to remember this day. It was just crazy. Most of my day went into troubleshooting InnoDB locking issues. At this point, the situation is all clear and I am very pleased with the results. Here are two graphs from MySQL Network Monitoring tool:

Keep manageable threads/connections and your server will be very happy.

Just noticed that Planet MySQL has a new layout. Overall, I like the change. The text seems much easier to read. Ads are now gone. For each active contributor, a score is shown. Don't know if the score is number of posts or what? Does anyone care to shed some light?

During my years as a Developer/DBA hearing the words "customizable" or "extensible" always brought a slight smile to my face, particularly when they were used in conjunction with a software sales pitch or evaluation. In fact, more often than not, while presented as product features, these words often really mean, "you can do some additional things with this software if you are willing to invest significant time and resources into doing so". Even today, some of the most popular open source and commercial software solutions provide application programming interfaces (APIs) and plug-ins that are really nothing more than containers that allow you to provide your own code. The goal being to make an off-the-shelf product more applicable to the problems you are attempting to solve without having to write a customized application from scratch.

Having gotten my feet with in regards to BLOBs while developing StuffDawg, I am starting to really see why some many people get eaten by the BLOB (and get spit out too!). The problem, at least that I have found, with BLOBs is that they do not seem to scale well. My experience makes me think that it is not just because of the fact that you are storing binary data in a database, but because of the overhead in the application code for having to deal with them.

For instance, suppose you are storing images in your database, say for a gallery. That is great and all, but how does one display the images in a web-browser? Using the <img src> tag I would assume. This is all well and good, but since the image does not exist on the file-system, you either have to put it there, or need to write a script that pulls the data from the …

The Guide To Understanding mysqlreport has been updated. As many people have noticed, since mysqlreport v3.2, Key Write and Read ratio went away, replaced by Write and Read hit percentage. I figured that “99.87%” was more intuitive than “0.0013″.

Pretty usually in our Perl projects we use third-party modules (CPAN or others) and sometimes we can find a bug in such module. What are our options in such cases? This small article describes some useful hints for Perl developers who have such problems.

(more…)