I had some fun yesterday with some odd performance problems. So I did a run with oprofile and got this:

561612   25.0417  /lib64/tls/libc-2.3.4.so memset
429457   19.1491  /usr/lib/debug/lib/modules/2.6.9-34.ELsmp/vmlinux clear_page
214268    9.5540  /usr/lib/debug/lib/modules/2.6.9-34.ELsmp/vmlinux do_page_fault
144293    6.4339  /usr/lib/debug/lib/modules/2.6.9-34.ELsmp/vmlinux do_no_page
94410     4.2097  /usr/lib/debug/lib/modules/2.6.9-34.ELsmp/vmlinux buffered_rmqueue
64998     2.8982  /lib64/tls/libc-2.3.4.so memcpy
59565     2.6559  /usr/lib/debug/lib/modules/2.6.9-34.ELsmp/vmlinux __down_read_trylock
59369     2.6472  /usr/lib/debug/lib/modules/2.6.9-34.ELsmp/vmlinux handle_mm_fault
47312     2.1096  /usr/lib/debug/lib/modules/2.6.9-34.ELsmp/vmlinux free_hot_cold_page
39161     1.7462  /usr/lib/debug/lib/modules/2.6.9-34.ELsmp/vmlinux release_pages
39140 …

I’ve just posted slides and (rather) rough presenter notes and image credits for my Age of Literate Machines presentation.

These are still draft slides. I’ll be working on an improved version in the coming weeks and months.

Many thanks to all who have sat through drafts of the talk (DES, Jen, Hilde, Markus, Monty, Rory, Tony).

• The Age of Literate Machines - Rough speaker?s notes and image credits (PDF)
• The Age of Literate …

Let’s say when you have to run a batch process monthly, you can survive with times like 10 minutes. I can imagine a lot of seasoned DBAs right now ROFL about my insignificant 10 minutes. The point here is I was developing this process and some test cases, so my usual trial/error methodology doesn’t scale very well with 10 minute offsets.

So I borrowed an idea from a colleague: why not moving all the database to memory? It’s not so big and I have 2G of ram. But, could I change all tables (~20) to MEMORY in one line or so?

Since this was a Ruby on Rails project, I used the rails console to be able to mix SQL and Ruby. My first try:

conn.tables.each do |t|
  conn.execute "ALTER TABLE #{t} ENGINE=MEMORY"
end

First error: foreign …

This is my second article about hacking MySQL. If you are interested in this topic, you may want to read my previous Hacking MySQL: SIGNAL support (I)

The problem

If I tell you there is a function called TRUNCATE, what do you think it does? which are its arguments?

For me, the obvious behaviour would be something like:

mysql> SELECT TRUNCATE(123.45);
+--------------------+
| TRUNCATE(123.45)   |
+--------------------+
|                123 |
+--------------------+
1 row in set (0.08 sec)

But the actual behaviour is this:

mysql> SELECT TRUNCATE(123.45);
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1

This is because …

I’ve been looking for an open source project to collaborate for some time now, and given the time I’m spending with MySQL lately and the expertise I’m gaining thanks to MySQL training, it looked like an obvious choice.

During the last advanced bootcamp, Tobias found bug #27894, which apparently was a simple fix. Dates in binlog were formatted as 736 instead of 070306 (for 2007-03-06). During the bootcamp I used my lonely nights at the hotel and came up with a patch, and some days later my first contribution was going into the main MySQL code.

The problem

Now I had to find something bigger. One of the things that most annoys me of MySQL is the lack of some way to abort a procedure or …

The NDB/Connectors have added support for Ruby, as well as Asynchronous Transaction support for Java, Python and Perl.

The Ruby support, of course, means that new you can interact with your MySQL Cluster installation using the NDBAPI from all your Ruby code.

The async stuff is especially cool, because it means you can send transactions to the Cluster and get responses by way of callbacks defined in the connector language. So you can do something like this:

class testaclass(object):

  def __init__(self, recAttr):
    self.recAttr=recAttr

  def __call__(self, ret, myTrans):
    print "value = ", self.recAttr.get_value()

#snip

myTrans = myNdb.startTransaction()

myOper = myTrans.getNdbOperation("mytablename")
myOper.readTuple(ndbapi.NdbOperation.LM_Read)

myOper.equal("ATTR1", 245755 )

myRecAttr= myOper.getValue("ATTR2")

a = testaclass(myRecAttr) …

I use del.icio.us a lot. I have one tag topost to save those things I find on the Internet which I want to write about later, but I have around 100 right now, so I was interested in picking random links from that list.

Bad news you can’t do that neither from the del.icio.us interface nor the API provided, so I took a different approach. I downloaded MySQLicious, a set of PHP scripts to mirror your delicious links into a local MySQL database. Once you edit your connection parameters, it’s quite easy to use:

$ ./mirrorlog.php
Update may be needed. Checking now.

Updated http://members.ebay.com/ws/eBayISAPI.dll?ViewUserPage&userid=*starlightstarbright*
Deleted eacb873e503b1b407c2ec69e541b8838

1 updated.

1 deleted.

You can …

Here’s a very rough pre-release of NdbObject, an ORM mapping for python that maps Objects to NDB directly with no SQL code.

NdbObject.tar.bz2

What if we made a storage engine that used MogileFS to store blobs/images?

I’m happy to release version 0.1 of the NDB/Python bindings. They are very young and not very well documented yet, but email me if you are interested in them and especially if you are interested in helping out. I’ll get a Trac system up and going (or something) or maybe sourceforge or savannah or google code or you know, whatever.

python-ndbapi-0.1.tar.bz2