Everybody knows that valgrind is great.

Well, I was observing a problem in some MySQL code, it looked like we were writing over some memory that we weren’t meant to be (as the structure hadn’t been initialised yet). But, seeing as this was memory that had been allocated off a MEM_ROOT (one of our memory allocators), valgrind wasn’t gonig to spit out anything.

This is because this bit of memory had already been allocated and subsequently “freed”, but then reallocated. The “free”ing overwrites the memory with garbage (which is what the MEM_ROOT code does) so that you should see crashes (and a pattern) when you do something bad.

The traditional way to troubleshoot this in to modify your memory allocator so that it just calls malloc() and free() all the time (and valgrind will trap them). We have some code in there to do that too. However, this requires …

My good friend Zak Greant spotted a news item at Techworld, where the European Commission appears to have changed its position on software patents.

Have a read, it's interesting and I hope that it really does indicate a new direction from the commission and not just one loose statement. If it is for real, that's absolutely great, but let's not be complacent and think "that's it". The pro-swpat lobbyists certainly won't give up, they have strong interests and stacks of cash driving them.
Of course it's not helpful to preach to the converted, but encouragement to continue on this path seems like a good idea. I'll happily give politicians kudos and encouragement when they do good stuff!

Also, if you haven't yet …

So what do I do on Memorial Day Weekend?

Well in theory I did not work.

Friday I went to go to see a SIFF, Seattle Internal Film Festival, movie in the evening. The fact that I can not remember what it was, and I think I gave it three stars means that I will not be recommending it to anyone. I have eight movies to see in three weeks (of which in the three weeks I will go to Boston, Korea, and probably California). Got home from the movie and started to port Everydevel to Apache 2.2 and MySQL 5.0. No one has bothered to do either and I am getting tired of continuing to set it up on old application stacks. The Apache part took a minor amount of reading just because of the changes in mod_perl2 from mod_perl. It was minor and mostly annoying stuff. I also setup Apache to just use pre-fork since I am personally not sure that I consider a threaded version of Apache with mod_perl to be safe. Watched Fire Fly while coding. …

Did you know that you can access tables on handler/engine level? I didn?t until I read the new training material for our new upcoming ?Performance Tuning? course. So, if you didn?t know the HANDLER command, book the course! There?s much more that you might now know yet.

Read http://dev.mysql.com/doc/refman/5.0/en/handler.html for more background.

root@localhost [world]> HANDLER City OPEN; Query OK, 0 rows affected (0.10 sec)

root@localhost [world]> HANDLER City READ FIRST; +----+-------+---------+----------+------------+ | Id | Name | Country | District | Population | +----+-------+---------+----------+------------+ | 1 | Kabul | AFG | Kabol | 1780000 | +----+-------+---------+----------+------------+ 1 …

We had discussion today which involved benchmarks of Join speed for MyISAM and Innodb storage engines for CPU bound workload, this is when data size is small enough to fit in memory and so buffer pool.

I tested very simple table, having with about 20.000 rows in it on 32bit Linux. The columns "id" "i" and "c" were populated with same integers so we can allow the same job to be done using different kinds of columns - primary key, integer indexed column and indexed char column. The query is also trivial - the point was to make sure it is not index covered query so it reads the rows and it does not return many rows. I varied the join clause to be id, i and C columns appropriately.

PLAIN TEXT SQL:

1. CREATE TABLE `t1` (
2.   `id` int(10) UNSIGNED NOT NULL DEFAULT '0',
3.   `i` int(10) UNSIGNED NOT NULL DEFAULT '0',
4.   `c` char(15) DEFAULT NULL,
…

Check out this cool INSERT ON DUPLICATE KEY UPDATE feature in MySQL 4.1. This will make some logging code I have much easier and cleaner than it currently is.

Back in March, I wrote about Partitioning in MySQL 5.1 not being usable, and since a few releases have gone out, and a few months have passed, I am re-visiting the issues I saw back then.

MySQL still doesn’t allow you to partition using a native DATE or DATETIME column. This is a major problem in my opinion, as it’s probably one of the most common uses for partitioning, but is only seen as a “feature request” by MySQL. Hopefully that will get fixed sometime in the near future.

I filed a couple more bugs today, the most serious of which was the inability to repair “crashed” partitions:

mysql> alter table ontime repair partition p_199609;
ERROR 145 (HY000): Table './flightstats/ontime#P#p_199609' is marked as
crashed and should be repaired

Here are the bugs I filed:

• …

INSERT ... ON DUPLICATE KEY UPDATE is very powerful but often forgotten MySQL feature. It was introduced in MySQL 4.1 but I still constantly see people unaware of it.

Myself I like this feature big deal because it is designed in truly MySQL style - very efficient solution for freqent task while keeping it beautiful and easy to use.

So what is this feature great for ? Well any kind of maintaining counters. If you're writing traffic accounting it could be traffic and number of packet passed for given port or IP address. For Web Applications it could be counting number of visits per page or IP address, number of times particular keyword was searched etc.

This functionality also makes it very easy to do incremental single pass log file processing and building summary tables.

Here is example:

PLAIN TEXT SQL:

1. CREATE TABLE ipstat(ip int UNSIGNED NOT NULL PRIMARY KEY,
…

Techworld.com reports:

The European Commission this week appeared to take a step back from its earlier position on the patentability of software, stating that computer programs are not patentable, and that patents on them may be struck down by the courts.

For the full story, see:
http://www.techworld.com/applications/news/index.cfm?newsid=6081

The 3rd international conference on GPL v3 will take place in Barcelona, Spain on June 22nd and 23rd. The international GPLv3 conferences are part of a year-long public consultation process to update the GNU General Public License.

Speakers include Richard Stallman, Eben Moglen, chairman at Software Freedom Law Center (SFLC) and Georg Greve, President of Free Software Foundation Europe (FSFE). Expert panelists from all parts of Europe, and from around the world will lead discussions on licence internationalisation, DRM, software patents, and adoption of the finished licence.

The current draft of the GPLv3 and resources explaining the background to the update are availble at http://gplv3.fsf.org/

The Conference’s schedule and further information will be published soon at
…