When I wrote this post about invalid utf8 characters I needed a way to convert the mysql message into a real identifier for me to take a look at. Below is a quick and dirty script to figure out the bad rows.

I wrote the script below in literally 2 mins. It's really basic and basically just CROSS JOINs two versions of the table one in utf8 the other in latin1 and reports back which string column is not correct. I haven't cleaned the script up, it's ugly and for my purposes will only exist for a short period of time.

test.$TABLE = is the original table latin1 for instance
- use mysqlimport this forces data to not get converted to latin1 from utf8 by setting the character set to binary (i.e. do no convert)

$DB.$TABLE = is the new utf8 table.

#!/usr/bin/php …

Some people might have lives, but I have a webserver.

I think I’ve now upgraded just about everything (software-wise) that’s upgradable on this machine:

• Apache 1.3.33 -> 1.3.37 (Thanks for hiding the win32 binaries under “Archives” when the *nix version is out in plain view, guys)
• PHP 5.0.3 -> 5.1.4 (This required ditching my old php.ini file and doing a new one from scratch)
• MySQL 5.1.8 -> 5.1.11 (Dead easy, even on Windows - yea, TEAM!)
• Perl 5.8.7 -> 5.8.8
• Python 2.3.2 -> 2.4.3
• Tcl 8.4.12 -> 8.5.0
• BlogCMS .3.4.6 -> WordPress 2.0.4 (The RSS feed was broken, I was getting tired of seeing my posts quoted elsewhere sans formatting, and every time I tried messing with the code, it just got worse)
• Singapore 0.9.11 -> 0.10.0 (The one part of BlogCMS that I still really liked after switching …

The Free Standards Group (FSG), a nonprofit organization dedicated to developing and promoting open source software standards, today announced that two of the most important and widely-used applications on Linux: RealPlayer and the MySQL database, are certifying to the Linux Standard Base. The result will be decreased build, porting and support efforts for the software developers and increased choice for end users. For the first time, a single RealPlayer or MySQL package will run on the many LSB-certified Linux distributions including Red Hat, Novell, Ubuntu, Red Flag and many others.

Quite a few people have chimed in on a recent discussion about PHP, MySQL, database abstraction layers, and performance. I think enough viewpoints have been covered that I don't need to comment, but one question I don't see answered is "what are the qualities of a good SQL abstraction layer?" I think it's a very interesting -- and complicated -- question. As it turns out, the term has several meanings, and I think it's important to understand them. In this article I explain one way to categorize different types of database abstraction layers, the goals for each, and some useful success criteria.

This is not about open source, but still, I found it quite inspiring.  On September 17, ultramarathon runner Dean Karnazes (sounds like Onassis), will embark on the ultimate endurance feat of running 50 marathons in 50 states in 50 days.  I met Karnazes last year when he was speaking at the Silicon Valley Marathon.  For a guy who is setting world records, he is incredibly humble.  He just happens to be able to run several hundred miles non-stop without sleep.  In fact, his record is 350 miles in 80 hours.  He's also won the Badwater Ultramarathon, arguably the toughest endurance run in the world: 135 miles through Death Valley over 3 mountain ranges where …

On August 28 — i.e., in two weeks from tomorrow — the European Parliament will return from its summer vacation. You can find the EP’s calendar here: There are different color codes, and those days which have no color at all are holidays and vacation days.

While the EP is not the only EU institution, it’s clearly one of the most important ones, and its return marks the end of what is usually the slowest part of the summer season in Brussels. Upon its return, the parliament is going to take a look at patent policy again, and is in particular going to evaluate the outcome of the European Commission’s patent policy hearing that took place in Brussels on July 12. In late May, several parliamentary groups (which are, in a simplified explanation, …

Travelling to Auckland (NZ) today, teaching a "MySQL 5.0 for DBAs" training course this week. I'll also be visiting AuckLUG on Monday evening. If you would like to catch up, just drop me a line (arjen @ you know where).

New lessons learnt at Brisbane airport...

• finally found a shop (after security that sells Double Coat TimTams;
  
• don't get melts or mochas from "BLD" - expensive and not-so-good;
  
• it's still a friendly & fast airport.

AirNZ still has good legspace even on their 747-400s (they also have 777s), and my 13" MacBook fits perfectly on the tray table. Speaking of the MacBook, they wanted to weigh my backpack and finally it weighted in at around 6kg including a bottle of water, a few books, etc. That's all new and good, as with my old Dell Inspiron 8500 (big widescreen, plus its power …

Quite a few people have chimed in on a recent discussion about PHP, MySQL, database abstraction layers, and performance. I think enough viewpoints have been covered that I don’t need to comment, but one question I don’t see answered is “what are the qualities of a good SQL abstraction layer?” I think it’s a very interesting—and complicated—question. As it turns out, the term has several meanings, and I think it’s important to understand them.

Setting up MySQL replication is fairly easy. Just tell the slaves which master they need to connect to and make sure the master is writing binlogs. Ok, it is a little more complicated, but in the end it's pretty straight-forward.

If however you have large number of slaves and a rather busy master machine, the network load can become significant. This is because all statements that are written to the binlog are transferred to all the slaves. They put them into their relay logs and asynchronously process them. Each slave may decide which statements need to be executed and which can be discarded. This is especially useful in setups where there are tables modified on the slaves that should not be replicated.

Use case

In a production scenario we have up to 35 slaves configured against a single master. The master does "full" binlogs, i. e. no exceptions are made. This is done on purpose, because the binlogs are backed up for …

mysql> CREATE TABLE `a` (
-> `a` text
-> ) ENGINE=TABLE_FUNCTIONS DEFAULT CHARSET=latin1 CONNECTION='/ usr/lib/libhello.so' ;
Query OK, 0 rows affected (0.04 sec)

mysql> select * from a;
+--------------+
| a |
+--------------+
| Hello World! |
| Hello World! |
+--------------+
2 rows in set (0.02 sec)

mysql>