... or how to safely run data manipulation statements in your database.Reading posts on Devshed's forums I sometime notice people doing maintenance work on their data without any safety net apart from occasional ages old backups ;-).Anyway I think there's no need for a restore if you just issued the wrong update query, I mean, transactions are here for this, it's just a matter of educating people

Maatkit contains essential command-line utilities for MySQL, such as a table checksum tool and query profiler. It provides missing features such as checking replicas for data consistency, with emphasis on quality and scriptability. This release contains major bug fixes and new features. Some of the changes are not backwards-compatible. It also contains new tools to help you discover replication replicas and move them around the replication hierarchy. Changelog for mk-archiver: 2008-03-16: version 1.

MySQL 5.1 introduces the plugin interface, a series of API that allow the load and unload of server components at runtime.
The idea behind this architecture is to make MySQL server really expandable.
Surprisingly, the latest engines being developed in house (Falcon and Maria) are not implemented using the plugin technology. The best example of how to take advantage of this interface comes from the outside. Paul McCullagh's PBXT is a transactional engine that can be loaded at run time.
To use it, you can either download the pre-compiled plugin library, and load it into your server, or get the source code and build the whole server from scratch.
Of course, the most interesting option is the pre-compiled binary. I downloaded the latest …

If you have a 12-server MySQL Cluster with: 1 Management Node 3 SQL Nodes 2 Data Node Groups, 4 Data Nodes per group And each machine is configured to allocate 1G of memory for its function, how much data (data + indexes) can you store in total in your cluster? You can guess, but you get bonus points if you [...]

I was wrong in my last post, it seems that all Sun database developers are now part of the same organisation, including PostgreSQL's Josh Berkus.

MySQL has the pluggable storage engine architecture, which is unique in the industry. The idea is you pick from among a suite of storage engines the most suitable one. PostgreSQL on the other hand has a plugin architecture for programming languages you can then use for stored procedures. And the cool thing about Open Source...

Someone went as far as to implement a PostgreSQL plugin of LOLCODE, a funny programming language I didn't know about until recently. So now you could do this with PostgreSQL:

read more …

Yesterday, we concluded the Sakila Express World Tour, more aptly named “Sakila Tour of Seven Top European MySQL AB Sites Using Decadence Airlines“. The last two were in Germany: Hamburg in the north, in Sun’s offices, and Munich in the south, in Hilton am Tucherpark.

In Hamburg, we went directly into the meat with three hardcore developers. Onboarding, intellectual property, and contractors were the hardcore topics.

Ulf, Jan and Kay experienced Izhevsk and Kiev vicariously through the blog

The discussions added plenty of colour to the picture for Julie and Dave. Julie has a great metaphor for explaining the purpose of Sakila Express: To understand how the integration message comes out in the other end, in the Telephone game (also known as Chinese whispers, in German as Stille Post, in Swedish as Ryska …

For large number of online applications once you implemented proper sharding you can consider your scaling problems solved - by getting more and more hardware you can grow. As I recently wrote it however does not mean it is the most optimal way by itself to do things.

The "classical" sharding involves partitioning by user_id,site_id or somethat similar. This allows to spread data more or less evenly across the boxes and use any number of boxes. However this may be not the most optimal approach by itself because not all data belonging to same user is equal.

Consider Blog or Forum as example - most likely few last posts will get majority of hits while things written year ago are accessed with much less frequency. You can often level off this significantly for reads by using caching (if things are accessed frequently …

It was good 2 days at PHP Quebec Conf here in Montreal.

A lot of great sessions!!!!
My highlights:
- Great session by Eli White – High Performance PHP & MySQL.
- Really nice 2 sessions by Marc Wandschneider (The I18N and memcached sessions)
Some sessions I’ve attended were really really bad. I’m not going to name names
But overall, I guess it was worth taking 2 days from work and going to the php conf.

Here is the part about Sun and PHP:

At the end of the conferance, there was a small QA session with the conferance speakers. At the begining everyone was shy to ask questions. So some asked phunny questions like, “What’s the next big thing you’d be developing in PHP 6? Is it PDO2?” things like that. Everybody started laughing and I’ve (tried) to make a joking …

Dear Metadata fans,

It’s been a while that I blogged about Pentaho Metadata. This is undeservedly so because the last couple of months, a lot of things have been moving. Most of that is not really visible to the end-user. The GUI part of the metadata suite was attacked last year and doesn’t really need all that much work. What we have been doing is extending the underlaying architecture by making it more flexible, more robust and easier to program from an API viewpoint. Most of that work has been in the capable hands of Pentaho rock star Will Gorman. The work he did last year for example was building in support for libformula (Open Office formula) by Pentaho reporting wizard Thomas Morgner.

Lately, Will and I started on …

It's been nearly a year since the last post, so you might naturally wonder if I am dead or stopping development of MySQLdb. Actually I've been sick for the last year and a half or so, and hadn't really been motivated enough to do anything. Here's what's been going on:

John Eikenberry has taken over development of ZMySQLDA, since I pretty much don't do anything with Zope these days. He's made a couple of releases on the way to a 3.0 release. As far as I know, ZMySQLDA is still only useful with Zope 2 as Zope 3 has a different architecture and comes with MySQL support directly.

Monty Taylor from MySQL AB has volunteered to help out on MySQLdb. I believe the way this is going to work out is he's going to be doing maintenance on the 1.2 branch, and get some minor bug fixes out there. In addition, he has a good start on a native (i.e. written in Python) …