[Update 2007/08/17: I just found out, that this entry got overwritten by another one. A corrupted MySQL key file might have caused this.]

As a developer, reading things on the web, you come by Joel Spolsky pretty quickly. Joel is a top-notch essayist with many a story to share from his days on the Microsoft Excel Team and his current company Fog Creek. Now, Joel cares for a good read. Being sick of bad writing he tried to make an example of good software writing. He compiled a list of essays from The Net he deemed worth publishing in a book. Joel introduces each essay with a few comments. The preface to the piece by Michael “Rands” Lopp says, in my words, when you're done with this, you'll be going …

Jan Kneschke has released a new tool called "MySQL Proxy".
This proxy is between the mysqld and the client and can handle load balancing, fail over, query rewriting and analysis.
It´s also possible to extend the proxy with LUA scripting which opens nice possibilities.
I´m thinking about adding Memcache-Support. But perhaps its better up to the application to decide, if the cache should be used or not.
At least it should be possible to implement load balancing/ failover in an easy way for a slave-farm.

I am subscribed to the memcached list. Memcached use (and interest in caching in general) has exploded it seems in the last couple of years. There are a lot of people that join the list that not only don’t have a lot of experience with memcached, but they don’t have experience with caching. We have been caching at dealnews since the beginning. We started using memcached at dealnews in early 2005.

One common concern that new users have is data not being updated the second it goes in the database. This is how it worked before they were caching. They are used to it working that way. So, when they start caching, they miss that instant gratification. We went through this at dealnews. Our content team could write a deal, go to the front page and see it right then. They could then move on with their lives. As we grew and it became apparent …

This weekend, Monty and I got together for a different kind of hacking session.

Instead of developing software, we were working on developing a set of rough principles and rules for running a Free Software/Open Source business. We both have a good amount of experience working with various FLOSS projects (like Mozilla, MySQL, PHP, etc.) and FLOSS companies (like eZ Systems, Mozilla, MySQL, Zend, etc.) and hope that we can put this experience to good use.

For me, this was a tremendous help - I’ve been putting off working on this for Foo Associates for some months now. It is much easier to work on meeting the needs of customers than it is to work on planning for the future.

The notes are still extremely rough, but both Monty and I want to post them so that people can discuss. Also, …

I just received word that my proposal (which was to present my Age of Literate Machines presentation) for FrOSCon has been accepted.

I’m pretty excited - the event should be fun and it will give me a good chance to see friends (including a good number of my German MySQL colleagues)

I’ve just posted slides and (rather) rough presenter notes and image credits for my Age of Literate Machines presentation.

These are still draft slides. I’ll be working on an improved version in the coming weeks and months.

Many thanks to all who have sat through drafts of the talk (DES, Jen, Hilde, Markus, Monty, Rory, Tony).

• The Age of Literate Machines - Rough speaker?s notes and image credits (PDF)
• The Age of Literate …

I rarely see the point of posting the slides of a presentation for people who didn't see the original presentation. Yet, this is often requested. I don't have a problem with posting my latest slides (PDF, 2.9M), but they are of little value without context and I do have a problem with posting things of little value, so here's the context.

Slide 1

Welcome to my presentation about CouchDb. This is a ca. 80 minute ride through the world of non-relational data storage.

Slide 2

I'm Jan, from Münster, Germany. I'm a developer focussing mostly on the web. If time permits I'm studying computational linguistics. I mostly do freelancing consultance work on the web and gained experience with scaling high traffic LAMP sites doing that. I'm also the co-founder of freisatz, a company bringing typographic bliss to everyone.

I had little time and no internet while preparing the slides. …

From the city centre of Zurich to the major connecting points for travelling (main station, airport, motorways) it takes you an average of 7 minutes to get there. There's no other city in Europe that as fast to get out of. People argue wether this is a good or bad thing.

The Webtuesday people I met here in Zurich were coming from a couple of independent web-development shops and they are working on some of the top 20 Swiss websites. From the feedback, the talk was well received; overall, they found it interesting to learn about a new technology they could actually use. I'll be putting up the slides for the talk soon.

What I found most interesting were the discussions after the talk, when a couple of guys spend some time trying to apply CouchDb to the problems they currently face. The first bit of interest is that …

So, I mentioned in my MySQL Cluster post that I found out that cluster and joins don’t get a long too well. There are a couple of tips I have for using joins or replacing them with other techniques that I thought I would share.

Here are some things I keep in mind when using the NDB engine. These may apply to the other engines as well.

1. Test using the NDB engine. Testing against another engine is not adequate.
2. Avoid JOIN if you can, test often if you can’t
3. Avoid subqueries always
4. Select using primary keys when at all possible. Always select using a key.
5. Be sure to include both your where and order by columns in your keys.

Try your join from several directions.

Sometimes, the way you join can affect the engine. This is more …

So, the whole world changed at dealnews when Yahoo! linked us. We realized that our current infrastructure was not scaling very well. We had to make a change.

The Problem

Even though we were using all sorts of cool techniques, the server architecture was really still just a bunch of web servers all serving the same content. In addition to that, our existing systems as the time used a pull method. When a request came in, memcache was checked, if the data was not there, it was fetched from our main MySQL server. So, when there is no data in the cache or when it expires, this was very bad. Like when Yahoo! hit us. Some cache item would expire and 60,000 users would hit a page and each page would try and create the cache item.

The Solution

I …