For large number of online applications once you implemented proper sharding you can consider your scaling problems solved - by getting more and more hardware you can grow. As I recently wrote it however does not mean it is the most optimal way by itself to do things.

The "classical" sharding involves partitioning by user_id,site_id or somethat similar. This allows to spread data more or less evenly across the boxes and use any number of boxes. However this may be not the most optimal approach by itself because not all data belonging to same user is equal.

Consider Blog or Forum as example - most likely few last posts will get majority of hits while things written year ago are accessed with much less frequency. You can often level off this significantly for reads by using caching (if things are accessed frequently …

It was good 2 days at PHP Quebec Conf here in Montreal.

A lot of great sessions!!!!
My highlights:
- Great session by Eli White – High Performance PHP & MySQL.
- Really nice 2 sessions by Marc Wandschneider (The I18N and memcached sessions)
Some sessions I’ve attended were really really bad. I’m not going to name names
But overall, I guess it was worth taking 2 days from work and going to the php conf.

Here is the part about Sun and PHP:

At the end of the conferance, there was a small QA session with the conferance speakers. At the begining everyone was shy to ask questions. So some asked phunny questions like, “What’s the next big thing you’d be developing in PHP 6? Is it PDO2?” things like that. Everybody started laughing and I’ve (tried) to make a joking …

Dear Metadata fans,

It’s been a while that I blogged about Pentaho Metadata. This is undeservedly so because the last couple of months, a lot of things have been moving. Most of that is not really visible to the end-user. The GUI part of the metadata suite was attacked last year and doesn’t really need all that much work. What we have been doing is extending the underlaying architecture by making it more flexible, more robust and easier to program from an API viewpoint. Most of that work has been in the capable hands of Pentaho rock star Will Gorman. The work he did last year for example was building in support for libformula (Open Office formula) by Pentaho reporting wizard Thomas Morgner.

Lately, Will and I started on …

It's been nearly a year since the last post, so you might naturally wonder if I am dead or stopping development of MySQLdb. Actually I've been sick for the last year and a half or so, and hadn't really been motivated enough to do anything. Here's what's been going on:

John Eikenberry has taken over development of ZMySQLDA, since I pretty much don't do anything with Zope these days. He's made a couple of releases on the way to a 3.0 release. As far as I know, ZMySQLDA is still only useful with Zope 2 as Zope 3 has a different architecture and comes with MySQL support directly.

Monty Taylor from MySQL AB has volunteered to help out on MySQLdb. I believe the way this is going to work out is he's going to be doing maintenance on the 1.2 branch, and get some minor bug fixes out there. In addition, he has a good start on a native (i.e. written in Python) …

Maybe I'm just out of the loop - but I had no idea IBM had an incubator project on PHP. Nice.

For many developers, MySQL and PHP go together like peanut butter and jam. MySQL has a long history of working with the PHP community and that's a commitment that will continue forever. If you've thought scripting languages weren't powerful enough for "real programming" take a look at recent facilities like PDO and the Zend Framework. MySQL has also developed a native PHP driver that has become very popular. This past week to celebrate our commitment to PHP, Sun and MySQL held a meetup / cocktail reception and job fair at PHP Quebec, one of the biggest PHP conferences. Jay Pipes,... READ MORE

Welcome to the 88th edition of Log Buffer, the weekly review of database blogs. SQL Server To begin, Simon Sabin, on SimonS Blog, offers the proposition: SQL Server tools suck, do you agree? He elaborates, “When I moved from Oracle 7 to SQL Server 6.5 I was amazed at the tools you got with SQL Server. [...]

If you have Monolith installed in a RedHat cluster environment and are wondering how to make the cron job scripts only execute on the active cluster node, this script is a working example. It needs to be installed and running on each cluster node. The only variable to change is “MYHOST” depending on which node the script is running on. This analyzes the output of /usr/sbin/clustat for active node status and service type.

Feel free to use this as a starting point for other RH cluster aware wrappers.

Here are the associated crontab entries for the monolith cluster aware wrappers.

# Monolith Monitor Wrapper Scripts
02 * * * * root /usr/local/bin/monolith_cluster_report-generator.sh > /dev/null 2>&1
*/15 * * * * root /usr/local/bin/monolith_cluster_agent.sh > /dev/null 2>&1
02 * * * * root /usr/local/bin/monolith_cluster_cronexec.sh > /dev/null 2>&1
…

I am a very strong proponent of Open Source (excluding my Macbook). Joining MySQL Inc was a wonderful achievement, being part of the team behind the most popular open source database. Leaving MySQL was not an easy decision due to the people, but the Sun transition and requirements did help. However it is no surprise I am joining another open source company - Primebase Technologies in Hamburg, Germany. My association with the MySQL Community will only be strengthened with my full work and support behind the PBXT and Blob Streaming pluggable storage engines for MySQL.

It is actually poetic that I am joing Primebase for I have the auspicious recognition while an active part of the MySQL community of introducing Paul …

Sometimes technology gets in the way of getting things done, and us tekkies must do what we can to make our data warehouse work for the purpose it is intended. Ralph Kimball makes 10 good points about the common pitfalls to avoid when implementing a data warehouse, as listed in his book "The Data Warehouse Toolkit", published by Wiley.Become immersed with technology rather than the requirements