So I have mentioned the BLOB Streaming Project quite a lot lately, and for good reason! Paul has recently released an alpha version of the engine itself, called MyBS. I took some time last weekend to benchmark it's performance and thought I would share my result. Keep in mind, these are rather crude, but I think they do get the point across.

For the tests I used a slightly modified version StuffDawg since it already stores images inside the database. Oddly enough, I found a few bugs with StuffDawg that I was able to patch up and will be releasing an update soon.

So before I explain things, here is the juicy stuff - the results:

tim@filedawg ~/tmp $ time wget --quiet http://stuffdawg/image.php?itemid=20   

real    0m0.767s
user    0m0.000s
sys     0m0.012s
tim@filedawg ~/tmp $ time wget --quiet …

Sometime in the next six to 12 months a bad guy will use one with its
web interface to trigger a bomb.

I would also place odds that within the next 12 to 24 months, some
kid will hack an alien ship to stop an invasion of earth with an iPhone.

Movies count.

And here you only thought you only read my blog for advice on
databases and hacking right?

Saw the latest Die Hard film today... it was good... but very painful...

I just noticed PeterZ's post on page counters:
http://www.mysqlperformanceblog.com/2007/07/01/implementing-efficient-counters-with-mysql/

Let me offer a different solution.

Do not do it.

Really, if you grow a site to any size at all you will discover that
the contention you create on updates is just too much. Sure, you can
optimize it by have the update done on a de-normalized table, but
unless you are using 5.1 with its row level replication, eventually
the replication locks will nail you.

Instead build an accesslog type table. You probably already have one,
aka you are recording your accesslog in your database. Just run a job
every so often that updates the counter on your object in its own
table (which will …

Last week The Open Road caught up with Justin Steinman @ Novell and Mike Olson @ Oracle to discover how open source factors into these companies' businesses. This time, we're switching gears a bit to talk with a company that sells services around software - both open source and proprietary - rather than a software company.

Being familiar with the interesting open source work happening at SAIC, I decided to talk with two members of its Open Source Community of Practice: Ryan Brunton, a developer within SAIC's Open Source Community of Practice, and Wayne Waddoups, vice president of Strategy, SAIC Office of Technology. SAIC has long worked with projects like Linux and MySQL, but it's the cutting edge work it's doing with open source applications and infrastructure that caught my eye. …

Glyn Moody has an excellent article in Redmond Magazine on open source and interoperability. As it turns out, sometimes it takes Microsoft to notify the open-source community that for all the great things we've done, we sometimes fall short. One area that open source had traditionally failed in was in stitching together an end-to-end solution, as Nick McGrath (a friend and a wonderful person) suggests:

Of all the accusations Microsoft has leveled over the years against open source, perhaps the least contentious is that it lacks the tight integration offered by Microsoft's own products. As Nick McGrath, director of platform strategy for Microsoft in the United Kingdom, puts it: "One of the problems I've seen with open-source software is it doesn't take on board some of the issues that customers have around interoperability and integration. …

When seeing that the MySQL Proxy was released, I decided to try to experiment with it since I see strong potential with this tool, both for replication and for other uses (recall that I'm a replication guy, so this is my primary focus). I'm actually on vacation, but this will of course not stop me from tinkering with things (I know, I'm just a hopeless case in this aspect ;) ).

After reporting a minor bug, I managed to build and run it with some sample scripts. I'm using Kubuntu Feisty, and had some initial problems, but it was actually pretty straightforward. I'll repeat the steps anyway, in case anybody else have problems.

1. Get the source from the repository svn co http://svn.mysql.com/svnpublic/mysql-proxy/ mysql-proxy
2. Make sure you have all packages necessary. Several of the packages …

On many web sites you would see a counter how many time given object - blog post, forum thread, image, movie etc was viewed. This is sometimes handy feature but it can be rather expensive from performance point of view.

The nasty thing with counters as they are implemented the most trivial way - they convert read load to write load. When you would simply fetch given object information now you do not only fetch the data but also update the view counter.

For smaller single system web site with no caching the overhead may well be insignificant - as you run update for the same row which just was fetched it does not cause any synchronous IO and background IO can be batched rather effectively.

As the system growths however and you implement some form of caching, ie memcache you end up reducing number of reads from database dramatically but writes still have to happen at full speed. It also really hurts if you choose replication …

Version 0.5.0 of MySQL Proxy was released a week ago, and in the meantime there have been 25 revisions.
If you want to try the latest version in your box, you can get the sources from the svn repository and build it yourself.

svn co http://svn.mysql.com/svnpublic/mysql-proxy/ mysql-proxy

Then, you can build and install this way.

./autogen.sh && ./configure && make && make install

That works beautifully on my Linux desktop.

So far, the good news. The bad ones is that on my Mac OS X laptop, things are not so smooth.
First, aclocal complained about missing macros. Jan was online, and he helped me fix the problem with an environment variable.
Then …

While checking out the new features due for MySQL 6.0 (Falcon) the other day, I happened to browse my way past the Certification pages to see if anything had changed - am I glad I did!

It was during my browsing that I noticed the Important Information that detailed a typsetting error with the voucher expiry date found in the MySQL 5.0 Certification Study Guide……..the same guide that had been sat on my lounge floor for the last few months…..

Quickly flipping through the pages I found the voucher that offered 25% off your exam and then went …

We choose our joys and sorrows long before we experience them --Kahlil Gibran

MPP is hardware growth (scalability) by adding small 2 to 4 CPU servers, preferably cheap, to the existing infrastructure. SMP is replacing the current hardware by getting a bigger, badder, meaner box.

ETL

While Ab Initio does provide hash partition based parallelism capabilities that can be used in MPP environments, the effective use of this approach is a non-trivial undertaking. MPP is rarely used for ETL processing. Contemporary update: Hadoop and Hive are coming up fast to provide a viable solution in this space.

Reporting

It is easy to split reporting load into smaller chunks …