We got a couple of new servers at Solfo recently which showed me one of the reasons virtualization is so popular now: Servers are too fast!

The "standard issue" CPU is now a quad-2.5GHz CPU, so in each server we have 20 GHZ CPU and 32GB ram (at less than $50 per gigabyte it's too cheap to not just fill it up and be done upgrading). Just a few years ago the CPUs we were getting were "only" dual 2GHz, for ~8GHz CPU per box. That's a big increase!

In each "tier" of the application (app servers, db servers, search servers) our main reason for having more than one or two servers is redundancy / high availability - never lack of CPU and rarely because we need more memory.

Here's from one of our webservers (virtualized with Xen with 6 of the 8 CPUs on the "real" hardware).

The big exception is the MySQL servers where we get constrained by I/O so we need a single …

Hmm... understandable from a pure business perspective, but not the best outcome for this engine. It didn't have the highest of profiles and buzz, but I know that users were trying it and liked some of its features. Orphaned open source code tends to not go anywhere, unless someone else picks it up and runs it as a project. See the original announcement with links/refs below:
Update on solidDB for MySQL
By: Dhiren Patel (dhiren) - 2008-03-03 12:12

Dear Community Members,

As you may know, Solid was recently acquired by IBM to strengthen IBM?s Information Management Data Server offerings. As a result, I want to take a moment to give you an important update on the status of solidDB for MySQL.

Those …

I'd call this the 3rd strike and everybody knows what happens next

Marc Fleury has some good answers to the most clueless industry reporter around, starting with:
Spring is touting itself as a JBoss replacement. Smart PR, but false. Spring is a development framework comprising wrappers and dependency injection on top of Hibernate and Tomcat runtimes, both developed, and monetized by JBoss.

You can drop some balls, no one can keep track of what's going on in Open Source land, it's difficult enough to track what's going on in MySQL, Drupal, Virtualization and Distribution land but if you realize you don't have the whole picture (like not …

Sorry for the delay, now on to range queries and lenient updates.  Let’s call them queries and updates, for short.  So far, I’ve shown that B-trees (and any of a number of other data structures) are very far from the “tight bound.” I’ll say a bound is a tight if it’s a lower bound and you can come up with data structure that matches it.

So how do we match the bandwidth bound for queries and updates?  I already mentioned in passing how to do this, but let’s look more closely.

Fast Updates

The way to get fast updates is to log them.  You can easily saturate disk bandwidth by writing out the insertion, deletion and update requests with no index. 

A query now will typically start by sorting the data.  Even a point query requires looking at all the data, but a range query requires looking at all the data log times (in order to sort it), or using a large amount of extra …

In a previous post I explained about BCP. I have just finished my latest and nearly final test, and all worked as expected. For about an hour certain front end servers where hitting a database shard in 1 datacenter while the rest of the front ends hit the same shard in another datacenter.

What makes this incredible is that now data from mySQL can be close to geo-graphic locations of the end user, without having to make changes to the front end application. So fail over is silent from a database perspective if an entire datacenter is down. Actions outside of the application are also replicating seamlessly and in order. Latency is high but the goal is not to have a WWW in one datacenter talk to a shard in another datacenter. So, latency is in effect not an issue especially if one uses Akami-DNS to geo-graphically loadbalance your user …

Some among us (myself included) once worried that IBM was joining with Oracle to besiege MySQL when it acquired SolidDB, one of MySQL's primary storage engines. It turns out, however, that IBM didn't have such nefarious plans.

In fact, it didn't (or doesn't) have plans for ...

(A guest blog by Danny O'Brien, the EFF's cultural ambassador)

As one of the Electronic Frontier Foundation's outreach folk, I have to concede that our message is not often about cyber-unicorns and crypto-ponies. We're often warning companies and hackers about what we see as upcoming threats to their rights, and urging them to take action. To give some examples: one year, I gave a Emerging Tech talk called "The Wheel of Plaintiffs", in which we span a giant Flash wheel to see who we thought in the audience might get sued next, and by whom. Another year, I heard a hacker come out of our software patent tutorial muttering "Man, that was the most depressing talk I've ever heard". I'm sure that if there was a box in the O'Reilly audience feedback forms that said "Speaker Made Me Reconsider Landscape Gardening …

It’s not open source, but those involved with data management might be interested in my first post over at the 451’s new Too Much Information blog as it tracks the progress of H-Store, the new OLTP database project from Michael Stonebraker, who you may remember from such database projects as Ingres and PostgreSQL.

Given Jonathan Schwartz’s proclamation that Sun will make further open source acquisitions, I’ve been putting some thought into likely targets and/or new directions opened up by the MySQL acquisitions. One likely target sector is the ecosystem of vendors that surround the MySQL database - such as clustering and HA software providers - as well as complementary technologies.

With that is mind it is interesting to see that the company has hired two key Python developers, Frank Wierzbicki and Ted Leung. As the Infoworld report states, this is similar to the way Sun …

I saw a post by Baron mentioning that his tool maatkit is best for handling situations where a master-master replication setup has got out of sync.

If you think Baron was blowing his own trumpet he has good reason to. I have used his mk-archiver tool as part of the Maatkit to make the problem of archiving and purging data much easier. This was much easier than rolling my own solution.

Anyhow. I