At Flickr adding capacity is easy. Since we are able to scale at a function of user growth, independent of hardware classes. This makes adding new hardware easy. I'm able to adjust the server weight on the FLY to give more users to a certain class of hardware over another.


Changing schema is easy as well and now I do it much safer. In the pass I would turn off a master for each partitioned dataset all at the same time: then stay up for 20+ hours and execute an alter across the entire server farm, that was off line.

Now I do it in stages and do it throughout the week. Its a little slower but I get to sleep. Additionally with the new method I am able to do more all at the same time-like rebuild the entire dataset for a partition.

Here are my steps for doing a change that takes more then 10 hours:

remove the servers from the site config
push my.cnf.maintenance to the servers …

One performance gotcha with MEMORY tables you might know about comes from the fact it is the only MySQL storage engine which defaults to HASH index type by default, instead of BTREE which makes indexes unusable for prefix matches or range lookups. This is however not performance gotcha I'm going to write about. There is one more thing you should be aware which again comes from the fact MEMORY tables use HASH indexes by default.

I've created rather similar test table:

PLAIN TEXT SQL:

1. CREATE TABLE `test` (
2.   `id` int(11) NOT NULL AUTO_INCREMENT,
3.   `c` tinyint(4) DEFAULT NULL,
4.   PRIMARY KEY  (`id`),
5.   KEY `c` (`c`)
6. ) ENGINE=MEMORY

and populated it with 1.000.000 rows ALL of them having same value for c column.

Now I'm performing random deletes by primary key (DELETE FROM test WHERE …

Microsoft has an offer open-source startups are having a hard time refusing. Should they?

(Credit: Microsoft (Sam Ramji))

That's the question I asked myself while reading Mary Jo Foley's excellent article that dissects Microsoft's open-source strategy. As it turns out, it's very similar to Microsoft's general partner strategy: embrace and envelope. (Or embrace, extend, and extinguish, as used to be Microsoft's marching orders.)

Microsoft is looking at open-source software (OSS) as just another flavor of independent software vendors (ISV) software. Microsoft's goal is to convince OSS vendors to port their software to Windows. But Microsoft doesn't want OSS software to just sit on top of Windows; the company wants this software to be tied into the Windows …

Welcome to the 82nd edition of Log Buffer, the weekly review of database blogs. Some MySQL news to start. They call the engine Maria. “They” being MySQL AB co-founder Monty Widenius, and Maria being his new storage engine for MySQL. On his new blog, Monty Says, Monty says Maria is, “. . . a crash-safe alternative [...]

1. Zend and Microsoft have done a lot of work in the last year to make PHP run well on Windows. I never understood why Microsoft devoted resources to that, but now it is clear: it's going to make technical integration between the two organizations go much more smoothly.
2. This will hopefully be the death knell of the awful "Windows Live" branding for consumer web stuff.
3. Maybe now somebody will release a .NET OpenID 2.0 library that actually works.
4. Big losers here from a tech supplier perspective: FreeBSD and MySQL.
5. Another potential big loser is Adobe; having a gigantic global audience will help with adoption of Microsoft's various Adobe-killing initiatives like Silverlight, which would never otherwise have penetrated the consumer web without a large built-in audience like Yahoo's.

…

1. Zend and Microsoft have done a lot of work in the last year to make PHP run well on Windows. I never understood why Microsoft devoted resources to that, but now it is clear: it's going to make technical integration between the two organizations go much more smoothly.
2. This will hopefully be the death knell of the awful "Windows Live" branding for consumer web stuff.
3. Maybe now somebody will release a .NET OpenID 2.0 library that actually works.
4. Big losers here from a tech supplier perspective: FreeBSD and MySQL.
5. Another potential big loser is Adobe; having a gigantic global audience will help with adoption of Microsoft's various Adobe-killing initiatives like Silverlight, which would never otherwise have penetrated the consumer web without a large built-in audience like Yahoo's.

…

… or, “Making Event Attendance Count”

Late last year, I gave a keynote at paired Finnish conferences MindTrek and OpenMind. While the events were well worth attending, afterwards I spent a few bleak hours thinking about the actual costs of my attendance. If I had left Canada just for these events (which, thankfully, I didn’t) then a naive estimation of costs would have been something like this:

• ~20 000 km of air travel (Vancouver to Frankfurt, Frankfurt to Helsinki. Return.)
• ~7 000+ CAD of costs (flights, hotels, taxis, meals, time) (borne by a combination of eZ Systems, the Mozilla Foundation and the …

Is 2008 going to be the Year of the Acquisition? Activity in 2007 was on the rise but now things seem to be at full speed.

• I remember when Alta Vista and Excite! were the hot search engines, my how the world has changed.  It looks like it’s narrowing down to a two horse race with Microsoft putting the moves on Yahoo! for about $44.6 billion (Notes from SearchEngineLand). I guess it’s really on now, Google versus Microsoft in a search engine death match.
• Yahoo! acquired Zimbra last year. I wonder …

MySQL 5.1 offers an extremely useful feature called information schema plug-ins. This feature allows dynamic runtime loading of a shared library into the MySQL server to implement a table in the information_schema database. The SQL standard (ISO/IEC 9075-11:2003) allows database implementations to extend the information_schema. MySQL 5.1 transfers the possibility to do this directly to privileged database users so they can extend the information_schema themselves, in any way they see fit.

In this article, we will demonstrate how to create a minimal "Hello, World!" MySQL information schema plugin. In a forthcoming article, we'll demonstrate how …

MySQL's Certification exams are given in Pearson VUE test centers or at the MySQL Users Conference. Ignoring the numbers from the UC, the growth in exams taken last year was impressive. The Americas and EMEA grew at 25% and 26%. Pacific grew 91%. And in Japan we grew an amazing 166%!

The UC numbers are numbing by themselves. We will be printing in the neighborhood of 500+ exams to cover the six testing periods during the conference. Considering how exciting the speakers and presentations are, having that many people dedicate time in a relatively small room to sit demanding exams is impressive.