It is always surprising what I learn when I go to a conference these days. Years ago, I could go to any talk and just suck it all in. Now, it is the little nuggets. The topics as a whole do more to confirm what I have already developed while running the Phorum project and building the infastructure for dealnews.com. That confirmation is still nice. You know you are not the only one that thought a particular solution was a good idea.

One of the confirmations I have had is that the big sites like Flickr, Wikipedia, Facebook and others don’t use exotic setups when it comes to their hardware and OS. During a keynote panel, they all commented that they did not do any virtualization on their servers. Most did not use SANs. Some ran older MySQL versions but some were running quite recent versions. I have kept thinking that I …

10,000 Tables Can?t Be Wrong: Designing a Highly Scalable MySQL Architecture for Write-intensive Applications by Richard Chart

Chose MySQL for performance and stability, and less important but still there, experience and support. Support is becoming increasingly more and more important.

Starting point: 1 appliance supporting 200 devices
Problem/Goal: Extensible architecture with deep host and app monitoring, over 1000 devices with 100 mgmt points each
Distributed collection over a WAN, with latency and security concerns
Current reality: several times the scale of the original goal
Commercial embedded product, so they actually pay for the embedded MySQL server

Future: The fundamentals are sound: next generation of the product moves up another order of magnitude

Data Characteristics
>90% writes
ACID not important
Resilient to loss, because gaps in data do not …

So, I am at the MySQL Conference this week with my Phorum co-developers. We got to talking last night about how old Phorum is. We knew it was about 10 years. We pulled up some old archived zip file of version 1.5 and found in the this in the comment block.


* Created 04/16/1998


Whoa! That means that yesterday was the 10th birthday of the Phorum project. I would guess that is the date I originally put the code up on my personal web site for people to download. I remember sending that email to the PHP General mailing list. I told people they could have the code if they would help debug it. Later I officially made a GPL license and then a BSD style license as I became more knowledgeable about the open source and free software world.

So, for kicks we decided to install version 1.6 on the phorum.org site. Keep in mind the release date for that was March 30, 1999. The only hurdles …

Here is the quick notes from the session Helping InnoDB scale on servers with many cores by Mark Callaghan from Google (mcallaghan at google dot com).

• we have a team now, to help scale MySQL to do the enhancements (9 people,  I hope yahoo management reads this)
• Overview
  • describe the problems on big servers
  • work done by InnoDB community
  • ask MySQL/InnoDB to fix the problems by taking the patches
• Community team
  • InnoDB/Oracle
  • Google MySQL team
  • InnoDB community
  • Percona - Peter and Vadim
…

The Future of MySQL by Monty Widenius and Jay Pipes.

• Why this talk
  • MySQL and Sun should become more transparent
  • Easier to discuss and act when you have facts
  • when user know the limitations, they know how to go around
• Threads
  • one connection/thread doesn’t work well
  • no priority threads
  • no way to ensure we have X threads
• Symptoms
  • Too many context switches
  • we are not using multi-core efficiently
  • does not scale that well after 4-8 cores
• Solution
  • –thread-handling=pool-of-threads (6.0) …

I've so far missed every single keynote and breakfast. Fortunately, all the good ones have been recorded.

I'm sitting here in Paul's PBXT talk, and it's really cool. It was really cool last year, and it's only gotten better.

When someone asks me about the various future MySQL storage engines, my answer always is "It depends on which is better for your load. Maria or PBXT, or both?"

Final slides from keynote delivered this morning at the MySQL user conference. Topic was protecting live MySQL databases.

(Slides render well in both OpenOffice and PowerPoint)

This was my reason #1 to attend the UC and it lived up to my expectations!
The Lost art of the self join was a truly enjoyable experience. Beat Vontobel delivered with confidence and humor a talk about a subject that most people would consider dull.
It was a feast for all attendees with an inclination for hacking.

The truly amazing thing in this presentation was not that Beat explained how to solve a Sudoku puzzle with one query which is a truly amazing feat, but that he explained how to use the join mechanism to implement a backtracking machine, similar to a regular expression algorithm.
The presentation was concluded with a live solution of a 6x6 sudoku puzzle. The reason for not using a 9x9 puzzle is …

Watching the MySQL uproar unfold brings to mind an array of random thoughts:

1. Microsoft really should acquire an open-source company. It would get credit for opening up, even if it closed off some parts of whatever project it acquired. Sun...? It's getting lambasted for no logical reason at all. It's not as if MySQL wasn't actively considering tweaks to its model before Sun acquired them, just as all open-source companies do. No one has settled on the exact right model yet.
   
2. While Microsoft and other "proprietary companies" move toward opening up, it would appear that there is some movement among "open-source companies" to close off. Maybe we'll meet in the middle?
   ...

I've been getting pings all morning about my thoughts on the Slashdot article:
http://developers.slashdot.org/article.pl?sid=08/04/16/2337224

Google reveals my thoughts on crippleware. I've never been particularly quiet on the subject of the "Open Source 2.0" branding cycle of trying to say that a "mostly free" open source project is open source. The entire movement of "mostly free" pretty much disturbs me.

I do not believe though that this subject is cut and dry. I am going to use Innodb as an example. I do not consider it crippleware, and I have been amazed and delighted at Oracle's ownership of it. They have made excellent stewards of the source code.

Do they have a closed source backup tool? Yes.

Do I have an issue with this? No.

Why not? We can use Innodb just fine …