I'm sitting here at the end of the MySQL Conference 2007 waiting for "Orcas" beta 1 to uninstall.  Yes, I know I should have used a VM but that's not the issue.  The issue is that I'm missing a great closing keynote on Yahoo Pipes while I wait for setup to "generate setup script".  Folks, it's been doing that for 30 minutes!

I'm running Vista Ultimate on this Inspiron 6000 but I know these absurdities are not limited to lower end hardware.  I've got a dual-Opteron box at my house and uncompressing zip files is a joke.  Right-clicking a 30 meg zip on my home system (SATA, 2 gig RAM, dual Opteron) often takes at least 10 minutes to tell me that the rest of the uncompress will take another 20.  Enter 7-zip stage left and the same uncompress is done in about 90 seconds.  I've code for almost 20 years and I can tell you that I would have to work very hard to write code this bad.  It has to be …

MySQL Conference 2007 Day 4 rolled quickly into the second keynote Scaling MySQL at YouTube by Paul Tuckfield.

The introduction by Paul Tuckfield was; “What do I know about anything, I was just the DBA at PayPal, now I’m just the DBA at youTube. There are only 3 DBA’s at YouTube.”

This talk had a number of great performance points, with various caching situations. Very interesting.

Scaling MySQL at YouTube

Top Reasons for YouTube Scalability

The technology stack:

• Python
• Memcache
• MySQL Replication

Caching outside the database is huge.

It a display of numbers of hits per day it was said “I can neither confirm or deny the interpretation will work here (using an Alexa graph)”. This …

My hierarchies session is done. It’s nice to be finished and free to relax.

The materials and video are available at http://www.openwin.org/mike/presentations/managing-hierarchies-in-mysql/

As the trend of online business is increasing day by day the work of hosting as well as website design companies is increasing as well. Top class business sites not only require dedicated web server but also strongly need ecommerce web site design. In order to facilitate their clients through proper services the business sites do require high web hosting bandwidth package as well. One can …

So one of the things I finally started to figure out, at least sorta, is the concept of VIEWS. I had known about them for some time but never really understood why to use them. I had thought they provided worse performance than by not using them and simply using JOINs in a query. Well...the results I found were somewhat surprising. When using a view to de-normalize a dataset used for my Blog, I noticed an improvement of about 4-6 seconds on average when running 80 threads with 10000 iterations than by using the standard query. I ran these tests multiple times with the same configuration and even tried different numbers of threads/iterations. In all cases, using the view was faster. For completeness, here is the original query and the updated one using the view:

Using the view:

SELECT * FROM blogposts_view 
WHERE postid=$randomNumber;

Without the view:

SELECT BlogPosts.title, BlogCategories.name, …

And just like that the 2007 MySQL Conference is over and folks are on their way back home, or off to vacations, or whatever comes next.

It has been a great week hearing how others are using technology to build good things. It has also been good to meet folks and have good conversation sharing ideas about what works, what doesn't work, and how to change things so they will work.

MySQL is in a really good place right now. I commented to more than one person that it is impressive the kinds of folks who are driving their business/application with MySQL and came to talk about it. These include (in no order) YouTube, digg, flickr, Yahoo!, Wikipedia, Google, NASA, US Navy, SugarCRM, TicketMaster, and more. It is hard to beat that list when it comes to proof that a technology is viable.

Here's to another good year of pushing the LAMP stack and open source to wider use and …

In my fourth day at the MySQL Conference and Expo 2007, I attended several great sessions, starting with my own. If you’re wondering why this is a day late, it’s because the conference ended in the late afternoon, and they almost immediately – within a half-hour or so – removed the free wireless Internet access in the hotel. That was uncool. Keynotes etc I skipped the opening keynotes to polish off last-minute changes to innotop and my presentation.

Closing keynote of the 2007 MySQL Conference is Pasha Sadri from Yahoo! talking about pipes.

Pasha starts with a story about looking for an apartment near a park (this happened to him a few years ago). It was very tedious to go looking on Craigslist every hour clicking on every apartment and looking to see if the apartment was near a park.

Made a 50-line Perl script to merge database between Craigslist and Yahoo! local to get data sets. This turned into Yahoo! pipes which provides a visual editor to take data sources and create a set of controls over that data source and output the data in a certain way.

If you haven't seen pipes it is hard to describe, but very cool.

For design, pipes relied heavily on the idea of Unix pipes. The goal to …

Adobe open sources Flex. MySQL AB releases new performance benchmarks. SugarCRM supports Oracle Unbreakable Linux. (and more)

Adobe to Open Source Flex, Adobe (Press Release)

MySQL AB Announces Record Cluster Database Benchmark Results, MySQL AB (Press Release)

SugarCRM Expands Support for Oracle Unbreakable Linux Support, SugarCRM (Press Release)

EMC and Oracle Join Forces Under the Oracle Unbreakable Linux Support Program, Oracle (Press Release)

…

mysqlconf mysqlconf07

It’s not easy to do a DW in MySQL — but it’s not impossible either. Easier to go to Teradata than to write your own.

DW characteristics:

1) Organic, evolves over time from OLTP systems — issues, locking, large queries, # of userss.

2) Starts as a copy of OLTP, but changes over time — schema evolution, replication lag, duplicate data issues

3) Custom — designed from the ground up for DW — issues with getting it started, growth, aggregations, backup.

4) How do you update the data in the warehouse? — write/update/read/delete, write/read/delete, or write only — which means that roll out requires partitions or merge tables.

The secret to DW is partitioning — can be based on:
data — date, groups …

In a community session yesterday at MySQL Conference 2007, I first heard about Dorsal Source. A collaboration between Solid DB and Proven Scaling that allows for community people to upload patches to MySQL, get it compiled across multiple platforms, and have a downloadable distribution available on H/W individual contributors will never have access to.

That’s a great idea. I also hope we get the opportunity to get compiling of patches into multiple versions, as well to get builds of a lot of patches together. Personally, I’m running 3 versions just to diagnose one problem. 5.0.36 with a custom binary change, 5.0.37 so I have SHOW PROFILE, and 5.0.33 so I have …