Closing keynote of MySQL UC 2006 is Milton Ngan from Weta Digital talking about using MySQL in making movies. I've heard Milton speak at OSCON in the past, some impressive movie stuff they do with CG.

Milton starts with a review of the movies that have come out of Weta and talks about the number of shots and number of processors they used to render those shots. King Kong had over 2000 shots and required 4,500 processors. Milton is doing this presentation on a Mac.

Weta now has over a petabyte of data on tapes for storing shots. Has 120 terrabytes worth of disks connected with 10Gb ethernet. Milton goes through some photos of the facilities, one photo has a row of cabinets that holds ~2,500 processors. Milton then goes through a number of snips from King Kong showing camera footage with CG layovers.

What uses MySQL at Weta? Production management. Storing details about …

Weta Digital, the company on Peter Jackson films
Heavenly Creatures (1994) nominated for
The Frighteners (1996)
in 1997, Peter Jackston started working on King Kong, but Universal canned it because there were a lot of monster and disaster movies.
Contact (1997) with Robert Zemeckis — the zero-gravity ride was done by Weta Digital
Then they did the Lord of the Rings trilogy (2001-3)
Van Helsing (2004) (just a few bits and pieces)
I, Robot (2004) nominated for best digital effects — technology for armies in Lord of the Rings was used here.
King Kong (2005) and trying to get what Peter Jackson wanted.

(make a chart!)
Size of a movie is based on a shot (camera doesn’t cut away). Visual effects movie typically had 500-1000 shots.

So I got to speak to MIlton Ngan right before his closing keynote. I don’t know where we got the idea that they (Weta Digital) ran Fedora, because all they’ve had are a bunch of Red Hat 9 machines as well as a few Debian. They seemingly are moving fully to Ubuntu real soon now. As always, sticking with MySQL.

Oh, and watch out for X-Men 3, out on May 19th…

Last session of the MySQL UC is Jim Starkey giving an introduction to the new Falcon storage engine. Jim is an icon in the database field, was the creator of MVCC and the BLOB data type. There's a can of Falcon beer for anyone who asks a good question.

What is Falcon

• transactional MySQL storage engine
• based on Netfrastructure database engine
• engine has been in mission critical apps for more than 4 years
• extended and integrated into MySQL

Falcon is NOT

• an InnoDB clone
• Firebird
• a Firebird clone
• a standalone database management management system
• Netfrastructure

Jim's been at this for a long time, there have been some changes since he wrote his first database at DEC:

• 
  
• Uni-processors to …

The slides for the talk I presented together with Zak on Application Clustering with MySQL & Squid is now available. The slides are in OpenDocument format so you need OpenOffice.org to view them. Any comments or suggestions welcome.
Update: slides are now also available in PDF format.

Jim Starkey

Falcon is based on the netfrastructure db engine
Netfrastructure has been deployed in mission critical apps for >4 years.
Extended and integrated into mysql environment.

What Falcon is NOT:
InnoDB clone
Firebird (open source derivative of Interbase db that Jim wrote years ago)
Firebird clone
Standalone Database Management System (it was, inside of netfrastructure engine)
Netfrastructure (netfra is much more with jvm and search, though these features may roll out later)

What Jim’s learned in 20 years
Disks are slower than CPU and memory than they were 25 years ago.
MVCC=Multi-generational concurrency control (how Jim named it, but someone changed it to “version”)
Putting record versions on disks are problematic
Web applications are better and for the future (religion) [I agree, though, for …

Afternoon presentation is Jacob Kaplan-Moss, an author on the django framework. I got to hang out with Jacob a bit in the speaker's lounge yesterday. Cool guy who's passionate about the framework and building maintainable applications. He's writing a Django book (with Adrian) for Apress so was fun to compare notes on experiences. Listening to Jacob makes me want to build a web application, even though I've got 100 other things on my plate.

Jasob starts with an introduction to the Lawrence, Kansas scene and newspaper (where he works). Much like the Snakes and Rubies presentation by Adrian Holovaty in Chicago last November. The Django story is pretty cool, and something that is entertaining to listen to.

Django is similar to MVC but is more like MTV (shows the MTV logo), or Model-View-Template.

Sergei Golubchik

Plugin API is new in MySQL 5.1, so you can plugin your own API commands.

Built-in versioning
Easy to maintain and distribute
Generic — allows you to load any functionality into mysqld

Plugins can add new status variables for SHOW STATUS
For the future, plugins will allow you to add new commandline options, new server variables, and new SQL keywords.

Plugin administration:
INSTALL PLUGIN foo SONAME 'bar.so'
UNINSTALL PLUGIN foo
SHOW PLUGINS
INFORMATION_SCHEMA.PLUGINS
–plugin-dir=/path/to/dir

Plugin types:
Storage Engines
Fulltext Search Parsers
code changes text before it goes to the FULLTEXT data
can be used to search non-plaintext data formats, such as pdf, doc, mp3
can be used to parse …

Mixi is a massive social networking site in Japan, nice user interface demo.

Began in late 2003 with 10,000 users and 600K pageviews a day. As it became clear that the site was viral, the question became how to scale out before getting overwhelmed.

In the first year they went from 600 users to 210,000 users.

2,000,000 users in the second year.

Now 3.7 million users, with 15,000 new users per day. Japan has 86.7 million users.

Site has 70% active users (logging in at least every 72 hours). Average user spends 3 hours 20 minutes on mixi a week. 35th on Alexa worldwide, 3rd in Japan.

Mixi now gets more traffic than Amazon Japan.

Mixi uses LAMP (perl), memcache, squid. Fedora for Linux.

Requests go mod_proxy to mod_perl, then either memcache or a set of MySQL servers divided by blogs, messages and other.

Using more than 100 servers, 10 new servers per month. All done …

Further to my earlier post Hacking MySQL Source (in a good way) in which I was having a compilation problem when re-compiling MySQL source (i.e. tt worked the first build, but failed on subsequent re-compiles, even with no changes).
I’ve been able to solve my re-compiling issue, with special thanks to Jan Kenschke of MySQL AB, who was near during the MySQL Quiz Show.

As suspected, there is no need to re-run the ./BUILD/compile-pentium-debug command as this cleans, a simple make command is sufficient (I needed to confirm no other command args where needed). By doing this, it also better highlighted the actual error, where previously is wasn’t. I didn’t keep a copy of the error, but in effect I needed to do the following in order to successfully recompile.

su - …