Sergei Golubchik

Plugin API is new in MySQL 5.1, so you can plugin your own API commands.

Built-in versioning
Easy to maintain and distribute
Generic — allows you to load any functionality into mysqld

Plugins can add new status variables for SHOW STATUS
For the future, plugins will allow you to add new commandline options, new server variables, and new SQL keywords.

Plugin administration:
INSTALL PLUGIN foo SONAME 'bar.so'
UNINSTALL PLUGIN foo
SHOW PLUGINS
INFORMATION_SCHEMA.PLUGINS
–plugin-dir=/path/to/dir

Plugin types:
Storage Engines
Fulltext Search Parsers
code changes text before it goes to the FULLTEXT data
can be used to search non-plaintext data formats, such as pdf, doc, mp3
can be used to parse …

Mixi is a massive social networking site in Japan, nice user interface demo.

Began in late 2003 with 10,000 users and 600K pageviews a day. As it became clear that the site was viral, the question became how to scale out before getting overwhelmed.

In the first year they went from 600 users to 210,000 users.

2,000,000 users in the second year.

Now 3.7 million users, with 15,000 new users per day. Japan has 86.7 million users.

Site has 70% active users (logging in at least every 72 hours). Average user spends 3 hours 20 minutes on mixi a week. 35th on Alexa worldwide, 3rd in Japan.

Mixi now gets more traffic than Amazon Japan.

Mixi uses LAMP (perl), memcache, squid. Fedora for Linux.

Requests go mod_proxy to mod_perl, then either memcache or a set of MySQL servers divided by blogs, messages and other.

Using more than 100 servers, 10 new servers per month. All done …

Further to my earlier post Hacking MySQL Source (in a good way) in which I was having a compilation problem when re-compiling MySQL source (i.e. tt worked the first build, but failed on subsequent re-compiles, even with no changes).
I’ve been able to solve my re-compiling issue, with special thanks to Jan Kenschke of MySQL AB, who was near during the MySQL Quiz Show.

As suspected, there is no need to re-run the ./BUILD/compile-pentium-debug command as this cleans, a simple make command is sufficient (I needed to confirm no other command args where needed). By doing this, it also better highlighted the actual error, where previously is wasn’t. I didn’t keep a copy of the error, but in effect I needed to do the following in order to successfully recompile.

su - …

By tim

Continuing my series of queries about how "Web 2.0" companies used databases, I asked Cal Henderson of Flickr to tell me "how the folksonomy model intersects with the traditional database. How do you manage a tag cloud?"

He replied: "lots of the 'web 2.0' feature set doesn't fit well with traditional normalised db schema design. denormalization (or heavy caching) is the only way to generate a tag cloud in milliseconds for hundereds of millions of tags. you can cache stuff that's slow to generate, but if it's so expensive to generate that you can't ever regenerate that view without pegging a whole database server then it's not going to work."

Here's the full text of my exchange with Cal:

The first question I asked was "what's your …

Jacom Nikom (MIT Lincoln Lab and Boston MySQL Meetup regular) is presenting about performance when processing sensor data stream.

Where does the data come from? Multiple locations, 10 pictures of different kinds of sattelite and telescope equipment. I get the sense that there is a ton of data that is being collected. Sensors send messages to a publish/subscribe middleware which apply algorithms, sends it through a historian and down to MySQL.

The historian creates a thread for each subscription. The historian processes the message and creates an INSERT query. This is happening from many sensors, each operating at as many as 100 messages/second.

When choosing a database they looked at benchmarks, tried to determine performance based on the predicted databases. Wanted to know how MySQL would respond to moving.

Jacob gets into some serious detail about performance on various versions of Linux, hardware, and MySQL. The …

This morning Jay mentioned in the MySQL UC keynote introduction that Beat Vontobel had helped save a woman's life earlier this week.

Without getting a lot of detail the story is that Beat was outside on the sidewalk and witnessed this major accident out on the road. He dropped everything and ran to the car where he checked pulse and breathing and talked to the person as she regained conciousness while the emergency response was on the way. Once they arrived he helped by moving to the back seat of the car and holding the driver's neck straight so the emergency response person could perform checks without risking movement of the spine.

Man, that's remarkable.

Everybody can contribute to the MySQL Open Source Community. Here are some ideas:

• You can start with completely a MySQL Conference Survey form (those forms that we are all avoiding).
• You can download, evaluate and test newer versions of MySQL Products.
• You can contribute to the MySQL Forums to ask questions and even give answers.
• Log Bugs & Enhancements on MySQL Products.
• Write about your experiences with MySQL in a Blog and add your Blog to PlanetMySQL. With now over 100 feeds, if everybody wrote something about MySQL in the next year, we would have over 1000 feeds next conference.
• Contribute your thoughts (via wiki), MySQL products, code snippets and …

Brian Aker — Director of Architecture

MySQL Server
SQL-Based, aiming to be SQL-03 compliant. Stable, scalable, easy to use, modular, high-performance RDBMS.

Client Library Support
Libmysql c-library (think OCI)
JDBC — type IV
ODBC
Perl DBD::DBI
PHP (built in)
ADO.Net, OleDB, Ruby, Erlang, Eiffel, Smalltalk and more provided by third parties.

So great because you can connect in many ways to different storage engines
Architecture

sql/ is for kernel
mysys/ is the portable runtime library. MySQL is ported to 52 platforms
(was 53 but Brian deleted OS/2 last week ). The portable runtime library wraps commands like pwrite, unlinking and renaming files so that operating system doesn’t matter.
mysql-test/ is for your test cases — run “mysql testrun –record”. Can take SQL from web …

I am sitting in Dorion's session about scaling the real time web. Technorati tags grew from 0 to 100 million in a year.

Technorati has about 10TB of core data in MySQL over about 20 machines and they use replication. With replication they add 100TB and 200 machines more. Currently growing at about 1TB per day in total.

A service oriented architecture to separate physical and logical access is used. Technorati uses commodity hardware and Open Source software.

Scaling Technorati Tags : Launched on Janurary 10 of 2005.

Tags are partitioned by entity (tags and posttags). Tags database is separate. Post information in one set of databases and tags information in another set of databases. A caching layer is also present. Overtime Technorati has blended the use of InnoDB and MyISAM based on use.

Currently around 120 million tags in a single table distributed using replication. …

Second keynote of Thursday morning (last day) at MySQL UC 2006 is Mitch Kapor talking about the Wikipedia. Mitch starts with a survey of the crowd. Most people in the crowd use the wikipedia, 15% of the crowd edits the wikipedia.

The typical response from a person that doesn't know about the wikipedia says "The Wikipedia can't possibly work..."

but it does

The mainstream media is very skeptical. Anyone can edit any article at any time. This radical openness is more open than open source projects. There is no review and anyone can post. The radical openness provides opportunities that have never been tried before.

Ideas that are challeneged by the Wikipedia:

• 
  
• someone has to be in charge - even the sysadmins are volunteer and do not keep a …