My talk was right after the PBXT storage engine talk. PBXT is darncool, it looks to be a leading contender for fast localstore nearly transactional data with lots of blobs and varchars. I was taking lots of notes, especially his "gotchas for storage engine developers" and his "streaming blobs" ideas.

Then a whole bunch of people came in for my talk, almost filling the room. There were a few startup problems with the projector and the mike, and we were off.

Unfortunately, I was scheduled opposite a talk on "Highly Available MySQL Cluster on Amazon EC2", which was annoying because I wanted to go to their talk, they wanted to come to mine, and there were likely a lot of people who wanted to go to both.

It …

Starting out bright and early again. I have a feeling Martin and Yannick might gang up on me for waking them up so early every morning and making sure we are on our way at 7am sharp, but it’s worth it. We did make it here for the keynotes. The first session was Freedom Businesses [...]

A Storage Engine for Amazon S3, Mark Atwood

It looks mighty interesting, as transfers to Amazon S3 are free. I think it’ll work well in America and places where bandwidth rocks, but I don’t see this working too well in Australia. Oh how I wish the Internets will improve.

Mark has got all his stuff online at A MySQL Storage Engine for AWS S3. He was also kind enough to upload most of the notes, which made my reporting easier, and don’t forget to view the presentation.

Traditional storage engines use the local disk.

Networked engines: Federated, ODBC, HTTP, MemCacheD and S3 storage engine.

What is S3?
…

So, probably like lots of people - i run a few web apps locally that I use for various purposes. In my case, this also includes some cool custom developed things.

I also use Zeroconf to easily discover all this foo around a network.

I run my critical mysql install by hand - it’s not constantly up. This is so, as somebody noticed (during Eben’s keynote at the MySQL Conference where he talked a lot about privacy) that one of the apps i run is entitled “tax”.

Since I’m somewhere other than at home, my mysql instance was stopped (much harder for people to grab the data out of it if the process isn’t running to begin with).

So yeah… good points - check what random people out on the network may have access to on your laptop - and know what you should not run as default (I’m careful there).

As some of you may now, I’ve been on the Bay Area for 10 days now, and I’m getting used to this. I took part on an Advanced Bootcamp last week to be able to deliver some new MySQL courses at Warp

I don’t have much pictures from the conference (I actually have a lot, but most of them are blurry or with bad lighting), but here’s a little sample from my last week.

MySQL Instructors at the Bootcamp

Mårten Mickos (CEO of MySQL) and Guy Kawasaki (VC and blogger) on stage at the opening Keynote

I’ve always been a big Flickr fan, and user, and love them even more now that their 2GB limit is gone. So this was a most interesting talk, and I think photographers and Flickr users alike will find it interesting what’s behind Flickr. Dathan also has a very interesting blog.

Federation at Flickr: Doing Billions of Queries Per Day, Dathan Pattishall

Database guy - query, performance, troubleshooting, application building, etc. Previously worked at Friendster in 2003.

Master-Slave lag - unable to keep up with the demand that Flickr was having. So they have multiple masters, the had multiple single points of failure (SPOF).

Everything had to be real time. Write intensive, so more than 1 master needed. No more slave lag. Serve pages fast, with many queries. Get rid of the SPOFs and be redundant.

Need ability to make live …

Yesterday my talk about "Scaling the world's largest photo blogging community" went very well and I couldn't be more happier. There were a lot of questions from the audience at the end which made me really happy as it was a clear sign that my presentation wasn't flying over their head :)

Thank you to all those who attended. I will be posting the slides to my talk later tonight (it sucks that blogger doesn't has file upload).

Charles Lee is presenting the session "Using Hibernate to Ease the Migration to Open Source Databases."

Most of the talk so far is Hyperic related.

EJBQL is a poor query language and has bad transaction handling. It also has pessimistic locking and lack of database tools. Entity Beans are proxy objects. Managers interface with entity beans.

Why migrate to Hibernate?

HQL and Criteria based queries supported by Hibernate are fully featured. It does Lazy fetching and doesn't loads objects until you need them. There is a straight forward transaction demarcation. it has secondary cache integration and is a pretty popular framework.

Hibernate POJOs and Transactions: Hibernate POJOs are actually detached from the framework and don't have to inherit the framework API. They are actual objects that can be passed around. Managers look up the POJOs and POJOs travel through the …

Listening to Dathan Pattishall talk about flickr at the 2007 MySQL User Conference. Dathan worked at AuctionWatch in 1999, then in 2003 worked at Friendster, now at Flickr.

Flickr was unable to keep up with demand. Replication was not working, too much slave lag. They came up with some requirements. Needed to support a write intensive site with multiple masters. There should be no single point of failures. Need to have real-time maintenance and be able to serve pages extremely fast.

At AuctionWatch they put folks on separate boxes. At Friendster they had an algorithm that spread folks across many machines. At Flickr they use federation, which is made up of shards, a global ring, and logic to connect shards.

Shards are a slice of a main database. Flickr uses active master-master replication but externalizing the auto increment process.

The global ring is a …

It seems Google couldn’t wait to have their code evaluated for merging into the main source tree, and decided to release it to the general public as patches to 4.0.26 for the community to evaluate.

What have we added and enhanced? The high availability features include support for semi-synchronous replication, mirroring the binlog from a master to [...]