A Storage Engine for Amazon S3, Mark Atwood

It looks mighty interesting, as transfers to Amazon S3 are free. I think it’ll work well in America and places where bandwidth rocks, but I don’t see this working too well in Australia. Oh how I wish the Internets will improve.

Mark has got all his stuff online at A MySQL Storage Engine for AWS S3. He was also kind enough to upload most of the notes, which made my reporting easier, and don’t forget to view the presentation.

Traditional storage engines use the local disk.

Networked engines: Federated, ODBC, HTTP, MemCacheD and S3 storage engine.

What is S3?
…

So, probably like lots of people - i run a few web apps locally that I use for various purposes. In my case, this also includes some cool custom developed things.

I also use Zeroconf to easily discover all this foo around a network.

I run my critical mysql install by hand - it’s not constantly up. This is so, as somebody noticed (during Eben’s keynote at the MySQL Conference where he talked a lot about privacy) that one of the apps i run is entitled “tax”.

Since I’m somewhere other than at home, my mysql instance was stopped (much harder for people to grab the data out of it if the process isn’t running to begin with).

So yeah… good points - check what random people out on the network may have access to on your laptop - and know what you should not run as default (I’m careful there).

As some of you may now, I’ve been on the Bay Area for 10 days now, and I’m getting used to this. I took part on an Advanced Bootcamp last week to be able to deliver some new MySQL courses at Warp

I don’t have much pictures from the conference (I actually have a lot, but most of them are blurry or with bad lighting), but here’s a little sample from my last week.

MySQL Instructors at the Bootcamp

Mårten Mickos (CEO of MySQL) and Guy Kawasaki (VC and blogger) on stage at the opening Keynote

I’ve always been a big Flickr fan, and user, and love them even more now that their 2GB limit is gone. So this was a most interesting talk, and I think photographers and Flickr users alike will find it interesting what’s behind Flickr. Dathan also has a very interesting blog.

Federation at Flickr: Doing Billions of Queries Per Day, Dathan Pattishall

Database guy - query, performance, troubleshooting, application building, etc. Previously worked at Friendster in 2003.

Master-Slave lag - unable to keep up with the demand that Flickr was having. So they have multiple masters, the had multiple single points of failure (SPOF).

Everything had to be real time. Write intensive, so more than 1 master needed. No more slave lag. Serve pages fast, with many queries. Get rid of the SPOFs and be redundant.

Need ability to make live …

Yesterday my talk about "Scaling the world's largest photo blogging community" went very well and I couldn't be more happier. There were a lot of questions from the audience at the end which made me really happy as it was a clear sign that my presentation wasn't flying over their head :)

Thank you to all those who attended. I will be posting the slides to my talk later tonight (it sucks that blogger doesn't has file upload).

Charles Lee is presenting the session "Using Hibernate to Ease the Migration to Open Source Databases."

Most of the talk so far is Hyperic related.

EJBQL is a poor query language and has bad transaction handling. It also has pessimistic locking and lack of database tools. Entity Beans are proxy objects. Managers interface with entity beans.

Why migrate to Hibernate?

HQL and Criteria based queries supported by Hibernate are fully featured. It does Lazy fetching and doesn't loads objects until you need them. There is a straight forward transaction demarcation. it has secondary cache integration and is a pretty popular framework.

Hibernate POJOs and Transactions: Hibernate POJOs are actually detached from the framework and don't have to inherit the framework API. They are actual objects that can be passed around. Managers look up the POJOs and POJOs travel through the …

Listening to Dathan Pattishall talk about flickr at the 2007 MySQL User Conference. Dathan worked at AuctionWatch in 1999, then in 2003 worked at Friendster, now at Flickr.

Flickr was unable to keep up with demand. Replication was not working, too much slave lag. They came up with some requirements. Needed to support a write intensive site with multiple masters. There should be no single point of failures. Need to have real-time maintenance and be able to serve pages extremely fast.

At AuctionWatch they put folks on separate boxes. At Friendster they had an algorithm that spread folks across many machines. At Flickr they use federation, which is made up of shards, a global ring, and logic to connect shards.

Shards are a slice of a main database. Flickr uses active master-master replication but externalizing the auto increment process.

The global ring is a …

It seems Google couldn’t wait to have their code evaluated for merging into the main source tree, and decided to release it to the general public as patches to 4.0.26 for the community to evaluate.

What have we added and enhanced? The high availability features include support for semi-synchronous replication, mirroring the binlog from a master to [...]

Came across this post this morning:

http://www.futhark.ch/index.php?action=comment&category=mysql&id=150

The Title is "Are you prepared for the attack of the MySQL worm?"

With all of the real security issues that exist on the Internet I dislike seeing people dream this stuff up.

The author mentions have a piece of SQL inject itself into an event (which you don't need, this could be done with a stored procedure and a sleep() call), and a Federated table.

Let us talk about why this is a non issue:

Databases do not live on the Internet. They live behind firewalls where access is strictly controlled. Databases are not mail servers, web servers, or finger servers. People do not allow others to access their databases directly.

…

Yes, you heard it here first. Monty is the world’s first MySQL Fellow. For outstanding work, being the co-Founder of MySQL AB, and for continuing excellent work. Standing ovation, might I add.

Some quick comments from the clash of the DB egos:
Heikki Tuuri - he worked for Solid, then went on to university, and learned everything about databases. He attributes it to stubbornness. Monty said he should make InnoDB open source, and he obeyed!

Mikael Ronstrom - “entrepreneurial ego in a large company”. He had a vision of what he wanted to do. Guess he was also a bit stubborn…

Jim Starkey had the most interesting hat ;-) And he probably made a couple of bucks from it…
“Does the United States still have anything to say in databases?”

Starting over is important, because you learn new things… Jim Starkey has written about 5 databases in his lifetime.

Nitro: really …