昨日開催された shibuya.pm #12 - NoSQL特集で使用したスライドを slideshare にアップロードしました。

高度に進化した分散データストアについてView more documents from kazuho.

　開発しているシャーディングミドルウェアである Incline と Pacific については YAPC::Asia 2009 を始めいろいろな所で話をする機会をいただいてきたので、今回は、なぜ RDBMS …

For couple of months I have been writing middlewares for database shards, and today I made a presentation covering them.  It includes the following.

• Incline - a trigger and queue based distributed materialized view manager
• Pacific - a set of perl scripts to manage MySQL shards, a MySQL shard can be split into two in less than 10 seconds of write blocking (and no read blocks)
• DBIx::ShardManager - a client API for accessing database shards using Incline and Pacific

With these middlewares I think it is no more difficult to write web applications that runs on database shards.  In fact IMHO it is as easy as writing a webapp that runs on a standalone database.

The presentation slides are available from slideshare.  If you have any question or suggestions, please leave a comment.  Thank you.

…

For the last weeks, I have been writing a tool called "Incline," a program that automatically maintains consistency between sharded MySQL databases.  The aim of the software is to free application developers from hand-writing code for keeping consistency between RDB nodes, so that they can concentrate on writing the application logic.

Background

Denormalization is unavoidable in a sharded RDB environment.  For example, when a message is sent from a user to another user, the information should be stored on a database node where the sender of the message belongs to, and on another node where the receiver does.  In most cases, denormalization logic is hand-written by web application developers, and since it has been a burden for creating large-scale web services.  Incline takes off the load from developers.  By reading the definition files, Incline keeps the tables on a sharded MySQL …

　先週、概要を紹介させていただいた Pacific について。まだ API をフリーズしていないつもりなのですが、だいぶ整ってきた気がするので、ざっくりまとめておきたいと思います。

インストール手順

1. Thrift をインストール注1
2. Pacific の svn レポジトリからチェックアウト
3. Perl ドライバを make (cd driver-perl && perl Makefile.PL && make all test install)
4. リゾルバを make (cd resolver && make)

テーブルのセットアップ手順

…