In addition to the Spider engine, Kentoku SHIBA has also created the …

This Thursday (October 15th, 13:00 UTC), Giuseppe Maxia will present the Spider Storage Engine. Here's from the abstract: Everybody needs sharding. Which is not easy to maintain. Being tied to the application layer, sharding is hard to export and to interact with. The Spider storage engine, a plugin for MySQL 5.1 and later, solves the problem in a transparent way. It is an extension of partitioning. Using this engine, the user can deal transparently with multiple backends in the server layer. This means that the data is accessible from any application without code changes. This lecture will briefly introduce MySQL partitioning, and then show how to create and use the Spider engine, with some practical …

This Thursday (October 15th, 13:00 UTC), Giuseppe Maxia will present the Spider Storage Engine. Here's from the abstract: Everybody needs sharding. Which is not easy to maintain. Being tied to the application layer, sharding is hard to export and to interact with. The Spider storage engine, a plugin for MySQL 5.1 and later, solves the problem in a transparent way. It is an extension of partitioning. Using this engine, the user can deal transparently with multiple backends in the server layer. This means that the data is accessible from any application without code changes. This lecture will briefly introduce MySQL partitioning, and then show how to create and use the Spider engine, with some practical …

This Thursday (October 15th, 13:00 UTC), Giuseppe Maxia will present the Spider Storage Engine. Here's from the abstract: Everybody needs sharding. Which is not easy to maintain. Being tied to the application layer, sharding is hard to export and to interact with. The Spider storage engine, a plugin for MySQL 5.1 and later, solves the problem in a transparent way. It is an extension of partitioning. Using this engine, the user can deal transparently with multiple backends in the server layer. This means that the data is accessible from any application without code changes. This lecture will briefly introduce MySQL partitioning, and then show how to create and use the Spider engine, with some practical …

I'm pleased to announce the release of Spider storage engine version 2.5(beta) and Vertical Partitioning storage engine version 0.5(alpha).
Spider is a Storage Engine for database sharding.
http://spiderformysql.com/
Vertical Partitioning is a Storage Engine for vertical partitioning for a table.
http://launchpad.net/vpformysql

The main changes in this version are following.
- The standalone compilation with Vertical Partitioning storage engine is now available.

- Vertical Partitioning can be used for multi-step partitioning.
  Example:
    create table a(a int primary key)engine=myisam;
    create table b(a int primary key)engine=vp comment 'tnl "a"';
    create table c(a …

I'm pleased to announce the release of Spider storage engine version 2.4(beta) and Vertical Partitioning storage engine version 0.4(alpha).
Spider is a Storage Engine for database sharding.
http://spiderformysql.com/
Vertical Partitioning is a Storage Engine for vertical partitioning for a table.
http://launchpad.net/vpformysql

The main changes in this version are following.
- "Engine-condition-pushdown" is available on Vertical Partitioning with Spider.

Please see "99_change_logs.txt" in the download documents for checking other changes.
Thanks to Gennady for bug report.

Enjoy!

I'm pleased to announce the release of Spider storage engine version 2.3(beta).
Spider is a Storage Engine for database sharding.
http://spiderformysql.com/

The main changes in this version are following.
- Add UDF parameter "connection_channel".
  This parameter is used for increasing background parallelism to same remote server by using multiple connections.

Please see "99_change_logs.txt" in the download documents for checking other changes.

Enjoy!

I'm pleased to announce the release of Spider storage engine version 2.2(beta).
Spider is a Storage Engine for database sharding.
http://spiderformysql.com/

The main changes in this version are following.
- Add UDF parameter "iop000".
  You can use "ignore" and "replace" at inserting records to temporary table list's tables using this parameter.

Please see "99_change_logs.txt" in the download documents for checking other changes.

Enjoy!

I'm pleased to announce the release of Spider storage engine version 2.1(beta).
http://spiderformysql.com/

The main changes in this version are following.
- Add UDFs "spider_direct_sql" and "spider_bg_direct_sql".
  These UDFs execute hand writing SQLs on remote server by using Spider's table link and store result sets in temporary tables.
  Main usages are following.
  1.Maintenance remote servers.
    (Create table on remote servers for Spider's link tables, etc.)
  2.Use every resources (excluding table) on remote servers.
  3.Mass update of the bulk data. (Update all rows in tables, etc.
    There are next advantages. It is possible to update it in parallel
    with multiple remote servers. The …

This is the Sharding for the masses: Introducing the SPIDER storage engine by Giuseppe Maxia, given at OpenSQLCamp, at FrOSCon, in August 2009. These are somewhat live notes, and the slides are available too.

Sharding for the masses View more documents from Giuseppe Maxia.

Why sharding? Scaling, of course. The MySQL way to solve this, is replication (even Yahoo! and Google use this).

When the master doesn’t have enough resources to cope with what you do (i.e. large data sets), replication chokes.

You can use proxies for sharding. There exists …