In a previous post, I wrote an overview about Spider for MySQL with its advantages and disadvantages. Now I’ll go through a simple example demonstrating how to implement Spider for MySQL.

System information: MySQL instances information (shards):

• Server: 3 VMs (here in after will be mentioned as mysqla “192.168.56.51”, mysqlb “192.168.56.52” and mysqlc “192.168.56.53”).
• OS: CentOS 6.4
• RAM: 512MB
• MySQL version: 5.5.43 (tar-balls)
• Download link: http://dev.mysql.com/get/Downloads/MySQL-5.5/mysql-5.5.43-linux2.6-x86_64.tar.gz

…

MySQL sharding is one of the most used and surely the most abused MySQL scaling technology. My April 2 Dzone article, “To Shard, or Not to Shard,” proved there is indeed quite an interest in this topic.

As such, I’m hosting a live webinar tomorrow (April 22) that will shed light on questions about sharding with MySQL. It’s titled: To Shard or Not to Shard That is the Question!

I’ll be answering questions such as:

• Is sharding right for your application or should you use other scaling technologies?
• If you’re sharding, what things do you need to consider and which questions do you need to have answered?
• What kind of specific technologies can assist you with sharding?

I hope …

Having big tables is one of the expected database problems, especially, for the fast growing database systems. In fact, big tables itself is not a problem but with big tables, the following problems are strongly expected:

1. Retrieving data from big tables is so slow.
2. It is a very hard job to maintain those tables like adding/removing an index, adding/dropping/modifying a column, … etc.
3. System resources, especially, the IO system might not be able to handle such huge traffic of writes and reads.
4. When it comes to the reporting queries, it might be a horrible nightmare!
5. Always cause disk space problem.

All the above problems will show up the need for scaling! So, let’s check out what are the possible solutions for that problem:

• MySQL Partitioning: Is a good solution but we will still face disk space and server resources problems. …

MongoDB 3.0 was recently released. Instead of focusing on what’s new – that is so easy to find, let’s rather talk about something that has not changed a lot since the early MongoDB days. This topic is sharding and most specifically: how to choose a good sharding key. Note that most of the discussion will also apply to MySQL, so if you are more interested in sharding than in MongoDB, it could still be worth reading.

When do you want to shard?

In general sharding is recommended with MongoDB as soon as any of these conditions is met:

• #1: A single server can no longer handle the write workload.
• #2: The working set no longer fits in memory.
• #3: The dataset is too large to easily fit in a single server.

Note that #1 and #2 are by far the most common reason why people need sharding. Also note that in the MySQL world, #2 does not imply that you need sharding.

…

I visited MongoDB Day in London on November 6. Here are a few observations:

App-Developer Centric. It is interesting to see how much MongoDB is about developers; the ops side is something which is a necessary evil developers have to deal with. The ops topics covered in principle that there are no topics about choices of operating systems or hardware for MongoDB beyond flash and more memory.

Development Stacks. Being application centric there was good coverage of the MongoDB-powered stacks – MEAN and METEOR specifically got attention. Especially the METEOR presentation by Henrik Ingo was cool – real-time view synchronization between the Web browser (or mobile app) and database as well as the same language for server-side and client-side development is a really great concept. Though …

This year’s MySQL Central at Oracle Open World was an exhilarating experience. In contrast to the previous year’s MySQL Connect events, MySQL have now got their own Central at the main Oracle Open World. In the previous years, we were always short on time and trying to get a lot of sessions into just two days was just to much. This time I could both present sessions, attend sessions by other users, and also to talk to people in the MySQL community: something that I really enjoy and also find very valuable to see where we should be heading.

This year, the “MySQL Fabric Team” representation on MySQL Central was me and Narayanan Venkateswaran, which is heading the sharding solution in MySQL Fabric. Together with the conference, we also released MySQL Fabric 1.5.2 as the GA release of MySQL Fabric 1.5 containing a few new features:

• Server …

Martin Arrieta and I gave an online presentation last week on “Putting MySQL Fabric To Use.” If you missed it, you can find a recording and the slides here, and the vagrant environment we used plus a transcript of the commands we ran here (be sure to check out the ‘sharding’ branch, as that’s what we used during the webinar).

Thank you all for attending and asking interesting questions. We were unable to answer all of them in the scheduled time, so here are our replies to all the questions.

What is GTID? And how does it relate to MySQL Fabric?
GTID stands for Global …

Martin and I have recently been blogging together about MySQL Fabric (in case you’ve missed this, you can find the first post of the series here), and on July 30th, we’re going to be presenting a webinar on this topic titled “Putting MySQL Fabric to Use.”

The focus of the webinar is to help you get started quickly on this technology, so we’ll include very few slides (mostly just a diagram or two) and then jump straight into shared screen mode, with lots of live console and …

This is the fourth post in our MySQL Fabric series. In case you’re joining us now, we started with an introductory post, and then discussed High Availability (HA) using MySQL Fabric here (Part 1) and here (Part 2). Today we will talk about how MySQL Fabric can help you scale out MySQL databases with sharding.

Introduction

At the time of writing, MySQL Fabric includes support for range- and hash-based sharding. As with HA, the functionality is split between client, through a MySQL Fabric-aware connector; and server, through the mysqlfabric utility and …

On 19th June 2014, Mats Kindahl and I presented a free webinar on why and how you should be using MySQL Fabric to add Sharding (scaling out reads & writes) and High Availability to MySQL. The webinar replay is available here. This blog post includes a transcript of the questions raised during the live webinar together with the responses given – if you’re questions aren’t answered already then please feel free to post them as comments here.

Abstract

MySQL Fabric is built around an extensible and open source framework for managing farms of MySQL Servers. Currently two features have been implemented – High Availability (built on top of MySQL Replication) and …