Showing entries 1 to 10 of 80
10 Older Entries »
Displaying posts with tag: sharding (reset)
MySQL Sharding with ProxySQL

This article demonstrates how MySQL sharding with ProxySQL works.

Recently a colleague of mine asked me to provide a simple example on how ProxySQL performs sharding.

In response, I’m writing this short tutorial in the hope it will illustrate ProxySQL’s sharding functionalities, and help people out there better understand how to use it.

ProxySQL is a very powerful platform that allows us to manipulate and manage our connections and queries in a simple but effective way. This article shows you how.

Before starting let’s clarify some basic concepts.

  • ProxySQL organizes its internal set of servers in Host Groups (HG), and each HG can be associated with users and Query Rules (QR)
  • Each QR can be final (apply = 1) or = let ProxySQL continue to parse other QRs
  • A QR can be a rewrite action, be a simple match, have a specific target HG, or be generic
  • QRs are defined …
[Read more]
Distributed Transactions in Vitess

With Vitess introducing sharding and allowing you to create cross-shard indexes, distributed transactions become unavoidable for certain workloads. Currently, Vitess only supports best-effort distributed transactions. So, it’s possible that a distributed commit only completes partially, leaving data in an inconsistent state.

At this point, 2 Phase Commit (2PC) is the only known protocol that allows you to give atomic guarantees for distributed transactions. For this protocol to work, a database must be able to support the ‘Prepare’ contract. However, not all databases provide such support. Also, some of the engines that do support it either do it incorrectly or inefficiently. Specifically, the pre-5.7 MySQL XA protocol works incorrectly for replication, and is therefore not usable.

The question was asked: Is it possible to build a Prepare protocol on top of a database that does not support it? The answer is: Yes, …

[Read more]
Setup a MongoDB replica/sharding set in seconds

In the MySQL world, we’re used to playing in the MySQL Sandbox. It allows us to deploy a testing replication environment in seconds, without a great deal of effort or navigating multiple virtual machines. It is a tool that we couldn’t live without in Support.

In this post I am going to walk through the different ways we have to deploy a MongoDB replica/sharding set test in a similar way. It is important to mention that this is not intended for production, but to be used for troubleshooting, learning or just playing around with replication.

Replica Set regression test’s diagnostic commands

MongoDB includes a .js that allows us to deploy a replication set from the MongoDB’s shell. Just run the following:

# mongo …
[Read more]
Announcing: MySQL Fabric 1.6.2 on Labs

The MySQL Fabric team is pleased to announce that a new version is now available on Labs. This labs release includes our latest developments to give you a taste of what we have been working on. In particular, it includes the following new features and improvements:

  • Multi-node Fabric support is now available!

Cloud Native MySQL Sharding with Vitess and Kubernetes

Cross-posted on Google Cloud Platform Blog.

Cloud native technologies like Kubernetes help you compose scalable services out of a sea of small logical units. In our last post, we introduced Vitess (an open-source project that powers YouTube's main database) as a way of turning MySQL into a scalable Kubernetes application. Our goal was to make scaling your persistent datastore in Kubernetes as simple as scaling stateless app servers - just run a single command to launch more …

[Read more]
Announcing: MySQL Fabric 1.5.5 GA is Available!

The MySQL Fabric team is pleased to announce that a new (GA) release is now available. This release includes several improvements in terms of stability and usability (bug fixes and feature enhancements). In a nutshell:

  • Two new types of users for backup and restore are now available allowing a better managements of the required privileges for each user type.

Spider for MySQL – Implementation

In a previous post, I wrote an overview about Spider for MySQL with its advantages and disadvantages. Now I’ll go through a simple example demonstrating how to implement Spider for MySQL.

System information: MySQL instances information (shards):

[Read more]
Considering Sharding with MySQL? Join my April 22 webinar. Questions welcome!

MySQL sharding is one of the most used and surely the most abused MySQL scaling technology. My April 2 Dzone article, “To Shard, or Not to Shard,” proved there is indeed quite an interest in this topic.

As such, I’m hosting a live webinar tomorrow (April 22) that will shed light on questions about sharding with MySQL. It’s titled: To Shard or Not to Shard That is the Question!

I’ll be answering questions such as:

  • Is sharding right for your application or should you use other scaling technologies?
  • If you’re sharding, what things do you need to consider and which questions do you need to have answered?
  • What kind of specific technologies can assist you with sharding?

I hope …

[Read more]
Spider for MySQL – Overview

Having big tables is one of the expected database problems, especially, for the fast growing database systems. In fact, big tables itself is not a problem but with big tables, the following problems are strongly expected:

  1. Retrieving data from big tables is so slow.
  2. It is a very hard job to maintain those tables like adding/removing an index, adding/dropping/modifying a column, … etc.
  3. System resources, especially, the IO system might not be able to handle such huge traffic of writes and reads.
  4. When it comes to the reporting queries, it might be a horrible nightmare!
  5. Always cause disk space problem.

All the above problems will show up the need for scaling! So, let’s check out what are the possible solutions for that problem:

  • MySQL Partitioning: Is a good solution but we will still face disk space and server resources problems. …
[Read more]
Choosing a good sharding key in MongoDB (and MySQL)

MongoDB 3.0 was recently released. Instead of focusing on what’s new – that is so easy to find, let’s rather talk about something that has not changed a lot since the early MongoDB days. This topic is sharding and most specifically: how to choose a good sharding key. Note that most of the discussion will also apply to MySQL, so if you are more interested in sharding than in MongoDB, it could still be worth reading.

When do you want to shard?

In general sharding is recommended with MongoDB as soon as any of these conditions is met:

  • #1: A single server can no longer handle the write workload.
  • #2: The working set no longer fits in memory.
  • #3: The dataset is too large to easily fit in a single server.

Note that #1 and #2 are by far the most common reason why people need sharding. Also note that in the MySQL world, #2 does not imply that you need sharding.

[Read more]
Showing entries 1 to 10 of 80
10 Older Entries »