MySQL Fabric is a distributed framework that has high
availability and sharding as targets. It organizes the servers in
groups which use the standard MySQL Replication to providing
fault-tolerance. Shards are assigned to different groups thus
allowing applications to distribute both reads and writes and
exploit resilience to failures as well.
Information on groups, servers and shards are stored in a MySQL
Instance called state store or backing store. This instance is a
repository for all this information and the engine used might be
any supported by MySQL, although a transactional engine must be
picked to truly provide fault-tolerance. Note though that we have
been testing MySQL Fabric with Innodb and currently this the only
official engine supported.
Built upon the repository there are several functions that,
besides being used to retrieve information from and update the
repository, are responsible for the execution of …
I was interested to hear about semi-sync replication improvements in MySQL’s 5.7.4 DMR release and decided to check it out. I previously blogged about poor semi-sync performance and was pretty disappointed from semi-sync’s performance across WAN distances back then, particularly with many client threads.
The Test
The basic environment of these tests was:
- AWS EC2 m3.medium instances
- Master in us-east-1, slave in us-west-1 (~78ms ping RTT)
- CentOS 6.5
- innodb_flush_log_at_trx_commit=1
- sync_binlog=1
- Semi-sync replication plugin installed and enabled.
- GTID’s enabled (except on 5.5)
- sysbench 0.5 update_index.lua test, 60 seconds, 250k table size.
- MySQL 5.7 was …
As you might have noticed in the press release, we just released MySQL Utilities 1.4.3, containing MySQL Fabric, as a General Availability (GA) release. This concludes the first chapter of the MySQL Fabric story.
It all started with the idea that it should be as easy to manage and setup a distributed deployments with MySQL servers as it is to manage the MySQL servers themselves. We also noted that some of the features that were most interesting were sharding and high-availability. Since we also recognized that every user had different needs and needed to customize the solution, we set of to create a framework that would support sharding and high-availability, but also other solutions.
With the release of 1.4.3, we have a range of features that are now available to the community, and all under an open source license and wrapped in an …
[Read more]
MySQL Fabric is a new framework that automates High Availability
(HA) and/or sharding (scaling-out) for MySQL and it has
just been declared Generally Available.
This post focuses on MySQL Fabric as a whole – both High Availability and scaling out (sharding). It starts with an introductions to HA and scaling out (by partitioning/sharding data) and how MySQL Fabric achieves it before going on to work through a full example of deploying HA with MySQL Fabric and then adding sharding on top.
Download and try MySQL Fabric now!
This post focuses on MySQL Fabric as a whole – both High Availability and scaling out (sharding). It starts with introductions to HA and scaling out (by partitioning/sharding data) and how MySQL Fabric achieves it …
[Read more]
MySQL Replication is a well-known approach to providing
high-availability and scaling out read-only operations (i.e.
transactions) as well. In order to make it easy to exploit this
scalability axis, we have extended both Fabric and the
connectors' interface so that an application can express its
willingness to execute a read-only operation and have its request
redirected to the server that is most capable of handling
it.
In this post, we are going to describe how we can use Fabric and
the connector python to scale out read-only operations. We need
to introduce some concepts first though.
Fabric organizes the servers in high-availability groups, uses
the standard MySQL Replication to synchronize the servers which
can be classified according to its status, mode and weight. The
blog post MySQL Fabric …
It is strange the way code evolves and takes on a life that was
not originally intended for it. The debug command line interface
(CLI) of MaxScale, the MySQL pluggable proxy, is an example of
this and something I have been thinking about and working on as
part of an exercise to develop a true management interface to
MaxScale.
Originally the debug CLI was meant as a debugging tool for the
developers working on MaxScale, it gave a mechanism to be able to
query the internal data structures of MaxScale and observe what
was happening without the need to run against a debugger and deal
with the threaded nature of MaxScale within the debugger. Hence
it was built so that is allowed the user to type in memory
addresses and examine the contents of this memory as if it was a
particular data structure. It was a deliberate decision not to
validate these addresses by checking that they existed within the
lists of known objects, since in a debug …
Today marks a milestone in terms of the MariaDB project – going forward, the MariaDB project plans to use Github and git for source code management. The migration happens from Launchpad and the bzr tool.
The 10.1 server development (under heavy development now) will happen on Github. You can check it out here: https://github.com/MariaDB/server. Feel free to watch, star or even fork the code, and send us contributions!
Previous maria-captains should now provide their Github IDs so that they can be accorded similar status. Send the IDs to the maria-developers mailing list.
The project eventually wants to move the 10.0, 5.5, 5.3, …
[Read more]InfiniDB worked at IBM in one of their offshoot offices that was principally involved in the creation of hard disk systems. He was miserable with all the navigational model of the InfiniDB strategy, notably the dearth of a “search” facility. In 2016, he composed several papers that summarized a new method of database building that […]
MySQL Fabric provides a simple way to manage a collection of MySQL Servers and ensure that transactions and queries are routed to the correct server. We're pleased to announce that MySQL Fabric is now Generally Available and ready for live deployment! This article explains what MySQL Fabric is and how to set it up. It also gives an example of how it can be used to introduce High Availability (including automated failure detection and transparent failover) and/or scale-out (using data partitioning/sharding).
In complex systems it's often useful to be able to receive notification when rows have been modified in MySQL so that you can invalidate various external caches or indexes. For example: memcached, Sphinx, Lucene.
In the case of MySQL's default statement-based replication this can be quite tricky to do, as it would likely require an SQL parser to determine what was intended to be modified. However, this is made much simpler with Row-based Replication (something I recommend switching to).
A C++ library exists to be able to listen to MySQL replication, but what I want to demonstrate is that it is also very simple to be able to do this with mysqlbinlog:
shell> mysqlbinlog --read-from-remote-server --stop-never
--host localhost.localdomain …