As I’ve mentioned in previous posts, TokuMX replication differs quite a bit from MongoDB’s replication. The differences are large enough such that we’ve completely redone some of MongoDB’s existing algorithms. One such area is how secondaries apply oplog data from a primary. In this post, I’ll explain how.

In designing how secondaries apply oplog data, we did not look closely at how MongoDB does it. In fact, I’ve currently forgotten all I’ve learned about MongoDB’s implementation, so I am not in a position to compare the two. I think I recall that MongoDB’s oplog idempotency was a key to their algorithms. Because we chose …

At MySQL, we have been working on simplifying the failover process
making it faster, more flexible and easier to use. In MySQL 5.6 we added
support for Global Transaction Identifiers (GTID), which was a huge leap in the
direction of easing the failover process hiding the details about
replication logs and positions. With MySQL 5.7.4, we are introducing a
new feature that further adds to flexibility and onliness- the user can
only shut down components that he needs to re-configure. 

What we allow with this new feature is to execute CHANGE MASTER TO
command without stopping slave altogether. We realized that stopping
slave altogether is not mandatory in all cases and doing that was more
of a cautious approach to switching master restricting more than what’s
required at times.

Lets dive deep into this to see what can be relaxed here. For this, lets …

The MySQL Utilities Team is pleased to announce the latest release candidate (RC) release of MySQL Utilities. This release includes a number of improvements for useabilty, stability, and a few enhancements. A complete list of all improvements can be found in our release_notes.

New Utilities!
We have also included two new utilities.

• The mysqlrplsync utility was added, which checks data consistency between servers in a replicated setup. 
• The mysqlrplms utility was added, which provides round-robin multi-source replication (a slave server continually cycles through multiple masters in order to store a consolidated data set).

How Can I Download MySQL Utilities?
You can download MySQL Utilities 1.4.2 from the following link using one of the pre-built …

One of the key platforms I’ve been testing on for the MySQL to Hadoop replication has been Cloudera, largely driven by customer requirements, but it’s also one of the easiest way to get started with Hadoop.

What I’m even more pleased about is the fact that we are proud to announce that Tungsten Replicator 3.0 is certified for use on the new Cloudera Enterprise 5 platform. That means that we’re sure that replicating your data from MySQL to Cloudera 5 and have it work without causing problems or difficulties on the Hadoop loading and materialisation.

Cloudera is a great product, and we’re very happy to be working so effectively with the new Cloudera Enterprise 5. Cloudera …

We are very happy to introduce an new MySQL utility called 'mysqlrplsync' that can check the data consistency of an active replication system. This utility is one of the new utilities included in MySQL Utilities release-1.4.2 RC. The other utility is the multi-source replication utility, mysqlrplms.

In a nutshell, the mysqlrplsync utility allows you to check the data consistency between servers in an active replication system. The utility reports missing databases and tables as well as data differences (per table) between the servers. A sophisticated synchronization algorithm that utilizes a table checksum is applied on the active replication servers to locate differences in the data.

Main FeaturesBesides permitting the data consistency check of an active replication system, here are the main features of the mysqlrplsync utility:

• Allows the comparison of a specific set of servers, providing …

Hopefully by now you have already seen that we are working on Hadoop replication. I’m happy to say that it is going really well. I’ve managed to push a few terabytes of data and different data sets through into Hadoop on Cloudera, HortonWorks, and Amazon’s Elastic MapReduce (EMR). For those who have been following my long association with the IBM InfoSphere BigInsights Hadoop product, and I’m pleased to say that it’s working there too. I’ve had to adapt Robert’s original script to work with the different versions of the underlying Hadoop tools and systems to make it compatible. The actual performance and process is unchanged; you just use a different JS-based batchloader script to work with different tools.

Robert has also been simplifying some of the core functionality, such as configuring some fixed pre-determined formats, so you no longer have to explicitly set the field and record separators.

I’ve also been …

A colleague and I have been looking at GTID on MySQL recently and you may be interested in the blog post that results from that. You can see it here. http://blog.booking.com/mysql-5.6-gtids-evaluation-and-online-migration.html.

March 27, 2014 By Severalnines

With datacenters being stretched by resource-intensive applications, more and more businesses are outgrowing their existing in-house capacity and having to reconfigure their IT operations. But how do you migrate a busy application to a totally new data center without downtime? How will the application scale in a virtualized cloud environment? And how do you guard against cloud server failures and keep a high level of uptime?

In this example, we will show you how to migrate a Web application (Wordpress) from a local data center to a AWS VPC. Without downtime even! 

Main steps:

• Setup the cloud servers (for web server + database)
• Install Galera Cluster and ClusterControl
• Setup MySQL replication between the single instance MySQL (local) to Galera Cluster (remote)
• Sync your application …

Background

Eventual consistency is a consistency model used in many large distributed databases which requires that all changes to a replicated piece of data eventually reach all affected replicas; conflict resolution is not handled and responsibility is pushed up to the application author in the event of conflicting updates [13].

Eventual consistency is a specific form of weak consistency; the storage system guarantees that if no new updates are made to the object, eventually all accesses will return the last updated value [14]. If no failures occur, the maximum size of the inconsistency window can be determined based on factors such as communication delays, the load on the system, and the number of replicas involved in the replication scheme [3].

A few examples of eventually consistent systems:

• DNS
• Asynchronous master/slave replication on an RDBMS e.g. MariaDB
• memcached in front …

At an extremely high level, replication in MongoDB and MySQL are similar. Both databases have exactly one machine, the primary (or master), that accepts writes from clients. With a single transaction (or atomic operation, in MongoDB’s case), the tables and oplog (or binary log in MySQL) are modified to reflect the change. The log captures what the change is so other secondaries (or slaves) can read the changes and process them, making the slaves identical to the master. (Note that I am NOT talking about multi-master replication.)

Underneath the covers, their implementations are quite different. And in peeking underneath the covers while developing TokuMX, I learned more about my favorite thing in …