Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 653 Next 30 Older Entries

Displaying posts with tag: Replication (reset)

How-to and Performance Impact of SSL-Encrypted Replication Traffic in Galera Cluster for MySQL
+0 Vote Up -0Vote Down
August 7, 2014 By Severalnines

 

Deploying Galera Clusters across WAN environments might lead to concerns around data privacy and security - especially as more organisations are having to comply with national and international regulations. You would not want hackers eavesdropping or intercepting replication traffic. Encrypted replication hides what is sent between the Galera nodes, and makes sure each node is only communicating to the ones it trusts. But how expensive is encryption?

 

In this blog, we will show you how to encrypt the replication traffic between your Galera nodes. We will also look into the performance impact of this

  [Read more...]
Orchestrator 1.0.5: refactoring masters, multi-master replication
+0 Vote Up -0Vote Down

Outbrain's orchestrator Version 1.0.5 is released.

Quick links: Orchestrator Manual, FAQ, Downloads

Orchestrator now supports refactoring of masters via master-master topologies. It now allows promoting slaves as co-masters and detachment of instances from a co-master topology, effectively allowing for replacing an active master.

Like this

Drag a master:

  [Read more...]
Orchestrator 1.0.4 released
+1 Vote Up -0Vote Down

Outbrain's orchestrator Version 1.0.4 is released.

Quick links: Orchestrator Manual, FAQ, Downloads

What's new?

Co-masters

orchestrator now does a much better visualization of Master-Master replication:

  [Read more...]
Prewarm your EBS backed EC2 MySQL slaves
+1 Vote Up -0Vote Down

This is the story of cold blocks and mismatched instances and how they will cause you pain and cost you money until you understand why.

Most of the clients that we support run on the Amazon cloud using either RDS or running MySQL on plain EC2 instances using (Provisioned IOPS) PIOPS EBS for data storage.

As expected the common architecture is running a master with one or more slaves handling the read traffic.

A common problem is that after the slaves are provisioned (normally created from an EBS snapshot) they lag badly due to slow IO performance.

Unfortunately what tends to be lost in the “speed of provisioning new resources” fetish is some limitations in terms of data persistence layer (EBS).

If you are using EBS and you have created the EBS volume from snapshot or created a new volume you have to pre-warm the EBS volume

  [Read more...]
MySQL Fabric – Part 1 – Installing
+2 Vote Up -0Vote Down
Send to Kindle

MySQL Fabric is a tool included on MySQL Utilities that helps you to manage your MySQL instances.
It works by basically adding a new layer between your application and MySQL instances, which can provide an easy way to use sharding and build a high available system.

For More information about what is MySQL Fabric, please follow the documentation.

To install our Fabric environment, we will have to configure 4 servers, I will use the follow names and IP on this tutorial:

fabric1 (192.168.0.200) - fabric
mysql1 (192.168.0.201) - mysql master
mysql2 (192.168.0.202) - mysql slave
mysql3 (192.168.0.203) - mysql slave

Note: I’m running CentOS


  [Read more...]
Setting up and Using MySQL Replication
Employee_Team +1 Vote Up -0Vote Down

MySQL Replication allows servers to copy changes from one instance to another. Take the MySQL for Database Administrators course to learn about replication, including:

  • How to set up a replication environment
  • Complex topologies
  • Replication types
  • Global transaction IDs (GTIDs)
  • MySQL Utilities
  • Logs and threads

Replication is one of the many topics covered in the MySQL for Database Administrators course. You can take this 5-day instructor-led course as  

  [Read more...]
MySQL Slave Scaling and more
+3 Vote Up -0Vote Down

Jean-François talks about binlog servers. Take a look here: http://blog.booking.com/mysql_slave_scaling_and_more.html

Using MySQL triggers and views in Amazon RDS
+1 Vote Up -0Vote Down

I recently had an opportunity to migrate a customer from a physical server into Amazon’s RDS environment. In this particular case the customers’ platform makes extensive use of MySQL triggers and views.  I came across two significant issues that prevented me from following Amazon’s documentation, which basically states “use mysqldump” but doesn’t call out a specific method of dealing with MySQL triggers and views.

Amazon Relational Database Service (Amazon RDS) is a great platform if you’re looking for complete hands-off management of your MySQL environment, but comes at a cost in the area of flexibility, i.e. you don’t have SUPER privilege and this brings up additional challenges.

  • You need to ensure you set log_bin_trust_function_creators=1 ( by default this is off, 0).
  • You need to
  •   [Read more...]
    Replicating Oracle Webinar Question Follow-up
    +0 Vote Up -0Vote Down

    We had really great webinar on Replicating to/from Oracle earliest this month, and you can view the recording of that Webinar here.

    A good sign of how great a Webinar was is the questions that come afterwards, and we didn’t get through them all. so here are all the questions and answers for the entire webinar.

    Q: What is the overhead of Replicator on source database with asynchronous CDC?

    A: With asynchronous operation there is no substantial CPU overhead (as with synchronous), but the amount of generated redo logs becomes bigger requiring more disk space and better log management to ensure that the space is used effectively.

    Q: Do you support migration from Solaris/Oracle to Linux/Oracle?

    A: The replication is

      [Read more...]
    Introducing Orchestrator: manage and visualize your MySQL replication topologies and get home for dinner
    +1 Vote Up -0Vote Down

    I'm happy to announce the availability of Outbrain's Orchestrator: MySQL replication management & visualization tool.

    • Orchestrator reads your replication topologies (give it one server - be it master or slave - in each topology, and it will reveal the rest).
    • It keeps a state of this topology.
    • It can continuously poll your servers to get an up to date topology map.
    • It visualizes the topology in a clear and slick D3 tree.
    • It allows you
      [Read more...]
    Semi-sync replication is not slow!
    Employee +2 Vote Up -0Vote Down

    If you read Yoshinori's post about Semi-sync at Facebook, he lists the objective of using semi-sync as an alternative to running full durability on a master. That is to say that once you can guarantee writes have safely been shipped across the network, you may not strictly need to guarantee that they are safe locally.

    This is something that I have been wanting to benchmark for a long time, and reading Jay's post about Semi-sync replication performance in MySQL 5.7 DMR4 and a conversation in last week's #dbhangops inspired me to explore this in more detail. For my tests, I will be using Master-Slave replication and three alternative definitions of

      [Read more...]
    Using deep nested replication topologies
    +0 Vote Up -0Vote Down

    If you're running more than a few slaves in a replication topology, you might choose to use deeply nested replication: slaves replicating from other slaves at 2, 3 or even 4 levels. There are pros and cons to such topologies, discussed below.

    A simple, small deep nested topology is depicted below (it is also a real production topology of ours):

    Two slaves, srv-4 and srv-8 act as local masters to yet other slaves. Why would we want to have this complexity?

    Pros

    • Reduce load on master: too many slaves replicating from single master means the master becomes loaded with serving the binary logs.
      [Read more...]
    Fun with Bugs #33 - bugs fixed in MySQL 5.6.19
    +0 Vote Up -0Vote Down
    It seems Oracle released MySQL 5.6.19 yesterday. So, it's time to check what community bug reports are fixed there.

    Let's start with InnoDB. We have the following bugs fixed:
    • Bug #72079, "Upgrade from 5.6.10 to 5.6.16 crashes and leaves unusable DB". Honestly I had not noted this bug, even though I upgrade several instances on Windows to each and every release since 5.6.8. Probably because I had no need to use FTS indexes till recently...
    • Bug #71014, about two extra memset calls that are now removed. I hope the patch bug report



      [Read more...]
    MySQL Fabric: Musings on Release 1.4.3
    Employee +2 Vote Up -0Vote Down
    As you might have noticed in the press release, we just released MySQL Utilities 1.4.3, containing MySQL Fabric, as a General Availability (GA) release. This concludes the first chapter of the MySQL Fabric story.

    It all started with the idea that it should be as easy to manage and setup a distributed deployments with MySQL servers as it is to manage the MySQL servers themselves. We also noted that some of the features that were most interesting were sharding and high-availability. Since we also recognized that every user had different needs and needed to customize the solution, we set of to create a framework that would support sharding and high-availability, but also other solutions.

    With the release of 1.4.3, we have a range of features that are now available to the community, and all under

      [Read more...]
    From Percona Server to MySQL and back to Percona Server: beware of crash safe replication info
    +0 Vote Up -0Vote Down

    We're migrating some of our "vanilla" MySQL 5.5 servers to Percona Server 5.5. One of the major incentives is the crash-safe replication feature, allowing slaves to die (power failure) and resume replication without losing position in relay logs.

    Whether or not we will migrate all our servers depends on further benchmarking; so far we've noticed unexpected results, but these are still premature to publish.

    However the fact that we are using both MySQL & Percona Server has led us into a peculiar situation which I'd like to share. We reseed our servers via LVM snapshots. If we need a new machine, or have a corrupted slave, we capture an image of a running slave and duplicate it, a process which takes

      [Read more...]
    Faking a slave: Subscribing to mysql row-based-replication changes
    Employee +0 Vote Up -1Vote Down

    In complex systems it's often useful to be able to receive notification when rows have been modified in MySQL so that you can invalidate various external caches or indexes. For example: memcached, Sphinx, Lucene.

    In the case of MySQL's default statement-based replication this can be quite tricky to do, as it would likely require an SQL parser to determine what was intended to be modified. However, this is made much simpler with Row-based Replication (something I recommend switching to).

    A C++ library exists to be able to listen to MySQL replication, but what I want to demonstrate is that it is also very simple to be able to do this with mysqlbinlog:


    shell> mysqlbinlog

      [Read more...]
    Errant transactions: Major hurdle for GTID-based failover in MySQL 5.6
    +1 Vote Up -0Vote Down

    I have previously written about the new replication protocol that comes with GTIDs in MySQL 5.6. Because of this new replication protocol, you can inadvertently create errant transactions that may turn any failover to a nightmare. Let’s see the problems and the potential solutions.

    In short

    • Errant transactions may cause all kinds of data corruption/replication errors when failing over.
    • Detection of errant transactions can be done with the GTID_SUBSET() and
      [Read more...]
    How does the Replication Synchronization Checker Work?
    +1 Vote Up -0Vote Down
    We recently introduced 'mysqlrplsync' in MySQL Utilities release-1.4.2 RC. This new utility allows users to check the data consistency of an active replication system. In this blog we provide more details about how 'mysqlrplsync' works.

    In an active replication topology, slaves may be slightly behind the master in processing events. Depending on the workload and capabilities of each slave, transactions may be applied at different times. Should this occur and something untoward happen to one of the slaves (such as a user making a manual change directly on the slave), a synchronization process may be required to ensure that the slaves have the same data - to manually catch up all of the slaves that are behind the master.

    The strategy we choose was to build on the top of the replication process and makes use of GTIDs; it works independently of the



      [Read more...]
    High Availability with MySQL Fabric: Part I
    +2 Vote Up -0Vote Down

    In our previous post, we introduced the MySQL Fabric utility and said we would dig deeper into it. This post is the first part of our test of MySQL Fabric’s High Availability (HA) functionality.

    Today, we’ll review MySQL Fabric’s HA concepts, and then walk you through the setup of a 3-node cluster with one Primary and two Secondaries, doing a few basic tests with it. In a second post, we will spend more time generating failure scenarios and documenting how Fabric handles them. (MySQL Fabric is an extensible framework to manage large farms of MySQL servers, with support for high-availability and sharding.)

    Before we begin, we recommend you read 

      [Read more...]
    GTIDs in MySQL 5.6: New replication protocol; new ways to break replication
    +1 Vote Up -0Vote Down

    One of the MySQL 5.6 features many people are interested in is Global Transactions IDs (GTIDs). This is for a good reason: Reconnecting a slave to a new master has always been a challenge while it is so trivial when GTIDs are enabled. However, using GTIDs is not only about replacing good old binlog file/position with unique identifiers, it is also using a new replication protocol. And if you are not aware of it, it can bite.

    Replication protocols: old vs new

    The old protocol is pretty straightforward: the slave connects to a given binary log file at a specific offset, and the master sends all the transactions from there.

    The new protocol is slightly different: the slave first sends the range of GTIDs it has

      [Read more...]
    MySQL 5.6 + GTID & MariaDB 10 replication
    +1 Vote Up -0Vote Down

    While at the keynote of Tomas Ulin at Percona Live MySQL Conference & Expo Santa Clara 2014, he asked the audience what they were running, and most of the audience was on MySQL 5.5 while about 15% of the audience was on MySQL 5.6. This number is steadily increasing I’m sure, so one thing that becomes important is that people will probably start turning on Global Transaction Identifiers (GTIDs). 

    As you may already know, MariaDB 10 has a different implementation of Global Transaction ID. To me, this poses a problem in a mixed use environment (or even a migration scenario). Which is why

      [Read more...]
    MySQL Fabric: Tales and Tails from Percona Live
    Employee +2 Vote Up -0Vote Down
    Going to Percona Live and presenting MySQL Fabric gave me the opportunity to meet a lot of people and get a lot of good feedback. I talked to developers from many different companies and got a lot of great feedback that will affect the priorities we make, so to all I spoke to I would like to say a great "Thank you!" for the interesting discussions that we had. Your feedback is very valuable. It was very interesting to read the comments on MySQL Fabric on MySQL Performance Blog. The article discuss the current version of MySQL Fabric distributed with MySQL Utilities  [Read more...]
    When you cannot replicate from master M to slave S
    +1 Vote Up -1Vote Down

    Working on some replication topology automation, here are some rules that will prevent you from replicating from a MySQL server M to a slave S:

    • M does not have binary logs (log_bin) enabled
    • M is itself a slave and does not have log_slave_updates enabled
    • M has a higher major version than S (e.g. M is 5.6 and S is 5.5, but do note log_bin_use_v1_row_events)
    • Both servers have same server_id
    • S has log_bin &
      [Read more...]
    The mystery of MySQL 5.6 excessive buffer pool flushing
    +1 Vote Up -0Vote Down

    I'm experimenting with upgrading to MySQL 5.6 and am experiencing an unexplained increase in disk I/O utilization. After discussing this with several people I'm publishing in the hope that someone has an enlightenment on this.

    We have a few dozens servers in a normal replication topology. On this particular replication topology we've already evaluated that STATEMENT based replication is faster than ROW based replication, and so we use SBR. We have two different workloads on our slaves, applied by two different HAProxy groups, on three different data centres. Hardware-wise, servers of two groups use either Virident SSD cards or normal SAS spindle disks.

    Our servers are I/O bound. A common query used by both workloads looks up data that does not necessarily have a hotspot, and is

      [Read more...]
    "Anemomaster": DML visibility. Your must-do for tomorrow
    +0 Vote Up -0Vote Down

    Here's our take of master DML query monitoring at Outbrain (presented April 2014). It took a half-day to code, implement, automate and deploy, and within the first hour of work we managed to catch multiple ill-doing services and scripts. You might want to try this out for yourself.

    What's this about?

    What queries do you monitor on your MySQL servers? Many don't monitor queries at all, and only look up slow queries on occasion, using pt-query-digest. Some monitor slow queries, where Anemometer (relying on pt-query-digest) is a very good tool. To the extreme, some monitor TCP traffic

      [Read more...]
    Making the MTR rpl suite GTID_MODE Agnostic
    Employee_Team +1 Vote Up -0Vote Down

    In MySQL 5.6 we introduced GTID_MODE as a new server option. A global transaction identifier (GTID) is a unique identifier created and associated with each transaction when it is committed on the server of origin (master). This identifier is unique not only to the server on which it originated, but is unique across all servers in a given replication setup. There is a 1-to-1 mapping between all transactions and all GTIDs. For additional information, please refer to the MySQL manual.

    Prior to 5.6.17 and 5.7.4, we had GTID specific replication (referred to as “rpl” within the MTR suite) regression tests, and we had to separately run

      [Read more...]
    How TokuMX Secondaries Work in Replication
    +0 Vote Up -0Vote Down

    As I’ve mentioned in previous posts, TokuMX replication differs quite a bit from MongoDB’s replication. The differences are large enough such that we’ve completely redone some of MongoDB’s existing algorithms. One such area is how secondaries apply oplog data from a primary. In this post, I’ll explain how.

    In designing how secondaries apply oplog data, we did not look closely at how MongoDB does it. In fact, I’ve currently forgotten all I’ve learned about MongoDB’s implementation, so I am not in a position to compare the two. I think I recall that MongoDB’s oplog idempotency was a key to their

      [Read more...]
    MySQL-5.7.4- Change master without stopping slave altogether
    Employee_Team +1 Vote Up -0Vote Down
    At MySQL, we have been working on simplifying the failover process
    making it faster, more flexible and easier to use. In MySQL 5.6 we added
    support for Global Transaction Identifiers (GTID), which was a huge leap in the
    direction of easing the failover process hiding the details about
    replication logs and positions. With MySQL 5.7.4, we are introducing a
    new feature that further adds to flexibility and onliness- the user can
    only shut down components that he needs to re-configure. 

    What we allow with this new feature is to execute CHANGE MASTER TO
    command without stopping slave altogether. We realized that stopping
    slave altogether is not mandatory in all cases and doing that was more
    of a cautious approach to switching master restricting more than what’s
    required at times.

    Lets dive









      [Read more...]

    New! MySQL Utilities release-1.4.2-RC
    Employee +2 Vote Up -0Vote Down
    The MySQL Utilities Team is pleased to announce the latest release candidate (RC) release of MySQL Utilities. This release includes a number of improvements for useabilty, stability, and a few enhancements. A complete list of all improvements can be found in our release_notes.

    New Utilities!


    We have also included two new utilities.
    • The mysqlrplsync utility was added, which checks data consistency between servers in a replicated setup. 
    • The mysqlrplms utility was added, which provides round-robin multi-source replication (a slave server continually cycles through multiple masters in order to store a consolidated data set).

    How Can I Download MySQL Utilities?


    You can download MySQL Utilities 1.4.2 from





      [Read more...]
    Tungsten Replicator 3.0 is Cloudera Enterprise 5 Certified
    +0 Vote Up -0Vote Down

    One of the key platforms I’ve been testing on for the MySQL to Hadoop replication has been Cloudera, largely driven by customer requirements, but it’s also one of the easiest way to get started with Hadoop.

    What I’m even more pleased about is the fact that we are proud to announce that Tungsten Replicator 3.0 is certified for use on the new Cloudera Enterprise 5 platform. That means that we’re sure that replicating your data from MySQL to Cloudera 5 and have it work without causing problems or difficulties on the Hadoop

      [Read more...]
    Showing entries 1 to 30 of 653 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.