Planet MySQL

Is Synchronous Data Replication over WAN Really a Viable Strategy?

+3

-0

posted by Robert Hodges on Sun 05 Aug 2012 21:27 UTC
Tags:

(edit) mysql, Replication, drizzle, cloud computing, tungsten

Synchronous data replication over long distances has the sort of seductive appeal that often characterizes bad ideas. Why wouldn't you want every local credit card transaction simultaneously stored on the other side of the planet far away from earthquake, storms and human foolishness? The answer is simple: conventional SQL applications interact poorly with synchronous replication over wide area networks (WANs).

I spent a couple of years down the synchronous replication rabbit hole in an earlier Continuent product. It was one of those experiences that make you a sadder but wiser person. This article digs into some of the problems with synchronous replication and shows why another approach, asynchronous multi-master replication, is currently a better way to manage databases connected by

[Read more...]

Replicate from Oracle to MySQL *without* GoldenGate

+2

-0

posted by Linas Virbalas on Fri 22 Jun 2012 13:59 UTC
Tags:

(edit) mysql, Oracle, Replication, tungsten, heterogeneous

Intro

Oracle is widely use to support back-end systems. On the other hand, MySQL is the "go-to" data management solution for the web-facing part of many businesses. If you have both Oracle and MySQL in-house, you may already also have the need to share data between them. In this article I'll describe software that my colleagues and I have been working on to move data from Oracle to MySQL in real-time without costing an arm and a leg.

Tungsten to the Rescue!

Latest Tungsten Replicator has many features, most of which are open-source, but the recent one for me is particularly exciting - thanks to the

[Read more...]

New MySQL & MariaDB Instructional Videos from SkySQL

+3

-0

posted by SkySQL on Wed 13 Jun 2012 11:37 UTC
Tags:

(edit) sphinx, cluster, MONyog, scaledb, tungsten, mariadb, infinidb, skysql, MHA

Are you looking to expand your knowledge about MySQL and MariaDB database solutions?

Well, you’re in luck! SkySQL is introducing an exclusive collection of educational videos featuring some of the industry’s leading experts on the MySQL database and related technologies. View informative, technical talks on a variety of topics, from the experts at SkySQL, MariaDB, Calpont InfiniDB, Continuent, ScaleDB, Severalnines, Sphinx, Webyog, and others.

Included in this collection is a video of Michael (Monty) Widenius and David Axmark, the original authors of the MySQL database, as they shared their insights on current and future trends pertaining to the world’s most popular open

+4

-0

posted by Robert Hodges on Mon 30 Apr 2012 07:57 UTC
Tags:

(edit) mysql, Replication, tungsten, mariadb

An increasing number of organizations run applications that depend on MySQL multi-master replication between remote sites. I have worked on several such implementations recently. This article summarizes the lessons from those experiences that seem most useful when deploying multi-master on existing as well as new applications.

Let's start by defining terms. Multi-master replication means that applications update the same tables on different masters, and the changes replicate automatically between those masters. Remote sites mean that the masters are separated by a wide area network (WAN), which implies high average network latency of 100ms or more. WAN network latency is also characterized by a long tail, ranging from seconds due to congestion to hours or even days

[Read more...]

Replication Is Bad for MySQL Temp Tables

+2

-0

posted by Robert Hodges on Mon 23 Apr 2012 02:46 UTC
Tags:

(edit) mysql, Replication, tungsten, mariadb

Experienced MySQL DBAs know that temp tables cause major problems for MySQL replication. It turns out the converse is also true: replication can cause major problems for temporary tables.

In a recent customer engagement we enabled Tungsten Replicator with a MySQL application that originally ran on a server that did not use replication. QA promptly discovered reports that previously ran in 10 seconds were now running in as many minutes. It turned out that the reports used temp tables to assemble data, and these were being written into the master binlog. This created bloated binlogs and extremely slow reports. We fixed the problem by enabling row replication (i.e., binlog-format=row in my.cnf).

A common DBA response to temp table problems is to try to

[Read more...]

Some lessons from MySQL Conference 2012

+9

-0

posted by Giuseppe Maxia on Mon 16 Apr 2012 22:15 UTC
Tags:

(edit) mysql, community, conference, Testing, Percona, sandbox, tungsten, 5.5, 5.6

The Percona Live MySQL Conference and Expo 2012 is over. Together with the SkySQL solutions day, it has kept me occupied for 4 full days, from early morning to late at night.

I have to say that I am pleased. The quality of the organization was very high, with a very good lineup of speakers and an excellent technical support.

As usual, I have learned a lot during this week, either directly, by attending talks, or indirectly, by meeting people who told me what was juicy at the talks that I had missed. And I have met new interesting people, and caught up with the people that I know already.

This conference was particularly intense also because I got myself involved in 5 talks, which was probably more than I should have. How did

+0

-0

posted by Robert Hodges on Tue 03 Apr 2012 20:33 UTC
Tags:

(edit) mysql, tungsten, SaaS, mariadb

Cloud databases have a memory problem. Continuent has been doing a lot of Amazon deployments lately, and it is becoming apparent that memory utilization in those environments is more than just an inconvenience. In this article I would like to discuss the memory problem that we see in customer implementations and some new features of Tungsten Enterprise that help alleviate it for MySQL.

The Cloud Memory Problem and Database Arrays

As I discussed in a recent article about prefetch, the amount of RAM allocated to the InnoDB buffer pool is one of the principle determinants of MySQL performance. The speed difference between using

[Read more...]

Presenting at Percona Live and SkySQL MariaDB Solutions Day in Santa Clara

+3

-0

posted by Robert Hodges on Tue 13 Mar 2012 04:53 UTC
Tags:

(edit) mysql, Replication, IT Industry, tungsten, mariadb

MySQL community conferences are alive and well in 2012. Percona has taken the initiative to host the yearly MySQL event at the Santa Clara Hyatt; it's now called Percona Live MySQL Conference and Expo. It runs from 10 through 12 April. But don't plan on going home Thursday night. On Friday 13 April you can also attend the SkySQL and MariaDB MySQL Solutions Day in the same location. And wait, that's not all! Drizzle Day is also on 13 April and also at the Hyatt, so you can catch up on what the Drizzle folks have been up to for the last 12 months.

Now for some specifics on the conferences where

[Read more...]

Cool technology and usability in Tungsten Enterprise

+5

-0

posted by Giuseppe Maxia on Tue 06 Mar 2012 09:03 UTC
Tags:

(edit) mysql, Replication, management, tungsten

When I joined Continuent, at the end of 2010, I was fascinated by the technology of its core products. Readers of this blog know that I have had my hands full with Tungsten Replicator, but what really turned me on was the flagship management suite, Tungsten Enterprise. After hammering at it for several months, and always marveling at the beauty of its technology, let me give a tour of the suite, so that you'll understand what's so exciting about it. First off, Tungsten Enterprise is not simply a replication tool. It is based on replication, but it is mostly a data management suite. Its aim is to reduce complexity for the user and to show a database cluster to the user as if it were a single server, always on, no matter what happens. The most amazing things that you will see in Tungsten Enterprise are

Automatic

+3

-0

posted by Giuseppe Maxia on Mon 05 Mar 2012 23:27 UTC
Tags:

(edit) mysql, Replication, database, release, usability, sandbox, tungsten

Tungsten Replicator 2.0.5 was released this week-end. The release notes have quite a long list of bug fixes. Thanks to all the ones who have submitted bug reports, and fixes! There are a couple of new features as well. The replicator includes now a slave prefetch service. Unlike parallel replication, this feature works fine with a single database, and provides performance improvements that in many cases solve the slave lagging problems. This was a bitch of a feature to get right. Many have tried it, many have experienced various degrees of success, and several failures. We started with the bold assertiveness of the brave after an exciting talk at [Read more...]

Solving Replication Lag with Tungsten Slave Prefetch

+2

-0

posted by Robert Hodges on Mon 27 Feb 2012 02:28 UTC
Tags:

(edit) mysql, Replication, tungsten, mariadb

Slave prefetch is an increasingly popular technique for speeding up native MySQL replication, with several tools already published to enable it, such as mk-slave-prefetch and Replication Booster. Tungsten Replicator is now joining the fray. This article explains how our implementation works, how to install and tune it, and how well it performs compared to unaided MySQL native replication as well as Tungsten parallel replication.

Understanding Slave Prefetch

Slow reads from storage are the principle reason for lagging MySQL replication. This seems paradoxical since at first glance the lag is caused by delayed updates. The

[Read more...]

MySQL and Friends schedule at FOSDEM 2012

+1

-0

posted by Giuseppe Maxia on Wed 18 Jan 2012 15:46 UTC
Tags:

(edit) mysql, Replication, Events, community, conference, FOSDEM, tungsten

The MySQL DevRoom at FOSDEM is ready. The schedule has been voted. Thanks to all who have participated. Now, let's make sure that the event is successful. The schedule is juicy, and not only because I have three talks in it! Sunday 2012-02-05 Event Speaker Room When All you need to know about migrations and you never dared to ask Ralf Gebhardt [Read more...]

Time to vote for MySQL sessions at FOSDEM

+1

-0

posted by Giuseppe Maxia on Mon 02 Jan 2012 22:16 UTC
Tags:

(edit) mysql, free software, database, community, conference, opensqlcamp, Presentations, FOSDEM, participation, tungsten

There is a room dedicated to MySQL at FOSDEM 2012. (Thanks to @lefred for organizing). The CfP has received 37 submissions, but there will be time slots only for 12 to 15 talks. So now it's up to the community. If you want to attend a particular talk, you should vote for it. Like in previous years, the selection of the talks is public. You can see the list of the proposals, with the instructions, which I repeat here. You can vote either publicly, using Twitter, or privately, by sending an email. Each talk proposal will be referred by the number [Read more...]

Testing new builds with MySQL-Sandbox 3.0.24

+3

-0

posted by Giuseppe Maxia on Mon 19 Dec 2011 14:53 UTC
Tags:

(edit) mysql, Replication, usability, Percona, sandbox, tungsten, upgrade, mariadb, recipes

MySQL::Sandbox 3.0.24 was released yesterday, with many new features.

More than vanilla MySQL

If you have missed my previous announcement, here's the gist of it. MySQL Sandbox can now deal with tarballs from either Percona Server or MariaDB. The main difference after this change is that you can now create a directory called <PREFIX>5.5.16 and make_sandbox will recognize it as well as the plain 5.5.16.

$ make_sandbox --export_binaries --add_prefix=ps \
   Percona-Server-5.5.11-rel20.2-114.Darwin.i386.tar.gz  \
   -- --sandbox_directory=msb_ps5_5_11
unpacking Percona-Server-5.5.11-rel20.2-114.Darwin.i386.tar.gz
[…]

+2

-0

posted by Giuseppe Maxia on Sat 10 Dec 2011 21:17 UTC
Tags:

(edit) mysql, Replication, bugs, tungsten

When an open source project becomes popular, bug reports start flocking in. This is both good and bad news for the project developers. The good news is that someone is using the product, and they are finding ways of breaking it that we didn't think of. The bad news is that most of the times the reporters assume that the developers have super human powers, and that they will find what's wrong by the simple mentioning that a given feature is not working as expected. Unfortunately, it doesn't work that way. An effective bug report should have enough information that the ones in charge will be able to reproduce it and examine in lab conditions to find the problem. When dealing with databases and database tools, there are several cases, from simple to complex. Let's cover them in order.

Installation issues

This is often a straightforward case of lack of [Read more...]

Why So Many Proprietary Rewrites of MySQL and InnoDB?

+3

-0

posted by Robert Hodges on Thu 17 Nov 2011 07:02 UTC
Tags:

(edit) mysql, tungsten

Every couple of weeks or so I get marketing email from a Continuent competitor advertising a closed-source clone of MySQL. It is said to be pumped up on illegal substances and therefore the solution to all my problems. I like this sort of spam because it makes it easier to track what the neighbors are up to. However it does bring up a question. Why are so many companies offering what amount to proprietary replacements of MySQL? This does not mean alternative builds like Percona or MariaDB. It means products like Clustrix, Schooner, or Xeround, which replace MySQL entirely, or like [Read more...]

451 CAOS Links 2011.11.08

+0

-0

posted by The 451 Group on Tue 08 Nov 2011 14:58 UTC
Tags:

(edit) links, software, enterprisedb, Linux, opensource, 451 group, 451caostheory, 451group, caostheory, matt aslett, mattaslett, matthew aslett, matthewaslett, open-source, The 451 Group, the451group, toad, continuent, Accel, tungsten, netapp, cloudera, rackspace, lucid imagination, SCO Group, gorilla logic, karmasphere, OpenStack, Abiquo, mapr, stackIQ, splunk, fonemonkey, ian skerrett, john swainson, quest

Cloudera raises $40m. Accel announces $100m fund. Rackspace takes OpenStack private. And more.

# Cloudera raised $40m in series D funding and announced a partnership with NetApp around its NetApp Open Solution for Hadoop.

# Accel Partners launched a $100m Big Data Fund to invest in Hadoop- and NoSQL-related vendors.

# Rackspace

[Read more...]

Replication stars

+3

-0

posted by Giuseppe Maxia on Fri 04 Nov 2011 15:58 UTC
Tags:

(edit) Open Source, Replication, Business, community, best practices, scalability, ha, master, challenge, conflict, innovation, continuent, tungsten, multi-master, multiple, conflict prevention

Working with replication, you come across many topologies, some of them sound and established, some of them less so, and some of them still in the realm of the hopeless wishes. I have been working with replication for almost 10 years now, and my wish list grew quite big during this time. In the last 12 months, though, while working at Continuent, some of the topologies that I wanted to work with have moved from the cloud of wishful thinking to the firm land of things that happen. My quest for star replication starts with the most common topology. One master, many slaves.

Fig 1. Master/Slave topology

+4

-0

posted by Robert Hodges on Mon 31 Oct 2011 14:40 UTC
Tags:

(edit) mysql, Replication, tungsten

Tungsten parallel apply on slaves, or parallel replication for short, has been available for about a year. Until recently we did not have many formal benchmarks of its performance. Fortunately the excellent Percona Live Conference in London accepted my talk on Tungsten parallel replication (slides available here), so Giuseppe Maxia and I finally allocated a block of time for systematic performance testing.

In a nutshell, the results were quite good. In the best cases Tungsten parallel apply out-performs single-threaded native replication by about 4.5 to 1. Both Giuseppe and I have verified this using slightly different

[Read more...]

Tungsten Replicator and MySQL Sandbox at Percona Live London 2011

+4

-0

posted by Giuseppe Maxia on Tue 18 Oct 2011 21:52 UTC
Tags:

(edit) mysql, Open Source, Replication, free software, community, conference, Presentation, Percona, London, sandbox, tungsten

I will be a speaker at Percona Live - London 2011, and I am looking forward to the event, which is packed with great content. A whopping 40 session of MySQL content, plus 3 keynotes and 14 tutorials. It's enough to keep every MySQL enthusiast busy. Continuent speakers will be particularly busy, as between me and Robert Hodges, we will be on stage four times on Tuesday, October 25th.

Robert, at 10:15am: Teaching an Old Dog New Tricks: Tungsten Enterprise Clusters for

+3

-0

posted by Robert Hodges on Fri 30 Sep 2011 04:57 UTC
Tags:

(edit) mysql, Replication, tungsten, NoSQL, mongodb

Proof-of-concept Tungsten support for MongoDB arrived last May, when I posted about our hackathon effort to replicate from MySQL to MongoDB. That code then lay fallow for a few months while we worked on other things like parallel replication, but the period of idleness has ended. Earlier this week I checked in fixes to Tungsten Replicator to add one-line installation support for MongoDB slaves.

MySQL to MongoDB replication will be officially supported in the Tungsten Replicator 2.0.5 build, which will be available in a few weeks. However, you can try out MySQL to

[Read more...]

Upgrading Tungsten Replicator: as easy as ...

+3

-0

posted by Giuseppe Maxia on Thu 22 Sep 2011 20:37 UTC
Tags:

(edit) mysql, Replication, update, usability, tungsten, upgrade, recipes

When I talked about the usability improvements of Tungsten Replicator, I did not mention the procedure for upgrading. I was reminded about it by a question in the TR mailing list, and since the question was very relevant, I updated the Tungsten Cookbook with some quick upgrading instructions. A quick upgrading procedure is as important as the installer. Since we release software quite often, either because we have scheduled features to release or because of bug fixes, users want to apply a new release to an existing installation without much fuss. You can do the upgrade with a very quick and painless procedure. Let's suppose that you have installed one Tungsten Replicator cluster using this command:


#
# using

+3

-0

posted by Giuseppe Maxia on Thu 22 Sep 2011 10:25 UTC
Tags:

(edit) mysql, Oracle, Open Source, Replication, community, conference, San Francisco, tungsten, command line

Oracle Open World 2011 is approaching. MySQL is very well represented. Sheeri has put together a simple table of all the MySQL sessions at OOW, which is more handy than the Oracle schedule. I will be speaking in three sessions on Sunday, October 2nd.

Sunday, 9am MySQL: Don't Be a Rookie Forever—Be in Command (Line)I have given this talk before, as a tutorial at the UC in 2010 and at FrOSCon one month ago. It is one of the most rewarding sessions ever. The attendee were very interested. This will be a short version of the

[Read more...]

Chaining Replication Clusters

+2

-0

posted by Giuseppe Maxia on Mon 19 Sep 2011 00:16 UTC
Tags:

(edit) mysql, Replication, tungsten, topology

MySQL built-in replication includes a concept called relay slave, which allows you to create hierarchical database clusters. You can do the same thing with Tungsten, and this can be done in more than one way. Let's start with two distinct clusters. We can follow the recipes in the Tungsten Cookbook to install a master / slave cluster in three separate hosts and a [Read more...]

Quick recipes for database cluster building

+4

-0

posted by Giuseppe Maxia on Fri 16 Sep 2011 21:33 UTC
Tags:

(edit) mysql, Replication, database, community, best practices, tungsten, recipes, multi-master, cookbook, topology

One lesson learned in more than two decades working in this industry is that most of the IT professionals are impatient, want to achieve results immediately, and, most importantly, they don't read documentation. Much as the average geek is happy to answer many requests with a dismissive RTFM, the same geeks are not as diligent when it comes to learning about new or updated technologies. For this reason, there is a kind of documentation that is very much appreciated by busy and impatient professionals: cookbooks. And I am not talking about food. Geeks are not known for being cooks (1) and they like fast food. I am talking about collection of technical recipes, short articles where a problem is briefly stated, and a direct solution is shown. Working with Tungsten Replicator, I am constantly amazed at all the things you [Read more...]

A Glance at Real-Time Replication From MySQL To Oracle

+5

-0

posted by Linas Virbalas on Tue 13 Sep 2011 06:46 UTC
Tags:

(edit) mysql, Oracle, Replication, tungsten, heterogeneous

Intro
The open source Tungsten Replicator is very powerful. It's exciting to see how a flexible initial design allows to push the boundaries of replication with each new release. The recently published System of Record approach for multi-master databases does exactly this for multi-master databases.

Nevertheless, in today's database environments it is often not enough to move data, however well it is done, between the nodes of a single DBMS vendor like MySQL. There is an entirely new set of challenges that appear when Oracle joins the Enterprise room.

[Read more...]

What's Next for Tungsten Replicator

+3

-0

posted by Robert Hodges on Thu 08 Sep 2011 19:15 UTC
Tags:

(edit) mysql, postgresql, Oracle, Replication, tungsten, mongodb

As Giuseppe Maxia recently posted we released Tungsten Replicator 2.0.4 this week. It has a raft of bug fixes and new features of which one-line installations are the single biggest improvement. I set up replicators dozens of times a day and having a single command for standard cluster topologies is a huge step forward. Kudos to Jeff Mace for getting this nailed down.

So what's next? You can get see what we are up to in general by looking at our issues list. We cannot do everything at once, but here are the current priorities for Tungsten Replicator 2.0.5.

Parallel replication speed

[Read more...]

The happiness of failing installations

+2

-0

posted by Giuseppe Maxia on Thu 08 Sep 2011 16:38 UTC
Tags:

(edit) mysql, Replication, database, tungsten

When you set-up the same software several times (for you or for your customers), you want that software to install quickly and reliably, and you are generally happy when everything works as expected.
In this context, a failing installation is when the installation process exits unexpectedly, and you are left with an error message and the prospect of looking at the manual to find out what was it.

A failing installation is unpleasant, you'd say, and I concur. But do you know what's more unpleasant than a failing installation? It's an installation that succeeds, only to fail silently the first time you try using the application.

Looking at this enhanced definition, it is no surprise that I assert to find happiness in failure. And I have practical reasons for my claim. When I first tried Tungsten

[Read more...]

Tungsten Replicator 2.0.4 released: usability and power

+7

-1

posted by Giuseppe Maxia on Tue 06 Sep 2011 22:00 UTC
Tags:

(edit) mysql, Replication, installation, cluster, Downloads, tungsten

It has been a bumpy ride, with dozens of issues opened and resolved, but we finally feel that Tungsten Replicator 2.0.4 is ready for prime time.There have been quite a lot of changes. Most notably, the replicator is much faster, especially when it comes to parallel replication, and it is much easier to install, thanks to its new integrated installer, which can validate all the requirements to install the replicator, and suggest remedies when the requirements aren't met. This new installer is so good, in fact, that calling it installer is an insult. It is a legitimate cluster builder, able to install a full fledged [Read more...]

A review of Tungsten Replicator: Part 1 – Installation

+9

-0

posted by Ben Mildren on Thu 01 Sep 2011 19:25 UTC
Tags:

(edit) mysql, Replication, Group Blog Posts, tungsten, Technical Blog

I’ve been following the development of Tungsten Replicator for quiet some time now, and recently was fortunate enough to find the time to take a look at the product in more detail. If you haven’t heard of Tungsten Replicator yet, it’s an open source database replication engine that can be used to complement or completely [...]

Search

Profile

Top Voted (Last Week)

Top Voted (Last Month)

MySQL Links

Top 20 Most Active Authors

Oracle MySQL Blogs

Top 10 Vendor Blogs

More than vanilla MySQL

Installation issues