Open source projects have a great way of pulling in original ideas. Take encrypting replicated SQL: it's a big deal when you are are not sure about network security. This problem comes up very quickly when you transmit data over WAN links or in shared/cloud environments.

I have been procrastinating on implementation of encryption in the Tungsten Replicator because it felt as if we were going to have to do some surgery for it to work correctly. (Another hypothesis is that I'm lazy.) However, this morning I was talking via Skype to Mark Stenbäck, an engineer from Finland whom we met through a recent conference. Mark had a great idea. Why not just write an event filter? An excellent thought...Here's how it could work.

Filters …

...Beta-4 to be precise. Downloads are available on the Continuent Forge. You can get more information about Tungsten in general from our community pages.

The Beta-4 build has a number of nice improvements. The best new feature is a utility to look and manage at events in the transaction history log. It's our version of mysqlbinlog but without any funny options to look at row updates.

Speaking of row updates, we now support all standard datatypes used in MySQL 5.1 row events. Error handling now works on the "WALL-E model"--if there's a serious error the Replicator goes into a restartable state called OFFLINE:ERROR and waits for you to bring it back on-line. Recovering from …

Time-delayed replication is a useful feature that allows a slave to lag a fixed amount of time behind a master, thus providing a time window to recover from disasters like deleting a 10 million line table by accident. You just run over to the slave, turn off replication, and recover lost data, as the delayed updates mean it has yet to see your deadly mistake. It's a simple way to protect your administrative honor as well as your job.

Time-delayed replication has been on the MySQL to-do list since at least 2001. It's currently scheduled for release 6.0 and the fix is included in recent OurDelta builds. However, there's a very simple way to get the feature with Tungsten Replicator filters. This works for unadulterated MySQL 5.0 and 5.1 releases.

I wrote about filters in a previous post on the …

Listening to Sheeri's presentation on MySQL 5.1, I saw that there are a few questions left unanswered. I am listing here some of the questions that I found interesting, plus a few from an early webinar on the same topic.

Q: does Partitioning physically split data?

A: No. Some engines (MyISAM, Archive) do a physical split, but this is not necessary, as you see if you apply partitioning to a InnoDB table. Partitioning is a logical split of data, for easy retrieval. It is completely transparent to the user.

Q: Can you set partitions to different servers?

A: No. Partitions are logical parts of one table within one server. Partitioning through the Federated engine is not supported.

Q: How efficient are Row-Based …

We have a lot of customers who do click analysis, site analytics, search engine marketing, online advertising, user behavior analysis, and many similar types of work.  The first thing these have in common is that they're generally some kind of loggable event.

The next characteristic of a lot of these systems (real or planned) is the desire for "real-time" analysis.  Our customers often want their systems to provide the freshest data to their own clients, with no delays.

Finally, the analysis is usually multi-dimensional.  The typical user wants to be able to generate summaries and reports in many different ways on demand, often to support the functionality of the application as well as to provide reports to their clients.  Clicks by day, by customer, top ads by clicks, top ads by click-through ratio, and so on for dozens of different types of slicing and dicing.

And as a result, one of the most common …

Our fiendish plot to provide advanced open source replication for MySQL and Oracle took another step forward yesterday. The Tungsten Replicator beta-3 build is available for download on our Forge site. This build is fully open source. Tungsten Replicator provides heterogeneous replication from MySQL to Oracle, seamless failover from a master to one of several slaves, event checksums, event filtering hooks, and a number of other useful replication features for MySQL and Oracle previously not offered outside of commercial products.

The beta-3 build has a number of important improvements:

• MySQL 5.1 row replication is largely working. We are still seeing problems with datetime and timestamp replication but most other datatypes work.
  
• The …

MySQL 5.1 is GA. Let the fear and loathing begin. In a recent post Monty describes a number of problems that he feels should have prevented a GA declaration at this time. I like Monty's forthrightness immensely and his words have strongly influenced our work to develop the Tungsten Replicator. That said, I must respectfully disagree with his opinion.

It's hard to comment on overall quality of 5.1, though I have yet to hit any bugs personally after using it intermittently for almost a year. However, we have done a lot of work with MySQL row replication. Monty points out several bugs in the row replication implementation. Frankly, they would not hold me back. Row replication has so many advantages in eliminating …

Anyway, it was verified independently, but there was a catch. The script in the master was using SET storage_engine=InnoDB to create the tables necessary for the test. That looked good enough to me. The script was indeed creating InnoDB tables on the master. The trouble was that the "SET" command is not replicated. Thus the …

Happy Thanksgiving and little holiday challenge for you.
Say you have a trigger on the slave which you would like to work differently, depending on whenever update is executed via replication thread vs updating table locally ? This can be helpful for example for auditing updates which were done directly instead of coming from the master and some other cases.
Suggest how you would do it by commenting

Entry posted by peter | 12 comments

Add to: | …

Sometimes I amaze myself in my capacity to make assumptions about how things should work, especially when it comes to test plans... ( You know what happens when we assume, right? )I had this great idea to setup a couple slaves off a master-master replication set something like this:MASTER A <--------------> MASTER B | | | | |