Showing entries 1 to 10 of 16
6 Older Entries »
Displaying posts with tag: replicator (reset)
The Fast Way to Import CSV Data Into a Tungsten Cluster

The Question Recently, a customer asked us:

After importing a new section of user data into our Tungsten cluster, we are seeing perpetually rising replication lag. We are sitting at 8.5hrs estimated convergence time after importing around 50 million rows and this lag is climbing continuously. We are currently migrating some of our users from a NoSQL database into our Tungsten cluster. We have a procedure to write out a bunch of CSV files after translating our old data into columns and then we recursively send them to the write master using the mysql client. Specifically our import SQL is doing LOAD DATA LOCAL INFILE and the reading in a large CSV file to do the import. We have 20k records per CSV file and we have 12 workers which insert them in parallel.

Simple Overview The Skinny

In cases like this, the slaves are having trouble with the database unable to keep up with the apply stage …

[Read more]
Introducing Tungsten Replicator (AMI): The High Performance Replication Engine For MySQL, MariaDB & Percona Server

Discover Tungsten Replicator (AMI) – the replication engine for MySQL, MariaDB & Percona Server, including Amazon RDS MySQL and Amazon Aurora We’re excited to announce the availability on the Amazon Marketplace of the new Tungsten Replicator (AMI). Tungsten Replicator (AMI) is a replication engine that provides high-performance and improved replication functionality over the native MySQL replication solution and provides an ability to apply real-time MySQL data feed into a range of analytics and big data databases. Tungsten Replicator (AMI) builds on the well-established, commercial stand-alone product Tungsten Replicator and offers the exact same functionality, but with the convenience that comes with an ad-hoc online service: cost-effective and rapid, automated deployment. Target Databases Available as of Today

[Read more]
Troubleshooting Data Differences in a MySQL Database Cluster

Overview The Skinny

From time to time we are asked how to check whether or not there are data discrepancies between Master/Slave nodes within a MySQL (or MariaDB) cluster that’s managed with Tungsten Clustering. This is always a challenging task, not least because we hope and believe that our replication mechanism would avoid such occurrences, that said there can be factors outside of our control that can appear to “corrupt” data – such as inadvertent execution of DML against a slave using a root level user account.

Tungsten Replicator, the core replication component in our Tungsten Clustering solution for MySQL (& MariaDB), is just that, a replicator – it takes transactions from the binary logs and replicates them around. The replicator isn’t a data synchronisation tool in that respect, the …

[Read more]
Tungsten Clustering 6.0.5 and Tungsten Replicator 6.0.5 Released

Continuent is pleased to announce that Tungsten Clustering 6.0.5 and Tungsten Replicator 6.0.5 are now available!

Our v6.0.5 release fixes a number of bugs and introduces some new features, with improvements across the board in a variety of different components.

Some of the key improvements include:

  • A new Clustering utility script has been added to the release, tungsten_reset_manager, which assists with the graceful reset of the manager’s dynamic state files on disk.
  • Fixed an issue where the tpm command would allocate inconsistent THL listener ports for the Composite Multimaster (CMM) topology.
  • MySQL ping commands are now reconnected/retried upon “server gone away” error (Proxy mode ONLY).
  • mysql_checker_query script was returning …
[Read more]
Tungsten Clustering 5.3.6 and Tungsten Replicator 5.3.6 Released

Continuent is pleased to announce that Tungsten Clustering 5.3.6 and Tungsten Replicator 5.3.6 are now available!

Our 5.3.6 release fixes a number of bugs and has been released to improve stability.

Highlights common to both products:

  • Now, instead of searching for a master with appropriate role (i.e. matching the slave preferred role) until timeout is reached, the replicator will loop twice before accepting connection to any host no matter what its role is. (CT-712)
  • Changing the state machine so that RESTORING is not a substate of OFFLINE:NORMAL, but instead of OFFLINE so that a transition from OFFLINE:NORMAL:RESTORING to ONLINE is not possible any longer. Now it will not be possible to transition from
    OFFLINE:RESTORING to ONLINE (CT-797)
  • When an heartbeat is inserted, it will now use the JVM timezone instead of hardcoded UTC. (CT-803)
  • Now skipping files that are not …
[Read more]
Replicating data into Clickhouse

Clickhouse is a relatively new analytics and datawarehouse engine that provides for very quick insertion and analysing of data. Like most analytics platforms it’s built on a column-oriented storage basis and unlike many alternatives is completely open source. It’s also exceedingly fast, even on relatively modest platforms.

Clickhouse does have some differences from some other environments, for example, data inserted cannot easily be updated, and it supports a number of different storage and table engine formats that are used to store and index the information. So how do we get into that from our MySQL transactional store?

Well, you can do dumps and loads, or you could use Tungsten Replicator to do that for you. The techniques I’m going to describe here are not in an active release, but use the same principles as other part of our data loading.

We’re going to use the CSV-based batch loading system that is …

[Read more]
Tungsten Clustering 6.0.4 and Tungsten Replicator 6.0.4 Released

Continuent is pleased to announce that Tungsten Clustering 6.0.4 and Tungsten Replicator 6.0.4 are now available!

Our v6.0.4 release fixes a number of bugs and introduces some new features, with improvements across the board in a variety of different components.

Some of the key improvements include:

  • When installing from an RPM, the installation would automatically restart the Connector during the installation. This behavior can now be controlled by setting the parameter --no-connectors within the configuration to prevent tpm from restarting the Connectors during the automated update processing.
  • Cross-site Replicators within a composite multimaster deployment can now be configured to point to slaves by default, and to prefer slaves over masters during operation using the new --policy-relay-from-slave=true option
  • You can now enable an option so …
[Read more]
Understanding THL, Events and Storage: Part 1

When Tungsten Replicator extracts data, the information that has been extracted is written down into the Tungsten History Log, or THL. These files are in a specific format and they are used to store all of the extracted information in a format that can easily be used to recreate and generate data in a target.

Each transaction from the source is written into the THL as an event, so within a single THL file there will be one or more events stored. For each event, we record information about the overall transaction, as well as then information about the transaction itself. That event can contain one or more statements, or rows, or both. Because we don’t want to get an ever increasing single file, the replicator will also divide up the THL into multiple files to tmake the data easier to manage.

We’ll get down into the details soon, until then, let’s start by looking at the basics of the THL, files and sequence numbers and how to …

[Read more]
We’re hiring: Database Clustering Technical Sales & Support Engineer

We are growing!

Consequently, Continuent needs additional talented staff.

We are currently looking for a person who would be a good fit for our “Database Clustering Technical Sales & Support Engineer” position, in the US Pacific timezone.

If you know someone who would be a good fit for this role and our company’s culture, please send them our way.

Continuent Clustering 5.3.3/5.3.4 and Tungsten Replicator 5.3.3/5.3.4 Released

Continuent is pleased to announce that Continuent Clustering 5.3.4 and Tungsten Replicator 5.3.4 are now available!

Release 5.3.4 was released shortly after 5.3.3 due to a specific bug in our reporting tool tpm diag. All of the major changes except this one fix are in the 5.3.3 release.

Our 5.3.3/4 release fixes a number of bugs and has been released to improve stability in certain parts of the products.

Highlights common to both products:

  • Fixed an issue with LOAD DATA INFILE
  • Replicator now outputs the filename of the file when using thl to show events
  • tpm diag has been improved the way we extract MySQL and support Net:SSH options

Highlights in the clustering product:

  • Tungsten Manager stability has been improved by identifying some memory leaks.
  • Tungsten Connector has fixed a number of bugs relating to bridge mode, …
[Read more]
Showing entries 1 to 10 of 16
6 Older Entries »