Showing entries 1 to 10 of 14
4 Older Entries »
Displaying posts with tag: replicator (reset)
Troubleshooting Data Differences in a MySQL Database Cluster

Overview The Skinny

From time to time we are asked how to check whether or not there are data discrepancies between Master/Slave nodes within a MySQL (or MariaDB) cluster that’s managed with Tungsten Clustering. This is always a challenging task, not least because we hope and believe that our replication mechanism would avoid such occurrences, that said there can be factors outside of our control that can appear to “corrupt” data – such as inadvertent execution of DML against a slave using a root level user account.

Tungsten Replicator, the core replication component in our Tungsten Clustering solution for MySQL (& MariaDB), is just that, a replicator – it takes transactions from the binary logs and replicates them around. The replicator isn’t a data synchronisation tool in that respect, the …

[Read more]
Tungsten Clustering 6.0.5 and Tungsten Replicator 6.0.5 Released

Continuent is pleased to announce that Tungsten Clustering 6.0.5 and Tungsten Replicator 6.0.5 are now available!

Our v6.0.5 release fixes a number of bugs and introduces some new features, with improvements across the board in a variety of different components.

Some of the key improvements include:

  • A new Clustering utility script has been added to the release, tungsten_reset_manager, which assists with the graceful reset of the manager’s dynamic state files on disk.
  • Fixed an issue where the tpm command would allocate inconsistent THL listener ports for the Composite Multimaster (CMM) topology.
  • MySQL ping commands are now reconnected/retried upon “server gone away” error (Proxy mode ONLY).
  • mysql_checker_query script was returning …
[Read more]
Tungsten Clustering 5.3.6 and Tungsten Replicator 5.3.6 Released

Continuent is pleased to announce that Tungsten Clustering 5.3.6 and Tungsten Replicator 5.3.6 are now available!

Our 5.3.6 release fixes a number of bugs and has been released to improve stability.

Highlights common to both products:

  • Now, instead of searching for a master with appropriate role (i.e. matching the slave preferred role) until timeout is reached, the replicator will loop twice before accepting connection to any host no matter what its role is. (CT-712)
  • Changing the state machine so that RESTORING is not a substate of OFFLINE:NORMAL, but instead of OFFLINE so that a transition from OFFLINE:NORMAL:RESTORING to ONLINE is not possible any longer. Now it will not be possible to transition from
    OFFLINE:RESTORING to ONLINE (CT-797)
  • When an heartbeat is inserted, it will now use the JVM timezone instead of hardcoded UTC. (CT-803)
  • Now skipping files that are not …
[Read more]
Replicating data into Clickhouse

Clickhouse is a relatively new analytics and datawarehouse engine that provides for very quick insertion and analysing of data. Like most analytics platforms it’s built on a column-oriented storage basis and unlike many alternatives is completely open source. It’s also exceedingly fast, even on relatively modest platforms.

Clickhouse does have some differences from some other environments, for example, data inserted cannot easily be updated, and it supports a number of different storage and table engine formats that are used to store and index the information. So how do we get into that from our MySQL transactional store?

Well, you can do dumps and loads, or you could use Tungsten Replicator to do that for you. The techniques I’m going to describe here are not in an active release, but use the same principles as other part of our data loading.

We’re going to use the CSV-based batch loading system that is …

[Read more]
Tungsten Clustering 6.0.4 and Tungsten Replicator 6.0.4 Released

Continuent is pleased to announce that Tungsten Clustering 6.0.4 and Tungsten Replicator 6.0.4 are now available!

Our v6.0.4 release fixes a number of bugs and introduces some new features, with improvements across the board in a variety of different components.

Some of the key improvements include:

  • When installing from an RPM, the installation would automatically restart the Connector during the installation. This behavior can now be controlled by setting the parameter --no-connectors within the configuration to prevent tpm from restarting the Connectors during the automated update processing.
  • Cross-site Replicators within a composite multimaster deployment can now be configured to point to slaves by default, and to prefer slaves over masters during operation using the new --policy-relay-from-slave=true option
  • You can now enable an option so …
[Read more]
Understanding THL, Events and Storage: Part 1

When Tungsten Replicator extracts data, the information that has been extracted is written down into the Tungsten History Log, or THL. These files are in a specific format and they are used to store all of the extracted information in a format that can easily be used to recreate and generate data in a target.

Each transaction from the source is written into the THL as an event, so within a single THL file there will be one or more events stored. For each event, we record information about the overall transaction, as well as then information about the transaction itself. That event can contain one or more statements, or rows, or both. Because we don’t want to get an ever increasing single file, the replicator will also divide up the THL into multiple files to tmake the data easier to manage.

We’ll get down into the details soon, until then, let’s start by looking at the basics of the THL, files and sequence numbers and how to …

[Read more]
We’re hiring: Database Clustering Technical Sales & Support Engineer

We are growing!

Consequently, Continuent needs additional talented staff.

We are currently looking for a person who would be a good fit for our “Database Clustering Technical Sales & Support Engineer” position, in the US Pacific timezone.

If you know someone who would be a good fit for this role and our company’s culture, please send them our way.

Continuent Clustering 5.3.3/5.3.4 and Tungsten Replicator 5.3.3/5.3.4 Released

Continuent is pleased to announce that Continuent Clustering 5.3.4 and Tungsten Replicator 5.3.4 are now available!

Release 5.3.4 was released shortly after 5.3.3 due to a specific bug in our reporting tool tpm diag. All of the major changes except this one fix are in the 5.3.3 release.

Our 5.3.3/4 release fixes a number of bugs and has been released to improve stability in certain parts of the products.

Highlights common to both products:

  • Fixed an issue with LOAD DATA INFILE
  • Replicator now outputs the filename of the file when using thl to show events
  • tpm diag has been improved the way we extract MySQL and support Net:SSH options

Highlights in the clustering product:

  • Tungsten Manager stability has been improved by identifying some memory leaks.
  • Tungsten Connector has fixed a number of bugs relating to bridge mode, …
[Read more]
Mastering Tungsten Replicator Series: Tasty THL Tips – unsafe_for_block_commit

The Tungsten Replicator is an extraordinarily powerful and flexible tool capable of moving vast volumes of data from source to target.

In this blog post we will discuss one specific aspect of the THL (Transaction History Log) – the METADATA unsafe_for_block_commit flag.

What do you mean, Unsafe?

In a recent customer support case, we were asked the meaning of the unsafe_for_block_commit flag. For example, list the event information for sequence number 3481394254:

[tungsten@tr2-mysql01 (sandbox) ~]$ thl list -seqno 3481394254 | more
SEQ# = 3481394254 / FRAG# = 0 (last frag)
- TIME = 2018-09-16 06:52:47.0
- EPOCH# = 3480364140
- EVENTID = mysql-bin.001068:0000000294739578;622252
- SOURCEID = tr2-mysql01.sandbox.yourdomain.com
- METADATA = [mysql_server_id=1;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;service=brm;shard=shard_1736]
- TYPE = …
[Read more]
Mastering Continuent Clustering Series: Tuning for High-Latency Links

What if I want the cluster to be less sensitive to network, especially WAN latency?

Continuent Clustering supports having clusters at multiple sites with active-active replication meshing them together.

This is extraordinarily powerful, yet at times high network latency can make it harder for messaging between the sites to arrive in a timely manner.

This is evidenced by seeing the following in the Manager log files named tmsvc.log:

2018/07/08 16:51:05 | db3 |  INFO [Rule_0604$u58$_DETECT_UNREACHABLE_REMOTE_SERVICE1555959201] - CONSEQUENCE: [Sun Jul 08 16:51:04 UTC 2018] CLUSTER global/omega(state=UNREACHABLE)
...
2018/07/08 16:51:42 | db3 |  INFO [Rule_2025$u58$_REPORT_COMPONENT_STATE_TRANSITIONS1542395297] - CLUSTER 'omega@global' STATE TRANSITION UNREACHABLE => ONLINE

The delta is 37 seconds in the above example between state=UNREACHABLE and UNREACHABLE => ONLINE

The default …

[Read more]
Showing entries 1 to 10 of 14
4 Older Entries »