Percona XtraDB Cluster (PXC) offers a great deal of flexibility when it comes to the state transfer (SST) options (used when a new node is automatically provisioned with data). For many environments, on-the-fly compression capability gives great benefits of saving network bandwidth during the process of sending sometimes terabytes of data. The usual choice for compression here is a built-in Percona XtraBackup compress option (using qpress internally), or options compressor/decompressor for the compression tool of choice. In the second case, the popular option is the gzip or its …[Read more]
4 Older Entries »
State Snapshot Transfer can be a very long and expensive process, depending on the size of your Percona XtraDB Cluster (PXC)/Galera cluster, as well as network and disk bandwidth. There are situations where it is needed though, like after long enough node separation, where the gcache on other members was too small to keep all the needed transactions.
Let’s see how we can avoid SST, yet recover fast and without even the need for doing a full backup from another node.
Below, I will present a simple scenario, where one of the cluster nodes was having a broken network for long enough that it will make …[Read more]
Normally when a node is taken out of the cluster for a short period of time (for maintenance or shutdown), gcache on other nodes of the cluster help donate the missing write-set(s) when the node rejoins. This approach works if you have configured a larger gcache, or the downtime is short enough. Both approaches aren’t good, especially for a production cluster. Also, a larger gcache for the server lifetime means blocking larger disk-space when the same job can be done with relative smaller disk-space.
Re-configuring gcache, on a potential DONOR node before downtime requires a node …[Read more]
One of the things I like about consulting at Percona is the opportunity to be exposed to unusual problems. I recently worked with a customer having issues getting SST to work with Percona XtraDB Cluster. A simple problem you would think. After four hours of debugging, my general feeling was that nothing made sense.
I added a bash trace to the SST script and it claimed MySQL died prematurely:
[ -n '' ]] + ps -p 11244 + wsrep_log_error 'Parent mysqld process (PID:11244) terminated unexpectedly.' + wsrep_log '[ERROR] Parent mysqld process (PID:11244) terminated unexpectedly.' ++ date '+%Y-%m-%d %H:%M:%S' + local readonly 'tst=2017-11-28 22:02:46'
At the same time, from the MySQL error log MySQL was …[Read more]
In this blog post, we’ll look at the performance of SST data transfer using encryption.
In my previous post, we reviewed SST data transfer in an unsecured environment. Now let’s take a closer look at a setup with encrypted network connections between the donor and joiner nodes.
The base setup is the same as the previous time:
- Database server: Percona XtraDB Cluster 5.7 on donor node
- Database: sysbench database – 100 tables 4M rows each (total ~122GB)
- Network: donor/joiner hosts are connected with dedicated 10Gbit LAN
- Hardware: donor/joiner hosts – boxes with 28 Cores+HT/RAM 256GB/Samsung SSD 850/Ubuntu 16.04
The setup details for the encryption aspects in our testing:
- Cryptography libraries: openssl-1.0.2, …
In this blog, we’ll look at evaluating the performance of an SST data transfer without encryption.
A State Snapshot Transfer (SST) operation is an important part of Percona XtraDB Cluster. It’s used to provision the joining node with all the necessary data. There are three methods of SST operation available: mysqldump, rsync, xtrabackup. The most advanced one – xtrabackup – is the default method for SST in Percona XtraDB Cluster.
We decided to evaluate the current state of xtrabackup, focusing on the process of transferring data between the donor and joiner nodes tp find out if there is any room for improvements or optimizations.
Taking into account that the security of the network connections used for Percona XtraDB Cluster deployment is one of the most important factors that affects SST performance, we will evaluate SST operations in two setups: without network encryption, and in …[Read more]
CentOS 7 Systemd issue with Percona Cluster is that SST fails to sync the nodes, unable to join cluster group and giving the misleading broken pipe 32 SIG errors.
The post How to Resolve Systemd Issue with Percona XtraDB Cluster on CentOS 7 appeared first on Datavail.
Percona XtraDB Cluster 5.6.32-25.17 is now the current release, based on the following:
- Percona Server 5.6.32-78.1
- Galera Replication library 3.17
- Codership wsrep API version 25
My last post was an introduction to Red Hat’s Ceph. As interesting and useful as it was, it wasn’t a practical example. Like most of the readers, I learn about and see the possibilities of technologies by burning my fingers on them. This post dives into a real and novel Ceph use case: handling of the Percona XtraDB Cluster SST operation using Ceph snapshots.
If you are familiar with Percona XtraDB Cluster, you know that a full state snapshot transfer (SST) is required to provision a new cluster node. Similarly, SST can also be triggered when a cluster node happens to have a corrupted …[Read more]
4 Older Entries »