… an expedition to uncover (and fix) database performance issues!

© 2021 Tiago L. Alves. All rights reserved.

One of the worse things that can happen when upgrading to a newer database version is discovering that the performance is not as good as before. Despite the effort put into gate-keeping MySQL NDB Cluster’s strict performance requirements, one of our customers found a performance regression when upgrading from our 7.4 version to our 7.6 version. How did that happen when our automated performance test suite failed to show it? To answer that, and fix the issue we enrolled on a database performance archaeology expedition…

When your performance is not good enough

MySQL NDB Cluster is an open-source in-memory distributed database developed for high-availability (99.999% or more) and predictable query times. It can be found at the core of gaming, banking, telecommunication, and online services. …

In this post, we will see, how easily user can take NDB Cluster backup and then restore it. NDB cluster supports online backups, which are taken while transactions are modifying the data being backed up. In NDB Cluster, each backup captures all of the table content stored in the cluster.

User can take backup in the following states:

• When the cluster is live and fully operational
• When the cluster is live, but in a degraded state:
  • Some data nodes are down
  • Some data nodes are restarting
• During read and write transactions
  

Users can restore backups in the following cluster environments:

• Restore to the same physical cluster
• Restore into a different physical cluster
• Restore into a different configuration cluster i.e. backup …

In this post we will see the unique features of MySQL Cluster i.e. Self healing and no single point of failure.

MySQL Cluster No Single Point Of Failure:

As we know that in MySQL Cluster, data nodes are logically grouped into nodegroups. All data nodes in a nodegroup contain the same data, kept in sync at all times. Different nodegroups contain different data. So in a single nodegroup, MySQL Cluster can sustain multiple node shutdown at the same time but at least one node must live in that nodegroup. Lets demonstrate this feature with a 4 replicas cluster where each nodegroup have 4 data nodes.

Lets create a MySQL Cluster with below environment.

• MySQL Cluster 8.0.22 version
• 2 Management servers
• 8 Data nodes servers
• 2 Mysqld servers
• 4 API nodes
• OS: Ubuntu 20.04 LTS
• NoOfReplicas = 4
  

…

In this post, we will see how easily we can install MySQL Cluster on different physical hosts. One can install MySQL Cluster on a single host too. If you are wondering what MySQL Cluster is then please look into my previous post.
MySQL Cluster Installation:

Lets create a MySQL Cluster with the following environment:

• MySQL Cluster 8.0.22 version

• 2 Management servers

• 4 Data nodes servers

• 2 Mysqld servers

• 4 API nodes

• OS: Ubuntu 20.04 LTS
  
  
  

Hostname and IP address of all the nodes in the MySQL Cluster:
Management Node1 (mgmt1):        Hostname: …

MySQL Cluster: MySQL Cluster is a real time, ACID-compliant transactional database. It is a combination of MySQL server and the NDB storage engine. Data is cached in memory and durably stored on disk. Indexed columns are always kept in memory while non-indexed columns can be kept in memory or disk. It was mainly designed for telecom databases with 99.999% availability and high performance. Unlike simpler sharded systems, MySQL Cluster transactions can transparently query and update all data in the system.

Key features of MySQL Cluster:

• MySQL Cluster is designed using a shared nothing architecture
  

• Support for large database sizes
  

  • With all columns kept in memory, can store upto few terabytes
    

  • With some columns kept on disk, can store upto few petabytes
    

  …

MySQL highly available solutions, InnoDB Cluster (it uses InnoDB storage engine and is based on Group Replication plugin) and NDB Cluster (NDB storage engine), offer high scalability and redundant topologies.

• InnoDB Cluster can be configured with up to 9 replicas, in single primary configuration or multi-primary.
• NDB Cluster instead, while being a much different solution, offers the chance to have …

For more information on how to create a service look at man systemd.serviceInstallDirectory for MCM installation is /opt Download latest tar package of MCM from edelivery.oracle.com or support.oracle.com (today this is package: MySQL Cluster Manager 1.4.8 TAR for Generic Linux x86 (64bit))Run commands below once you have downloaded the MCM package:

sudo su -
cd /opt
tar xzf /tmp/mcm-1.4.8-linux-glibc2.12-x86-64bit.tar.gz
mv mcm-1.4.8-linux-glibc2.12-x86-64bit mcm
cp /opt/mcm/mcm1.4.8/etc/mcmd.ini /opt/mcm/
mkdir /opt/mcm/mcm_data

Set manager-directory to folder /opt/mcm/mcm_data in configuration file:

cat /opt/mcm/mcmd.ini | sed 's/^##manager-directory.*/manager-directory = \/opt\/mcm\/mcm_data/' >> /opt/mcm/mcmd.ini

…

What a great event ! I got very positive feedback for this extended preFOSDEM MySQL Day 2020.

Before my summary, I wanted to thank again all speakers and attendees. Without you this event won’t even exist. And thank you to ICAB for hosting us yet another time !

As you may know, we had 2 parallel tracks for 2 consecutive days. We had very valuable technical content from the MySQL developers and also from the external community members who talked about their favorite database.

There was a large interest in MySQL NDB Cluster and it was obvious that the nice presentations we had, encouraged people to try the worlds highest performance open source in-memory database (and they can even test it on their laptop with DBDeployer). The NDB users were also happy with the new stuff and impressive numbers that Bernd …

The schedule of the preFOSDEM Day is now available !

We had a lot of proposals to deal with. Also this is a MySQL event where we, the MySQL Team has the possibility to show to you, our Community, all what we have working on to improve MySQL but also new stuff. We also invite some of our friends from the MySQL Community to talk about their experience.

I think we did a good selection and propose you new content. We are extremely happy to have Saverio Miroddi from TicketSolve talking about MySQL 8.0, Uber talking about InnoDB Cluster, and Facebook about Binlog.

As you can see, we will have 2 rooms, where one will be dedicated mostly to SQL and Optimizer topics but also tutorials.

You can also see that this year we also want to put some spot lights to MySQL NDB Cluster. You will see what is it, how is it used and what’s new. We will also have 2 community speakers sharing their MySQL NDB knowledge: Giuseppe Maxia, …

In this part we are going to analyze a bit more complex query than before.
This query is a 6-way join.

The query is:
SELECT
        supp_nation,
        cust_nation,
        l_year,
        SUM(volume) AS revenue
FROM
        (
                SELECT
                        n1.n_name AS supp_nation,
                        n2.n_name AS cust_nation,
                        extract(year FROM l_shipdate) as l_year,
                        l_extendedprice * (1 - l_discount) AS …