Mydumper supports streaming of backups right from version 0.11.3 and the latest version Mydumper 0.12.3 it enabled its support for compressed streaming backup. This was the most awaited feature added to Mydumper, making it a more powerful tool for data migration to RDS or Cloud.

If you are hearing mydumper for the first time, then let’s have a quick catch-up on what Mydumper is and what it does exactly.

Mydumper is a multithread logical backup and restores tool for MySQL and its forks. To know more you can refer to our previous blogs/presentations below.

• https://mydbops.wordpress.com/2016/01/31/faster-logical-backup-with-mydumper/
• …

Aurora Serverless V2 is generally available around the corner recently 21-04-22 for MySQL 8 and PostgreSQL, with promising features that overcome the V1 disadvantages. Below are those major features

Features

• Online auto instance upsize (vertical scaling)
• Read scaling (Supports up to 15 Read-replica)
• Supports mixed-configuration cluster ie, the master can be normal Aurora(provisioned) and readers can be in serverlessv2 and vice versa
• MultiAZ capability (HA)
• Aurora global databases (DR)
• Scaling based on memory pressure
• Vertically Scales while SQL is running
• Public IP allowed
• Works with custom port
• Compatible with Aurora version 3.02.0 ie., >= MySQL 8.0.23 (only supported)
…

In the previous blog, we have gone through about network-level security in RDS. In this blog, we will see about the Database level security in RDS.

After network-level restriction to host, we can’t allow a user to connect to the database from anywhere over the internet. We need to restrict user access at the Database level as well. Need to create a user with a restricted host along with a strong password to avoid cracking of password. In RDS we have a special feature named Identity and Access Management (IAM).

We need to monitor the user activity as well in the Database. Because the user may wrongly execute the query in the server which leads to data loss or production outage. The user activity has to be monitored as per the compliances. We can achieve this by enabling the audit log in the RDS.

By …

Nowadays everything is getting digitalized and migration toward the cloud is at its peak. There is a high chance of data leaks if we don’t tighten the security of the Database servers. It is mandatory to secure the database by restricting access to Database. Although we have restricted the access. We need to monitor the activity of the user to prevent the unwanted usage of data.

Security will be split into three layers

1. Network-level security.
2. OS level security.
3. Database level security.

OS level security will be handled by the AWS team. Since It is managed by the AWS Team. So all the security patching, minor version upgrades of OS, and kernel tuning will be governed by the AWS infra team.

Network-level security and database-level security are owned by the end user.

While exporting the table with MySQL native utility, we don’t have any control on the process, and also there will be no progress update as well on the process completion. So when exporting the larger table will consume high resource utilization and also the disk space usage will also be high.

MySQL shell utility will make the process easier. It will export the table and we can import the data back with a parallel thread and also will provide the current progress status on export/import progress.

util.exportTable() utility was introduced in Shell – 8.0.22 version, will export the data in a controlled manner. We can store the data in either local or Cloud Infrastructure Object Storage bucket as well.

We will see about the compression ratio along with the time taken for native MySQL vs Shell utility

Feature :

• Compression
• Progress status
• Supported output …

We all tried various alternative methods for modifying the table structure, but pt-online-schema-change (pt-osc) is the most convenient and preferred method for performing the alter online. It has more granular control too. But it may lead to data loss if proper precautionary steps are not taken care of.

In this blog, we are going to modify a column to a unique key using pt-osc, below I have shared the table structure.

mysql> show create table test\G
* 1. row *
Table: test
Create Table: CREATE TABLE test (
Personid int(11) NOT NULL AUTO_INCREMENT,
LastName varchar(255) NOT NULL,
FirstName varchar(255) DEFAULT NULL,
Age int(11) DEFAULT NULL,
PRIMARY KEY (Personid)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

I have inserted the data of 1000 rows using …

Aurora has a salient feature “Parallel query“, Which will be more beneficial for analytical workload environments.

Before going to deep dive on this particular feature, let us understand the basis of Aurora.

Aurora Archiecture

Key feature

• The key feature of Aurora is simple data synchronisation among the nodes. The sync latency will be too low when compared to RDS because the synchronisation is happening on storage volumes among the nodes. Also all the server will available in different zone, even when a zone goes down we can able to maintain will other server present in other zone with auto failure.

• Auto healing volume, Each …

Over the decades we have been reading the MySQL error log from the server system file, if there are any issues in MySQL or any unknown restart happened , generally we look at the mysql error log.

By default MySQL error log can be found in the default path /var/log/mysqld.log , or it can be explicitly configured using the variable log_error.

Few drawbacks using MySQL error log as FILE

• Possibility of missing genuine errors while reading lengthy information.
• Filtering of errors for the particular date and timeframes.
• Cannot provide the DB server access to developers because of fear of mishandling DB servers.

To overcome the above issues , from MySQL …

Recently one of our customers ran into an issue, wherein a bad actor(code) from the application had made the wrong update to 16 M records of a critical table in the database, causing the entire production process to go down. The application Team was able to find the bad actor and block it, our Remote DBA was involved in the Data Recovery/Rollback.

Here I would like to discuss possible recovery methods for the above said scenario

Delayed Slave:

A simple and effective way to recover is by using a delayed slave, RDS started supporting this feature from version 5.6.40 and 5.7.22 i.e., you can induce a SQL thread delay-interval for applying the writes to a slave, detailed implementation is covered in our blog here. It’s …

In my day to day tasks as a DB Engineer at Mydbops we have been using multiple MySQL tools for multiple use cases to ensure an optimal performance and availability for servers managed by our Remote DBA Team.

A tool like pt-online-schema can be used for any DDL changes ( Overview to DDL algorithm ), if any tool which needs to scheduled for longer period we tend to use screen or cron.

Some of the problems we face when we demonise the process or use screen for running processes.

• The daemon process gets killed when the server reboot happens. …