Showing entries 1 to 10 of 88
10 Older Entries »
Displaying posts with tag: troubleshooting (reset)
Dolphie – “Rerecord not fadeaway”

After installing & configuring Dolphie, let’s take a look into how we can “re-record not fadeaway” and avoid using a VHS tape.

One of the coolest features is being able to go back in time with Dolphie and analyze what was happening at a specific moment.

This feature requires recording so we can replay.

Setting Dolphie up for recording mode.

I’m really just going to share the links to the the github site and organize my steps so someone else might want to rinse’n’repeat or “replay”. I’m just a mere messenger.

It works via the Daemon mode. …

[Read more]
Using Dolphie for MySQL focus monitoring & performance tuning.

I have been looking into “dolphie” lately, and have to say, “thanks Charles!”. I actually first saw dolphie via Lefred’s MySQL Belgian Days and installed it just after looking into the slidedeck. But never got around to looking further… until now.

We can use so many different observability (o11y?) tools to get notifications, alerts, react, generate reports, etc. from so many different companies, using agents, proxies, repositories, and so on and so forth. And after exchanging experiences with dolphie’s author himself, Charles, the idea here is to go that little bit further in …

[Read more]
Some InnoDB Cluster troubleshooting commands

Different ways to get the status:

mysqlsh --login-path=icadmin -h$MYROUTER1 -- cluster status

mysqlsh --login-path=icadmin -h${HOSTNAME} --redirect-primary -- cluster status

mysqlsh icadmin:'P4ssw0rD'@db01:3306 -- cluster status --extended=0

mysqlsh icadmin@$MYROUTER1:3306 -- cluster status --extended=1

watch -n 5 “mysqlsh --login-path=icadmin -h$MYROUTER1 -- cluster status”

Set the Primary Instance (switching):

mysqlsh --login-path=icadmin -h${HOSTNAME} --redirect-primary -- cluster set_primary_instance "db01"

Obtaining MySQL InnoDB Cluster basics:

select cluster_id, cluster_name, description, cluster_type, primary_mode, clusterset_id from mysql_innodb_cluster_metadata.clusters;

Members of our cluster:

select * from performance_schema.replication_group_members order by MEMBER_ROLE;

Local & Remote Trans Q’s:

[Read more]
How to Troubleshoot a MySQL Replica IO Thread that is Stuck in a Connecting State

Discover how to troubleshoot a MySQL replica IO thread stuck in a connecting state. Learn about the replication architecture, security group rules for AWS EC2 instances, and how to address common issues like network restrictions and bind address configuration.

MySQL is a powerful database management and a widely used cloud database service. One of its key features is the ability to create replicas of a master database to improve its availability and scalability. However, at times the IO thread in a MySQL replica may get stuck in a connecting state, which can cause replication issues and affect the overall data consistency …

[Read more]
Prevent ProxySQL from directing traffic to broken MySQL replica

ProxySQL is an open-source MySQL proxy server, meaning it serves as an intermediary between a MySQL server and the applications that access its databases. ProxySQL can improve performance by distributing traffic among a pool of multiple database servers.

Consider 2 slaves are routed under Proxysql , In any one of the slave, if the replication is broken, we could still see the traffic routing to the broken replication slave. We can make Proxy to not send traffic to broken replication slave, by setting appropriate value to the variable mysql-monitor_slave_lag_when_null

[Read more]
Beware of your Backup before upgrading MySQL 8.0.30

MySQL 8.0.30 was released on 26-07-2022. It has a good list of features.

  • Generated Invisible Primary Key
  • Dynamic Redo log Sizing
  • Instruments to monitor group replication memory
  • Support for Ubuntu 22.04

A most notable feature for Database Administrators and database reliability engineers is Dynamic redo logging.

The more widely adopted open-source backup tool is Xtrabackup. The change in the redo log design has affected this hot backup tool.

2022-10-04T18:40:08.211998+05:30 0 [Note] [MY-011825] [Xtrabackup] recognized server arguments: --datadir=/var/lib/mysql 
2022-10-04T18:40:08.212264+05:30 0 [Note] [MY-011825] [Xtrabackup] recognized client arguments: …
[Read more]
Troubleshooting XA transactions in MySQL

This is one of the exciting troubleshooting related to XA transactions let’s dive in. Let me provide a few backgrounds.

For one of our clients, when we try to drop a table, it is waiting for metadata lock. On debugging it is one of the XA transactions is holding the shared write lock and causing metadata lock on all the other local transactions.

Let us view the metadata locks from the Performance Schema.

mysql> select OBJECT_TYPE,OBJECT_SCHEMA,OBJECT_NAME, LOCK_TYPE,LOCK_STATUS,SOURCE from performance_schema.metadata_locks\G
******************* 1. row *********************
OBJECT_TYPE: TABLE
OBJECT_SCHEMA: sbtest
OBJECT_NAME: sbtest_table1
LOCK_TYPE: SHARED_WRITE
LOCK_STATUS: GRANTED

SOURCE: xa.cc:284
******************* 2. row *********************
OBJECT_TYPE: TABLE …
[Read more]
MySQL “No space left on device from storage engine”

We have planned for archiving the data to improve the DB performance and to reclaim the space. We were evaluating Compression in InnoDB and TokuDB. To find out the best compression method. We started benchmarking the compression ratio between InnoDB and TokuDB.

Everything goes well for some time, but after a few hours got an error message that can’t able to insert the data into the TokuDB table due to storage being full. It is so strange and the host has enough free space.


Table structure:-

mysql> show create table mydbops.tokudb\G
*************************** 1. row ***************************
       Table: tokudb
Create Table: CREATE TABLE `tokudb` (
  `ID` int DEFAULT NULL,
  `Name` longtext,
  `Image` blob
) ENGINE=TokuDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
1 row in set (2.18 sec)

mysql> show create table mydbops.innodb\G
*************************** 1. row …
[Read more]
Handling MySQL case sensitive column in pt-archiver

To copy the data of the particular column of the table to another table/server, We have an option to export the data as CSV and import the data back to a different table. But when the table size is large and we need to copy the data only for the required data to the target table will cause the load in the server since the table scanning is huge.

To overcome this, we have the pt-archiver copy the data from the source table to the destination as a whole or only for required columns. And also we can do this in a controlled manner as well. So there will be no performance impact even on the production time.

Source table structure :

mysql> show create table source\G
*************************** 1. row ***************************
       Table: source
Create Table: CREATE TABLE `source` (
  `id` int unsigned NOT NULL …
[Read more]
Troubleshooting an unique key addition during pt-online-schema-change

We all tried various alternative methods for modifying the table structure, but pt-online-schema-change (pt-osc) is the most convenient and preferred method for performing the alter online. It has more granular control too. But it may lead to data loss if proper precautionary steps are not taken care of.

In this blog, we are going to modify a column to a unique key using pt-osc, below I have shared the table structure.

mysql> show create table test\G
* 1. row *
Table: test
Create Table: CREATE TABLE test (
Personid int(11) NOT NULL AUTO_INCREMENT,
LastName varchar(255) NOT NULL,
FirstName varchar(255) DEFAULT NULL,
Age int(11) DEFAULT NULL,
PRIMARY KEY (Personid)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

I have inserted the data of 1000 rows using …

[Read more]
Showing entries 1 to 10 of 88
10 Older Entries »