Used to be that replication lag was as simple as Seconds_Behind_Master (renamed to Seconds_Behind_Source). But with multi-threaded replication (MTR) this is no longer the case. It’s time to relearn replication lag monitoring using Performance Schema tables.

With multi-threaded replication (MTR), a replica can commit transactions in the same order as the source, or not. This is determined by sysvar replica_preserve_commit_order (RPCO). As of MySQL v8.0.27 (released October 2021) it’s ON by default, but it was OFF by default for several years prior. In either case, it’s relatively new compared to 20+ years of single-threaded replication for which commit order was not an issue or option. But with MTR, it’s important to understand the affects of RPCO, especially with respect to the focus of this three-part series: replication lag.

After installing & configuring Dolphie, let’s take a look into how we can “re-record not fadeaway” and avoid using a VHS tape.

One of the coolest features is being able to go back in time with Dolphie and analyze what was happening at a specific moment.

This feature requires recording so we can replay.

Setting Dolphie up for recording mode.

I’m really just going to share the links to the the github site and organize my steps so someone else might want to rinse’n’repeat or “replay”. I’m just a mere messenger.

It works via the Daemon mode. …

I have been looking into “dolphie” lately, and have to say, “thanks Charles!”. I actually first saw dolphie via Lefred’s MySQL Belgian Days and installed it just after looking into the slidedeck. But never got around to looking further… until now.

We can use so many different observability (o11y?) tools to get notifications, alerts, react, generate reports, etc. from so many different companies, using agents, proxies, repositories, and so on and so forth. And after exchanging experiences with dolphie’s author himself, Charles, the idea here is to go that little bit further in …

MySQL 8.0 and newer change and improve how we measure and monitor replication lag. Even though multi-threaded replication (MTR) has been on by default for the last three years (since v8.0.27 released October 2021), the industry has been steeped in single-threaded replication for nearly 30 years. As a result, replication lag with MTR is a complicated topic because it depends on version, configuration, and more. This three-part series provides a detailed understanding, starting from what was originally an unrelated feature: binary log group commit.

Just thought I’d share a script I use daily and helps me redirect my attention if needed.

This is but a mere pointer, guideline and starting point in any task. I just thought I’d share and hope someone else’s day becomes slightly easier thanks to some brief investigation and command tweaking.

Now the really handy thing here is that I only hard code the router01 node name, as I’m using that as a potential endpoint (thinking cloud, XaaS, etc…) where it could also be a VIP, LBR or similar. It’s the entry point so I can query the P_S table error_log so I can get different views and act accordingly.

For example:

• First, give me the InnoDB Cluster ordered server list so I can take a step back from my usual pains and worries, and see the architecture view. And make me type “Y” or similar to move on. Here if there were any server missing, I’d see the summary right away so I don’t really need to …

Replication has been the core functionality, allowing high availability in MySQL for decades already. However, you may still encounter replication errors that keep you awake at night. One of the most common and challenging to deal with starts with: “Got fatal error 1236 from source when reading data from binary log“. This blog post is […]

Replication being slow—replication lag—is a common complaint, but MySQL replication is actually really fast. Let’s run a controlled experiment and peek inside the Performance Schema binary logs to see why.

Replication being slow—replication lag—is a common complaint, but MySQL replication is actually really fast. Let’s run a controlled experiment and peek inside the Performance Schema and binary logs to see why.

Replication being slow—replication lag—is a common complaint, but MySQL replication is actually really fast. Let’s run a controlled experiment and peek inside the Performance Schema and binary logs to see why.