Coming from the MySQL world, I’m used to being able to easily determine the replication delay (in seconds) via the SHOW SLAVE STATUS command: mysql> show slave status\G *************************** 1. row *************************** ... Seconds_Behind_Master: 0 ... 1 row in set (0.00 sec) Unfortunately, there is no such comparable command in PostgreSQL. The official docs propose [...]
Why is it so important to close connections to databases if there’s no explicit need to keep them open (which usually the case)?
Symptoms
- Icinga reports high usage of allowed connections (>90%) on master MySQL server.
- No running queries or any specific in processlist. All the connections are sleeping from the same servers.
- The number of connections was 20 times more than the usual
Root cause
There was tremendous amount of queries doing “copying to tmp table” on the slaves. The connections were opened to both master and slaves and waiting for the slaves to answer the query the connections to master was kept opened. So the sleeping connections was piled up to the limit of max connections.
Lesson
- When troubleshooting always check slaves too.
- Close unused connections especially for masters.
- Keep your query optimized and …
If you ever had a replication slave that is severely behind, you probably noticed that it’s not catching up with a busy master at a steady pace. Instead, the “Seconds behind master” is going up and down so you can’t really tell whether the replica is catching up or not by looking at just few samples, unless these are spread apart. And even then you can’t tell at a glance when it is going to catch up.
Normally, the “severely behind” thing should not happen, but it does often happen in our consulting practice:
- sometimes replication would break and then it needs to catch up after it is fixed,
- other times new replication slave is built from a backup which is normally hours behind,
- or, it could be that replication slave became too slow to catch up due to missing index
Whatever the case is, single question I am being asked by the customer every time this happens is this: …
[Read more]MySQL data rules the cloud, but recent experience shows us that there's no substitute for maintaining copies of data, across availability zones, when it comes to Amazon Web Services (AWS) data resilience.
In this video (recording of our 8/23/12 webcast), we survey technologies for maintaining real-time copies of your data and the pros & cons of each. We conclude with a live demonstration of a
There has been a lot of talk about MySQL 5.6 replication improvements. With few exceptions, what I have seen was either marketing messages or hearsay. This means that few people have really tried out the new features to see whether they meet the users needs.
As usual, I did try the new version in my environment. I like to form my own opinion based on experiments, and so I have been trying out these features since they have appeared in early milestones.
What follows is a list of (potentially) surprising results that
you may get when using MySQL 5.6.
All the examples are made using MySQL 5.6.6.
Gotcha #1 : too much noise
I have already mentioned that MySQL 5.6 is too verbose when creating data directory. This also means that your error log may have way more information than you'd like to get. …
[Read more]
Synchronous data replication over long distances has the sort of
seductive appeal that often characterizes bad ideas. Why
wouldn't you want every local credit card transaction
simultaneously stored on the other side of the planet far away
from earthquake, storms and human foolishness? The answer
is simple: conventional SQL applications interact poorly with
synchronous replication over wide area networks (WANs).
I spent a couple of years down the synchronous replication rabbit
hole in an earlier Continuent product. It was one of those
experiences that make you a sadder but wiser person. This
article digs into some of the problems with synchronous
replication and shows why another approach, asynchronous
multi-master replication, is currently a better way to manage
databases connected by long-haul networks.
Synchronous Replication between …
I have seen a few posts on DBA.SE (where I answer a lot of questions) recommending the use of semi-synchronous replication in MySQL 5.5 over a WAN as a way to improve the reliability of replication. My gut reaction was that this is a very bad idea with even a tiny write load, but I wanted to test it out to confirm. Please note that I do not mean to disparage the author of those posts, a user whom I have great respect for.
What is semi-synchronous replication?
The short version is that one slave has to acknowledge receipt of the binary log event before the query returns. The slave doesn’t have to execute it before returning control so it’s still an asynchronous commit. …
[Read more]Monitoring MySQL’s replication has always been a bit hit and miss, especially when trying to detect whether a slave is becoming overloaded or not. There’s been a few ways to do this in the past:
- Monitor the Seconds_behind_master variable from SHOW SLAVE STATUS, which is the difference between the timestamp passed down from the master that the event was executed, to the time that the SQL thread started executing the event.
- Use something like mk-heartbeat, which operates in the same way as Seconds_Behind_Master (trying to show you the actual time difference between the master and slave), and is a little more robust in complex replication chains, and other situations where Seconds_Behind_Master falls down (such as when the IO thread is lagging). …
We found a simple XA transaction that crashes MySQL 5.5 replication. This simple transaction inserts a row into an InnoDB table and a TokuDB table. The bug was caused by a flaw in the logging code exposed by the transaction’s use of two XA storage engines (TokuDB and InnoDB). This bug was fixed in the TokuDB 6.0.1 release.
Here are some details. Suppose that a database contains the following tables.
create table t1 (a int) engine=InnoDB
create table t2 (a int) engine=TokuDB
The following transaction
begin
insert into t1 values (1)
insert into t2 values (2)
commit
causes the replication slave to crash.
The crash occurs when mysqld tries to dereference a NULL pointer.
#4 0x000000000088e203 in MYSQL_BIN_LOG::log_and_order (this=0x14b8640, thd=0x7f7758000af0, xid=161, all=true, need_prepare_ordered=false, need_commit_ordered=true) at …
[Read more]Read the original article at Accidental DBA’s Guide to MySQL Management
So you’ve been tasked with managing the MySQL databases in your environment, but you’re not sure where to start. Here’s the quick & dirty guide. Oh yeah, and for those who love our stuff, take a look to your right. See that subscribe button? Grab our newsletter!
1. Installation
The “yum” tool is your friend. If you’re using debian, you’ll use apt-get but it’s very similar. You can do a “yum list” to see what packages are available. We prefer to use the Percona distribution of MySQL. It’s fully compatible with stock MySQL distribution, but usually a bit ahead in terms of tweak and fixes. Also if you’re …
[Read more]