A database is showing replication delay, and so are all the other instances of the same replication hierarchy, all of which reside in Openstack.
Shortly before 21:30 the database begins to lag, until around 23:45, when it starts to catch up, slowly. After 00:30, we gain delay again, plateau and then around 01:45, we catch up.
The database is moving deep into replication delay sometimes. It does not do that on bare metal.
The VM is a nice hardware blade simulation, 16C/32T, 128 GB of memory.
The data is on persistent volume backed by Ceph.
# df -Th /mysql/<hierarchyname>/
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/vg00-mysqlVol xfs 1.8T 1.1T 766G 58% /mysql/<hierarchyname>
# vgs
VG #PV #LV #SN Attr VSize VFree
sysvm 1 8 0 …[Read more]