We’ve recently been dealing with an interesting customer case where a PXC version upgrade from 5.7 to 8.0 seemed to trigger a completely unexpected problem. Even though the workload and MySQL settings did not change, all PXC members began exhibiting sparse but regular bursts of massive disk writes. Yet the MySQL side workload statistics that we gathered during those bursts did not explain the volume of them at all!
Analyzing these bursts further with pidstat and iostat confirmed that the mysqld process was indeed the sole source of excessive disk writes, and digging further showed the responsible threads to be the Galera receiver thread responsible for subscribing to the replication stream, as well as the main connection threads on the writer node, or Galera applier …
[Read more]