There is much more to write about all the work we do at Facebook with memory management efficiency on our systems, but there was this one detour investigation in the middle of 2012 that I had to revisit recently courtesy of Wikipedia.

There are lots of factors that make machines page out memory segments into disk, thus slowing everything down and locking software up – from file system cache pressure to runaway memory leaks to kernel drivers being greedy. But certain swap-out scenarios are confusing – systems seem to have lots of memory available, with proper settings file system cache should not cause swapping, and obviously in production environment all the memory leaks are ironed out.

And yet in mid-2012 we noticed that our new kernel machines were swapping out for no obvious reason. When it comes to swapping, MySQL community will always point to Jeremy’s post on …

I was busy testing a PXC cluster today when suddenly was buffled with a confusing error:

140108 23:33:39 innobackupex: Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_group=xtrabackup' (using password: NO).
innobackupex: Error: Failed to connect to MySQL server as DBD::mysql module is not installed at /usr/bin/innobackupex line 2913.

OK, so my first though was DBD::mysql is missing, but as I checked:

[root@pxc03 keepalived]# yum list installed| grep perl-DBD-MySQL
perl-DBD-MySQL.x86_64 4.013-3.el6 @base

After some digging, it’s not actually the module that is missing – it was one of the module’s dependency which is the mysql client library.

[root@pxc03 keepalived]# yum deplist perl-DBD-MySQL.x86_64|grep mysql
dependency: libmysqlclient.so.16()(64bit)
provider: mysql-libs.x86_64 5.1.71-1.el6
dependency: libmysqlclient.so.16(libmysqlclient_16)(64bit)
provider: mysql-libs.x86_64 …

I built a new image on VMWare Fusion for my class, which required installing MySQL 5.6 on Fedora, Version 20. If you don’t know how to add your user to the sudoers list, you should check this older and recently updated blog post.

1. Download the MySQL Yum Repository and launch the downloaded RPM.

1. Install MySQL on Fedora, Version 20, which you can find with the following command:

shell> rpm -qa | grep mysql
mysql-community-release-fc20-5.noarch

The fc20-5 changes with point releases, but assuming that you’re installing the fc20-5 release:

      shell> sudo yum localinstall …

In short, during an optimization or a MySQL query involving a lot of replace(replace(replace…))) I decided to compile PREG as lib_mysqludf_preg from UDF Repository for MySQL in a view to use PCRE functions directly in MySQL.

Too bad MySQL still cant not use functions for index creation, but this is not the theme of this post.

So, fist install MariaDB 10.0.6:
set the MariaDB 10.0.6 repo, I used the ftp mirror at ftp.ulak.net.tr

root@seik-centos-01:[Wed Dec 18 23:55:12][/tmp]$ cat /etc/yum.repos.d/MariaDB.repo 
# MariaDB 10.0 CentOS repository list - created 2013-12-18 18:07 UTC
# http://mariadb.org/mariadb/repositories/
[mariadb]
name = MariaDB
#baseurl = http://yum.mariadb.org/10.0/centos6-amd64
baseurl = ftp://ftp.ulak.net.tr/pub/MariaDB/mariadb-10.0.6/yum/centos6-amd64/
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1 …

Because most MySQL production systems probably run on Linux, I’ve decided to place the most important Linux tuning tips that will help improve MySQL performance. There is nothing new here, most of them are well known, however, I’ve decided to collect those Linux configuration tips into 1 blog post.

Filesystem

• ext4 (or xfs), mount with noatime
• Scheduler – use deadline or noop

# echo deadline >/sys/block/sda/queue/scheduler
add "elevator=deadline" to grub.conf

(For more info see Linux Schedulers in TPCC like benchmark)

Memory

• Swappiness and NUMA:

# echo 0 > /proc/sys/vm/swappiness
add "vm.swappiness = 0" to /etc/sysctl.conf

• Set numa interleave all

numactl …

You may think that you already know what's the opposite of "DISABLED", but with MySQL Event Scheduler you'll be wrong.

In fact MySQL Event Scheduler may have three different states[1][2]:

DISABLED -  The Event Scheduler thread does not run [1]. In addition, the Event Scheduler state cannot be changed at runtime.
OFF (default) - The Event Scheduler thread does not run [1]. When the Event Scheduler is OFF it can be started by setting the value of event_scheduler to ON.
ON - The Event Scheduler is started; the event scheduler thread runs and executes all scheduled events.

So if you're going to find it in the DISABLED state and instinctively set it to ENABLED you'll end up with a non-starting MySQL daemon.
Be warned and stay safe out there!

[1]: http://dev.mysql.com/doc/refman/5.5/en/events-configuration.html
[2]: When the Event Scheduler is not running does not appear …

In general, I chose Google Cloud Storage to store web sites MySQL backups due to its price and speed of upload/download in real time

I used the Google native tool – gsutil , innobackupex and some bash

in short : the /etc and local MySQL  backup

#!/bin/sh
# Barcelona Tue Nov 22 17 16:30:36 CEST 2013

days_to_keep=3
NFS=/home/mysql.backups/
exportDate=`date +%Y-%m-%d.%H.%M.%S`
export_DIR=${NFS}/${HOSTNAME}.${exportDate}
test ! -d "${export_DIR}" && echo "$(date) : creating ${export_DIR}" && mkdir -p "${export_DIR}"
export_MySQL_DIR=${export_DIR}/mysql.bckp
export_ETC_DIR=${export_DIR}/etc.bckp
# backup the /etc directory
rsync -avh …

this is a short script for daily backup of the OpenStack MySQL and the /etc direcotry of the control node

[root@dev-epg-rhos-01 BACKUP]# cat /root/bin/epg.innobackupex.openstack.sh
#!/bin/sh
# done for epgmad4@tid.es
# Barcelona Thu Oct 17 16:30:36 CEST 2013

days_to_keep=7
NFS=/BACKUP
exportDate=`date +%Y-%m-%d.%H.%M.%S`
export_DIR=${NFS}/${HOSTNAME}.${exportDate}
test ! -d "${export_DIR}" && echo "$(date) : creating ${export_DIR}" && mkdir -p "${export_DIR}"
export_MySQL_DIR=${export_DIR}/mysql.bckp
export_ETC_DIR=${export_DIR}/etc.bckp
rsync -avh /etc ${export_ETC_DIR}
echo "=========================================================================================================" >> ${export_DIR}/README.restore.with.innobackupex
echo "HOW to restore this FULL mysql backup" >> ${export_DIR}/README.restore.with.innobackupex
echo "=========================================================================================================" >> …

MySQL is one of the most famous Open Source Relational Database Management System (RDBMS) created by Michael Widenius. Default choice in systems as WordPress and used by top companies as Google and Wikipedia. Due its simplicity and large community, it is largely used by developers and webmasters around the world. MySQL is under dual license: […]

Thanks to everyone who came. Definitely check out the recording below!

Heyo everybody!

Coming up this Wednesday, October 2nd, 2013 at 12:00pm pacific (19:00 GMT) is another exciting #DBHangOps. Join the google hangout to discuss:

• MySQL Utilities from Chuck Bell!
• How do you constrain over-consumption of resources by competing applications?
• Network vs. Local storage for MySQL
  • How do you use this for HA? Do you use DRBD or floating VIPs?
• Secure connections to MySQL (using SSL) (requested by Daniel)
  • Do you have experience with this? What’s the performance impact?
• PCI Certification with MySQL — what changed for you?
• MySQL-isms!
  • (From Gerry) Sub SELECTs — Why aren’t these as mature as other databases?
  …