I recently troubleshoot an interesting MySQL crash, and I think it is worth sharing (with the related bugs). MySQL crashed when the disk was full, you can see the free disk graph below. The Y-axis is in the tens of GiB scale and the X-axis is in the hour scale. Can you guess what happened ?
Just to make sure we agree on the meaning of the graph above, let's describe it
#!/bin/sh
# get.database.sizes.sh
# by dragkh
# Wed, 02 Mar 2011 17:59:44 +0100
# it will work on linux boxes with working mysql
ps axu \
| grep datadir \
| grep mysql \
| grep -v grep \
| grep var \
| sed 's/^.*--datadir=//; s/ .*$//' \
| sort \
| uniq \
| while read crap
do
echo ""
sized=$(du -s $crap | awk '{print $1}')
avail=$(df | grep $(dirname $crap) | awk '{print $2}')
echo -e "$crap\t$sized\t$avail" | awk '{printf ("%s:\tUsed: %6.2fG \tDISK: %6.2fG\t Usage: %6.2f%% \n",$1,$2/1024/1024,$3/1024/1024,($2/$3)*100)}'
find $crap/ -maxdepth 1 -type d ! -type l ! -path "$crap/" -printf '"%p"\n' | xargs --no-run-if-empty du -s | sort -nk 1,9 | awk -vavail=$avail '{crapy="";for(i=2; i<=NF; i++) {crapy=crapy" "$i;};printf (" -- %s\tUsed: %6.2fG \tDISK:%6.2fG Usage:%6.2f%%\n",crapy,$1/1024/1024,avail/1024/1024,($1/avail)*100)}'
done
result looks like
root@xxxx:[Wed Mar 02 16:54:59]: /var/lib/mysql/aaaa/datafiles: Used: 26.43G …[Read more]