Showing entries 1 to 7
Displaying posts with tag: awk (reset)
On Importing InnoDB Tablespaces and Row Formats

Let me start with a short summary and then proceed with a long story, code snippets, hexdumps, links and awk functions converted from the source code of MariaDB server. This blog post can be summarized as follows:

  • One can find row_format used to create table explicitly in the .frm file (or the outputs of SHOW CREATE TABLE or SHOW TABLE STATUS). Internals manual may help to find out where is it stored and source code reading helps to find the way to interpret the values.
  • For InnoDB tables created without specifying the row_format explicitly neither logical backup nor .frm file itself contains the information about the row format used. There are 4 of them (Redundant, Compact, Dynamic and Compressed). The one used implicitly is defined by current value of the …
[Read more]
Useful sed / awk liners for MySQL

Listing some useful sed / awk liners to use with MySQL. I use these on occasion.

sed, awk & grep have many overlapping features. Some simple tasks can be performed by either. For example, stripping empty lines can be performed by either:

grep '.'
awk '/./'
sed '/./!d'
grep -v '^$'
awk '!/^$/'
sed '/^$/d'

It's a matter of taste & convention which tool and variation to use. So for any script I suggest, there may be many variations, possibly cleaner, shorter; feel free to comment.


The output of mysqldump is in particular useful when one wishes to make transformation on data or metadata.

  • Convert MyISAM tables to InnoDB:
mysqldump | sed -e 's/^) ENGINE=MyISAM/) …
[Read more]
Linux bash: check the MySQL database disk usage
# by dragkh
# Wed, 02 Mar 2011 17:59:44 +0100
# it will work on linux boxes with working mysql 
ps axu \
| grep datadir \
| grep mysql \
| grep -v grep \
| grep var \
| sed 's/^.*--datadir=//; s/ .*$//'   \
| sort \
| uniq  \
| while read crap
echo ""
sized=$(du -s $crap | awk '{print $1}')
avail=$(df | grep $(dirname $crap) | awk '{print $2}')
echo -e  "$crap\t$sized\t$avail" | awk '{printf ("%s:\tUsed: %6.2fG \tDISK: %6.2fG\t Usage: %6.2f%% \n",$1,$2/1024/1024,$3/1024/1024,($2/$3)*100)}'
find $crap/ -maxdepth  1 -type d ! -type l  ! -path "$crap/"   -printf '"%p"\n'  | xargs  --no-run-if-empty  du -s | sort -nk 1,9 | awk -vavail=$avail '{crapy="";for(i=2; i<=NF; i++) {crapy=crapy" "$i;};printf (" -- %s\tUsed: %6.2fG \tDISK:%6.2fG Usage:%6.2f%%\n",crapy,$1/1024/1024,avail/1024/1024,($1/avail)*100)}'

result looks like

root@xxxx:[Wed Mar 02 16:54:59]:
/var/lib/mysql/aaaa/datafiles:               Used:  26.43G …
[Read more]
Poor Man's Profiler using Solaris' pstack

Recently I was working with the output of pstack from a hung MySQL server and wanted to use Poor Man's Profiler in order to combine stack traces. Unfortunately, the awk magic expects the output from gdb's thread apply all bt output.

gdb output:

Thread 10 (Thread 0xa644db90 (LWP 26275)):
#0 0xb7f47410 in __kernel_vsyscall ()
#1 0xb7f33b1a in do_sigwait () from /lib/tls/i686/cmov/
#2 0xb7f33bbf in sigwait () from /lib/tls/i686/cmov/
#3 0x081cc4fc in signal_hand ()
#4 0xb7f2b4fb in start_thread () from /lib/tls/i686/cmov/
#5 0xb7d25e5e in clone () from /lib/tls/i686/cmov/

Thread 9 (Thread 0xa641cb90 (LWP 26273)):
#0 0xb7f47410 in __kernel_vsyscall ()
#1 0xb7d1e881 in select …
[Read more]
Liveblogging: Senior Skills: Grok awk

[author's note: personally, I use awk a bunch in MySQL DBA work, for tasks like scrubbing data from a production export for use in qa/dev, but usually have to resort to Perl for really complex stuff, but now I know how to do .]

By default, fields are separated by any number of spaces. The -F option to awk changes the separator on commandline.
Print the first field, fields are separated by a colon.
awk -F: '{print $1}' /etc/passwd

Print the first and fifth field:
awk -F: '{$print $1,$5}' /etc/passwd

Can pattern match and use files, so you can replace:
grep foo /etc/passwd | awk -F: '{print $1,$5}'
awk -F: '/foo/ {print $1,$5}' /etc/passwd

NF = built in variable (no $) used to mean “field number”
This will print the first and last fields of lines where the first …

[Read more]
mysqlbinlog --server-id before MySQL 5.1? awk to the rescue!

Recently I had an interesting issue crop up. Due to an unfortunate migration incident in which involved master/master replication and not checking to see if replication was caught up, we ended up with an infinite replication loop of a number of SQL statements. awk helped immensely in the aftermath cleanup.

The basics of the replication infinite loop were (more…)

Observing the MySQL Query Log

Debugging an existing application can be hard to bootstrap. Sometimes it just helps to observe the queries a web application is sending to the database. Unfortunately, the MySQL Query log does not directly tell the user which query goes to which database.
Continue reading "Observing the MySQL Query Log"

Showing entries 1 to 7