well, I incremented max_allowed_packet from 16M to 512M
anyway, I got the same error on the next clean import.
so decided to find a workaround.
so, how to get the data separated from the ddl statements:

# get the tables names into the insert statement, its better to have that in file for future usage
sed '/^INSERT INTO/!d;s/ VALUES.*$//' jiradb.20130118.sql | sort | uniq > tablas.como.nombres.txt

then how to get the data separated:

root@jiragg:[Fri Jan 18 15:26:33]:[/usr/local/BACKUP]$ cat make.inserts.sh
#!/bin/sh
# trim function thank to http://stackoverflow.com/questions/369758/how-to-trim-whitespace-from-bash-variable
# and http://codesnippets.joyent.com/posts/show/1816
trim() {
    local var=$1
    var="${var#"${var%%[![:space:]]*}"}"   # remove leading whitespace characters
    var="${var%"${var##*[![:space:]]}"}"   # remove trailing whitespace characters
    echo -n "$var"
}

while read tabname
do …

We assume the new disk is /dev/sdb1 formatted as ext3
and it will be mounted as /data

# 0. make sure there is no mysqlm mysql data directory :
yum remove mysql mysql-server -y 
test -d /data/mysql/ && rm -rf /data/mysql/
test -d /var/lib/mysql/ && rm -rf /var/lib/mysql/

# 1. install Mysql
yum install mysql mysql-server -y

# 2. check the mysql  status
service mysqld status

# 3. start the mysqld if not started
service mysqld start

# 4. check the mysql status again
service mysqld status

# 5. stop mysqld in case its started, and check thre is n mysql process:
service mysqld stop
ps axu | grep mysql

# 6. make sure the /data partition is added to the /etc/fstab. If not add it:
test  `cat  /etc/fstab | grep /data | wc -l ` -eq 0  && echo "/dev/sdb1 /data ext3  defaults 1 1" >>  /etc/fstab

# 7. make sure the /data partition is mounted, 
test  `cat /proc/mounts  | grep /data | grep -v grep | wc -l` -eq 0 && mount /data

# 8. …

1. install Cmake

wget  http://www.cmake.org/files/v2.8/cmake-2.8.10.1.tar.gz
tar xvfz cmake-2.8.10.1.tar.gz
cd cmake-2.8.10.1/
./configure
gmake -j3


2. install libaio

gmake -wget  http://www.kernel.org/pub/linux/kernel/people/bcrl/aio/libaio-0.3.92.tar.gz
tar xvfz libaio-0.3.92.tar.gz
cd libaio-0.3.92/
make prefix=/usr/
make prefix=/usr/ installj3 install

3. install MariaDB 5.5.28
go to https://downloads.mariadb.org/mariadb/5.5.28/ OR

wget https://downloads.mariadb.org/f/mariadb-5.5.28/kvm-tarbake-jaunty-x86/mariadb-5.5.28.tar.gz/from/http:/mariadb.ulak.net.tr/
tar xvf mariadb-5.5.28.tar.gz
cd mariadb-5.5.28/

the build will fail :

root@bubu:[Tue Nov 27 10:15:45]:[/opt/installs/mariadb-5.5.28]$ ./BUILD/compile-pentium-max
testing pentium3 ... ok
+++ /bin/rm -rf configure
+++ /bin/rm -rf CMakeCache.txt …

I write a fair number of scripts on this site and have posted a lot of code over the years. Generally if I am not pasting the code to be viewed on the webpage then I link to a file that a user can download; which leads to a lot of mish-mash code that doesn’t have a home. I’ve always kept the code files in a private SVN repo over the years but have recently moved them all to BitBucket Git repositories. So here they are: lots of code samples and useful bits of programming to save time.

Generic Shell Scripts: https://bitbucket.org/themattreid/generic-bash-scripts/src
Generic Python Scripts: https://bitbucket.org/themattreid/generic-python-scripts/src
Generic MySQL Stored Procs: …

The other day I needed to run a simple mysql job to backup and delete some database records on a live server. Being a live server, it is important to make sure you aren't asking the database to take on jobs that could potentially lock it up. Better to run a batch job. Running a batch is simple. You can call it right from the mysql console with:

source [path_to]/[the_batch_script].sql

But what if there are millions of records that need deleting? Bash shell script to the rescue.

Here is the idea of the SQL job that needed to get run a few times:

START TRANSACTION;

/* Find what you want to delete and put a LIMIT on your batch size */
CREATE TEMPORARY TABLE records_to_delete_temp SELECT id from `records` where ..... limit 1000;

/* Creating backup table to archive spam orders */
CREATE TABLE IF NOT EXISTS `records_backup` LIKE `records`;
INSERT INTO `records_backup` SELECT * from `records` where id in (select …

Converting from uw-mailboxes (mbx) to Unix format (dovecot)

It took me by surprise how the mailbox formats had changed, when I switched to Dovecot which is the best IMAP/POP3 mail program, in my opinion. It handles large (enormous) mailboxes with many (hundreds of) users. Caching makes things go fast again.

Here is a simple bash utility, to take all users, and convert all mailboxes & folders.

cd /home
for u in *
do
if [ -d /home/$u/Mail ]; then
echo "User: $u"
cd /home/$u/Mail

read more

I read somewhere that you should do extra watering of lawn (after sunset), if the temperature exceeded 80° F during the day.

So, a really quick hack works like this:

A bash script
#!/bin/bash

TEMP="`/usr/bin/mysql -ss open2300 -e \"SELECT FLOOR(MAX(temp_out)) FROM weather GROUP BY (rec_date) ORDER BY rec_date DESC LIMIT 1;\"`"
if [ "$TEMP" -ge 80 ]; then
/root/water-on.exp
/root/water2.exp 120 1
/root/water3.exp 120 1
fi

read more

Since I have been discussing scripting lately I thought I would continue with another topic I touched on briefly - backups.

I have written and modified the following script over the last few years. I have used it (and continue to use it) with multiple clients. It uses Percona's Xtrabackup to take the backup (although it can be easily modified to use mysqldump instead).

First the script

-----------------------------------------------------------------------------------------------------------------

#!/bin/bash
SAVEIFS=$IFS
IFS=$""
day_of_week=`date +%a`
backup_dir=/mysql-backup/
logfile=/root/backup_log.txt
report=/tmp/report.txt
servername=slave1
email=bmurphy@paragon-cs.com
password=`cat /root/.ssh/.backup_password`

# run backup
echo ' ' > $report
echo 'The backup is now beginning:' >> $report
echo ' ' …

I've been involved in a number of interviews over the last few weeks as a client has been loking for a MySQL DBA. When you are looking for position as a DBA in a large scale environement there are some very important things you have to know.

You absolutely must know a scripting language. In a smaller environment this often isn't necessary. You will live and die by this in a large environment. I asked every applicant one specific question..if you had to change a mysql server variable on a pool of 100 mysql servers how would you do this? It's easy when it's one,two or even a dozen servers. just log in, change the my.cnf and change it "on the fly" if you can. Restart mysql if you can't.

You going to do that to 100 servers? It will take all day and be prone to failures. Scripting is the key here. Even just bash shell scripting can be very powerful. In another post I will cover a simple bash script to loop through a …

MongoDB Backup types and strategies are neatly explained in its documentation, which you can check here. In case you are not familiar with MongoDB backup types and strategies, please have a look at its documentation.

What I am describing here is a simple script which we are using since months to take MongoDB backup and transfer it over to our Backup server. Here are few things its doing:

• As we have multiple MongoDB Replica Sets, the script identify current replica set and check whether current server is Master or Slave, exit if its Master. We take backup only from Slave host.
• Take Backup using mongodump command.
• Upon successful completion of dump, transfer that to our Backup server. Ensure that ssh key based authentication is setup between both servers to implement seamless and secure transfer. It creates new directory based on …