Here are some quick and easy bash commands to solve every day problems I run into. Comment and leave some of your own if you like. I might update this post with new ones over time. These are just some common ones.

Iterate through directory listing and remove the file extension from each file
ls -1 | while read each; do new=`echo $each |sed 's/\(.*\)\..*/\1/'` && echo $new && mv "$each" "$new"; done

Output relevant process info, and nothing else
ps axo "user,pid,ppid,%cpu,%mem,tty,stime,state,command"| grep -v "grep" | grep $your-string-here

Setup a SOCKS5 proxy on localhost port 5050, to tunnel all traffic through a destination server
ssh -N -D 5050 username@destination_server'

Setup a SOCKS5 proxy via a remote TOR connection, using local port 5050 and remote TOR port 9050
ssh -L 5050:127.0.0.1:9050 username@destination_server'

Display text or code file contents to screen but don't display any # comment lines
sed -e '/^#/d' $1 < …

As you may expect from open source world thingy, almost every Linux distribution has developed it’s own way to manage our favourite RDBMS service. Yet none is perfect, or even some of them seems to not work in real server scenario1.

In this post I’m trying to compare and point out most annoying aspects of initialization scripts that I had to face in production.

In ‘old days’ probably all Linux distributions used to start and stop services using so called init scripts usually written in Unix shell (sh or Bash). But situation is not so simple these days anymore.

Folks started to think about improving things, like making system initialization faster by parallelization of starting services. So Upstart was developed in …

Do you get write laziness in the command line everything what you need to connect to a MySQL server every time?

It may take less than minute, but sometimes one minute is vital (especially if we’re near the end of the world):

ivancp@ubuntu$ mysql -u root -p -h mysqlhost database

When we are hurry, these commands often fail several times per minute.

The solution: we can create shortcuts with bash alias commands in file ~/.bashrc :

# File ~ /. Bashrc
 
# Command "my" to connect to a local server
alias my='mysql -u root -p'
 
# Command "my2" to connect to a remote server
alias my2='mysql -u root -h 192.168.1.56 -p'

Next time if you want to access the local server just type the command my [database name] , there only ask for database password. You can use any command aliases, I prefer  “my” and “my2” they are short and useful.

But if you have several servers comes …

#!/bin/sh
# get.database.sizes.sh
# by dragkh
# Wed, 02 Mar 2011 17:59:44 +0100
# it will work on linux boxes with working mysql 
ps axu \
| grep datadir \
| grep mysql \
| grep -v grep \
| grep var \
| sed 's/^.*--datadir=//; s/ .*$//'   \
| sort \
| uniq  \
| while read crap
do 
echo ""
sized=$(du -s $crap | awk '{print $1}')
avail=$(df | grep $(dirname $crap) | awk '{print $2}')
echo -e  "$crap\t$sized\t$avail" | awk '{printf ("%s:\tUsed: %6.2fG \tDISK: %6.2fG\t Usage: %6.2f%% \n",$1,$2/1024/1024,$3/1024/1024,($2/$3)*100)}'
find $crap/ -maxdepth  1 -type d ! -type l  ! -path "$crap/"   -printf '"%p"\n'  | xargs  --no-run-if-empty  du -s | sort -nk 1,9 | awk -vavail=$avail '{crapy="";for(i=2; i<=NF; i++) {crapy=crapy" "$i;};printf (" -- %s\tUsed: %6.2fG \tDISK:%6.2fG Usage:%6.2f%%\n",crapy,$1/1024/1024,avail/1024/1024,($1/avail)*100)}'
done

result looks like

root@xxxx:[Wed Mar 02 16:54:59]:
/var/lib/mysql/aaaa/datafiles:               Used:  26.43G …

I’ve always been a big fan of having a customized .bashrc file. The one I distribute to all of my servers has aliases for quick commands to save me time on the command line, functions that get work done when aliases are too simplistic, reporting for the server for each cli login, and of course a formatted and colored prompt (for terms that support colors). I also change certain aspects and commands based on the operating system since I’m not always on a redhat box or linux at all. Here’s my bashrc file – maybe you have some fun additions that you’d like to share. What saves you time on the command line?

I am currently working on a project to deploy new website builds to a
small number of servers. I needed something simple and reliable that could
be built in a very short period of time. I decided to whip something up in
bash with the intent of refining it in Python later.

As I began to write this code, I realized that it probably would have been
quicker to do it in Python from the start. I decided to stick with bash as
somewhat of an academic exercise. The vast majority of these deployment
scripts were trivial; check the code out of git, create a manifest, package
it up, spew it to the servers, etc, etc. The problem came during the last
step. We decided to use a symlink to point to the active build out of a
number of builds that could be available on the server at any given time.
Since all of our servers should be running the exact same version of the
build, it …

Here’s a simple answer to a simple question. “How do I run a backup of MySQL to another machine without writing to the local server’s filesystem?” – this is especially useful if you are running out of space on the local server and cannot write a temporary file to the filesystem during backups.

Method one – this writes a remote file.
mysqldump [options] [db_name|--all-databases]| gzip -c | ssh user@host.com "cat > /path/to/new/file.sql.gz"

Method two – this writes directly into a remote mysql server
mysqldump [options] [db_name|--all-databases]| mysql --host=[remote host] –user=root –password=[pass] [db_name]

If you’re using the new Arthemia theme for WordPress you might notice that there are two areas of the theme that can have articles promoted to; namely Headline and Featured sections. This is controlled by category association. Basically you have a post and if you want it in the Headline area of the theme you attach the category “headline” to it, similarly for the featured section. Now, let’s say you don’t want to manually change this all the time since it can be time consuming to promote posts to those categories if you want rotating content.

Here’s a simple solution. In this bash script I connect to MySQL and remove the current associations from posts and then randomly choose posts to be promoted to the Headline and Featured categories. This can be modified for other ideas you might have involving categories/posts/randomized …

If you own a Nokia N900 cellular device you might be interested in the ability to control all of your IM accounts from the command line. For those that do not know, the N900 runs Maemo Linux and is capable of running MySQL embedded if you so choose. Here’s a quick script I wrote to provide that functionality for IM accounts. It’s at the bottom of the page, called “im-connections”.

wiki: http://wiki.maemo.org/N900_Mission_Control#Set_all_SIP_accounts_to_online_or_offline
pastebin: http://pastebin.com/qAC57E1N

The primary responsibility of MySQL professionals is to establish and run proper backup and recovery plans. The most used method to backup a MySQL database is the mysqldump utility. This mysqldump utility creates a backup file for one or more MySQL databases that consists of DDL/DML statements needed to recreate the databases with their data. To [...]