As any server farm scales out, it becomes increasingly difficult to Watch All The Things™. I’ve been watching the progress of LogStash+ElasticSearch+Kibana (also known as an ELK stack) for a while and gave it a go this weekend. The trick for me was wanting to run each element inside of a separate Docker container so that I have easily portable elements to scale out with.

A step back. What is Docker? Docker is a container (using LXC) around an application. In short, you install Docker, start a container using a base image (CentOS, Ubuntu, etc.) and then run the container, dropping you into a shell. From here, you configure your application, then save your container. You …

Over the past couple of years, several tools have come up to solve some of the long standing problems for developers, DBAs and operations teams. This list is no where near exhaustive and your mileage may vary.

These tools should let you get up and coding, saving your code to a central place.

I run (Mac/Windows), I need a Linux server to learn some of this cool stuff, right?

Not exactly. VirtualBox is a virtualization project for multiple platforms (Mac, Win, Linux, Solaris) that allows you to run a guest OS right from your laptop or desktop. There are repositories of images (alternative here) that you can use to get yourself going quickly, or you can provision your own.

I set up (cool open source package here) and now my team …

http://webscalesql.org/

yeah , big buzz around that one

So I decided to check the install process:

1. Clone the repo from

root@webscalesql-5.6.clean:[Mon Mar 31 11:37:11][~]$ cd /opt/
root@webscalesql-5.6.clean:[Mon Mar 31 11:37:15][/opt]$ mkdir installs
root@webscalesql-5.6.clean:[Mon Mar 31 11:37:17][/opt]$ cd installs/
root@webscalesql-5.6.clean:[Mon Mar 31 11:37:19][/opt/installs]$ git clone https://github.com/webscalesql/webscalesql-5.6.git
Initialized empty Git repository in /opt/installs/webscalesql-5.6/.git/
remote: Counting objects: 30397, done.
remote: Compressing objects: 100% (12678/12678), done.
remote: Total 30397 (delta 18716), reused 27620 (delta 16936)
Receiving objects: 100% (30397/30397), 47.99 MiB | 460 KiB/s, done.
Resolving deltas: 100% (18716/18716), done.

2. …

I needed to test the new server audit plug-in on latest MariaDB 10.0.9
More info about that handy plug in you may find here : http://www.skysql.com/downloads/mariadb-audit-plugin
Take a moment to read the info, along with that nice blog : http://www.skysql.com/blogs/ralf-gebhardt/activating-auditing-mariadb-and-mysql-5-minutes

In short:

root@mariadb-10.0.9:[Mon Mar 24 07:51:54][/tmp/]$ lftp http://ftp.igh.cnrs.fr/pub/mariadb/mariadb-10.0.9/yum/
cd ok, cwd=/pub/mariadb/mariadb-10.0.9/yum                                   
lftp ftp.igh.cnrs.fr:/pub/mariadb/mariadb-10.0.9/yum> bookmark add mariadb-10.0.9
lftp ftp.igh.cnrs.fr:/pub/mariadb/mariadb-10.0.9/yum> ls
drwxr-xr-x  --  ..                   
drwxr-xr-x            - …

While installing mysql-workbench-community-6.0.9 on Ubuntu 13.04 if you get below error.

root@Dell-XPS:/home/nilnandan/Downloads# dpkg -i mysql-workbench-community-6.0.9-1ubu1204-amd64.deb
Selecting previously unselected package mysql-workbench-community.
(Reading database … 203112 files and directories currently installed.)
Unpacking mysql-workbench-community (from mysql-workbench-community-6.0.9-1ubu1204-amd64.deb) …
dpkg: dependency problems prevent configuration of mysql-workbench-community:
 mysql-workbench-community depends on libctemplate0; however:
  Package libctemplate0 is not installed.
 mysql-workbench-community depends on libgtkmm-2.4-1c2a (>= 1:2.24.0); however:
  Package libgtkmm-2.4-1c2a is not installed.
…

Percona Toolkit has a great tool, pt-query-digest, that can use tcpdump data. Capturing raw tcp data can be taxing on a server, however, when you see the following message:

64000 packets received by filter
12000 packets dropped by kernel

When there is a significant amount of user cpu% being used, the kernel will drop packets you are trying to capture, leading to a partial picture and missing data. I’ve found that if you write it using the native tcpdump format, it’s more efficient and you drop less. There are also recommendations on Stack Overflow on how to help prevent this.

To perform a capture for a specific length of time, here’s the trick I …

Continuent, a leading provider of database clustering and replication software has five (5) new positions open:

Build/Test Engineer Senior Database Availability and Clustering Engineer Senior Database Replication Engineer Data Replication Sales Engineer Clustering and Replication Test Development Engineer

If you want to get in on the ground floor of a growing company in a challenging field

So, back in late 2008, rather soon after we got to start working on Drizzle full time, someone discovered unicodesnowmanforyou.com, or:

☃

Since we had decided that Drizzle was going to be UTF-8 everywhere,(after seeing for years how hard it was for people to get character sets correct in MySQL) we soon added ☃.test to the tree, which tried a few interesting things:

CREATE TABLE ☃; CREATE DATABASE ☃; etc etc

Because what better to show off UTF-8 than using odd Unicode characters for table names, database names and file names. Well… it turns out we were all good except if you attempted to check out the source tree on Solaris. It was some combination of Python, Bazaar and Solaris that meant you just got python stacktraces and no source tree. So, if you look now it’s actually …

 Many people think buffered write (write()/pwrite()) is fast because it does not do disk access. But this is not always true. Buffered write sometimes does disk access by itself, or waits for some disk accesses by other threads. Here are three common cases where write() takes longer time (== causing stalls).
1. Read Modify Write Suppose the following logic. Opening aaa.dat without O_DIRECT/O_SYNC, writing 1000 bytes sequentially for 100,000 times, then flushing by fsync().

  fd=open("aaa.dat", O_WRONLY);
  for(i=0; i< 100000; i++) {
    write(fd, buf, 1000);
  }
  fsync(fd);

 You might think each write() will finish fast enough (at least less than 0.1ms) because it shouldn't do any disk access. But it is not always true.
 Operating System manages I/O by page. It's 4KB for most Linux environments. If you'd modify 1000 bytes of the 4KB page from offset 0, Linux first needs to read the 4KB …

Over on the codecentric blog I published an article on SSH Two-Factor Authentication with Duo Security. You are welcome to check it out.