Today I will describe the setup of a Hadoop / HDSF multi-node cluster on Debian Lenny with a redundant Namenode using DRBD and Heartbeat, four Datanodes and Tasktracker, a Backup- Checkpointnode and Rack awareness.

Hadoop Cluster Setup on Debian Lenny purposes

This article descibes how to setup a hadoop (version 0.21.0) cluster on debian lenny (version 5.x). I will not describe how to use MapReduce.

general

Hadoop is a framework for distributed computing written in Java. The project includs the following subprojects:

• HDFS: A distributed file system
• MapReduce: A framework for distributed large data processing

list of references

• http://en.wikipedia.org/wiki/Hadoop
• http://hadoop.apache.org/
• …

When I blogged about microblogging using Twitter a while ago, I got some suggestions regarding Identi.ca in the comments. I did not know about Identi.ca at that time, but decided to give it a try. Identi.ca is a service like Twitter based on StatusNet, an open source microblogging platform. After using both services in parallel for a while, with constant frustration about the unreliable Identi.ca -> Twitter gateway, I now finally decided to completely drop my account on Twitter.

Unfortunately something got miserably wrong while playing with the Identi.ca -> Twitter gateway last week, I somehow lost all my subscribers and subscriptions, so please …

How To Set Up MySQL Database Replication With SSL Encryption On Debian Lenny

This tutorial describes how to set up database replication in MySQL using an SSL connection for encryption (to make it impossible for hackers to sniff out passwords and data transferred between the master and slave). MySQL replication allows you to have an exact copy of a database from a master server on another server (slave), and all updates to the database on the master server are immediately replicated to the database on the slave server so that both databases are in sync. This is not a backup policy because an accidentally issued DELETE command will also be carried out on the slave; but replication can help protect against hardware failures though.

• Tweet

Installing A Web, Email And MySQL Database Cluster (Mirror) On Debian 5.0 With ISPConfig 3

This tutorial describes the installation of a clustered Web, Email, Database and DNS server to be used for redundancy, high availability and load balancing on Debian 5 with the ISPConfig 3 control panel. GlusterFS will be used to mirror the data between the servers and ISPConfig for mirroring the configuration files. I will use a setup of two servers here for demonstration purposes but the setup can scale to a higher number of servers with only minor modifications in the GlusterFS configuration files.

A couple of new webinar recordings are available from our web site:

• Upgrading to Pacemaker on Debian squeeze tells you all there is to know about upgrading to the Pacemaker cluster manager on the upcoming Debian release, “squeeze”.
• High Availability for Oracle databases shows you how to leverage the Linux cluster stack to deploy highly available Oracle databases.

Like our live webinars, the recordings are of course free of charge. Enjoy!

…

Apache And MySQL Monitoring With Bijk On Debian Lenny

This tutorial describes how you can monitor your server with the tool Bijk. Bijk creates online 30 graphs about load, CPU, memory, traffic, Apache, NginX, PostreSQL and others with alerts. Bijk can be used on Debian, Ubuntu, CentOS, RedHat and Gentoo. In this article I will explain how to install Bijk on Debian.

For those of you who haven’t yet registered, this is our reminder for today’s Clustering in Debian webinar at 1400 UTC. If you’re planning to run Pacemaker on the upcoming Debian squeeze release, don’t miss this!

…

Martin Loschwitz, the longest-serving Debian Developer in our ranks, will present a walk-through of going from Linux clustering on Debian lenny (with Heartbeat 2.1.3, shudder) to squeeze (with Heartbeat 3 and Pacemaker). In this webinar you will learn everything you need to know about a painless migration to the new Linux cluster stack.

This webinar will be held on Monday, June 7, at 1400 UTC. You must provide a valid email address to receive a meeting key. Register below!

The widget …

Sphinx As MySQL Storage Engine (SphinxSE)

SphinX is a great full-text search engine for MySQL. Installing the Sphinx daemon was straightforward as you can compile it from the source or use a .DEB/.RPM package but SphinxSE was a little bit tricky since it needed to be installed as a plugin on a running MySQL server. So if you use Debian or Centos and install your MySQL from a .deb or .rpm package this is how you do it.

The full Linux cluster stack, including Pacemaker, Heartbeat, Corosync and related packages, cleared all hurdles and migrated to the Debian testing repository this morning. This means that Pacemaker and both messaging layers it supports will be in the upcoming Debian release, codenamed squeeze.

Credit goes to the debian-ha maintainers crowd for making this possible: Martin Loschwitz, …