What is Ansible?

Ansible is a configuration management and deployment system, like Puppet, Capistrano, Fabric, and Chef. Its aim is to be radically simple and let you use your existing scripts to help with cluster configuration and software deployment whenever possible. Here are the ways that Ansible differentiates itself.

Simplicity

Ansible does not include a client/server architecture with pull-based clients (although in more recent versions, it does include pull-based configuration and deployment). Rather, it uses pre-existing network infrastructure: SSH. Every company has SSH installed on their cluster servers, and Ansible simply rides on top of this infrastructure to get the code and configuration out to the nodes.

Language

  [Read more...]

Amazon announced high I/O instances today. This is huge for anyone with a database larger than available memory, as it’s been a complete nightmare dealing with EBS up till now. Now your Cassandra, MongoDB, MySQL, or whatever your using should be able to perform well without requiring keeping your entire dataset in memory.

With each instance you get 2x1TB of disk. In this tutorial I’ll be setting it up as a RAID0 to get a single 2TB disk which should deliver excellent performance.

Before you get started, make sure you’ve got mdadm installed:

apt-get install mdadm

To begin, check fdisk and make sure your 1TB drives are mounted.

root@ip-10-140-128-232:~# fdisk -l

Disk /dev/xvda1: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044

  [Read more...]

MongoDB Backup types and strategies are neatly explained in its documentation, which you can check here. In case you are not familiar with MongoDB backup types and strategies, please have a look at its documentation.

What I am describing here is a simple script which we are using since months to take MongoDB backup and transfer it over to our Backup server. Here are few things its doing:

• As we have multiple MongoDB Replica Sets, the script identify current replica set and check whether current server is Master or Slave, exit if its Master. We take backup only from Slave host.
• Take Backup using mongodump command.
• Upon successful completion of dump, transfer that to our Backup server. Ensure that ssh key based authentication is setup between both servers to implement seamless and

  [Read more...]

As you might have noticed, I'm getting further away from MySQL here. This is just how things are I guess, I just do much less work with MySQL these days. The first migration was from MySQL to MongoDB, which was some time back. This was pretty successful, but note that we still have some data in MySQL, but the bulk of the data is in MongoDB right now.

Running any database on Amazon (and we run all databases on Amazon or on Amazon RDS service) may be costly, depending on how you utilize your resources. The recently announced Amazon DynamoDB is Amazons NoSQL service offering, but it is not like MongoDB with a twist, far from it. If you have read what I have written about MongoDB, I have now and then complained about the lack of functionality, but to be honest, I have learnt to live with MongoDB and it's shortcomings, and have started to like many of the

  [Read more...]

Read the original article at A handy guide for PHP and MongoDB Web Development

What makes a beginner’s guide handy is when it speaks to your intuition. It anticipates the burning questions that follow from a newbie trying to grasp new concepts and it quickly answers them. PHP and MongoDB Web Development – Beginner’s Guide is one such guide.

I hadn’t heard of Packt Publishing or Rubayeet Islam before picking up this title and I must say I’m impressed. Based in Birmingham, with offices in Mumbai, part of Packt’s

  [Read more...]

It is my pleasure to announce that I will be presenting on Tungsten Replicator next Wednesday, January 11th at the Los Angeles MySQL Meetup. The presentation title is Fast, Flexible, and Fun--The Tungsten Replicator Magical Mystery Tour. This talk is going to be fun for two reasons.

First, it's a great opportunity to meet people in the LA MySQL community and talk about my favorite replication software. Tungsten is like a Swiss Army Knife for data replication.  It solves a wide range of problems involving HA, scaling, and data movement.   The presentation gives a quick intro to the replicator, then surveys how to use the most interesting features, including parallel slave

  [Read more...]

First of all, I wish you a happy new year.
Many things happened last year, it was really exciting to be involved in the MySQL ecosystem.
I hope this enthusiasm will be increased this year, up to you !

To start the year, I propose the MySQL[plus] Awards 2011
It will only take 5 minutes to fill out these polls.
Answer with your heart first and then with your experience with some of these tools or services.

Polls will be closed January 31, so, vote now !
For “other” answers, please,  let me a comment with details.

Don’t hesitate to submit proposal for tools or services in the comments.

  [Read more...]

Here are some of the fake O'Reilly book covers I mentioned in a prior post.  These have been optimized for use as black & white Kindle screensaver wallpaper images.  If you haven't done so already, you can install a Kindle screensaver hack with a couple of downloads. 

Update: I've embedded a slideshow from PicasaWeb, but it requires Flash.  If you don't see it you can click on the links below to go directly to

  [Read more...]

We have seen the "warning" against using MongoDB a few times now, and I have to say that this reminds me of other such warnings:

• 640 K is enough
  
• Relational databases are too slow (John Cullinane) (see for example: http://www.cbronline.com/news/ellison_makes_pre_emptive_strike_to_save_oracle_from_fate_of_cullinet)
• Linux is for nerds only.
• Virtual computing is waaay too slow.
• There is no reason anyone would want a computer in their home. (Ken Olsen, DEC)
• Who the h*ll wants to hear actors talk?. Warner

In a sense, most of them were right. If you had, in the 1920's, asked the movie going public if they wanted "talkies", chances are most of them would have said no. If you had told my mom and

  [Read more...]

Last time I wrote about MongoDB for MySQL DBAs I described some of the basics of MongoDB querying, and this time I'll follow that up with some more on querying.

As we saw last time, the basic format of a MongoDB query is:
db.find(<query>,<attributes>)
Note that you do NOT replace db with the name of the database you want to query here, you just make the database you want to use the current one and issue the query, such as:
> use test
> db.mycoll.find()
The example above will find all objects in the mycoll collection, and will include all the object attributes and also the key (_id), like this:
{ "_id" : ObjectId("4eb0634807b16556bf46b214"), "c1" : 1 }
{ "_id" : ObjectId("4eb0634a07b16556bf46b215"),








  [Read more...]

Last time I wrote about MongoDB for MySQL DBAs I described some of the basics of MongoDB querying, and this time I'll follow that up with some more on querying.

As we saw last time, the basic format of a MongoDB query is:
db.find(,)
Note that you do NOT replace db with the name of the database you want to query here, you just make the database you want to use the current one and issue the query, such as:
> use test
> db.mycoll.find()
The example above will find all objects in the mycoll collection, and will include all the object attributes and also the key (_id), like this:
{ "_id" : ObjectId("4eb0634807b16556bf46b214"), "c1" : 1 }
{ "_id" : ObjectId("4eb0634a07b16556bf46b215"), "c2" : 1 }
{ "_id" :









  [Read more...]

In this second part of a series of blogs on using a NoSQL database, in this case MongoDB, aimed at MySQL users, I will describe querying a bit. And it's probably not what you expected if you haven't tried a NoSQL database or MongoDB more specifically, before. The first blogpost on this subject is here.

At the heart of accessing MongoDB is JavaScript, more specifically it uses the Mozilla SpiderMonkey JavaScript engine. As I wrote in the previous blog on this subject, JavScript is all over the place in MongoB, largely you can look at MonngoDB as JavaScript with HUGE space for variables, and although many would look at this as gross oversimplification, it works

  [Read more...]

As you probably know, I have been doing RDBMS work for many years, some 25+ years by now. At Recorded Future I am the database architect, and although an RDBMS is used extensively, MySQL in this case, we are looking at options, and are currently doing more and more work using a NoSQL Solution, probably te most popular one by now, namely MongoDB.

And before you complan: NoSQL is not a good term, but someone with a longer NoSQL background should then find something better, not yours truly. And for all intents and purposes, you know what I am talking about, right?

I plan to post a few MongoDB posts here, how it looks like from an RDBMS dudes (like myself) POV. This is the first installment, but there will be more. I should also point out that I am no expert in NoSQL technologies in general, nor specifically in MongoDB, but I am trying, and our



  [Read more...]

Proof-of-concept Tungsten support for MongoDB arrived last May, when I posted about our hackathon effort to replicate from MySQL to MongoDB.  That code then lay fallow for a few months while we worked on other things like parallel replication, but the period of idleness has ended.  Earlier this week I checked in fixes to Tungsten Replicator to add one-line installation support for MongoDB slaves.

MySQL to MongoDB replication will be officially supported in the Tungsten Replicator 2.0.5 build, which will be available in a few weeks.  However, you can try out MySQL to

  [Read more...]

A special extended edition of Tech Messages for 2011-08-31 through 2011-09-21:

• Lab Overview
  This guy is serious about his lab!
• Throughput
  All sorts of useful information here. My favorite is the first table which includes bandwidth and packet throughput for different kids of circuits.
• Must see: 5 Steps to Scaling MongoDB (Or Any DB) in 8 Minutes
  These tips are relevant no matter what database system you are using.

  [Read more...]

As Giuseppe Maxia recently posted we released Tungsten Replicator 2.0.4 this week.  It has a raft of bug fixes and new features of which one-line installations are the single biggest improvement.  I set up replicators dozens of times a day and having a single command for standard cluster topologies is a huge step forward.  Kudos to Jeff Mace for getting this nailed down.

So what's next?  You can get see what we are up to in general by looking at our issues list.  We cannot do everything at once, but here are the current priorities for Tungsten Replicator 2.0.5.

• Parallel replication speed

  [Read more...]

Backing up databases has never been fun, not as fun as having a cool English Ale on the balcony on a hot summar day anyway, but MongoDB takes this one step further when it comes to annoyances.

In general, I often feel that many Open Source projects start with good intentions for what the project should do and how, but then more stuff is added as you require it, and suddenly what started out as a simple and fast application for a narrow usecase, suddenly turns into a bit of a mess. And the issue might well be that building fast, compact software for a specialized usecase, as they start out, is not the same as writing generic software, with a wide range of use cases, code that can easily be maintained and enhanced as we go along. And why should it not be like that? In many cases, this is just fine and the limited usecase is just what the project sets

  [Read more...]

I found myself reading NoSQL is a Premature Optimization a few minutes ago and threw up in my mouth a little. That article is so far off base that I’m not even sure where to start, so I guess I’ll go in order.

In fact, I would argue that starting with NoSQL because you think you might someday have enough traffic and scale to warrant it is a premature optimization, and as such, should be avoided by smaller and even medium sized organizations.  You will have plenty of time to switch to NoSQL as and if it becomes helpful.  Until that time, NoSQL is an expensive distraction you don’t need.

Uhm… WHAT?!

I’ve spent more than a few years using MySQL and have been using some NoSQL systems for the last year or so in a fairly busy environment. And

  [Read more...]

“Our experience from PNUTS also tells that these systems are hard to build: performance, but also scaleout, elasticity, failure handling, replication. You can’t afford to take any of these for granted when choosing a system. We wanted to find a way to call these out.” – Adam Silberstein and Raghu Ramakrishnan, Yahoo! Research. ___________________________________ A [...]

My Colleague Linas Virbalas has just crossed the boundary between real and virtual and has started a blog, titled Flying Clusters.
Linas is a gifted developer who is taking care of the special projects. One of such projects is replication between MySQL and PostgreSQL, which works quite well.Another project, which has just started, is about providing PostgreSQL with Advanced Logical Replication using Tungsten replicator. As you probably know, recent versions of PostgreSQL can do physical
  [Read more...]

The Open Database Camp 2011 is near. In 9 days, the welcome party starts, and then the conference itself gets going.
If you are coming earlier than Friday, May 6th, you can either use public transportation or book a private seat with a volunteer in the car pooling page. Please help the organizers: post your arrival and departure dates and times, so we may be able to help you even outside the official conference days.
About the conference itself, as everyone should know, it's a

  [Read more...]