This is a follow up post, describing the implementation details of Hadoop Applier, and steps to configure and install it. Hadoop Applier integrates MySQL with Hadoop providing the real-time replication of INSERTs to HDFS, and hence can be consumed by the data stores working on top of Hadoop. You can know more about the design rationale and per-requisites in the previous post.

Design and Implementation:

Hadoop Applier replicates rows inserted into a table in MySQL to the Hadoop Distributed File System(HDFS). It uses an API provided by libhdfs, a C library to manipulate files in HDFS.

The library comes pre-compiled with Hadoop distributions. It connects to the MySQL master (or read …

Enabling Real-Time MySQL to HDFS Integration

Batch processing delivered by Map/Reduce remains central to Apache Hadoop, but as the pressure to gain competitive advantage from “speed of thought” analytics grows, so Hadoop itself is undergoing significant evolution. The development of technologies allowing real time queries, such as Apache Drill, Cloudera Impala and the Stinger Initiative are emerging, supported by new generations of resource management with Apache YARN

To support this growing emphasis on real-time operations, we are releasing a new …

MySQL Conference & Expo

2013

LIVE BLOG!

This year again, the MySQL Conference and Expo, hosted by Percona, smells like a wonderful event.
I haven’t the chance to attend this event but I would like to share with you the soul of this one.
I’ll try to share latest news, tweets and slides from the event in this live post.

Stay tuned and come back!

Breaking news!

• LATEST : See you next year!
• MySQL 5.7 is coming… : http://bit.ly/11zENgT
• SkySQL Merges With MariaDB : http://bit.ly/15DRtHL
• Wikipedia adopts MariaDB :  …

We discuss INFORMATION_SCHEMA with plenty of examples on useful queries. In this week's Ear Candy, we talk about resident versus virtual memory on Linux, and in At the Movies has a video about MongoDB.

Events
DB Hangops in May will be Wed May 8th and 22nd noon pacific time.
Upcoming MySQL tech tours

Training
SkySQL Trainings
Tungsten University trainings

read more

Good morning Percona Live visitors! Attached to this post you can find a spreadsheet (both LibreOffice or Excel, as you prefer) that you can use towards the end of my tutorial. I've also attached the slides so you can download a copy of them.

Every few months, I get the fun job of announcing what’s new in TokuDB®, but this time is special. With Version 7, TokuDB for MySQL and MariaDB is going open source.

The free Community Edition is fully functional and fully performant. It has all the compression you’ve come to expect from TokuDB. It has hot schema changes: no-down-time column insertion, deletion, renaming, etc., as well as index creation. It has clustering secondary keys. We are also announcing an Enterprise Edition (coming soon) with additional benefits, such as a support package and advanced backup and recovery tools.

Making TokuDB open source is a natural next step for Tokutek’s involvement in the MySQL community. So far, Tokutek has been involved in the community in many ways:

• We’ve contributed a number of  …

To recover a dropped or corrupt table with Percona Data Recovery Tool for InnoDB you need two things: media with records(ibdata1, *.ibd, disk image, etc.) and a table structure. Indeed, there is no information about the table structure in an InnoDB page. Normally we either recover the structure from .frm files or take it from some old backup.

A new tool sys_parser can recover the table structure from InnoDB dictionary.

Why do we need a new tool anyway? It is absolutely critical to have an accurate table definition to ensure a successful recovery. Even an unnoticeable difference like NULL or NOT NULL can shift all values by a byte and thus will spoil the result. That’s why I prefer the structure from .frm …

If the following items items describe what you need in a high-availability solution, then MySQL Cluster is for you:

• High scale, reads and writes
• 99.999% availability
• Real-time
• SQL and NoSQL
• Low TCO

And what better way to get started on MySQL Cluster than taking the authentic MySQL Cluster training course.

In this 3-day course, you learn important cluster concepts and get hands-on experience installing, configuring and managing a cluster. Some events already on the schedule for this course include:

Switching roles

To get a taste of the power of Tungsten Replicator, we will show how to switch roles. This is a controlled operation (as opposed to fail-over), where we can decide when to switch and which nodes are involved.

In our topology, host1 is the master, and we have three slaves. We can either ask for a switch and let the script select the first available slave, or tell the script which slave should be promoted. The script will show us the steps needed to perform the operation.

IMPORTANT! Please note that this operation is not risk free. Tungsten replicator is a simple replication system, not a complete management tool like Continuent Tungsten. WIth the replicator, you must make sure that the applications have stopped writing to the master before starting the switch, and then you should address the application to the new master when the …

Intro

Tungsten Replicator is an open source tool that does high performance replication across database servers. It was designed to replace MySQL replication, although it also supports replication from and to Oracle and other systems. In this article, we will only cover MySQL replication, both simple and multi-master.

Preparing for installation

To follow the material in this article, you will need a recent build of Tungsten Replicator. You can get the latest ones from http://bit.ly/tr20_builds. In this article, we are using build 2.0.8-167.

Before starting any installation, you should make sure that you have satisfied all the prerequisites. Don't underestimate the list. Any missing …