This is a follow up post, describing the implementation details of Hadoop Applier
, and steps to configure and install it. Hadoop Applier integrates MySQL with Hadoop providing the real-time replication of INSERTs to HDFS, and hence can be consumed by the data stores working on top of Hadoop. You can know more about the design rationale and per-requisites in the previous post
. Design and Implementation:
Hadoop Applier replicates rows inserted into a table in MySQL to the Hadoop Distributed File System(HDFS
). It uses an API provided by libhdfs
, a C library to manipulate files in HDFS.
The library comes pre-compiled with Hadoop distributions.It [Read more...]