We are just starting to get into the swing of setting up our new training and webinar schedule. Initially, there will be one Webinar session (typically on a Wednesday) and one training session (on a Tuesday) every week from now. We’ll be covering a variety of different topics at each. Typically our webinars will be about products and features, comparisons to other products, mixed in with product news (new releases, new features) and individual sessions based on what is going on at Continuent and the market in general. Our training, by comparison, is going to be a hands-on, step-by-step sequence covering all of the different areas of our product. So we’ll cover everything from the basics of how the products work, how to deploy them, typical functionality (switching, start/stop, etc), and troubleshooting. All of the sessions are going to be recorded and we’ll produce a suitable archive page so that you can go and view the past sessions. Need a …[Read more]
10 Older Entries »
Hello again everybody.
Well, I promised it a couple of weeks ago, and I’m sorry it has been so long (I’ve been working on other fun stuff in addition to this). But I’m pleased to say that we now have a fully working applier that takes data from an incoming THL stream, whether that is Oracle or MySQL, and converts that into a JSON document and message for distribution over a Kafka topic.
Currently, the configuration is organised with the following parameters:
- The topic name is set according to the incoming schema and table. You can optionally add a prefix. So, for example, if you have a table ‘invoices’ in the schema ‘sales’, your Kafka topic will be sales_invoices, or if you’ve added a prefix, ‘myprefix_schema_table’.
- Data is marshalled into a JSON document as part of the message, and the structure is to have a bunch of metadata and then an embedded record. You’ll see an …
Replication between Tungsten clusters
The process I will describe in this post will allow you to configure replication between Tungsten clusters. The most common use case I have seen for this is a dedicated ETL cluster.
The setup will look like this:
So we will have a composite datasource (compositeprod) composed of east and west clusters, and two ETL clusters, one on each side.
The described setup allows failover or switchover within a single datacenter (e.g. db1.east -> db2.east) or to the Disaster Recovery Site, (e.g. db1.east -> db1.west) for the core cluster.
At the time of this article’s publication it is not possible to replicate between two composite clusters, so each ETL cluster needs to be standalone. This feature is expected for Tungsten 5.0 release.
We will install a standalone replicator on the ETL hosts to bring data in …[Read more]
It had been in the making for long time. Google announced that Google Code would be closing, and since then the Continuent team has been hard at work to handle the transition. You can guess it: this operation would have been quicker if it had been done by a small company like we were one year ago, but being part of a large corporation introduces some constraints that have affected our schedule.
However, our wish has always been, and still is, to keep Tungsten Replicator as an open source product, with full functionalities and with the full benefits that the open source development model offers.
Today, Tungsten Replicator is available on GitHub as …[Read more]
Replication is the process that transfers data from an active master to a slave server, which reproduces the data stream to achieve, as best as possible, a faithful copy of the data in the master.
To check replication health, you may start with sampling the service, i.e. committing some Sentinel value in the master and retrieving it from the slave.
Sentinel data: Tap tap… Is this thing on?
If you want to make sure that replication is working, the easiest test is using replication itself to see if data is being copied across from the master to the slaves. The method is easy:
- Make sure that the data you want to see is NOT in the master or in the slave. If you skip this step, you may think that replication is working, while in fact it may not.
- Either create a table …
Computer science is like an enormous tool box you can rummage
through whenever you have a problem to solve. Most of the tools
are sturdy and practical, like algorithms for B-trees. Some are
also elegant, like consistent hashing in Dynamo. Finally there
are some tools that you never quite figure out even after years
of reflection. That piece of steel you are looking at could be
Excalibur. Or it could be a rusty knife.
The CAP theorem falls into the last category, at least for me. It was a major topic in the blogosphere a few years ago and Google Trends shows steadily increasing interest in the term since 2010. It's not my goal to explain CAP fully--a good informal description is …
I blogged a couple of weeks ago about my upcoming MySQL/Hadoop talk at SCaLE 12X.
Unfortunately I had to cancel. A few days after writing the
article I came down with an eye problem that is fixed but
prevents me from flying anywhere for a few weeks. That's a pity
as I was definitely looking forward to attending the conference
and explaining how Tungsten replicates transactions from MySQL
Meanwhile, we are still moving at full steam with Hadoop-related work at Continuent, which is the basis for the next major replication release, Tungsten Replicator 3.0.0. Binary builds and documentation will go up in a few days. There will also be many more public talks about Hadoop support, starting in April at …
On the road again - FOSSAsia
It has been a few busy months until now. I have moved from Italy to Thailand, and the move has been my first priority, keeping me from attending FOSDEM and interacting with social media. Now I start catching my breath, and looking around for new events to attend. But before I get into this, let’s make a few things clear:
- I am still working for Continuent. Actually, it’s because of my company flexibility that I could move to a different country (a different continent, 6 time zones away) without much trouble. Thanks, Continuent! (BTW: Continuent is hiring! )
- I am still involved with MySQL activities, events, and community matters. I just happen to be in a different time zone, where direct talk with people in Europe and …
It's my pleasure to be presenting at SCaLE
12X on the subject of real-time data loading from MySQL to Hadoop.
This is the first public talk on work at Continuent that
enables Tungsten Replicator to move transactions from
MySQL to HDFS (Hadoop Distributed File System). I will
explain how replication to Hadoop works, how to set it up, and
offer a few words on constructing views of MySQL data using tools
As usual with replication everything we are doing on Hadoop replication is open source. Builds and documentation will be publicly available …
PerconaLive 2014 program is published
After a few months of submissions and reviews, the program for PerconaLive MySQL Conference and Expo 2014 is published. The conference will be held in Santa Clara, from April 1 to 4, 2014.
Registration with early bird discount is available until February 2nd. If you plan to attend, this is probably the best time to act.
I will be presenting twice at the conference:
- MySQL Replication: Advanced features in all flavors it’s a 3 hours tutorial where I compare the …
10 Older Entries »