Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 70 Next 30 Older Entries

Displaying posts with tag: continuent (reset)

Tungsten Replicator 3.0 is Cloudera Enterprise 5 Certified
+0 Vote Up -0Vote Down

One of the key platforms I’ve been testing on for the MySQL to Hadoop replication has been Cloudera, largely driven by customer requirements, but it’s also one of the easiest way to get started with Hadoop.

What I’m even more pleased about is the fact that we are proud to announce that Tungsten Replicator 3.0 is certified for use on the new Cloudera Enterprise 5 platform. That means that we’re sure that replicating your data from MySQL to Cloudera 5 and have it work without causing problems or difficulties on the Hadoop

  [Read more...]
Continuent Replication to Hadoop – Now in Stereo!
+0 Vote Up -0Vote Down

Hopefully by now you have already seen that we are working on Hadoop replication. I’m happy to say that it is going really well. I’ve managed to push a few terabytes of data and different data sets through into Hadoop on Cloudera, HortonWorks, and Amazon’s Elastic MapReduce (EMR). For those who have been following my long association with the IBM InfoSphere BigInsights Hadoop product, and I’m pleased to say that it’s working there too. I’ve had to adapt Robert’s original script to work with the different versions of the underlying Hadoop tools and systems to make it compatible. The actual performance and process is unchanged; you just use a different JS-based batchloader script to work with different tools.

Robert has also been simplifying some of the core functionality, such as configuring some fixed pre-determined

  [Read more...]
Real-Time Data Loading from MySQL to Hadoop using Tungsten Replicator 3.0 Webinar
+0 Vote Up -0Vote Down

To follow-up and describe some of the methods and techniques behind replicating into Hadoop from MySQL in real-time, and how this can be combined into your data workflow, Continuent are running a webinar with me presenting that will go over the details and provide a demo of the data replication process.

Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0

Hadoop is an increasingly popular means of analyzing transaction data from MySQL. Up until now mechanisms for moving data between MySQL and Hadoop have been rather limited. The new Continuent Tungsten Replicator 3.0 provides enterprise-quality replication from MySQL to Hadoop. Tungsten Replicator 3.0 is 100% open source, released under a GPL V2 license, and available for download at

  [Read more...]
Parallel Extractor for Provisioning
+0 Vote Up -0Vote Down

Coming up as a new feature in Tungsten Replicator (and written by our replicator expert Stephane Giron) is the ability to provision a new database by using data from an existing database. This new feature comes in the form of a tool called the Parallel Extractor.

The principles are very simple. On the master side:

  • Start the master replicator offline.
  • Switch the replicator to the online provision state.
  • The master replicator pulls the data out of the existing database and writes that information into the Transaction History Log (THL). At this point, the normal replicator thread is not extracting events from the source database.
  • Once the parallel replication has completed, the replicator switches over to normal extraction mode, and starts writing change data into the THL.

On the slave side, the THL events are read

  [Read more...]
MC at Percona Live San Francisco 2014
+0 Vote Up -0Vote Down

Now I’m back in the MySQL fold, I’ve got the opportunity to speak at Percona Live again. I’ve always enjoyed speaking at this conference (back when it was known by another name…), although I need to up my game and do the 6 talks I did back in 2009.

On the Tuesday afternoon, tutorials day, I’m running a half-day session with my replication colleague Linas Virbalas. This will be similar to the session I did at Percona Live London, and cover some of the more advanced content on replication, including, but not limited to:

  • Filters
  • JavaScript Filtering
  • Some fun and practical filters
  • Heterogeneous replication from MySQL out to MongoDB, Vertica, Oracle and Hadoop

I might even choose to demo

  [Read more...]
MySQL to Hadoop Step-By-Step
+0 Vote Up -0Vote Down

We had a great webinar on Thursday about replicating from MySQL to Hadoop (watch the whole thing). It was great, but one of the questions at the end was ‘is there an easy way to test’.

Sadly we can’t go giving out convenient ready-to-run downloads of these things because of licensing and and other complexities, so I want to try and make it as simple and straightforward as possible by giving you the directions to complete. I’m going to be point to the Continuent Documentation every now and then so this is not too crowded, but we should get through it pretty easily.

Major Decisions

For this to work: 

  • We’ll setup two VMs, one the master
  [Read more...]
Real-Time Replication from MySQL to Cassandra
+1 Vote Up -0Vote Down

Earlier this month I blogged about our new Hadoop applier, I published the docs for that this week (http://docs.continuent.com/tungsten-replicator-3.0/deployment-hadoop.html) as part of the Tungsten Replicator 3.0 documentation (http://docs.continuent.com/tungsten-replicator-3.0/index.html). It contains some additional interesting nuggets that will appear in future blog posts.

The main part of that functionality that performs the actual applier for Hadoop is based around a JavaScript applier engine – there will eventually be docs for that as part of the Batch Applier content (

  [Read more...]
On the road again - FOSSAsia
+0 Vote Up -0Vote Down

On the road again - FOSSAsia

It has been a few busy months until now. I have moved from Italy to Thailand, and the move has been my first priority, keeping me from attending FOSDEM and interacting with social media. Now I start catching my breath, and looking around for new events to attend. But before I get into this, let’s make a few things clear:

  • I am still working for Continuent. Actually, it’s because of my company flexibility that I could move to a different country (a different continent, 6 time zones away) without much trouble. Thanks, Continuent! (BTW: Continuent is hiring! )
  • I am still involved with MySQL activities, events, and community matters. I just happen to be in a different time zone, where direct talk with
  [Read more...]
Getting Data into Hadoop in real-time
+0 Vote Up -0Vote Down

Moving data between databases is hard. Without ever intending it, I seem to have spent a lifetime working on solutions for getting data into and out of databases, but more frequently between. In fact, my first job out of university was migrating data from BRS/Text, a free-text database (probably what we would call a NoSQL) into a more structured Oracle.

Today I spend some of my time working in Big Data, more often than not, migrating information from existing data stores into Big Data so that they can be analysed, something I covered in more detail here:

http://www.ibm.com/developerworks/library/bd-sqltohadoop1/index.html
http://www.ibm.com/developerworks/library/bd-sqltohadoop2/index.html


  [Read more...]
Anonymizing Data During Replication
+0 Vote Up -0Vote Down

If you happen to work with personal data, chances are you are subject to SOX (Sarbanes-Oxley) whether you like it or not.

One of the worst aspects of this is that if you want to be able to analyse your data and you replicate out to another host, you have to find a way of anonymizing the information. There are of course lots of ways of doing this, but if you are replicating the data, why not anonymize it during the replication?

Of the many cool features in Tungsten Replicator, one of my favorites is filtering. This allows you to process the stream of changes that are coming from the data extracted from the master and perform operations on it. We use it a lot in the replicator for ignoring tables, schemas and columns, and for ensuring that we have the correct information within the THL.

Given this, let’s use it to anonymize the data as it is being

  [Read more...]
Tungsten-Replicator 2.1.1 with better installation and built-in security
+1 Vote Up -0Vote Down


UPDATE 2013-08-30: Tungsten 2.1.2 was released.

UPDATE 2013-08-23: We have found a few problems that happen when replicating with RBR and temporal columns. We will have to publish an updated bugfix release quite soon.

Tungsten Replicator 2.1.1 is out. Key features in this release are:

  • A better installer, of which we have already given a preview in tpm, the multi-master composer. The new installer allows faster and more powerful deployments of both single and multiple masters topologies. And it also allows the next feature:
  • Secured communication layer. Now the replicator data and

  [Read more...]
Developing Applications for use with Continuent Tungsten and Tungsten Replicator in SDJ
+1 Vote Up -0Vote Down

I’ve just had a new article published with the Software Developers Journal talking about how you can write applications to take full advantage of Continuent Tungsten and Tungsten Replicator.

As a developer of an application there really isn’t a problem better than finding that you have to scale up the application and the database that supports it to handle the increased load. The main bottleneck to most expansion is the database server and in many modern environments that replication is based around MySQL. Application servers are easy to add on to the front-end of your environment.

Read: Qt5 – How to Become a Professional

  [Read more...]
Welcome Tungsten Replicator 2.1.0!
+3 Vote Up -0Vote Down

Overview


First off, the important news. Tungsten Replicator 2.1.0 was released today.
You can download it and give it a try right now.


Second, I would say that I am quite surprised at how much we have done in this release. The previous release (2.0.7) was in February, which is just a few months ago, and yet it looks like ages when I see the list of improvements, new features and bug fixes in the Release Notes. I did not realized it until I ran my last batch of checks to test the upgrade from the previous release, which I hadn’t run for quite a long





  [Read more...]
Percona Live MySQL Conference 2013 wrap-up
+2 Vote Up -0Vote Down

The Percona Live MySQL Conference & Expo 2013 was April 22-25 in Santa Clara, California. This was Percona’s second year organizing the conference and we were very pleased with the event and the feedback (check the #perconalive hashtag for a sampling of the great comments such as this from Tom Krouper or this from John Goulah or this from Jeremy Tinley or this from SF MySQL Meetup).

  [Read more...]
The Data Day, A few days: April 22-26 2013
+0 Vote Up -0Vote Down

Pivotal launches. SkySQL and Mony Program merge. And much, much more

Our report on the changes in the MySQL ecosystem is now available for 451 clients and non-clients alike at bit.ly/451mysql

— Matt Aslett (@maslett) April 25, 2013

For 451 Research clients: VMware expands Serengeti’s horizons with updated Hadoop virtualization project bit.ly/17muQFI

— Matt Aslett (@maslett) April 26, 2013

For 451 Research clients: SkySQL, Monty Program merge to support MariaDB following formation of MariaDB Foundation bit.ly/10dsdjf

  [Read more...]
Percona Live 2013, MySQL, Continuent and an ever-healthy Ecosystem
+0 Vote Up -0Vote Down

I’m sitting here in the lounge at SFO thinking back on the last week, the majority of which has been spent meeting my new workmates and attending the Percona MySQL conference.

For me it has been as much of a family reunion as it has been about seeing the wonderful things going on in MySQL.

Having joined Continuent last month after an ‘absence’ in NoSQL land of almost 2.5 years, joining the MySQL community again just felt like coming home after a long absence. And that’s no bad thing. On a very personal level it was great to see so many of my old friends, many of whom were not only pleased to see me, but pleased to see me working back in the MySQL fold. Evidently many people think this is where I belong.

What was great to see is that the MySQL community is alive and

  [Read more...]
A great day! MySQL Sandbox and Continuent get a MySQL Community Award
+3 Vote Up -0Vote Down

It was a busy day at Percona Live MySQL Conference and Expo 2013. After the regular sessions, there were the traditional MySQL Community Awards, and two entities close to me were among the winners.

  • My company, Continuent won the award as Corporate Contributor of the year;
  • My pet project of the last 7 years, MySQL Sandbox, won the award as Application of the year 2013.

Thanks to the MySQL community for these awards. They give me (and my colleagues) a goos feeling of appreciation.

The day was even more successful for four major occurrences that contributed to my wellness:

  [Read more...]
Biggest MySQL related news in the last 24 hours
+5 Vote Up -0Vote Down

For me, the biggest news in the last 24 hours so far has been:

  • SkySQL merges with Monty Program, developers of MariaDB. This of course affects me directly and leads to a change in affiliation in a few months.
  • TokuDB goes opensource. I think this is really big news. Beyond just the fact that it can now be a storage engine in the main MariaDB tree, I love the work they’re doing to extend it to be an engine for MongoDB as well.
  • Continuent
  •   [Read more...]
    Continuent Tungsten at SkySQL & Cloud Database Solutions Day, Friday 4/26
    +1 Vote Up -0Vote Down
    Join Continuent, SkySQL and MariaDB for a day of free tutorials and interactive discussions around the MySQL and Maria databases, the Cloud, and High Availability, immediately following Percona Live: MySQL User Conference and Expo 2013. Robert Hodges (CEO, Continuent) and Edward Archibald (CTO, Continuent) will be talking about "Scalable MySQL Operation in the Cloud with Continuent Tungsten" at
    Keynotes, BOFs, and the Community Networking Reception at Percona Live MySQL Conference and Expo
    +1 Vote Up -0Vote Down

    The Percona Live MySQL Conference and Expo begins next Monday and runs April 22-25, 2013. Attendees will see great keynotes from leaders in the industry including representatives from Oracle, Amazon Web Services, HP, Continuent, and Percona. They can also participate in thought provoking Birds of a Feather sessions on Tuesday night and the Wednesday night Community Networking Reception will be fun and entertaining with the presentation of the Community Awards and the Lightning Talks.

    If you

      [Read more...]
    Joining Continuent
    +0 Vote Up -0Vote Down

    I’ve just completed my first month here at Continuent, strangely back into the MySQL ecosystem which I have been working in for some time before I joined CouchOne, and then Couchbase, two and half years ago. Making the move back to MySQL is both an experience, and somehow, comfortable…

    Continuent produce technology that makes for easier replication between MySQL servers and, more importantly, more flexible solutions when you need to scale out by providing connector and management functionality for your MySQL cluster. That means that you can easily backup, add slaves, and create complex replication scenarios such as multi-master, and even multiple-site, multiple-master topologies. This functionality is split over two products, Continuent Tungsten, which is the cluster management product, and the open source Tungsten Replicator, which provides the basic replication

      [Read more...]
    Multi-master data conflicts - Part 2: dealing with conflicts
    +2 Vote Up -0Vote Down

    In the first part of this article we examined the types of conflicts and their causes. In this part, we will analyse some of the methods available to deal with conflicts.

    Pessimistic locking (or: conflicts won't happen)

    Applicability: synchronous clusters with 2pc

    We've covered this topic in the previous article, but it's worth repeating. If you use a synchronous cluster, you don't have conflicts. For example, MySQL Cluster ensures consistent data with updates coming from different nodes. However, MySQL Cluster is not a replacement for a MySQL server, and it has severe limitations.


    Optimistic locking

    Applicability: synchronous clusters without 2pc (Galera)

    Conflicting transactions proceed on different


      [Read more...]
    Multi-master data conflicts - Part 1: understanding the problem
    +3 Vote Up -0Vote Down

    What is a conflict?

    Readers of this blog know that one of my favorite tools, Tungsten Replicator, can easily create multi-master replication topologies, such as all-masters, star, fan-in. While this is good news for system designers and ambitious DBAs, it also brings some inconvenience. When you allow updates to happen in more than one master, you risk having conflicts. You may have heard this term before. For the sake of clarity, let's define what conflicts are, before analyzing each case in detail.

    You have a conflict when several sources (masters) update concurrently the same data in asynchronous replication.

    It's important to stress that this happens with asynchronous replication. In a truly synchronous cluster, where all data is kept consistent through

      [Read more...]
    Sessions at Percona Live MySQL Conference 2013: fun, competition, novelties, and a free pass
    +3 Vote Up -0Vote Down

    The Percona Live MySQL Conference and Expo 2013 is almost 1 month away. It's time to start planning, set the expectations, and decide what to attend. This post will give a roundup of some of the sessions that I recommend attending and I look forward to.

    First, the unexpected!

    After much talk and disbelief, here they come! Oracle (http://www.mysql.com) engineers will participate to the Percona Live conference. This is wonderful! Their participation was requested by the organizers, by the attendees, and by community advocates, who all told the Oracle management how important it is to be in this conference. Finally, they have

      [Read more...]
    Introducing Data Fabric Design for Commodity SQL Databases
    +3 Vote Up -0Vote Down
    Extract from THE SCALE-OUT BLOG by Robert Hodges (CEO, Continuent)http://scale-out-blog.blogspot.com Data management is undergoing a revolution. Many businesses now depend on data sets that vastly exceed the capacity of DBMS servers. Applications operate 24x7 in complex cloud environments using small and relatively unreliable VMs. Managers need to act on new information from those systems in
    Meet you in London - Percona Live MySQL Conference
    +0 Vote Up -0Vote Down
    Continuent is proud to sponsor Percona Live MySQL Conference: London 2012!  Don't miss these five (5) talks by our database replication and clustering stars: Keynote: Future-Proofing MySQL for the World-Wide Data Revolution, by Robert Hodges Why, What, and How of Data Warehouses for MySQL, by Robert Hodges Multi-master, Multi-site MySQL Databases Made Easy with Continuent Tungsten, by Robert
    Overwhelming response from the MySQL community in Barcelona
    +4 Vote Up -0Vote Down

    Within hours of my post about meeting the MySQL community in Barcelona, we got several offers to help, and within one day, an event was created and agreed upon.

    Thanks!

    Today the event was posted at Evenbrite. It will take place on Tuesday, November 13th, at 7pm. It will be a one hour talk about State of the art in MySQL high availability and replication, followed by one hour of Q&A, networking, beer, and snacks.

    Registration is necessary, because the seats are limited. If you want to attend, you should register as soon as possible!

    Haute disponibilité MySQL, par Continuent
    +0 Vote Up -1Vote Down
    MySQL community in Barcelona - Let's meet in November
    +3 Vote Up -0Vote Down

    Update: With the enthusiastic responses that we have received from Barcelona, there is now an event defined at Evenbrite. If you are in Barcelona on November 13th, please register!



    My company, Continuent, will have an engineering meeting in Barcelona, Spain, from November 11th to 16th.

    We are meeting because, as we all work from home, we need to get in touch face-to-face at least a few times a year, and every time we try choosing a nice, inspiring place where we can both work and relax. This time, the choice went to Barcelona, which happens to be one of my favorite towns.

    Now comes the community. We build software that is mainly directed to MySQL users (although we also dedicate much effort to replication to and from Oracle, Postgres,



      [Read more...]
    Tungsten Replicator cookbook. Advanced replication topologies made easy
    +2 Vote Up -0Vote Down

    I have been asked many times to provide an easy way of deploying fan-in and star schema replication schemas. So far, I have been delayed by more pressing duties.

    Now the time has come. Since we are about to release a new version of Tungsten Replicator, I made the effort of putting together the steps for an easy deployment.

    Recipes

    The package (with downloads and svn code available at Tungsten-Replicator Toolbox) includes some juicy goodies. There are recipes to install.

    • Master/slave, the classic replication topology. Nothing fancy, but with the tools mentioned in the next section, it becomes as valuable as the other topologies.
    • All-masters. This is the Tungsten no-SPOF topology. Every node is a master,

      [Read more...]
    Showing entries 1 to 30 of 70 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.