Planet MySQL

Clustering MySQL instances with Oracle Clusterware 11gR2

+0

-0

posted by Ilmar Kerm on Wed 09 Nov 2011 14:14 UTC
Tags:

(edit) mysql, cluster

I've been doing lately quite many database clustering implementations; Oracle RAC and since we have many MySQL instances in production, had to find a good way to make MySQL instances highly available also.

One good solution for this is managing MySQL instances with clusterware and since we are planning to use Oracle RAC on Oracle Enterprise Linux anyway, then Oracle Clusterware is an excellent candidate for this task. Also... Oracle Clusterware is included with Oracle Enterprise Linux at no additional charge.

Requirements I had:

Multiple MySQL instances running in the same cluster, in case of node failure affected MySQL instances are moved to any other surviving node (least loaded)
Different MySQL instances may run

[Read more...]

MySQL Cluster, and NoSQL

+0

-0

posted by MySQL Community on Wed 02 Nov 2011 08:27 UTC
Tags:

(edit) mysql, podcast, MySQL Cluster, cluster, NoSQL

Those are the topics we cover in the latest episode of our “Meet The MySQL Experts” podcast.

Mat Keep and Bernd Ocklin talk about new database requirements, and walk us through what's new in the second Development Milestone Release of MySQL Cluster 7.2, including impressive performance improvements, new NoSQL access via memcached, cross data center scalability, and more...

Enjoy the podcast!

MySQL Cluster, and NoSQL

+1

-0

posted by MySQL Community on Wed 02 Nov 2011 03:27 UTC
Tags:

(edit) mysql, podcast, MySQL Cluster, cluster, NoSQL

Those are the topics we cover in the latest episode of our “Meet The MySQL Experts” podcast.

Mat Keep and Bernd Ocklin talk about new database requirements, and walk us through what's new in the second Development Milestone Release of MySQL Cluster 7.2, including impressive performance improvements, new NoSQL access via memcached, cross data center scalability, and more...

Enjoy the podcast!

Eventual Consistency - detecting conflicts

+3

-0

posted by Frazer Clement on Thu 20 Oct 2011 00:05 UTC
Tags:

(edit) mysql, Replication, design, cluster, NoSQL, distributed-systems, active-active

In my previous posts I introduced two new conflict detection functions, NDB$EPOCH and NDB$EPOCH_TRANS without explaining how these functions actually detect conflicts? To simplify the explanation I'll initially consider two circularly replicating MySQL Servers, A and B, rather than two replicating Clusters, but the principles are the same.

Commit ordering

Avoiding conflicts requires that data is only modified on one Server at

[Read more...]

Eventual consistency with transactions

+3

-0

posted by Frazer Clement on Mon 10 Oct 2011 00:26 UTC
Tags:

(edit) mysql, Replication, design, cluster, NoSQL, distributed-systems, latency-hiding, active-active

In my last post I described the motivation for the new NDB$EPOCH conflict detection function in MySQL Cluster. This function detects when a row has been concurrently updated on two asynchronously replicating MySQL Cluster databases, and takes steps to keep the databases in alignment.

With NDB$EPOCH, conflicts are detected and handled on a row granularity, as opposed to column granularity, as this is the granularity of the epoch metadata used to detect conflicts. Dealing

[Read more...]

MySQL Cluster 7.2 (DMR2): NoSQL, Key/Value, Memcached

+0

-0

posted by Oracle MySQL Group on Fri 07 Oct 2011 05:26 UTC
Tags:

(edit) mysql, database, memcached, MySQL Cluster, cluster, ha, NoSQL, auto-sharding

0 0 1 977 5569 Homework 46 13 6533 14.0 Normal 0 false false false EN-US JA X-NONE

70x Higher Performance, Cross Data Center Scalability and New NoSQL Interface

Its been an exciting week for all involved with MySQL Cluster, with the announcement of the

+1

-0

posted by MySQL Community on Fri 07 Oct 2011 00:26 UTC
Tags:

(edit) mysql, database, memcached, MySQL Cluster, cluster, ha, NoSQL, auto-sharding

0 0 1 977 5569 Homework 46 13 6533 14.0 Normal 0 false false false EN-US JA X-NONE

70x Higher Performance, Cross Data Center Scalability and New NoSQL Interface

Its been an exciting week for all involved with MySQL Cluster, with the announcement of the

+0

-0

posted by Oracle MySQL Group on Mon 03 Oct 2011 20:09 UTC
Tags:

(edit) mysql, database, scaling, MySQL Cluster, cluster, ha, Clustering, multi-site

Well actually….no. The second Development Milestone Release of MySQL Cluster 7.2 introduces support for what we call “Multi-Site Clustering”. In this post, I’ll provide an overview of this new capability, and considerations you need to make when considering it as a deployment option to scale geographically dispersed database services.

You can read more about MySQL Cluster 7.2.1 in the article posted on the MySQL Developer Zone.

MySQL Cluster has long offered Geographic Replication, distributing clusters to remote data centers to reduce the affects of geographic latency by pushing data closer to the user, as well as providing a capability for

+4

-0

posted by MySQL Community on Mon 03 Oct 2011 15:09 UTC
Tags:

(edit) mysql, database, scaling, MySQL Cluster, cluster, ha, Clustering, multi-site

Well actually….no. The second Development Milestone Release of MySQL Cluster 7.2 introduces support for what we call “Multi-Site Clustering”. In this post, I’ll provide an overview of this new capability, and considerations you need to make when considering it as a deployment option to scale geographically dispersed database services.

You can read more about MySQL Cluster 7.2.1 in the article posted on the MySQL Developer Zone.

MySQL Cluster has long offered Geographic Replication, distributing clusters to remote data centers to reduce the affects of geographic latency by pushing data closer to the user, as well as providing a capability for disaster

[Read more...]

Eventual consistency with MySQL

+2

-0

posted by Frazer Clement on Mon 03 Oct 2011 12:50 UTC
Tags:

(edit) mysql, Replication, design, cluster, NoSQL, distributed-systems, latency-hiding, active-active

tl;dr : New 'automatic' optimistic conflict detection functions available giving the best of both optimistic and pessimistic replication on the same data

MySQL replication supports a number of topologies, and one of the most interesting is an active-active, or master-master topology, where two or more Servers accept read and write traffic, with asynchronous replication between them.

This topology has a number of attractions, including :

Potentially higher availability
Potentially low impact on read/write latency
Service availability insensitive to replication

[Read more...]

MySQL HA Solutions: New Guide Available

+0

-0

posted by Oracle MySQL Group on Thu 29 Sep 2011 09:53 UTC
Tags:

(edit) mysql, Replication, database, cluster, ha, high, Availability, mat, keep

0 0 1 271 1547 Homework 12 3 1815 14.0 Normal 0 false false false EN-US JA X-NONE

Databases are the center of today’s web, enterprise and embedded applications, storing and protecting an organization’s most valuable assets and supporting business-critical applications. Just minutes

[Read more...]

MySQL HA Solutions: New Guide Available

+5

-0

posted by Oracle MySQL Group on Thu 29 Sep 2011 04:53 UTC
Tags:

(edit) mysql, Replication, database, cluster, ha, high, Availability, mat, keep

0 0 1 271 1547 Homework 12 3 1815 14.0 Normal 0 false false false EN-US JA X-NONE

Databases are the center of today’s web, enterprise and embedded applications, storing and protecting an organization’s most valuable assets and supporting business-critical applications. Just minutes

+1

-0

posted by SkySQL on Tue 27 Sep 2011 21:44 UTC
Tags:

(edit) mysql, cluster, Training, ha

Very soon after SkySQL Ab was created, we added an Advanced MySQL Cluster course to our training offering. This addition was made as a customer who was already quite experienced with MySQL cluster wanted training. This course was, however, deemed to be too complex and with too much pre-requisites to add to our public offering so it was left as a special treat for customers who had deep cluster knowledge. We decided that we would instead create a more general purpose MySQL Cluster course with less prerequisite knowledge required as soon as possible.
Many months have passed and a lot of other courses have been created since, but we now finally release our version of a general purpose MySQL Cluster training course, called Administering MySQL Cluster. As the name suggests this 3-day course
[Read more...]

Tungsten Replicator 2.0.4 released: usability and power

+7

-1

posted by Giuseppe Maxia on Tue 06 Sep 2011 22:00 UTC
Tags:

(edit) mysql, Replication, installation, cluster, Downloads, tungsten

It has been a bumpy ride, with dozens of issues opened and resolved, but we finally feel that Tungsten Replicator 2.0.4 is ready for prime time.There have been quite a lot of changes. Most notably, the replicator is much faster, especially when it comes to parallel replication, and it is much easier to install, thanks to its new integrated installer, which can validate all the requirements to install the replicator, and suggest remedies when the requirements aren't met. This new installer is so good, in fact, that calling it installer is an insult. It is a legitimate cluster builder, able to install a full fledged [Read more...]

MySQL with Windows Server 2008 R2 Failover Clustering

+6

-0

posted by Andrew Morgan on Tue 06 Sep 2011 12:18 UTC
Tags:

(edit) mysql, Windows, cluster, ha, High Availability

Windows Server 2008 R2 Failover Clustering

Oracle has announced support for running MySQL on Windows Server Failover Clustering (WSFC); with so many people developing and deploying MySQL on Windows, this offers a great option to add High Availability to MySQL deployments if you don’t want to go as far as deploying MySQL Cluster.

This post will give a brief overview of how to set things up but for all of the gory details a new white paper MySQL with Windows Server 2008 R2 Failover Clustering

+3

-0

posted by Oracle MySQL Group on Fri 05 Aug 2011 08:15 UTC
Tags:

(edit) mysql, Web, memcached, scaling, MySQL Cluster, cluster, NoSQL, not-only-sql

0 0 1 281 1602 Homework 13 3 1880 14.0 Normal 0 false false false EN-US JA X-NONE

Supporting successful services on the web means scaling your back-end databases across multiple dimensions. This blog focuses on scaling access methods to your data using SQL and/or NoSQL interfaces.

[Read more...]

MySQL Cluster Manager hands on

+3

-0

posted by Ronen Baram on Wed 27 Jul 2011 12:35 UTC
Tags:

(edit) mysql, enterprise, cluster, ha, manager

MySQL Cluster is, without doubt, the most interesting MySQL product Oracle offers to the people out there. It’s the flagship, the holy grail, based on the knowledge and technology developed doing our well known MySQL Server. I’m not going to go through why MySQL Cluster is so great, that you can find anywhere. I’m going [...]

Scaling Web Databases, Part 2: Adding Nodes, Evolving Schema with Zero Downtime

+3

-0

posted by Oracle MySQL Group on Thu 21 Jul 2011 11:49 UTC
Tags:

(edit) mysql, Databases, Web, scaling, MySQL Cluster, cluster, schema, upgrades, on-line

In my previous post, I discussed scaling web database performance in MySQL Cluster using auto-sharding and active/active geographic replication - enabling users to scale both within and across data centers.

I also mentioned that while scaling write-performance of any web service is critical, it is only 1 of multiple dimensions to scalability, which include:

- The need to scale operational agility to keep pace with demand. This means being able to add capacity and performance to the database, and to evolve the schema – all without downtime;

- The need to scale queries by having flexibility in the APIs used to access the database – including SQL and NoSQL interfaces;

- The need to scale the database while maintaining continuous availability.

+1

-0

posted by Geert Vanderkelen on Tue 19 Jul 2011 14:22 UTC
Tags:

(edit) mysql, work, cluster, howto

This post explains how to debug MySQL Cluster 7.1, installed using the RPM packages, using gdb on a Linux box (Red Hat, Oracle Enterprise Linux, CentOS, ..).

When a data node crashes lots of information goes into the error log, trace files and out log. However, it makes sometimes sense when you can repeat the crash, to run the data node in debug mode, or using gdb.

First, using RPMs and a Linux distribution, make sure you have the ‘debuginfo’ package installed. For example, for Red Hat or Oracle Enterprise Linux on a 64-bit machine, this package would be called: MySQL-Cluster-gpl-debuginfo-7.1.15-1.rhel5.x86_64.rpm .

Create a file with the following commands, we will name it ‘ndbd.gdb’:

set

+5

-0

posted by MySQL Community on Mon 18 Jul 2011 09:13 UTC
Tags:

(edit) mysql, database, MySQL Cluster, cluster, manager, Clustering, management, clusters

0 0 1 710 4053 Homework 33 9 4754 14.0 Normal 0 false false false EN-US JA X-NONE

Clustered computing brings with it many benefits: high performance, high availability, scalable infrastructure, etc. But it also brings with it more complexity.

Why?

Well, by its very nature, there

+3

-0

posted by Oracle MySQL Group on Tue 14 Jun 2011 13:50 UTC
Tags:

(edit) mysql, Web, database, MySQL Cluster, cluster, scale, mat, auto-sharding, keep

0 0 1 322 1841 Homework 15 4 2159 14.0 Normal 0 false false false EN-US JA X-NONE

The realities of today’s successful web services are creating new demands that many legacy databases were just not designed to handle:

- The need to scale writes, as well as reads, both within and across

+2

-0

posted by MySQL Community on Wed 18 May 2011 06:50 UTC
Tags:

(edit) mysql, Web, database, MySQL Cluster, cluster, scale, session, management, clusters

Join us for a live webinar (http://www.mysql.com/news-and-events/web-seminars/display-630.html) and download a new whitepaper (http://www.mysql.com/why-mysql/white-papers/mysql_wp_session_mngmnt.php) where we discuss how to realize new value from data collected during web session management.

Session management has long been a key component of any web infrastructure – enhancing the user browsing experience through improved reliability, reduced latency and tighter security.

Increasingly organizations are looking to unlock more value from session management to further improve user loyalty (i.e. making the web service more “sticky”) and improve monetization of web services. There are two distinct developments that offer the promise of unlocking more value from session data:
1. Provide highly personalized browsing experiences by

+2

-0

posted by Geert Vanderkelen on Mon 02 May 2011 11:34 UTC
Tags:

(edit) mysql, work, cluster, howto

There is a log file called ndb_<NodeID>_out.log created by the MySQL Cluster (http://www.mysql.com/products/cluster/) data nodes which can become quite big overtime. There is, unlike the cluster logs created by the management nodes, no rotation build in. So you have to revert to the basics and copy the file away, truncating the old one.

For example, if you want to ‘rotate’ the log file of data node with NodeID 3:

shell> mv ndb_3_out.log.1.gz ndb_3_out.log.2.gz
shell> cp ndb_3_out.log ndb_3_out.log.1
shell> cat /dev/null > ndb_3_out.log
shell> gzip ndb_3_out.log.1

It’s not elegant, and you might lose some entries, but it will help you keeping disk usage minimal. If you don’t need the log at all, just line 3 would do the trick.

You can use

+2

-0

posted by Geert Vanderkelen of team blog MySQL Support Team on Sun 03 Apr 2011 11:05 UTC
Tags:

(edit) mysql, work, cluster, howto

The MySQL Cluster data node log files can become very big. The best solution is to actually fix the underlying problem. But if you know what you are doing, you can work around it and filter out these annoying log entries.

An example of ‘annoying’ entries is when you run MySQL Cluster on virtual machines (not good!) and disks and OS can’t follow any more; a few lines from the ndb_X_out.log:

2011-04-03 10:52:31 [ndbd] WARNING  -- Ndb kernel thread 0 is stuck in: Scanning Timers elapsed=100
2011-04-03 10:52:31 [ndbd] INFO     -- timerHandlingLab now: 1301820751642 sent: 1301820751395 diff: 247
2011-04-03 10:52:31 [ndbd] INFO     -- Watchdog: User time: 296  System time: 536
2011-04-03 10:52:31 [ndbd] INFO     -- Watchdog: User time: 296  System time: 536
2011-04-03 10:52:31

+3

-0

posted by Frazer Clement on Sat 02 Apr 2011 00:05 UTC
Tags:

(edit) mysql, design, cluster, parallel, distributed-systems, latency-hiding

Unlike most other MySQL storage engines, Ndb does not perform all of its work in the MySQLD process. The Ndb table handler maps Storage Engine Api calls onto NdbApi calls, which eventually result in communication with data nodes. In terms of layers, we have SQL -> Handler Api -> NdbApi -> Communication. At each of these layer boundaries, the mapping between operations at the upper layer to operations at the lower layer is non trivial, based on runtime state, statistics, optimisations etc.

The MySQL status variables can be used to understand the behaviour of the MySQL Server in terms of user commands processed, and also how these map to some of the Storage Engine Handler Api calls.

Status variables

[Read more...]

MySQL Cluster online scaling

+7

-0

posted by Frazer Clement on Sun 27 Mar 2011 23:41 UTC
Tags:

(edit) mysql, design, cluster, distributed-systems

Most people looking at a diagram showing the Cluster architecture soon want to know if the system can scale online. Api nodes such as MySQLD processes can be added online, and the storage capacity of existing data nodes can be increased online, but it was not always possible to add new data nodes to the cluster without an initial system restart requiring a backup and restore.

An online add node and data repartitioning feature was finally implemented in MySQL Cluster 7.0. It's not clear how often users actually do scale their Clusters online, but it certainly is a cool thing to be able to do.

There are two parts to the feature :

Online add an empty data node to an existing cluster

Online rebalance existing data across the existing and new data nodes

Adding an empty data node to a cluster sounds trivial, but is actually fairly

[Read more...]

Data distribution in MySQL Cluster

+5

-0

posted by Frazer Clement on Sat 26 Mar 2011 00:43 UTC
Tags:

(edit) mysql, design, cluster, parallel, distributed-systems, message-passing

MySQL Cluster distributes rows amongst the data nodes in a cluster, and also provides data replication. How does this work? What are the trade offs?

Table fragments

Tables are 'horizontally fragmented' into table fragments each containing a disjoint subset of the rows of the table. The union of rows in all table fragments is the set of rows in the table. Rows are always identified by their primary key. Tables with no primary key are given a hidden primary key by MySQLD.

By default, one table fragment is created for each data node in the cluster at the time the table is created.

Node groups and Fragment replicas

The data nodes in a cluster are logically divided into Node groups. The size of each Node group is controlled by the NoOfReplicas parameter. All data nodes in a Node group store the same data. In

[Read more...]

Oracle Solaris Cluster 3.3 available

+2

-2

posted by Sun Cluster Engineering on Fri 18 Mar 2011 20:08 UTC
Tags:

(edit) sun, cluster, solaris, recovery, disaster, availabiity, suncluster

On September 8, 2010 Oracle announced the availability of Oracle Solaris Cluster 3.3

Oracle Solaris Cluster 3.3, built on the solid foundation of Oracle Solaris, offers the 
most extensive Oracle enterprise High Availability and Disaster Recovery solutions for the 
largest portfolio of mission-critical applications.

Integrated and thoroughly tested with Oracle's Sun servers, storage, connectivity 
solutions and Solaris 10 features, Oracle Solaris Cluster is now qualified with Solaris 
Trusted Extensions, supports Infiniband for general networking or storage usage, and can 
be deployed with Oracle Unified Storage in Campus Cluster configurations. It extends its 
applications support to new Oracle applications such as Oracle Business Intelligence, 
PeopleSoft, TimesTen, and MySQL Cluster.

The single, integrated HA and DR solution enables multi-tier deployments in virtualized 
environments.

[Read more...]

Oracle Solaris Cluster 3.3 available

+0

-0

posted by Sun Cluster Engineering on Fri 18 Mar 2011 13:08 UTC
Tags:

(edit) Oracle, sun, cluster, solaris, recovery, disaster, availabiity, suncluster

On September 8, 2010 Oracle announced the availability of Oracle Solaris Cluster 3.3

Oracle Solaris Cluster 3.3, built on the solid foundation of Oracle Solaris, offers the 
most extensive Oracle enterprise High Availability and Disaster Recovery solutions for the 
largest portfolio of mission-critical applications.

Integrated and thoroughly tested with Oracle's Sun servers, storage, connectivity 
solutions and Solaris 10 features, Oracle Solaris Cluster is now qualified with Solaris 
Trusted Extensions, supports Infiniband for general networking or storage usage, and can 
be deployed with Oracle Unified Storage in Campus Cluster configurations. It extends its 
applications support to new Oracle applications such as Oracle Business Intelligence, 
PeopleSoft, TimesTen, and MySQL Cluster.

The single, integrated HA and DR solution enables multi-tier deployments in virtualized

[Read more...]

Oracle Solaris Cluster 3.3 available

+0

-0

posted by Sun Cluster Engineering on Fri 18 Mar 2011 13:08 UTC
Tags:

(edit) Oracle, sun, cluster, solaris, recovery, disaster, availabiity, suncluster, Oracle Solaris Cluster

On September 8, 2010 Oracle announced the availability of Oracle Solaris Cluster 3.3

Oracle Solaris Cluster 3.3, built on the solid foundation of Oracle Solaris, offers the 
most extensive Oracle enterprise High Availability and Disaster Recovery solutions for the 
largest portfolio of mission-critical applications.

Integrated and thoroughly tested with Oracle's Sun servers, storage, connectivity 
solutions and Solaris 10 features, Oracle Solaris Cluster is now qualified with Solaris 
Trusted Extensions, supports Infiniband for general networking or storage usage, and can 
be deployed with Oracle Unified Storage in Campus Cluster configurations. It extends its 
applications support to new Oracle applications such as Oracle Business Intelligence, 
PeopleSoft, TimesTen, and MySQL Cluster.

The single, integrated HA and DR solution enables multi-tier deployments in virtualized

[Read more...]

Search

Profile

Top Voted (Last Week)

Top Voted (Last Month)

Top 20 Most Active Authors

Top 10 Most Active Team Blogs

Oracle MySQL Blogs

MySQL Links

Other Links

Planetarium