Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 39 Next 9 Older Entries

Displaying posts with tag: ndb (reset)

MySQL Cluster on Raspberry Pi - Sub-second failover
Employee +1 Vote Up -0Vote Down
MySQL Cluster claims to achieve sub-second failover without any data loss for commited transactions. And I always wanted to show this in a demo. Now we created that demo finally. See Mark's blog and Keith's blog for setting up MySQL Cluster on RaspberryPi.
The nice thing about the RPis is that you can easily pull the plug to test failover. Ok, that is only one possible failure scenario but for sure the most obvious and more impressive than "kill -9".


That demo application is constantly using the database for storing new lines, removing old lines and reading all line data for the graphical view. There is no caching. It uses JDBC directly.
To document the setup here is the config.ini file




  [Read more...]
Ghosts of MySQL Past, Part 8: The First Fork.
+1 Vote Up -0Vote Down

This is the 8th installment in the rather long series that started with Part 1 about a month ago.

Back in 2006, we were in the situation where MySQL 5.0 had taken forever, and the first “GA” release was not suitable for production. Looking towards MySQL 5.1, it was also unlikely to be out any time soon. The MySQL Cluster team had customers that needed new features in a stable release. The majority of users didn’t use the MySQL server at all, they directly used the C++ NDB API for the vast majority of queries – so the vast majority of release blocker bugs in the MySQL server would not affect the production

  [Read more...]
Percona XtraDB Cluster – A Drop-in-place Clustering Solution for MySQL
+0 Vote Up -0Vote Down
Emphasis on clustering solutions comes up quite a lot when talking to customers about High Availability. The reason is because clustering is supposed to provide an easier solution for maintaining high availability and so that you do not have to rely on other tools and techniques outside of the database server. I thought it would be good to share the gist of many of my discussions around clustering, in the form of a blog post. So here I will be doing a high-level comparison between MySQL NDB Cluster and Percona XtraDB Cluster.
High-availability options for MySQL, October 2013 update
+1 Vote Up -0Vote Down

The technologies allowing to build highly-available (HA) MySQL solutions are in constant evolution and they cover very different needs and use cases. In order to help people choose the best HA solution for their needs, we decided, Jay Janssen and I, to publish, on a regular basis (hopefully, this is the first), an update on the most common technologies and their state, with a focus on what type of workloads suite them best. We restricted ourselves to the open source solutions that provide automatic failover. Of course, don’t simply look at the number of Positives/Negatives items, they don’t have the same values. Should you pick any of these technologies, heavy testing is mandatory, HA is never beyond scenario that have been tested.

Percona XtraDB  [Read more...]

The MySQL Cluster storage engine
+2 Vote Up -0Vote Down

This is one close to my heart. I’ve recently written on other storage engines: Where are they now: MySQL Storage EnginesThe MERGE storage engine: not dead, just resting…. or forgotten and The MEMORY storage engine. Today, it’s the turn of MySQL Cluster.

Like InnoDB, MySQL Cluster started outside of MySQL. Those of you paying attention at home may notice a correlation between storage engines not written exclusively for MySQL and being at all successful.

NDB

  [Read more...]
NoSQL Memcached API for MySQL: Latest Updates
Employee_Team +4 Vote Up -0Vote Down

With data volumes exploding, it is vital to be able to ingest and query data at high speed. For this reason, MySQL has implemented NoSQL interfaces directly to the InnoDB and MySQL Cluster (http://www.mysql.com/products/cluster/) (NDB) storage engines, which bypass the SQL layer completely. Without SQL parsing and optimization, Key-Value data can be written directly to MySQL tables up to 9x faster, while maintaining ACID guarantees.

In addition, users can continue to run complex queries with SQL across the same data set, providing real-time analytics to the business or anonymizing sensitive data before loading to big data platforms such as Hadoop, while still maintaining all of the advantages of their existing relational database infrastructure.

This and more is discussed in the latest Guide to MySQL and NoSQL

  [Read more...]
Some corrections and additions to my simple KVS tests.
+0 Vote Up -0Vote Down
This is the first follow-up to my post on a simple test of KVS alternatives. To recap, I tested a simple single table schema in MySQL using the NDB and InnoDB storage engines. To have a Key-Value store to compare with, I did the same test in MongoDB. All tests were done of the same system, an 8-core AMD Linux box with 16 Gb RAM. The tests consisted of reading 1.000.000 rows, out of the total 105.000.000 in the table, distributed over 100 threads 10 times, a total of 10.000.000 rows read then. The test program I use makes sure that the same random ID's of the table are reused each time and the same are used for all servers.

Now, firstly, after some checking I realized that I had not fully cached the InnoDB engine, so it was doing a

  [Read more...]
How to Stop Playing “Hop and Seek”: MySQL Cluster and TokuDB, Part 2
+1 Vote Up -0Vote Down

In my last post, I wrote that I observed many similarities between TokuDB and MySQL Cluster. Many features that benefit TokuDB also benefit MySQL Cluster, and vice versa, with Hot Column Addition and Deletion (HCAD) being an example. Over my next few posts, I expand on some more of these possibly unexpected similarities.

Today I want to focus on optimizer support for clustering keys. Both MySQL Cluster and TokuDB can benefit from the MySQL optimizer supporting clustering keys. For TokuDB, the

  [Read more...]
MySQL Cluster on Raspberry Pi
+5 Vote Up -1Vote Down

Earlier this week, Andrew Morgan wrote a piece on running MySQL Cluster on Raspberry Pi. Since the term “Cluster” is hideously overloaded, I’ll note that we’re talking about the NDB cluster storage engine here, a very specific architecture originally acquired by MySQL AB from Ericsson (telco).

Raspberry Pi is a new single-board computer based on the ARM processor series (same stuff that powers most mobile phones these days), and it can run Linux without any fuss. Interfaces include Ethernet, USB, and HDMI video, and the cost is $25-50. I’m looking

  [Read more...]
Dynarr256 for DBACC -or- The death of MAX_ROWS
Employee +1 Vote Up -0Vote Down
Back in 2006 we became aware of problems storing large numbers of rows in a single table in cluster. Johan Andersson and Yves Trudeau have each blogged about the problem and the common workaround here and here.  We've since then done some cleanup to provide a more proper "Table is full" error message when running into this problem.

As explained in the referenced blog posts, the problem is the result of a limitation on the size of the hash index of each partition. The hash index for each partition would allow at most ~49 million records. By default an ndbd or ndbmtd node have only 1 local query handler (LQH) block and thus 1 partition per node.  The ndbmtd nodes having

  [Read more...]
Surprises in store with ndb_restore
+1 Vote Up -1Vote Down

While doing some routine fiddling regarding some topic I've now forgotten, I discovered that ndb_restore was doing something quite surprising. It's been common wisdom for some time that one can use ndb_restore -m to restore metadata into a new cluster and automatically have your data re-partitioned across the data nodes in the destination cluster. In fact, this was the recommended procedure for adding nodes to a cluster before online add node came along. Since MySQL Cluster 7.0, though, ndb_restore hasn't behaved that way, though that change in behavior doesn't seem to be documented and most don't know that the change ever took place.

I'll go through some of the methods you can use to find information about the partitioning strategy for an NDB table, talk a bit about why ndb_restore stopped working the way most everyone expected

  [Read more...]
MySQL Cluster by Ralf Gebhardt
+0 Vote Up -0Vote Down

Ralf Gebhardt of SkySQL.

Cluster: shared nothing architecture (no single point of failure), synchronous replication between nodes, ACID transactions, row level locking. In-memory storage (some data can be stored on disk, but indexes must be in-memory). Checkpointing to disk for durability. It supports two types of indexes – ordered T-trees, unique hash indexes. Online operations like adding node groups, software upgrades, table alterations. Quick standard architecture diagram displayed about MySQL Cluster.

Network partitioning protocol is designed to avoid a split brain scenario. Is there at least one node from each node group? If not then this part cannot continue – graceful shutdown. Are all nodes present from any node group? If so, then this is the only viable cluster – continue to operate. Ask the

  [Read more...]
How to replace a NDB node on EC2
+4 Vote Up -0Vote Down

NDB cluster is a very interesting solution in term of high availability since there are no single point of failure. In an environment like EC2, where a node can disappear almost without notice, one would think that it is a good fit.

It is indeed a good fit but reality is a bit trickier. The main issue we faced is that IPs are dynamic in EC2 so when an instance restarts, it gets a new IP. What the problem with a new IP? Just change the IP in the cluster config and perform an rolling restart! no? In fact this will not work, since the cluster is already in degraded mode, restarting the surviving node of the degraded node group (NoOfReplicas=2) will cause the NDB cluster to shutdown.

This can be solved by using host names instead of IPs in the config.ini file. What needs to be done is to define, in /etc/hosts, on entry per cluster member.

  [Read more...]
MySQL Cluster 7.2 Development Milestone Release - NoSQL with Memcached and 20x Higher JOIN Performance
Employee_Team +4 Vote Up -0Vote Down

The announcements at the Oracle Collaborate and O'Reilly MySQL conferences mark an exciting milestone in the development of the MySQL Cluster database.

MySQL Cluster is already proven as a write-scalable, real-time transactional database, combining 99.999% availability with the low TCO of open source. With a distributed, multi-master architecture and no single point of failure, MySQL Cluster scales horizontally on commodity hardware to serve read and write intensive workloads.

With these enhancements announced in the Development Milestone Release, MySQL Cluster can be extended to serve a broader range of workloads.

Summary of Key Enhancements

The MySQL Cluster 7.2 Development Milestone Release and latest labs.mysql.com builds deliver enhancements based on input from the

  [Read more...]
NoSQL to MySQL with Memcached
Employee_Team +6 Vote Up -0Vote Down

The ever increasing performance demands of web-based services has generated significant interest in providing NoSQL access methods to MySQL - enabling users to maintain all of the advantages of their existing relational database infrastructure, while providing blazing fast performance for simple queries, using an API to complement regular SQL access to their data.

The HandlerSocket development at DeNA is a great example of community innovation, with a solution implemented as a custom plug-in and protocol for the MySQL server daemon.

We are hearing the community say they want NotOnly SQL - they want their trusted SQL RDBMS - plus, they want NoSQL techniques to access that data. So, we are previewing our NotOnlySQL solution for MySQL - delivered via memcached - with implementations to

  [Read more...]
MySQL Cluster 7.2 Development Milestone Release - NoSQL with Memcached and 20x Higher JOIN Performance
Employee_Team +0 Vote Up -0Vote Down

The announcements at the Oracle Collaborate and O'Reilly MySQL conferences mark an exciting milestone in the development of the MySQL Cluster database.

MySQL Cluster is already proven as a write-scalable, real-time transactional database, combining 99.999% availability with the low TCO of open source. With a distributed, multi-master architecture and no single point of failure, MySQL Cluster scales horizontally on commodity hardware to serve read and write intensive workloads.

With these enhancements announced in the Development Milestone Release, MySQL Cluster can be extended to serve a broader range of workloads.

Summary of Key Enhancements

The MySQL Cluster 7.2 Development Milestone Release and latest labs.mysql.com builds deliver enhancements based on input from the community

  [Read more...]
NoSQL to MySQL with Memcached
Employee_Team +0 Vote Up -0Vote Down

The ever increasing performance demands of web-based services has generated significant interest in providing NoSQL access methods to MySQL - enabling users to maintain all of the advantages of their existing relational database infrastructure, while providing blazing fast performance for simple queries, using an API to complement regular SQL access to their data.

The HandlerSocket development at DeNA is a great example of community innovation, with a solution implemented as a custom plug-in and protocol for the MySQL server daemon.

We are hearing the community say they want NotOnly SQL - they want their trusted SQL RDBMS - plus, they want NoSQL techniques to access that data. So, we are previewing our NotOnlySQL solution for MySQL - delivered via memcached - with implementations to access

  [Read more...]
Review of High Availability MySQL Cookbook by Packt Publishing
+1 Vote Up -0Vote Down

A few months ago, I reviewed MySQL Admin Cookbook. Today I am reviewing High Availability MySQL Cookbook from Packt Publishing by Alex Davies. Overall, I found the book to contain some good hidden Gems.

The book is a mixture of MySQL Cluster (NDB), Replication schemes, some performance tuning, some minor kernel tweaking, and some more exotic approaches to common High Availability problems. Overall, I found this book very informative and a good read.
Now the specifics, the book starts out on NDB and stays focused on



  [Read more...]
AlsoSQL
+3 Vote Up -0Vote Down

So there’s a bit of a swelling around the idea of NoSQL. That is, databases that don’t have an SQL interface in front of them – with the promise of better performance. With a well designed backend, this is no doubt the case.

A flexible query language is rather useful though. I think we’ll see the rise of AlsoSQL. That is systems that present a fast and simple protocol along with a SQL interface.

This hybrid system has seen use for many years. MySQL Cluster (http://mysql.com/products/database/cluster/) is one such example. SQL through MySQL Server, NoSQL through NDB API.

With Drizzle, I feel we’ll be in a pretty good position to offer non-sql based protocols and access methods to existing storage engines.

Continuing the journey
+3 Vote Up -0Vote Down

A couple of months ago (December 1st for those playing along at home) it marked five years to the day that I started at MySQL AB (http://www.mysql.com) (now Sun, now Oracle). A good part of me is really surprised it was for that long and other parts surprised it wasn’t longer. Through MySQL and Sun, I met some pretty amazing people, worked with some really smart ones and formed really solid and awesome friendships. Of course, not everything was perfect (sometimes not even close), but we did have some fun.

Up until November 2008 (that’s 3 years and 11 months for those playing at home) I worked on MySQL Cluster (http://www.mysql.com/cluster). Still love the product and love how much better we’re making Drizzle so it’ll be the best SQL interface to NDB :)

The ideas behind Drizzle

  [Read more...]
NDB$INFO with SQL hits beta
+4 Vote Up -0Vote Down

Bernhard blogged over at http://ocklin.blogspot.com/2010/02/mysql-cluster-711-is-there.html that MySQL Cluster 7.1.1 Beta has been released. The big feature (from my point of view) is the SQL interface on top of NDB$INFO. This means there is now full infrastructure from the NDB data nodes right out to SQL in the MySQL Server for adding monitoring to any bit of the internals of the data nodes.

Drizzle FRM replacement: the table proto
+0 Vote Up -0Vote Down

Drizzle originally inherited the FRM file from MySQL (which inherited it from UNIREG). The FRM file stores metadata about a table; what columns it has, what type those columns are, what indexes, any default values, comments etc are all stored in the FRM. In the days of MyISAM, this worked relatively well. The row data was stored in table.MYD, indexes on top of it in table.MYI and information about the format of the row was
in table.FRM. Since MyISAM itself wasn’t crash safe, it didn’t really matter if creating/deleting the FRM file along with the table was either.

As more sophisticated engines were introduced (e.g. InnoDB) that had their own data dictionary, there started to be more of a problem. There were now two places storing information about a table: the FRM file and the data dictionary specific to the engine. Even if the data


  [Read more...]
NDB API examples, the trivia to make ndbapi_scan works!
+0 Vote Up -0Vote Down

When I did NDB Cluster engagements, it happened a few time that I had to cover the NDB API. Once, a customer asked an example on how to perform a simple scan. Remembering that some examples are within the NDB source tree, under ./storage/ndb/ndbapi-examples/ and I decided to take a look, why writing something when a good example already exists... I was not expecting what I found!

1. Missing headers


$ make
g++ -g -O0 -c -Wall -fno-rtti -fno-exceptions -I/usr/include -I../../../../include -I../../../../storage/ndb/include -I../../../../storage/ndb/include/ndbapi ndbapi_scan.cpp
ndbapi_scan.cpp: In constructor ‘Car::Car()’:
ndbapi_scan.cpp:111: error: ‘memset’ was not declared in this scope
ndbapi_scan.cpp: In function ‘void drop_table(MYSQL&)’:
ndbapi_scan.cpp:124: error:





  [Read more...]
New Open Query training days in Australia
+0 Vote Up -0Vote Down

The favourite Open Query course modules as well as reworked and brand new ones, with November/December 2009 dates for Brisbane, Sydney, Canberra and Melbourne listed below. You can register for days/modules individually, to suit your time, budget and current needs. Your trainers are Sean, Ray and Arjen (see OQ people).

For the Canberra and Melbourne days which are DBA/HA, registrations for all of the modules in a series before 15 October will receive a copy of the “High Performance MySQL” book (normal bookstore price is AUD 105).

Canberra

  • Thu 5 Nov: MySQL High Availability – Strategy and Tools
  • Fri 6 Nov: MySQL Cluster
  [Read more...]
NDB Kernel size over releases
+1 Vote Up -0Vote Down

So Jonas pointed out that the NDB kernel hasn’t changed too much in size over releases. Let’s have a look:

In fact, the size went down slightly from 4.1 to 5.0. In this, 6.4 and 7.0 are the same thing but appear twice for completeness.

You can see the raw results in the spreadsheet here.

Feedback from MySQL Cluster tutorial
+0 Vote Up -0Vote Down

Way back on Monday (at the MySQL Conference and Expo), I gave a full day tutorial on MySQL Cluster (http://www.mysql.com/cluster). I awoke early in the morning to a “oh ha ha” URL in an IM; but no, it wasn’t jetlag playing tricks with me. Luckily, this didn’t take much (if anything) away from the purpose of the day: teaching people about NDB.

Distracting-and-this-time-really-annoying-thing-of-the-day-2: It seems that O’Reilly had cut back on power this year, and there were no power boards in the room. A full day interactive tutorial, and nowhere to plug in laptops. Hrrm.. Luckily, having over the many years I’ve been speaking at

  [Read more...]
Data on MySQL Performance
Employee +1 Vote Up -0Vote Down
If you like to sift through tons of benchmark data about various
MySQL versions, Dimitri at the Sun Benchmark Labs have published
a serious amount of benchmark data in a report published here.

The report shows that the new MySQL 5.4.0 release
have a very good performance. The report also shows how the day
of a developer of performance improvements and the massive amount
of benchmark data that needs to be analysed and sifted through
to understand the impact of new performance improvements.

I personally met Dimitri the first time in 2002 when I was working
together with him for a couple of weeks on a benchmark on NDB Cluster
(the storage engine of MySQL Cluster). Our goal then was to perform
1 million reads per second on a 72-cpu












  [Read more...]
MySQL Cluster Tutorial
+0 Vote Up -0Vote Down

This year I am again giving a MySQL Cluster Tutorial at the MySQL Conference and Expo. As those who have attended before can tell you, this is a hands on tutorial. I don’t just stand up the front and talk at you for a day, that would be very boring (for all of us). While there is a good amount of presented material (there is a decent amount of theory to get through), there is a large component that involves setting up a cluster, putting data in, getting data out, backup, restore.

So if you’re wanting to learn about MySQL Cluster in a nice and friendly hands-on environment, I can recommend coming to my tutorial.

The tutorial isn’t the be-all and end-all tutorial. It does not teach you everything. It does give you a decent introduction though.

linux.conf.au 2009 wrap-up (incl Open Source Databases Mini-conf): Day 0-1
+0 Vote Up -0Vote Down

It’s no secret that I love linux.conf.au. My first was linux.conf.au 2003, in Perth and I’ve been to every one since (there are at least two people who’ve been to every single one, including CALU as it was called in 1999).

I’ve been on the board of Linux Australia for some insane proportion of the years since then (joining in 2003). Linux Australia is the not-for-profit community organisation that puts on linux.conf.au. It’s all volunteers and amazingly enough we have more than one group of people wanting to put on linux.conf.au each year!

This year, we Marched South to

  [Read more...]
row id in MySQL and Drizzle (and the engines)
+0 Vote Up -0Vote Down

Some database engines have a fundamental concept of a row id. The row id is everything you need to know to locate a row. Common uses include secondary indexes (key is what’s indexed, value is rowid which you then use to lookup the row).

One design is the InnoDB method of having secondary indexes have the value in the index be the primary key of the row. Another is to store the rowid instead. Usually (or often… or sometimes…) rowid is much smaller than the pkey of the row. This is how innodb can answer some queries just out of the index. If it used rowid, it may involve more IO to answer the query. All this is irrelevant if you never want just the primary key from a secondary index.

Some engines are designed from the start to have rowid, others it’s added later (e.g. NDB).

Anyway… all beside the point. Did you

  [Read more...]
Showing entries 1 to 30 of 39 Next 9 Older Entries

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.