English
| Showing entries 1 to 25 |
I know, I haven’t been posting much lately. 5.5 upgrades got postponed due to the new storage platform needing my immediate attention and being a speaker at the Percona Live conference in April also needs a lot of attention.
One of the things I want to try out is running multiple MySQL instances on the same machine. The concept remained in the back of my mind ever since I attended Ryan Thiessen’s presentation on the MySQL conference 2011 but we never actually got a proper usecase for it. Well, with the new storage platform it would be really beneficial so an excellent use case to try it out! So what have I been
[Read more...]
-0
Due to one of the machines being configured a little bit too close to the edge we had to restart it to downsize its innodb_buffer_pool_size a little bit. We tested the innodb_lru_dump_restore directive with this machine. Results were very promising: it wrote the dump every 5 minutes (we set the variable to 300 seconds) and after MySQL had restarted it reloaded the dump.
However: since MySQL is already started fully before it even starts to load the LRU dump it means MySQL is already available to the outside world. This means in a HA environment it would already be going to perform poorly due to the torrent of queries coming in. This means either the loading of the LRU dump needs to be done up front by a change in Percona Server or we need to alter MMM not to do anything with the server untill it has loaded the LRU dump.
Challenges, challenges…
Henrik just wrote a good article on different ways of achieving high availability with MySQL. I was going to respond in the comments, but decided it is better not to post such a long comment there.
One of the questions I think is useful to ask is what kind of high availability is desired. It is quite possible for a group of several people to stand in a hallway and talk about high availability, all of them apparently discussing the same thing but really talking about very different things.
Henrik says “At MySQL/Sun we recommended against asynchronous replication as a HA solution so that was the end of it as far as MMM was concerned. Instead we recommended DRBD, shared disk or MySQL Cluster based solutions.” Notice that all of those are synchronous
[Read more...]This month on Database Journal we talk about multi-master MySQL using circular replication to achieve high availability.
This is a Request for Input. Dual MySQL masters with MMM in a single datacentre are in common use, and other setups like DRBD and of course VM/SAN based failover solutions are conceptually straightforward also. Thus, achieving various forms of resilience within a single data-centre is doable and not costly.
Doing the same across multiple (let’s for simplicity sake limit it to two) datacentres is another matter. MySQL replication works well across longer links, and it can use MySQL’s in-built SSL or tools like stunnel. Of course it needs to be kept an eye on, as usual, but since it’s asynchronous the latency between the datacentres is not a big issue (apart from the fact that the second server gets up-to-date a little bit later).
But as those who have tried will know, having a client (application server) connection to a MySQL instance in a remote data-centre
[Read more...]A lot of systems are relatively easy to make HA (highly available). You just slap them into a well-known HA framework such as Linux-HA and you’re done. But databases are different, especially replicated databases, especially replicated MySQL.
The reason has to do with some properties that hold for many systems, but not for most databases. Most systems that you want to make HA are relatively lightweight and interchangeable, with little to zero statefulness, easy to start, easy to stop, don’t care a lot about storage (or at least don’t write a lot of data; that’s usually delegated to the database), and there’s little or no harm done if you ruthlessly behead them. The classic example is a web server or even most application
Walter and I are giving a tutorial on Monday morning, MySQL (and MariaDB) Dual Master Setups with MMM, I believe there are still some seats available – tutorials are a bit extra when you register for the conference, so you do need to sign up if you want to be there! It’s a hands-on tutorial/workshop, we’ll be setting up multiple clusters with dual master and the whole rest of the MMM fun, using VMs on your laptops and a separate wired network. Nothing beats messing with something live, breaking it, and seeing what happens!
Then on Tuesday afternoon (5:15pm, Ballroom F), Antony and I will do a session on the
[Read more...]The next presentation by Piotr Biel from Percona was on Multi-Master Manager for MySQL.
The introduction included a discussion of the popular MySQL HA solutions including:
A key problem that was clarified in the talk is the discussion of Multi-Master and this IS NOT master-master. You only write to a single node. With MySQL is this critical because MySQL replication does not manage collision detection.
The MMM Cluster Elements are:
And the Application Components are:
MMM works with 3 layers.
-0
There is a nagios plugin available on the MMM's google-code page, but if you didn't find it yet, here it is:
http://code.google.com/p/check-mysql-all/wiki/check_mmm
You can call this plugin over nrpe. I'm already working on to fork a version which more useful with passive checks.
This plugin was developed by Ryan Lowe (Percona).
I have been very busy here in Malaysia this week. On thursday, I was asked to do a MySQL University session on MMM. The preparation was very stressful. There was no good wifi to be found until literally a few hours before the session (Big thank you to Gurdip at APIIT for providing a space and exceptional help!). On top of that, dimdim, the software used by MySQL for their sessions doesn’t seem to want to work on Linux (particularly the speaker part). I ended up using a laptop borrowed from APIIT with Vista and IE. Feels kind of counter-intuitive for a company in the FOSS business.
The session went very well and here is the resulting recording of the MMM talk on the mysqlforge page.
But that wasn’t the end of the MMM-promotion week: I happened to be allowed to present at the
[Read more...]This Thursday (October 22nd, 13:00 UTC), Walter Heck (of Open Query) will present Dual Master Setups With MMM. MMM (Multi-Master Replication Manager for MySQL) is a set of flexible scripts to perform monitoring/failover and management of MySQL master-master replication configurations (with only one node writable at any time). Session slides (PDF).
The toolset also has the ability to read balance standard master/slave configurations with any number of slaves, so you can use it to move virtual IP addresses around a group of servers depending on whether they are behind in replication. For more
information, see mysql-mmm.org.
For MySQL University sessions you point your
[Read more...]
This Thursday (October 22nd, 13:00 UTC), Walter Heck will present Dual Master Setups With MMM. MMM (Multi-Master Replication Manager for MySQL) is a set of flexible scripts to perform monitoring/failover and management of MySQL master-master replication configurations (with only one node writable at any time). The toolset also has the ability to read balance standard master/slave configurations with any number of slaves, so you can use it to move virtual IP addresses around a group of servers depending on whether they are behind in replication. For more information, see http://mysql-mmm.org/.
For MySQL University sessions,
[Read more...]
This Thursday (October 22nd, 13:00 UTC), Walter Heck will present Dual Master Setups With MMM. MMM (Multi-Master Replication Manager for MySQL) is a set of flexible scripts to perform monitoring/failover and management of MySQL master-master replication configurations (with only one node writable at any time). The toolset also has the ability to read balance standard master/slave configurations with any number of slaves, so you can use it to move virtual IP addresses around a group of servers depending on whether they are behind in replication. For more information, see http://mysql-mmm.org/.
For MySQL University
[Read more...]
This Thursday (October 22nd, 13:00 UTC), Walter Heck will present Dual Master Setups With MMM. MMM (Multi-Master Replication Manager for MySQL) is a set of flexible scripts to perform monitoring/failover and management of MySQL master-master replication configurations (with only one node writable at any time). The toolset also has the ability to read balance standard master/slave configurations with any number of slaves, so you can use it to move virtual IP addresses around a group of servers depending on whether they are behind in replication. For more information, see http://mysql-mmm.org/.
For MySQL University
[Read more...]This is a “dogfood” type story (see below for explanation of the term)… Open Query has ideas on resilient architecture which it teaches (training) and recommends (consulting, support) to clients and the general public (blog, conferences, user group talks). Like many other businesses, when we first started we set up our infrastructure quickly and on the cheap, and it’s grown since. That’s how things grow naturally, and is as always a trade-off between keeping your business running and developing while also improving infrastructure (business processes and technical).
Quite a few months ago we also started investing (mostly time) in the technical infrastructure, and slowly moving the various systems across to new servers and splitting things up along the way. Around the same time, the main webserver frequently became unresponsive. I’ll spare
[Read more...]We all knew that we are risking with MMM. Risking, and placing availability as a more important like consistency. But non of us can risk loosing data forever but we show using it, regarding to our conversations think: "I can fix my data later on, but I can’t turn back time and prevent the downtime. (Pascal Hofmann@xaprb.com)".
As I wrote before about staying online, now let me write about how to stay consistent.
We all know, mmm is not like a key of salvation, but its getting close to it 
. While MySQL doesn't support multi-master-slave environments from it's source code, we will sleep badly wondering on the safety of our precious databases.
But its not just about MMM, a few days ago we ran in to a well known InnoDB
[Read more...]I’ve been thinking recently about the failure scenarios of MySQL replication clusters, such as master-master pairs or master-master-with-slaves. There are a few tools that are designed to help manage failover and load balancing in such clusters, by moving virtual IP addresses around. The ones I’m familiar with don’t always do the right thing when an irregularity is detected. I’ve been debating what the best way to do replication clustering with automatic failover really is.
I’d like to hear your thoughts on the following question: what types of scenarios require what kind of response from such a tool?
I can think of a number of failures. Let me give just a few simple examples in a master-master pair:
Problem: Query overload on the writable master makes mysqld unresponsive Do nothing. [Read more...]In this blog, I would like to mark myself as a power-user of MMM. For more then 3 years now I'm engineering high-traffic websites and none of them was a small project. In the beginning I worked behind the livejasmin.com project (warning: NSFW!) which is an Alexa top100 site and now I'm working at ustream.tv which is in the top500 (so far). In this area, your greatest enemy is downtime. Thousands of users want to see your dynamic content by executing thousand of queries against your MySQL servers and what they don't have is the patience and they don't even tolerate critical security-upgrades. In both sites I mentioned above there is no point in time during the day at which less then 10k visitors are hitting the sites. Now one question comes up: How can you do an upgrade on your MySQL servers or how can you alter a larger table?
So you have a new feature and that means
This is the first post on the brand spanking new MMM for MySQL Community Blog. Expect posts on new features for this great project. Expect user experiences.
So, your question is of course: what is MMM for MySQL and why haven't I heard of this before?
Well, MMM for MySQL is a project that strives to provide a HA Solution for MySQL, with automatic failover between multiple masters and slaves. If set up properly, you don't have to worry anymore over downtime for upgrading to a new version of MySQL, or a new version of your application (=> large alter tables, addition of indexes, etc.)
The project has recently reached it's version 2, which is a complete rewrite from scratch. We are now slowly seeing this being put into production, with some success stories making us happy already.
I won't make it too long here, but look forward to hear more from us!
-0
It seems pretty common to find customers install DRBD for the wrong reasons. There are many pros/cons to compare DRBD to replication, but I've managed to cut down my spiel I give to customers to these two points:
So if you are installing DRBD with the aim of purely "availability", and are not worried about losing that last write on the crash to your master database that (hopefully) happens only once every few years, you may be using the wrong technology.
While the prized "1 minute failover" is possible in DRBD, it doesn't
[Read more...]
-0
New alpha release 1.0-pre4 of the MySQL Master-Master Replication Manager. This release has lots of major fixes and I’m glad to announce first sponsored port of mmm to non-linux platform - it has been ported to Solaris 10. So, here are our changes in this version:
| Showing entries 1 to 25 |
