Read the original article at How to Optimize MySQL UNION For High Speed

Join 6100 others and follow Sean Hull on twitter @hullsean. There are two ways to speedup UNIONs in a MySQL database. First use UNION ALL if at all possible, and second try to push down your conditions. 1. UNION ALL is much faster than UNION How does a UNION work? Imagine you have two tables [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at The Most Important AWS Feature for Performance and Scalability

Join 6100 others and follow Sean Hull on twitter @hullsean. The Foundation of Speed All servers use disk to store files. Operating system libraries, webserver & application code, and most importantly databases all use disk constantly. So disk speed is crucial to server speed. Also check out: Five more things Deadly to Scalability. Disk Performance [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Five More Things Deadly to Scalability

Join 6000 others and follow Sean Hull on twitter @hullsean. 1. Slow Disk I/O – RAID 5 – Multi-tenant EBS Disk is the grounding of all your servers, and the base of their performance. True with larger and larger main memory, much is available in cache, a server still needs to constantly read from disk [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Make MySQL clustering work for you

We’ve told you all about MySQL mult-master replication’s limitations. If you write to two masters it is bound to fail for myriad reasons. Now what? Do what the pros do that’s what. A. Don’t write to both masters Using multi-master replication works great as long as you do so in active-passive mode. Never write to [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at No tools to reconcile MySQL with two masters

Here’s the last nail in the coffin. We hope this convinces you not to write to multiple masters with MySQL replication. Reason 10 – No tools to reconcile inconsistent masters For all the endless reasons we’ve already outlined MySQL replication is prone to failure. We know it’s going to happen, you now know too. When [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Temp tables can break replication

If you’re not convinced yet that writing to dual masters is a bad idea, we have a couple more reasons. 9. Temp tables break replication after restart MySQL’s replication is sensitive to temporary tables. You shouldn’t use them. If your queries create them to work against, and a node crashes, the temp tables will be [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Crashed nodes corrupt your MySQL cluster

Writing to two masters is like walking around with a loaded shotgun. Eventually one of your instances will fail and when it does, replications position & synchronization information could easily become corrupt! Reason 8 – Crashed nodes cause big problems MySQL instances, unfortunately can crash. When that happens, they don’t always sync the replication position [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Transaction isolation breaks when writing two masters

Continuing our discussion of multi-master replication, we hit on five more reasons why writing to two masters aka active-active replication is very dangerous. Click through to the end for multi-master solutions that work with MySQL. Reason 6 – You lose transaction isolation This may sound like a theoretical point to some. But hopefully we can [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Active-active replication reduces operational flexibility

Among the myriad data integrity and corruption risks associated with active-active replication, you also lose out on configurability and operational flexibility. Reason 7 – Can’t add nodes easily The often touted solution to use auto_increment_increment and auto_increment_offset fixes you to a given setup forever. You can set it up with a discrete two nodes, all [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at How to make MySQL multi-master work for you

Coming soon! Want more? Grab our Scalable Startups monthly for more tips and special content. Here’s a sample Related posts:Transaction isolation breaks when writing two masters MySQL requires an authoritative master to build slaves MySQL needs single master to check data integrity A master isn’t born but made Why does MySQL replication fail?

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at MySQL needs single master to check data integrity

MySQL slaves can drift out of sync. Many of our clients are surprised to find some data differences in their replication topology, once we do some checking and sniffing around. Such checks require a single reliable or authoritative master to compare against. Click through to the end for multi-master solutions that work with MySQL. Reason [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at MySQL requires an authoritative master to build slaves

In MySQL database operations, you often need to rebuild slaves. They fail for a lot of different reasons, fall out of sync or crash. When this happens you may find you need to reclone and start fresh. This is normally done by finding your authoritative master database, and doing a hotbackup. Click through to the [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Limitations of MySQL row-based replication

MySQL offers a few different options for how you perform replication. Statement-based has been around a lot longer, and though it has some troublesome characteristics they’re known well and can be managed. What’s more it supports online schema changes with multi-master active-passive setup. We recommend this solution. Row-based replication is newer. It attempts to address [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Why does MySQL replication fail?

When considering active-active multi-master, you must consider it’s foundation technology. Although MySQL replication is straightforward to setup, it can fail in a myriad of ways. Most of those are known and well understood. We can solve them only if we use the technology in the standard way. Click through to the end for multi-master solutions [...]

For more articles like these go to Sean Hull's Scalable Startups

As alway with new technology there is always different approaches regarding the adoption. You can try to use the bleeding edge features or start with a very standard configuration. My personal advise to new users is to start with the most basic configuration.

This allow you to get familiar with the fundamentals : - how [...]

Most of the time, when people say “scalability” they mean any of dozens of things. Most of the time, when I say it I mean exactly one precisely defined thing. However, I don’t claim that’s the only correct use of “scalability.” There is another, in particular, that I think is very important to understand: the inherent limitations of the system. This second one doesn’t have a single mathematical definition, but it’s vital nonetheless.

I’ll frame the discussion by asking this: how scalable is your database?

Using the two definitions I like to use the most, I answer the question in this way.

On February 28 at 1pm EST I’ll be taking part in a webinar, sponsored by ScaleBase, on strategies for scaling MySQL.

Scalability is one of the primary drivers we’ve seen for database users considering alternatives to traditional relational databases. That could mean adopting an entirely new database for new projects or – more likely for existing applications – looking at various strategies for improving the scalability of an existing database.

During the webinar I will be joined by Doron Levari and Paul Campaniello, both from ScaleBase, which enables applications to scale without disruption to the existing infrastructure. We’ll be discussing, amongst other things:

• Scaling-out your MySQL databases
• New

I’ve written a lot about modeling MySQL with the USL, and I like it best of all the scalability models I’ve seen, but it’s not the only way to think about scalability. I was aware that New Relic supports a scalability chart, so I decided to take a peek at that. Here’s a screenshot of the chart, from their blog:

Here’s how it works. It plots response time (or database time, or CPU) as the dependent variable, versus throughput as the independent variable. There’s a line through it to indicate the general shape. Samples are charted as points in a scatter plot. The points are color-coded by the time of day. Outliers

Last time I said that you can set a starting value for the USL’s coefficient of performance and let your modeling software (R, gnuplot, etc) manipulate this as part of the regression to find the best fit. However, there is a subtlety in the USL model that you need to be aware of. Here is a picture of the low-end of the curve:

The graph shows the USL model as the blue curve and linear scalability as the black line. Notice that at concurrencies less than 1, the value of the USL function is actually greater than the linear scalability function. This deserves some thought and explanation, because it can cause problems.

If you think about

Last time I said that the USL has a forgotten third coefficient, the coefficient of performance. This is the same thing as the system’s throughput at concurrency=1, or C(1). How do you determine this coefficient? There are at least three ways.

Neil Gunther’s writings, or at least those that I’ve read and remember, say that you should set it equal to your measurement of C(1). Most of his writing discusses a handful of measurements of the system: one at concurrency 1, and at least 4 to 6 at higher concurrencies. I can’t remember a time when he’s discussed taking more than one measurement of throughput at each level of concurrency, so I think the assumption is that you’re going to take a single

Read the original article at Why your cloud is speeding for a scalability cliff

Also find Sean Hull’s ramblings on twitter @hullsean. Don’t believe me that you’re headed for the cliff? A startup scales up to no avail Towards the end of 2012 I worked with an internet startup in the online education space. Their web application was not unusual, built in PHP and using Linux, Apache & Mysql [...]

For more articles like these go to Sean Hull's Scalable Startups

If you’re familiar with Neil Gunther’s Universal Scalability Law, you may have heard it said that there are two coefficients, variously called alpha and beta or sigma and kappa. There are actually three coefficients, though. See?

No, you don’t see it — but it’s actually there, as a hidden “1″ multiplied by N in the numerator on the right-hand side. When you’re using the USL to model a system’s scalability, you need to use the C(1), the “capacity at one,” as a multiplier. I call this the coefficient of performance. It’s rarely 1; it’s usually thousands.

To illustrate why this matters, consider two systems’ throughput as load increases:

Read the original article at Why do people leave consulting?

As a long time freelancer, it’s a question that’s intrigued me for some time. I do have some theories… First, definitions… I’m not talking about working for a large consulting firm. Although this role may be called “consultant”, my meaning is consultant as sole proprietor, entrepreneur, gun for hire or lone wolf. 1. Make more [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Anatomy of a Performance Review

A lot of firms come to us with a specific scalability problem. “Our user base is growing rapidly and the website is falling over!” Or they’re selling more widgets, “Our shopping cart is slowing down and we’re seeing users abandon their purchases”. These are real startup growing pains, so what to do?

We like to take a measured approach with these types of challenges, so we thought it would be helpful to run through a hypothetical scenario and see how we work.

Having trouble with scalability? Check out our

Read the original article at Why you should attend Percona Live 2012

What I loved about Percona Live 2011 Last year I was excited to go to Percona Live for the first time in NYC. I arrived just in time to hear Harrison Fisk from Facebook speak about some of the awesome tweaks they’re running with MySQL there. It’s not everyday that you get to hear from [...]

For more articles like these go to Sean Hull's Scalable Startups

Read the original article at Upcoming for Scalable Startups

Just back from the Labor Day holiday, and ready to dive back in.

I thought this would be a great time to outline some of our upcoming topics so here goes…

1. Why Oracle usability sucks

- a rant about Oracle’s weak points

In the meantime take a peek at our piece on why we wrote the book on Oracle & Open Source. We ruminate on trends in the datacenter and take a stab at Oracle’s future.

2. Why relational databases don’t scale

- Is there any such thing as automatic scalability?
- What blocks scalability?
- Are NoSQL databases magic?

Also one of our articles that went viral –