In case you’ve been living under a rock for the last 5 years, the NoSQL movement has changed. There was a time when everyone — EVERYONE — was dumping on relational databases, and MySQL in particular. Nonsense like “SQL itself is inherently unscalable” routinely came out of the mouths of otherwise usually sensible people. But that’s cooled off a little bit, thank heavens.

And what’s the new hotness? Well, Big Data, of course! But I digress. In the world of databases, it’s move over NoSQL, heeeeeere’s NewSQL. I’m talkin’ NuoDB, Clustrix, MySQL Cluster (NDB), and so forth. A lot of people now recognize that it wasn’t SQL or the relational model that was a problem — it was the implementations that had some issues. The pendulum has swung a little away from vilifying SQL, and we don’t talk about NoSQL as much as we talk about document-oriented or …

I’ve written a lot about modeling MySQL with the USL, and I like it best of all the scalability models I’ve seen, but it’s not the only way to think about scalability. I was aware that New Relic supports a scalability chart, so I decided to take a peek at that. Here’s a screenshot of the chart, from their blog:

Here’s how it works. It plots response time (or database time, or CPU) as the dependent variable, versus throughput as the independent variable. There’s a line through it to indicate the general shape. Samples are charted as points in a scatter plot. The points are color-coded by the time of day. Outliers are automatically removed.

The focus on response time is really good. That’s one of the things I like about New Relic. While most systems show people status counters, and imply …

Last time I said that you can set a starting value for the USL’s coefficient of performance and let your modeling software (R, gnuplot, etc) manipulate this as part of the regression to find the best fit. However, there is a subtlety in the USL model that you need to be aware of. Here is a picture of the low-end of the curve:

The graph shows the USL model as the blue curve and linear scalability as the black line. Notice that at concurrencies less than 1, the value of the USL function is actually greater than the linear scalability function. This deserves some thought and explanation, because it can cause problems.

If you think about it, concurrency between one and zero is impossible. In fact, concurrency is not a smooth function, it is a step function. There can be zero requests resident in the system, one …

Last time I said that the USL has a forgotten third coefficient, the coefficient of performance. This is the same thing as the system’s throughput at concurrency=1, or C(1). How do you determine this coefficient? There are at least three ways.

Neil Gunther’s writings, or at least those that I’ve read and remember, say that you should set it equal to your measurement of C(1). Most of his writing discusses a handful of measurements of the system: one at concurrency 1, and at least 4 to 6 at higher concurrencies. I can’t remember a time when he’s discussed taking more than one measurement of throughput at each level of concurrency, so I think the assumption is that you’re going to take a single measurement at various concurrencies (or, in the case of hardware scalability, units of hardware), and …

If you’re familiar with Neil Gunther’s Universal Scalability Law, you may have heard it said that there are two coefficients, variously called alpha and beta or sigma and kappa. There are actually three coefficients, though. See?

No, you don’t see it — but it’s actually there, as a hidden “1″ multiplied by N in the numerator on the right-hand side. When you’re using the USL to model a system’s scalability, you need to use the C(1), the “capacity at one,” as a multiplier. I call this the coefficient of performance. It’s rarely 1; it’s usually thousands.

To illustrate why this matters, consider two systems’ throughput as load increases:

The green line and the blue line are both linearly scalable systems. Add twice the concurrency, get twice the throughput. But the slope of the lines is different. The green system can do three times as much work as the blue system, even though it’s no more …

Moral of the upcoming story: when your systems rely on someone else’s software, make sure you test it thoroughly to understand how it works.

I found a couple of interesting things about the go-mysql-driver driver for Go. One is that it prepares every query before executing it, and then closes it afterwards, if you’re just using the db.Query() or db.QueryRow() functionality. There is zero benefit to this; Bill Karwin has probably listed it as an antipattern somewhere. I asked for one query, but I got three.

The other is that it doesn’t open the database connection when you call db.Open(). You can call that function and get no error. The first query on the resulting “db” object will actually connect to MySQL. Thus it’s actually kind of like the other major opensource Go/MySQL driver ( …

Those who’ve been around the MySQL world are probably aware of the much-discussed topics of GPL licensing, dual licensing, and in particular, licensing of the client libraries (also called connectors or drivers) and the FOSS exception to that licensing. This is newly relevant with the announcement of a permissively-licensed MySQL-compatible client library for MariaDB.

The difference is that this time there’s been some question about the provenance and history of the source code. Some people asked me about this. Some of them were aware of a relatively obscure detail: there’ve been permissively licensed MySQL client libraries for years, in the form of libdrizzle, a BSD-licensed library for the Drizzle fork of MySQL.

Here are some of the thoughts that seemed to be going through peoples’ minds:

• This changes everything, …

I was just bitten by failing to catch a MySQL warning. It’s the old familiar tune: I inserted 100 characters into a VARCHAR(50) and it didn’t throw an error*. Of course, then subsequent SELECT statements didn’t find the value I inserted.

What’s different this time is that I was using Go as the client. There is no single official MySQL driver for Go, although there are several good-quality community-maintained ones. I was using one of those through the official Go database interface, which is a simple and lightweight way to interact with relational databases. This interface will generate errors, but I didn’t think about warnings. This is funny, because usually I’m paranoid about capturing warnings from MySQL and treating them as errors.

After I discovered my mistake, I realized that Go’s database interface doesn’t provide a way to observe the warnings …

I spoke last Friday at DevIgnition about what’s coming in the MySQL 5.6 release. My slides are embedded below.

Further Reading:

• Presentation uploaded for Maatkit talk at MySQL Conference
• Videos and slides for MySQL Conference 2010
• Surge 2011 slides, recap
• Slides for my MySQL Conference talks are online
• …

Back when Sun Microsystems was setting, some of the programmers who had been involved with the popular and well-known open source MySQL database started a fork of the project called MariaDB.

read more