We’ve had a couple of issues with some of our server infrastructure recently, which have affected portions of our customer base. In this blog post I want to explain what has happened, why, and what we’re doing to correct and prevent it.

I am writing a combined report of these issues because the first one wasn’t fully understood when the second one happened, and because the issues largely have the same contributing factors.

I apologize to our customers who have been impacted. Monitoring is supposed to be more highly available than the monitored systems. I know firsthand how damaging it can be when you can’t access your monitoring data. I take this very seriously and the whole team is working hard to prevent it from recurring.

Summary of Incidents

• On November 15, 2015, from 19:15 until 22:30 Eastern time, some customer data ingest was delayed. Up to 25% of customer environments were affected at …

This is part 2 of our blog on how to troubleshoot Galera cluster - SST errors, and problems with network streaming. In part 1 of the blog, we covered issues ranging from node crashes to clusters that won’t restart, network splits and inconsistent data. Note that the issues described are all examples inspired from real life incidents in production environments.

This is the eighteenth installment in the ‘Become a MySQL DBA’ blog series. Our previous posts in the DBA series include:

• Troubleshooting Galera cluster issues - part 1,
• …

works like charm, thanks

Indexes are usually built by way of a data structure; typically, that structure takes the form of a “tree.” Most commonly, the structure of choice is a B-Tree, which is a hierarchical organization defined by the arrangement and interactions of its nodes.

High in a B-Tree’s hierarchy, you find a root node. Each of the items in that root node points to a collection of items in a child node. This can go on for multiple levels. Eventually, you reach the leaf nodes, where the data itself is stored. Those leaf nodes point back to the rows in the main table. To be technology specific, for instance, in MyISAM, those leaf nodes have offsets in the main data file – the main data file in a MyISAM table is just a sequential collection of rows.

The purpose of an index is to allow a user to very quickly traverse just such a structure – whatever form that structure or algorithm might take – and to locate and examine rows or ranges of …

The MariaDB project is pleased to announce the immediate availability of MariaDB Galera Cluster 10.0.22 and updates to MariaDB Connector/J, MariaDB Connector/C, and MariaDB Connector/ODBC. See the release notes and changelogs for details on each release. Download MariaDB Galera Cluster 10.0.22 Release Notes Changelog What is MariaDB Galera Cluster? MariaDB APT and YUM Repository Configuration […]

The post MariaDB Galera Cluster 10.0.22 and Connector updates appeared first on MariaDB.org.

Thu, 2015-11-19 09:03markusmakela

We’ve gotten a few questions on detailed installation tutorials for specific systems. So I decided to write one for CentOS 7 which is a pretty common OS. For this I’m using a fresh installation of CentOS 7 on a virtual machine.

Installing needed packages

The first things we need to get are tools required for getting MaxScale and building it. The Development Tools group in Yum has most of the required tools we need to build MaxScale. We also need a few other packages to build MaxScale.

The list of required packages for each distribution can be found from the documentation: Building-MaxScale-from-Source-Code

Installing packages:

yum groupinstall "Development Tools"
yum install gcc gcc-c++ ncurses-devel …

InnoDB holepunch experiments After excellent blogs by Mark Callaghan (see links below), I decided to use some of my time to experiment how different filesystems behave if the holepunch feature is used in MariaDB 10.1. First of all, MariaDB 10.1 does not use holepunch by default even if a table is page compressed (a term […]

The post InnoDB holepunch compression vs the filesystem in MariaDB 10.1 appeared first on MariaDB.org.

I am always fascinated by the cleanliness of UNIX. One tool only should do one thing, but it has to be the best in that way. The operating system itself will glue all the modules together and give you a complex feel of a system, you don’t have to take care of huge, bloated software, don’t have to deal with mysterious bugs, which are appearing random. Just small bricks of clever software and the rest are on you.

Recently I broke this, and frankly, I am not sure if it was a bad decision, or not.

With MySQL, if you want a point in time recovery after a disaster, you should back up not only the database but the binary logs themselves, as I have mentioned it in my last blog post. I added a feature to the binlogstreamer: it can clean up the binary logs after a given amount of time.

Let’s see how many ways we have to get rid of the old, unwanted …

Amazon EBS volumes come with a very cool feature called "lazy loading". In a nutshell: if a volume is created from an existing snapshot, it can become available almost immediately without waiting for all data to be restored. This allows for extremely fast provisioning of large data sets as long as you don't explicitly require the entire data set to be present before you start using it.
When an EBS volume is restored from snapshot, its blocks are fetched from Amazon S3. It happens either lazily in the background or explicitly on demand (think of a pagefault-like mechanism) and of course, fetching pieces of data from Amazon S3 is going to be one-two orders of magnitude slower than reading blocks directly from a volume.
In this short article, I will try to give you an idea of how this may impact the crash recovery time of your MySQL databases. Why talk about this? Depending on the workload and data set layout, crash recovery of a MySQL …

PinLater is an asynchronous job execution system we built as a replacement for our original solution based on Pyres. We’ve been using PinLater in production for over two years, and it’s one of our most mission critical systems. Usage of PinLater spans from core actions like Pinning, image thumbnail generation and when Pinners follow other Pinners, boards or topics, to larger batch operations like email delivery, push notifications and visual discovery workflows. We now operate more than 10 different clusters all on Amazon EC2. In aggregate, there are more than 500 job queues processing north of six million jobs per minute. Today we’re excited to announce PinLater as our latest …