In this blog post, we’ll discuss services monitoring using probabilistic fault detection.

Let’s admit it, the task of monitoring services is one of the most difficult. It is time-consuming, error-prone and difficult to automate. The usual monitoring approach has been pretty straightforward in the last few years: setup a service like Nagios, or pay money to get a cloud-based monitoring tool. Then choose the metrics you are interested in and set the thresholds. This is a manual process that works when you have a small number of services and servers, and you know exactly how they behave and what you should monitor. These days, we have hundred of servers with thousands of services sending us millions of metrics. That is the first problem: the manual approach to configuration doesn’t work.

That is not the only problem. We know that no two servers perform the same because no two servers have exactly the …

As intuitive and streamlined as ecommerce technology might seem from the user's perspective, it involves so much data that engineering ingenuity and smart database management must constantly deliver in order to keep up. At organizations like Shopify—responsible for the easy and reliable transactions at top brands around the world—that excellence of performance involves deep monitoring of their MySQL core and their Redis caching infrastructure, plus insightful query profiling, packet captures, and the admittance of developers to platforms that measure database performance.

Shopify’s motto is “Make commerce better for everyone.” That mantra applies whether the shopping's done online, on mobile, or in-store. For Shopify's engineering team, better means a fast, reliable application that delivers a positive …

MySQL 5.7 introduced the LOGICAL_CLOCK type of multi-threaded slave (MTS).  When using this type of parallel replication (and when slave_parallel_workers is greater than zero), slaves use information from the binary logs (written by the master) to run transactions in parallel.  However, enabling parallel replication on slaves might not be enough to get a higher replication throughput (VividCortex

ProxySQL is a high performance proxy for MySQL and its forks. One of the key features is its ability to handle hundreds of thousands of connections with very low overhead. Datadog is a monitoring service for cloud-scale applications, bringing together data from servers, databases, tools, and services to present a unified view of an entire stack.

Datadog does not yet provide an integration for ProxySQL. So I decided to write an integration by forking the Datadog agent. Read my detailed blog post on TwinDB Blog to learn how to use the ProxySQL-Datadog integration.

The post Monitoring ProxySQL using Datadog appeared first on ovais.tariq.

Last week VividCortex's Preetam Jinka published a post on his personal blog examining how our engineering team had overcome a problem with MySQL replication by using a new parallelization policy introduced in MySQL 5.7: LOGICAL_CLOCK.

Image Credit

The solution we developed—which achieves faster replication via group commit and a carefully calibrated delay—can offer huge replication improvements, but its implementation isn't immediately obvious or intuitive. We thought it worthwhile to provide a fuller description of how we arrived at the solution Preetam outlined.

Updated 1/11/2017

Quite some time ago, when we added the ability to monitor MySQL query performance statistics from MySQL’s PERFORMANCE_SCHEMA, we learned a number of valuable lessons, still worth sharing. There are definitely right and wrong ways to do it. If you are looking to the P_S tables for monitoring MySQL query performance, this blog post might save you some time and mistakes.

Image Credit

What Is The Performance Schema?

First, a quick introduction. The Performance Schema includes a set of tables that give information on how statements are performing. Most of the P_S tables follow a set of predictable conventions: there’s a set of tables with a limited set of full-granularity current and/or historical data, which is aggregated into tables that accumulate over time. In the case …

MySQL continues to be one of the most popular databases used in cloud-native applications. In fact, MySQL is so popular that other cloud databases such as AWS Aurora maintain wire protocol compatibility with MySQL. For SREs and DevOps engineers running MySQL database in production, it is crucial to understand how to monitor MySQL. MySQL poor health can lead to cascading effects on other application components. For example, slow queries can impact page load times for an application, or missing indexes can result in high-latency and application time-outs. By effectively monitoring the performance of databases and query executions, SREs and DevOps can identify if there are bottlenecks in the database tier which affect the overall application performance. With this appreciation for the importance of MySQL monitoring, let us quickly survey what techniques are commonly used for MySQL monitoring and then discuss …

This blog post discusses an information script for the Percona Monitoring and Management (PMM) tool.

In recent news, we announced the fresh-of-the-press Percona Monitoring and Management (or PMM for short) platform. Given the interaction of the different components that together make up PMM, I developed a script that helps provide you information about the status of your PMM installation.

You can use this script yourself, or one of our support might point you to this page to obtain the information they need to troubleshoot an issue you are experiencing.

You will likely want to execute this script once on the PMM …

Recently, I worked on a customer case where the customer needed to monitor the checksum via Nagios monitoring. The pmp-check-pt-table-checksum plugin from Percona Monitoring Plugins for MySQL achieves this goal. I thought it was worth a blogpost.

pmp-check-pt-table-checksum

 alerts you when the pt-table-checksum tool from Percona Toolkit finds data drifts on …

Percona announces the release of Percona Monitoring and Management 1.0.2 Beta on 28 July 2016.

Like prior versions, PMM is distributed through Docker Hub and is free to download. Full instructions for download and installation of the server and client are available in the documentation.

Notable changes to the tool include:

• Upgraded to Grafana 3.1.0.
• Upgraded to Prometheus 1.0.1.
• Set default metrics retention to 30 days.
• Eliminated port 9001. Now the container uses only one configurable port, 80 by default.
• Eliminated the need to specify ADDRESS variable …