While implementing ClickHouse for query executions statistics storage in Percona Monitoring and Management (PMM),  we were faced with a question of choosing the data type for metrics we store. It came down to this question: what is the difference in performance and space usage between Uint32, Uint64, Float32, and Float64 column types?

To test this, I created a test table with an abbreviated and simplified version of the main table in our ClickHouse Schema.

The “number of queries” is stored four times in four different columns to be able to benchmark queries referencing different columns.  We can do this with ClickHouse because it is a column store and it works only with columns referenced by the query. This method would not be appropriate for testing on …

Since the concept of cloud was born, there has been strong growth in the number of migrations to this environment. However, not everything that shines is gold.

As the demand grows, so does the costs. We can find ourselves in a situation where our monthly cloud expenses are very high and, in this case, it may make sense to migrate back to an on-prem environment.

The costs may not be the only reason. There might be security or compliance requirements, or we may need to have more control of our systems. Knowing what happens at a lower level can help us better optimize things.

AWS not only give us the environment, it also provides us with monitoring and management tools to run our system in the cloud. So, it can be really hard to migrate to an on-prem environment and recreate all these tools to …

If you work with MySQL or PostgreSQL, you probably at least heard about PMM (Percona Monitoring and Management). If you want to test it, you may also be interested in this article: I’ll show you how to setup a MySQL and a MariaDB instance and monitor them in the quickest way. Which is Docker.

Docker is also my favourite way of running PMM in production. To be clear, I wouldn’t run a production DBMS in Docker. But now we are talking to test something quickly on your laptop, without wasting too much time. And I don’t know a faster way.

Note: PMM is not a single piece of software. It is a solution created by making use of existing software projects, like Grafana, Prometheus and Orchestrator. In this post I want to add this complexity, even if the cost is a …

Mydbops database meetup was held on 26-01-2019 and 48 members from open source database community has attended this event. We had talks on MySQL and MongoDB databases.

Open source database enthusiastic from companies like OLA Cabs, Searce Inc, PayTm, Quikr, Zensar, Grab, SAP labs, Travel Yarri, Meddiff and a few has travelled from other cities of India like Hyderabad and Chennai.

The Welcome talk was given my Selva venkatesh of Mydbops. He spoke about the open source database community and welcomed all the participants and speakers.

Followed by it Karthik P R , CEO/ Founder, Mydbops made his presentation on “InnoDB Performance Optimisation” . InnoDB is the prime engine in MySQL.This talk focuses on the InnoDB Architecture in MySQL 8.0 and its fine tuning. He also answered the various questions raised …

Rails 6.0.0.beta1 is out and you may have already tested it. We all have heard about the main features such as multi-database connectivity, Action Mailbox & Action Text merge, parallelized testing, Action Cable testing etc. But there’s also a ton of other cool features that I found interesting.

Requirements change

With each major release comes new requirements, starting with Ruby which is now required with a minimal version of 2.5.0 instead of 2.2.2 for Rails 5.2. Our databases also get an upgrade with 5.5.8 for MySQL, 9.3 for PostgreSQL and 3.8 for SQLite.

Webpacker as default

Webpacker has been merged in Rails 5.1 and provides a modern asset pipeline with the integration of Webpack for your javascript files. Before Rails 6 you had to generate your app with the --webpacker option to use it, now Webpacker is the default and it’s a good first step for a modern asset pipeline on Rails in …

For those not familiar with it, FOSDEM is an amazing, free entry, full on celebration of open source that takes place in Brussels, Belgium every year. This year the event was held over the first weekend of February. Fringe events, such as the Pre-FOSDEM MySQL day hosted by Oracle MySQL, and the community dinner that follows, provide an opportunity to network.

In case you didn’t make it to FOSDEM this year, here are links to Percona’s presentations from the event. Organizers video and share online every talk from every dev room, a phenomenal achievement in itself. All credit to the volunteers who run this show.

Database Dev Room: Hugepages and databases presented by Fernando Laudares …

How to Monitor MySQL Databases?

Operational visibility is a must in any production environment. It is crucial to be able to identify any issues as soon as possible, otherwise you may end up in serious troubles as an undetected issue can cause serious service disruption or downtime. MySQL Enterprise Monitor is one of the oldest monitoring products for MySQL on the market, and is available as part of an commercial enterprise subscription agreement from Oracle.In this blog post we will take a look at MySQL Enterprise Monitor and the kind of insight it provides into MySQL.

Installation

First of all, MySQL Enterprise Monitor is part of MySQL Enterprise Edition, a …

Today I am going to introduce you all to an awesome tool GH-OST. We are using it since many months to Alter tables online. Believe me it saves lot of time and efforts while altering big tables in MySQL.
gh-ost has been developed at GitHub. To answer a problem we faced with ongoing, continuous production schema changes requiring modifications to MySQL tables. gh-ost changes the existing online table migration paradigm by providing a low impact, controllable, auditable, operations friendly solution.
gh-ost stands for GitHub’s Online Schema Transmogrifier/Transfigurator/Transformer/Thingy.
While migrating table gh-ost creates a ghost table in the likeness of your original table, migrate that table while …

Vitess is a sharding and scaling solution for MySQL that can make thousands of databases appear to be a single monolithic database. Vitess is interesting, and promising, in a number of ways. It’s addressing an extremely hard set of problems that’s been tried many times before with mixed results, but evidence points to Vitess succeeding where others have thus far failed.

Vitess solves three fundamental, hard problems for its target user base:

• Sharding. Sharding is essential for scaling traditional relational databases that weren’t built natively to operate as a distributed cluster of many nodes forming a single database. Every large-scale MySQL deployment—and there are thousands—uses sharding, bar none.
• Kubernetes deployment. You can’t just run a database like MySQL on Kubernetes without a robust suite of operational tooling for high availability. MySQL …

In my previous post I have explain how we are running maxwell as service in Ubuntu. Now in this post I will be explaining how we achieved HA for Backend Instances (MySQL Hosts), Maxwell was using to read binlog.
So the stack we have introduced to achieve high availability contains HAProxy (http://www.haproxy.org/) and GTID enabled slaves. HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications.
We have installed HAProxy on the local machine where we are running Maxwell and configured maxwell to connect to localhost where we have used HAProxy primary-secondary backup strategy using below config …