Showing entries 1 to 10 of 96
10 Older Entries »
Displaying posts with tag: scaling (reset)
Dockerizing MySQL step by step.


As a Database Engineer in Mydbops. We tend to solve multiple complex problems for our esteemed customers. To control the System resources and scale up /down based on needed we are evaluating Dockers and Kubernetes.

Docker is a set of platform as a service products that uses OS-level virtualization to deliver software in packages called Containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels.It’s more lightweight than standard Containers and boots up in seconds.

Docker also is easy to use when you need a simple, single instance. What is great about Docker though is that it allows configuring multiple versions of MySQL.

Docker Installation:

Docker can be installed with yum repository or apt-get repository based on your linux …

[Read more]
Using SKIP LOCK For Queue Processing in MySQL

A small thing that provides a huge help.

The other day I was writing some code to process a very large amount of items coming from a social media API. My items were ending in a queue in MySQL and then needed to be processed and eventually moved.

The task was not so strange,  but what I have to do is to develop a queue processor.  Now when you need to process a queue you have two types of queue: static and dynamic.

The static comes in a batch of N number of items in a given time interval and is normally easier to process given you have a defined number of items that you can split in chunks and process in parallel.

The dynamic is… well… more challenging. One option is to wait to have a predefined number of items, and then process them as if they were a static queue.

But this approach is not very good, given it is possible that it will delay a lot …

[Read more]
Tungsten Clustering Makes The 2020 DBTA Top Trending Products List

We’re delighted to be able to share that Tungsten Clustering – our flagship product – is named in the DBTA 2020 List of Trend Setting Products!

Congratulations to all the products and their teams that were named in the 2020 list.

We have been at the forefront of the market need since 2004 with our solutions for platform agnostic, highly available, globally scaling, clustered MySQL databases that are driving businesses to the cloud (whether hybrid or not) today; and our software solutions are the expression of that.

Tungsten Clustering allows enterprises running business-critical MySQL database applications to cost-effectively …

[Read more]
Geo-Scale MySQL for Continuous Global Operations & Fast Response Times

Geo-scale MySQL – or how to build a global, multi-region MySQL cloud back-end capable of serving several hundred million player accounts

This blog introduces a series of blogs we’ll be publishing over the next few months that discuss a number of different customer use cases that our solutions support and that centre around achieving continuous MySQL operations with commercial-grade high availability (HA), geographically redundant disaster recovery (DR) and global scaling.

This first use case looks at a customer of ours who are a global gaming company with several hundred million world-wide player accounts.

What is the challenge?

How to reliably, and fast, cater to hundreds of millions of game players around the world? The challenge here is to serve a game application for a geographically-distributed audience; in other words, a pretty unique challenge.

It requires fast, local response times …

[Read more]
MySQL 8.0 Clone Plugin and its internal process.

MySQL 8 has recently released clone plugin which makes DBA’s task of rebuilding the DB servers more easy.

  • Cloning is a process of creating an exact copy of the original. In technical terms cloning alias to (Backup + Recovery), MySQL database cloning requires a sequence of actions to be performed manually or in a scripted fashion with and without the tools involved.
  • Cloning is the first step when you want to configure the replication slave or Joining a new server to the InnoDB cluster. There was no native support for auto provisioning earlier. Percona XtraDB Cluster (MySQL + Galera Cluster) does cloning using xtrabackup tool by default when a new node joins the cluster.
  • Now MySQL simplified this task, In this post, We will see how to clone the database using clone plugin and its internals.
[Read more]
TaskMax limit affects MySQL connections

Recently we had been bitten by a Systemd limitation at the “Tasks” created per-unit ie., process. This includes both the kernel threads and user-space threads, with each thread counting individually.

Am writing this blog as a reference for someone who might come across this limitation.

We have been actively working on migration DB instances, from one DC to the newly built DC .The instances on the newer DC were provisioned with the latest hardware and latest Debian OS. Below is the detailed spec of the system.

RAM             : 244G
Core             : 44Core
HardDisk.   : SSD
IOPS             : 120K
OS         …

[Read more]
Will IO Size Affect your RDS Performance?​

During our recent consulting with one of our client, We came across an interesting issue on RDS. The baseline is that “Low IO size on your RDS instance can affect your DB performance”.  Yes, It’s IO size, Not IOPS.

We had our production systems running on RDS MySQL with a single master, 3 replicas. All instances are of same type db.m4.4xlarge with same parameter group configuration and the disk size is 1.5 TB. According to the AWS user guide, each of these instances can support up to 4500 (sustained IOPS) guaranteed IOPS.

Find below the Write IOPS graph for all the instances.

It’s understood that Write IOPS / pattern on Master can vary when compared with Slave, due to a lot of factors like binlog row format, log writing etc. But it has to be almost similar for all the slaves given that it …

[Read more]
MySQL Partition Manager (Yahoo!) in a nutshell

Partitioning is a way of splitting the actual data down into separate .ibd files (data compartments) in the file system based on the defined ranges using the partitioning key. It can help us with maintaining the enormous amount of data in different partitions without much hassle.

In this blog post, we are going to see how to manage table partitioning using yahoo partition manager.

Needs for partitioning:

  • Enhanced data retrieval ( reduced IO ) with smaller B+Tree.
  • Easy Archival or Purge by dropping or truncate  of partition
  • Lesser fragmentation, hence avoiding frequent table optimization.

Partitions management activity like adding/Dropping partition is made easy and automated by …

[Read more]
2018-11-15: Announcing: Scaling MySQL with TiDB, Vitess and MySQL Cluster at Madrid MySQL Users Group

[ English ] – texto en español abajo We’re pleased to announce the next Madrid MySQL Users Group meetup which will take place on the 15th of November at 19:00.  Sign up details can be found here.  There’ll also be a similar meetup in Amsterdam on the 12th November hosted by a colleague. Details here. […]

Presentation : Evolution of DBA’s in the Cloud Era

Mydbops team has presented at 15th edition Open source India Conference happened at Bangalore on 12-10-2018. Manosh malai ( Senior Devops/ DB Consultant ) and Kabilesh P R ( Co-Founder Mydbops ) has made presentation on “Evolution of DBA in the Cloud Era“. They have shared the value addition that DBA’s can bring into any organisation who have adopted to cloud ( DBaaS ). They have shared a case study of their consulting experience too.

Showing entries 1 to 10 of 96
10 Older Entries »