Showing entries 1 to 10 of 26
10 Older Entries »
Displaying posts with tag: aurora (reset)
Creating an External Replica of AWS Aurora MySQL with Mydumper

Oftentimes, we need to replicate between Amazon Aurora and an external MySQL server. The idea is to start by taking a point-in-time copy of the dataset. Next, we can configure MySQL replication to roll it forward and keep the data up-to-date.

This process is documented by Amazon, however, it relies on the mysqldump method to create the initial copy of the data. If the dataset is in the high GB/TB range, this single-threaded method could take a very long time. Similarly, there are ways to improve the import phase (which can easily take 2x the time of the export).

Let’s explore some tricks to significantly improve the speed of this process.

Preparation Steps

The first step is to enable binary logs in Aurora. Go to the Cluster-level parameter group and make sure binlog_format

[Read more]
Debezium MySQL Snapshot For AWS RDS Aurora From Backup Snaphot

I have published enough Debezium MySQL connector tutorials for taking snapshots from Read Replica. To continue my research I wanted to do something for AWS RDS Aurora as well. But aurora is not using binlog bases replication. So we can’t use the list of tutorials that I published already. In Aurora, we can get the binlog file name and its position from its snapshot of the source Cluster. So I used a snapshot for loading the historical data, and once it’s loaded we can resume the CDC from the main cluster.

Requirements:

  1. Running aurora cluster.
  2. Aurora cluster must have binlogs enabled.
  3. Make binlog retention period to a minimum 3 days(its a best practice).
  4. Debezium connector should be able to access both the clusters.
  5. Make sure you have different security …
[Read more]
Handling Bi-Directional Replication between Tungsten Clusters and AWS Aurora

Overview The Skinny

In this blog post, we explore the correct way to implement bi-directional Tungsten Replication between AWS Aurora and Tungsten Clustering for MySQL databases.

Background The Story

When we are approached by a prospect interested in using our solutions, we are proud of our pre-sales process by which that we engage at a very deep technical level to ensure the we provide the best possible solution to meet with the prospect’s requirements. This involves an in-depth hands-on POC, in addition to the significant time and effort we spend building and testing the solution architectures in our lab environment as part of the proposal process.

From time to time, we are presented with requirements that are not always quite so straight forward. Just recently we faced such a situation. A …

[Read more]
Adaptive Hash Index on AWS Aurora

Recently I had a case where queries against Aurora Reader were 2-3 times slower than on the Writer node. In this blog post, we are going to discuss why.

I am not going to go into the details of how Aurora works, as there are other blog posts discussing that. Here I am only going to focus on one part.

The Problem

My customer reported there is a huge performance difference between the Reader and the Writer node just by running selects. I was a bit surprised, as the select queries should run locally on the reader node, the dataset could fit easily in memory, there were no reads on disk level, and everything looked fine.

I was trying to rule out every option when one of my colleagues mentioned I should have a look at the InnoDB_Adaptive_Hash_Indexes. He was right – it …

[Read more]
Supercharge your Reporting: Aurora Autoscaling and Custom Endpoints

Whenever we talk about the scalability of databases its end up with a lot of discussions and effort to implement. Some of you may even argue that it is a bad idea to auto scale the transactional databases. But the pace of innovation in databases — particularly on a world with public cloud, is breathtaking. AWS Aurora is a game changer database engine in DBaaS for Open Source Databases. It provides performance, reliability, availability, and Scalability. With Aurora features like custom endpoints and loadbalancing across replicas, on can explore some interesting use cases. In this post, we will discuss how we solved a customer’s problem by using scalability features of Aurora. We will focus on provisioned Aurora on AWS, not Aurora Serverless.

We have a customer who was doing all the reporting and massive read …

[Read more]
Create Aurora Read Replica With AWS CLI/Lambda Python

Today I was working for a scaleable solution in Aurora. Im going to publish that blog post soon in Searce Blog. As a part of this solution, I want to create Aurora read replicas programmatically. So we have done the create aurora read replica with AWS CLI and Lambda with Python. If you refer the …

The post Create Aurora Read Replica With AWS CLI/Lambda Python appeared first on SQLgossip.

Create Aurora Read Replica With AWS CLI/Lambda Python

Today I was working for a scaleable solution in Aurora. Im going to publish that blog post soon in Searce Blog. As a part of this solution, I want to create Aurora read replicas programmatically. So we have done the create aurora read replica with AWS CLI and Lambda with Python. If you refer the AWS Doc, they mentioned there is no separate module for creating Aurora Read replica in boto3 or cli. Instead we can use create-db-instance. Many people may confused with this term. This blog will help them to create aurora read replicas using AWS CLI and Lambda.

Required Permission:

If you are running this code via lambda then you can use the below policy.

  • Lambda - Create a new role for lambda and attach inline policy
  • EC2 Role - Create a new …
[Read more]
Unforeseen use case of my GTID work: replicating from AWS Aurora to Google CloudSQL

A colleague brought an article to my attention.  I did not see it on Planet MySQL where I get most of the MySQL news (or it did not catch my eye there).  As it is interesting replication stuff, I think it is important to bring it to the attention of the MySQL Community, so I am writing this short post.

The surprising part for me is that it uses my 4-year-old work for online migration to GTID

Amazon Aurora Serverless - Features, Limitations, Glitches

Amazon Aurora Serverless — Features, Limitations, Glitches

Finally after an year AWS announced the AWS Aurora Serverless MySQL compatibility. I was expecting that they will release this in the reInvent 2018, but it live now. So now I’m expecting more new features for Aurora servers in the reInvent 2018. I have played with this baby and understood whats is doing and what we can do with that.

Here is the blog from AWS about Aurora Serverless

Aurora Serverless MySQL Generally Available | Amazon Web Services

What is Aurora Serverless?

Aurora serverless provides seamlessly scale up and down its compute and memory. You can pay for how much resources that you have been used. No needs of creating and managing the Read Replica and HA(Multi-AZ).

How Aurora Serverless works? Source: AWS

[Read more]
RDS Aurora MySQL Cost

I promised to do a pricing post on the Amazon RDS Aurora MySQL pricing, so here we go.  All pricing is noted in USD (we’ll explain why)

We compared pricing of equivalent EC2+EBS server instances, and verified our calculation model with Amazon’s own calculator and examples.  We use the pricing for Australia (Sydney data centre). Following are the relevant Amazon pricing pages from which we took the pricing numbers, formulae, and calculation examples:

[Read more]
Showing entries 1 to 10 of 26
10 Older Entries »