Showing entries 1 to 10 of 1329
10 Older Entries »
Displaying posts with tag: Open Source (reset)
Orchestrator fails to start after reboot using SystemD

While testing in an orchestrator lab I saw that none of my Orchestrator on-raft nodes were coming online after a reboot.

This is the status report from SystemD.

$ sudo systemctl status orchestrator
* orchestrator.service - orchestrator: MySQL replication management and visualization
   Loaded: loaded (/etc/systemd/system/orchestrator.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Fri 2020-04-03 09:30:05 UTC; 30s ago
     Docs: https://github.com/github/orchestrator
 Main PID: 957 (code=exited, status=1/FAILURE)

Apr 03 09:30:05 orchestrator-1 systemd[1]: Started orchestrator: MySQL replication management and visualization.
Apr 03 09:30:05 orchestrator-1 orchestrator[957]: 2020-04-03 09:30:05 ERROR dial tcp 127.0.0.1:3306: connect: connection refused
Apr 03 09:30:05 orchestrator-1 orchestrator[957]: 2020-04-03 09:30:05 FATAL dial tcp 127.0.0.1:3306: connect: connection refused
Apr 03 09:30:05 orchestrator-1 …
[Read more]
Sysbench and the Random Distribution Effect

What You May Not Know About Random Number Generation in Sysbench

Sysbench is a well known and largely used tool to perform benchmarking. Originally written by Peter Zaitsev in early 2000, it has become a de facto standard when performing testing and benchmarking. Nowadays it is maintained by Alexey Kopytov and can be found in Github at https://github.com/akopytov/sysbench.

What I have noticed though, is that while widely-used, some aspects of sysbench are not really familiar to many. For instance, the easy way to expand/modify the MySQL tests is using the lua extension, or the embedded way it handles the random number generation. 

Why This Article? 

I wrote this article with the intent to show how easy it can be to customize sysbench to make it what you need. There are many different ways to extend sysbench use, and one of these is …

[Read more]
A Linux Tool to Monitor Progress of MySQL Data Imports and Exports.

One problem I have experienced during logical exporting and importing databases is the tools don’t output any progress indicator (or anything at all really) and using the -v (verbose) switch outputs every single MySQL command being run from the import file. As, importing and exporting of databases  is one of the most frequent activity being performed , I Just felt the need of some tool that could help me check the approximate progress of import and export of databases going on.

So, I searched and fortunately could find a tool named Pipeviewer (PV) which allows a user to see the progress of data through a pipeline, by giving information such as time elapsed, percentage completed (with progress bar), current throughput rate, total data transferred, and ETA .

Installation :

On Centos and RHEL :

[root@vm1 vagrant]# yum install pv

[root@vm1 …

[Read more]
Configuring a Read-Only Web Interface for Orchestrator

In the MySQL ecosystem, orchestrator is the most popular and well-respected high availability and topology management tool, integrating well with other solutions such as ProxySQL. It facilitates automatic (or manual) discovery, refactoring and recovery of a replicated MySQL environment, and comes complete with both command-line (CLI) and web interfaces for both humans and machines to interact with.

As we all know, humans are prone to errors and as such accidents can happen, particularly when humans and computers interact with each other! Recently, one of these situations related to the web interface of orchestrator during topology refactoring with its drag-and-drop capabilities, where a drop occurred unintentionally and thus had an impact on replication.

When …

[Read more]
3 Step Migration of MySQL data to Clickhouse for faster analytics.

Recently one of our client approach Mydbops with Query slowness on a MySQL environment . They deployed the new code for generate the huge reports for the year end analytics data . After the deployment the queries were extremely slow and they struggled lot , then they approached us for the solution. After the analysis, their OLAP database as expected it was IO bound with 100% disk IOPS utilised during the report generation. So, the queries were starving for the Disk IO slows the process .

Problem statement :

  • Reports are majorly focused on two larger log tables ( emp_Report_model , emp_details ) .
  • The report generator (procedure) is using the count(*) statement to stimulate the aggregated data on each call. It is required for their business purpose .
  • Count(*) is terribly slow in MySQL ( Using MySQL 5.7 ) as it …
[Read more]
The state of Orchestrator, 2020 (spoiler: healthy)

This post serves as a pointer to my previous announcement about The state of Orchestrator, 2020.

Thank you to Tom Krouper who applied his operational engineer expertise to content publishing problems.

How to Run Orchestrator on FreeBSD

In this post, I am going to show you how to run Orchestrator on FreeBSD. The instructions have been tested in FreeBSD 11.3 but the general steps should apply to other versions as well.

At the time of this writing, Orchestrator doesn’t provide FreeBSD binaries, so we will need to compile it.

Preparing the Environment

The first step is to install the prerequisites. Let’s start by installing git:

[vagrant@freebsd ~]$ sudo pkg update
Updating FreeBSD repository catalogue...
Fetching meta.txz: 100% 944 B 0.9kB/s 00:01
Fetching packagesite.txz: 100% 6 MiB 492.3kB/s 00:13
Processing entries: 100%
FreeBSD repository update completed. 31526 packages processed.
All repositories are up to date.

[vagrant@freebsd ~]$ sudo pkg install git
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
New …
[Read more]
Automating MySQL schema migrations with GitHub Actions and more

In the past year, GitHub engineers shipped GitHub Packages, Actions, Sponsors, Mobile, security advisories and updates, notifications, code navigation, and more. Needless to say, the development pace at GitHub is accelerated.

With MySQL serving our backends, updating code requires changes to the underlying database schema. New features may require new tables, columns, changes to existing columns or indexes, dropping unused tables, and so on. On average, we have two schema migrations running daily on our production servers. Some days we have a half dozen migrations to run. We’ll cover how this amounted to a significant toil on the database infrastructure team, and how we searched for a solution to automate the manual parts of the process.

What’s in a migration?

At first …

[Read more]
Seriously Inconsistent Table Information in MySQL 8.0 Information_Schema

In MySQL 8, Information Schema was basically re-written to utilize the all-New Data Dictionary which made it faster and better in many ways. Yet it also introduces some very counterintuitive behaviors when it comes to how actual and consistent data is.

Here is a simple test.  Let’s create a table and insert a bunch of rows in it… and then check table size:

mysql> show table status \G
*************************** 1. row ***************************
           Name: t
         Engine: InnoDB
        Version: 10
     Row_format: Dynamic
           Rows: 0
 Avg_row_length: 0
    Data_length: 16384
Max_data_length: 0
   Index_length: 0
      Data_free: 0
 Auto_increment: NULL
    Create_time: 2019-12-27 01:04:37
    Update_time: NULL
     Check_time: NULL
      Collation: utf8mb4_0900_ai_ci
       Checksum: NULL
 Create_options:
        Comment:
1 row in set (0.02 sec)

mysql> select count(*) from t;
+----------+
| count(*) |
+----------+
| …
[Read more]
How to Create Your Own Repositories for Packages

For Linux, the most common way to distribute software is binary packages in the rpm or deb format. Most packages are included in the official distribution repositories or 3rd party software repositories. Nevertheless, there are some cases where you need to install just a few standalone packages.   You might be able to use the local package install tools, namely dpkg or rpm, however, there are cases where packages can’t be installed due to the dependencies and you need to install all dependencies manually. It might take some time and isn’t always an easy process. But there is a solution that can help – you can create your own local repository and deploy your packages to it.

Let’s discuss how to create your local repositories to make your life easier.

RPM-Based Distributions

RPM-based operating systems work with rpm packages and the most common package manager for them is yum.   While newer …

[Read more]
Showing entries 1 to 10 of 1329
10 Older Entries »