Showing entries 1 to 5
Displaying posts with tag: purge (reset)
About the new warnings generated during 'PURGE BINARY LOGS' execution

In this blog, I will discuss about warnings that users might face if they are trying to purge active/in_use binary log which are introduced in MySQL-5.6.12/MySQL-5.7.2 releases.  Before jumping directly to that, I will be covering few basic things about PURGE BINARY LOGS command, about safe way to purge the binary logs on a replication forum. If you are good at basics on "PURGE BINARY LOGS" , you can choose to directly move to the section (at the end of the blog) where I was talking about these new warnings.

A binary log file contains “events” that describe database changes occurred on a MySQL Server (typically called Master) that enabled binary logging using --log-bin. On a Master server, the files that are related to binary logging are binary log files (files with binlog_base_name.6_digit serial_number) and a binary log index file (file with …

[Read more]
Circular Partitioning scheme for a fully Automated Archival / Purge process with Events

Partitioning The advantage of partitioning is that we can drop / truncate the partitions which is real quick and done in seconds. It does lock the table (metadata lock) during this operation but only for few seconds. That duration depends on the number of pages in use in the buffer pool. With MySQL 5.6 it is possible to exchange the partitions , which helps in moving archived data to another table with similar structure.
Fully Automated archival / purge 

Consider a fast growing table which is partitioned to help the queries and help quick archival/purge. The queries if specify the partitioning filter can limit the search to relevant partitions and can considerably reduce the query time. Purging old data to reclaim the storage space can be readily achieved by truncating the old partitions. Archival has multiple techniques but with MySQL 5.6 exchanging the partitions make archival a real …

[Read more]
MySQL 5.6: Multi threaded purge


What does purge exactly do and why is it needed? If you have ever wondered then read on. It is really a type of garbage collector. When a user issues a DML like “DELETE FROM t WHERE c = 1;”, InnoDB doesn’t remove the matching record. This is what happens under the hood:

  1. It marks the record as deleted by setting a bit in the control bits of the record.
  2. Stores the before image of the modified columns to the UNDO log
  3. Updates the system columns DB_TRX_ID and DB_ROLL_PTR in the clustered index record. DB_TRX_ID identifies the transaction that made the last change, and DB_ROLL_PTR points to the new UNDO log record. This UNDO log record contains the old values of DB_TRX_ID and DB_ROLL_PTR, possibly pointing to an older transaction and undo log entry.

From …

[Read more]
Purging binary logs.

Being a MySQL DBA , one faces a common issue in replication environment -> Disk space issue on master, since the number of binary logs have increased. Now, one of the solution to this would be using expire_logs_days parameter in your mysql config file. But what if, the slave is lagging by few hours or if the slave is broken since few days and the binary logs are removed due to the parameter set. Whenever the salve comes up, it will go bonkers, knowing that the binary log where it last stopped no more exists.
I faced this issue a couple of times until I decided to automate it using a script. Herewith I am attaching the URL to my python script which can run regularly in cron. Features :

  • Checks the slaves connected to the master (I have limit it to 3 for now.)
  • Checks the last binary log file which is being used by the slave.
[Read more]
Using the event scheduler to purge the process list

Two of the most common tasks for database administrators are cleaning the process list from unresponsive queries and remove idle connections that are filling the connection pool.
Both tasks are related to poor usage of the database. In a perfect world, users would only run queries designed, tested, and benchmarked by the DBA or the project manager, and the application servers would never allocate more connections than planned.

But users are human, and an unpredictable amount of unplanned events can happen everywhere. When I was consulting, the above cases were quite common.
Before MySQL 5.1, the only method to clean up the process list was by hand, or using a cron job to do it from time to time.
MySQL 5.1 …

[Read more]
Showing entries 1 to 5