Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 209 Next 30 Older Entries

Displaying posts with tag: Backup (reset)

Single database backup and restore with MEB
+3 Vote Up -0Vote Down
I was recently asked about if MySQL Enterprise Backup would be able to restore single databases.

My initial answer was that this was complicated, but might be doable with the Transportable Table Space (TTS) option.

But first let's go back to the basics. A common way of working with mysqldump is to get a list of databases and then loop through the databases and dump the data and schema to a SQL file. But both backups and restores will take a lot of time if the size of the database grows. And it's a luke-warm backup at best instead of a hot backup. So that's why we have MySQL Enterprise Backup.

MySQL Enterprise Backup allows you to make a hot backup of InnoDB tables by copying the datafiles while watching the InnoDB redo log files.

On disk the data from the InnoDB storage engine consists of a system tablespace (one of more







  [Read more...]
MySQL backup and cleanup bash scripts with mydumper
+1 Vote Up -0Vote Down

1. Backup script

#!/bin/sh
# Fri Jun 27 10:44:49 2014
# done by dragkh
# usage: 
# cat /etc/cron.d/backupmysql 
# 0  3  *  *  *       root    /root/bin/clean.backup.hyperion.mysql.mydumper.daily.sh >>  /var/log/clean.backup.${HOSTNAME}.mysql.mydumper.daily.log 2>&1
# 35  3  *  *  *       root    /root/bin/backup.hyperion.mysql.mydumper.daily.sh >> /var/log/backup.${HOSTNAME}.mysql.mydumper.daily.log 2>&1

ROOT_BACKUP_DIR="/home/mydumper"

seik_date () {
if [ -z $1 ]
then
# cdate=`date +%Y-%m-%d\ %H:%M:%S\ %Z`; export cdate; echo $cdate
cdate=`date -R`; export cdate; echo $cdate
else

if [ -z ${2} ]
then
cdate=`date +%Y-%m-%d.%H.%M.%S`; export cdate; echo $cdate
else
cdate=`date "+%Y-%m-%d %H:%M:%S"`; export cdate; echo $cdate
fi

fi
}


function check_dir {
 test ! -d
  [Read more...]
MySQL Enterprise Backup Improved Compression Algorithm for 3.10
Employee +1 Vote Up -0Vote Down

Background:

Prior to version 3.10, MySQL Enterprise Backup (MEB) used zlib compression for in-memory compression of datafiles. The compression worked by splitting the innodb datafiles into fixed size blocks and compressing each block independently.After searching on the web we found there are many compression algorithms available which can be used for compression. This triggered the idea of testing the performance of available compression algorithms. If the benchmark shows improved performance we can make backup and/or restore faster by adding the new compression algorithm to MEB.

Implementation :

The idea to implement the algorithms procceded as follows .

1. Select a "long list" of algorithms based on literature and what Google and other databases are using.
2. Create a

  [Read more...]
MySQL Enterprise Backup 3.10: Teasing compression.
Employee +0 Vote Up -0Vote Down

Ok, so I wanted to look into the new compression options of MEB 3.10.

And I would like to share my tests with you. Remember, they’re just this, tests, so please feel free to copy n paste and obtain your own results and conclusions, and should I say it, baselines, in order to compare future behaviour, on your own system.

An Oracle Linux 6.3 virtual machine with 3Gb RAM, 2 virtual threads, on a 1x quad core, windows laptop. Not pretty, but hey.

So, these tests are solely about backup. I’ll do restore when I get some *more* time.

 

First up, lets compare like with like, i.e. MEB version 3.9 & 3.10:

Let’s make this interesting, hence, want to use as much resources available as possible, read, write, process threads and number of buffers.

mysqlbackup --user=root --password=oracle
  [Read more...]
MySQL Enterprise Backup 3.10: Teasing compression.
Employee +0 Vote Up -0Vote Down

Ok, so I wanted to look into the new compression options of MEB 3.10.

And I would like to share my tests with you. Remember, they’re just this, tests, so please feel free to copy n paste and obtain your own results and conclusions, and should I say it, baselines, in order to compare future behaviour, on your own system.

An Oracle Linux 6.3 virtual machine with 3Gb RAM, 2 virtual threads, on a 1x quad core, windows laptop. Not pretty, but hey.

So, these tests are solely about backup. I’ll do restore when I get some *more* time.

 

First up, lets compare like with like, i.e. MEB version 3.9 & 3.10:

Let’s make this interesting, hence, want to use as much resources available as possible, read, write, process threads and number of buffers.

mysqlbackup --user=root --password=oracle
  [Read more...]
Data Encryption with MySQL Enterprise Backup 3.10
Employee +0 Vote Up -0Vote Down

Introduction

MySQL Enterprise Backup (MEB) 3.10 introduces support for encrypted backups by allowing backup images, or single-file backups, to be encrypted. However, backups stored in multiple files in a backup directory can not be encrypted.

Any MEB command that produces a backup image can be optionally requested to encrypt it. The encrypted backup image can be stored in a file or tape in the same way as an unencrypted backup image. Similarly, any MEB command that reads data from a backup image accepts also an encrypted backup image. This means that encrypted backups can be used in all the same situations as unencrypted backup images.

MEB encrypts data with Advanced Encryption Standard (AES) algorithm in CBC mode with 256-bit keys. AES is a symmetric block cipher which means that the same key is used both for encryption and

  [Read more...]
Offline checksum validation for directory and Image backup using MySQL Enterprise Backup
Employee +0 Vote Up -0Vote Down
Data integrity: ------------------- Data integrity refers to maintaining and assuring the accuracy and consistency of data over its entire life-cycle. Every organization whether it's small or large want to make sure their data is consistent and error free. Data might move to other media/ different storage system for performance, speed, scalability or any other business reasons. So we want to make sure data is not corrupt while migration/movement. Data integrity is a policy which enterprise can enforce to be confident about their own data.
The overall intent of any data integrity technique is to ensure data is recorded exactly as  intended and upon retrieval later, ensure data is the same as it was when originally recorded.

Objective:


  [Read more...]
Zyme Relies on MySQL Enterprise Edition to Deliver High Quality Global Channel Insights to Customers
Employee_Team +1 Vote Up -0Vote Down

Zyme, based in Redwood Shores, California, is the global leading provider of Channel Data Management (CDM) solutions to companies selling through indirect channels. For high tech and consumer electronic products alone, over $1 trillion USD worth of goods are flowing through those indirect sales channels every year. However, when companies sell products through multi-tier channel partners and retailers around the world, it has proven to be challenging in acquiring global, standardized channel inventory and sales data cost-effectively. As a result, companies lacking of such critical information often miss the opportunities to make timely and accurate business decisions either to increase revenue, reduce costs or to prevent losses.

Having a vision to solve such channel visibility problems for customers including Symantec,

  [Read more...]
Introducing Trite: A tool for automating import of InnoDB tablespaces
+0 Vote Up -0Vote Down

Mysqldump is a fantastic tool for backing up and restoring small and medium sized MySQL tables and databases quickly. However, when databases surge into the multi-terabyte range restoring from logical backups is inefficient. It can take a significant amount of time to insert a hundred million plus rows to a single table, even with very fast I/O. Programs like MySQL Enterprise Backup and Percona XtraBackup allow non-blocking binary copies of your InnoDB tables to be taken while it is online and processing requests. XtraBackup also has an export feature that allows InnoDB file per tablespaces to be detached from the shared table space and imported
  [Read more...]
–use-tts backup & restore
Employee +0 Vote Up -0Vote Down

In addition to my recent post, I just had to go into using the –use-tts for specific tables and selective backup sets.

As all my schemas were employeesn, I thought it would be a good idea to run:

mysqlbackup -uroot --socket=/tmp/mysql.sock --backup-dir=/home/mysql/MEB/restore \
--with-timestamp --use-tts --include=employees* backup

as I want all the tables. If I only wanted a specific table, say ‘salaries’ I could have done:

mysqlbackup -uroot --socket=/tmp/mysql.sock --backup-dir=/home/mysql/MEB/restore \
--with-timestamp --use-tts --include=employees*\.*salaries backup

and backup just the ‘salaries’ tables but from all the employeesn databases.

then.. be a mean little dba:

drop database employees;
drop database employees10;
drop database
  [Read more...]
MySQL Enterprise Backup: parallel config & backup n restore results.
Employee +0 Vote Up -0Vote Down

In this post I go into some performance metrics and time spent on using MySQL Enterprise Backup instead of mysqldump, and seeing how far I could go with some parallel configuration.

Setup:

It’s on an old laptop:

–Ubuntu 12.04 LTS, 32bit Intel Pentium M 1.86Ghz, 2Gb –Source disk:  internal 80Gb ATA ST9808211A –Destination:  external 1Tb SAMSUNG HD103SI –MySQL Enterprise Edition 5.6.15 –MySQL Enterprise Backup 3.9.0 –Employees sample database duplicated via MySQL Utilities 1.3.6 (on Win7 PC) to generate a ~5Gb MySQL Server. And to simulate data size, I used the MySQL Utilities:
mysqldbcopy --source=root:pass@host:3356 --destination=root:pass@host:3356 employees:employees1 \
employees:employees2 employees:employees3 employees:employees4 ...
  [Read more...]
MySQL Enterprise Backup: parallel config & backup n restore results.
Employee +0 Vote Up -0Vote Down

In this post I go into some performance metrics and time spent on using MySQL Enterprise Backup instead of mysqldump, and seeing how far I could go with some parallel configuration.

Setup:

It’s on an old laptop:

–Ubuntu 12.04 LTS, 32bit Intel Pentium M 1.86Ghz, 2Gb –Source disk:  internal 80Gb ATA ST9808211A –Destination:  external 1Tb SAMSUNG HD103SI –MySQL Enterprise Edition 5.6.15 –MySQL Enterprise Backup 3.9.0 –Employees sample database duplicated via MySQL Utilities 1.3.6 (on Win7 PC) to generate a ~5Gb MySQL Server. And to simulate data size, I used the MySQL Utilities:
mysqldbcopy --source=root:pass@host:3356 --destination=root:pass@host:3356 employees:employees1 \
employees:employees2 employees:employees3 employees:employees4 ...
  [Read more...]
MariaDB Database Disk snapshot backups on Windows
+4 Vote Up -0Vote Down
Using disk snapshots to perform MariaDB backups has become more and more common, but this seems to have been limited to Linux and Cloud environments. There seems that there has been a notion that Snapshots cannot be done on native Windows, the way we can do snapshots using LVM on Linux for example. This is actually False, if you use the NTFS filesystem snapshoting is built in and has been so for a long time, the base for it is the Volume Snapshot Service (VSS) which was introduced as far back as Windows XP.

Using Snapshots to backup on Linux or Windows works well, assuming that the storage engine you use can recover on startup, the backups created this was are physically consistent but not logically, but InnoDB will recover this. Also note that any warnings about using snapshots with MyISAM also

  [Read more...]
how MySQL engineering broke the backups
+0 Vote Up -0Vote Down

MySQL has exceptional track of record by introducing minor fixes that cause major breakages. Though usually I could blame naiveté of engineers, who did not really ever have to deal with production implications, but lately I can start sensing various business implications against open-source offerings.

As an original author of mydumper I really cannot get out of my mind that 5.5 and 5.6 metadata locking changes are there to screw with anyone who is building a backup solution using stable snapshot views of MySQL (for example, mysqldump –single-transaction, the golden standard of backing things up in MySQL world).

As seen in a bug #71017 (palindrome!) filed by my esteemed colleague Eric, newly

  [Read more...]
How to store MySQL innobackupex backups at Google Cloud Storage
+0 Vote Up -0Vote Down

In general, I chose Google Cloud Storage to store web sites MySQL backups due to its price and speed of upload/download in real time

I used the Google native tool – gsutil , innobackupex and some bash

in short : the /etc and local MySQL  backup

#!/bin/sh
# Barcelona Tue Nov 22 17 16:30:36 CEST 2013

days_to_keep=3
NFS=/home/mysql.backups/
exportDate=`date +%Y-%m-%d.%H.%M.%S`
export_DIR=${NFS}/${HOSTNAME}.${exportDate}
test ! -d "${export_DIR}" && echo "$(date) : creating ${export_DIR}" && mkdir -p
  [Read more...]
MEB integration with Workbench
Employee +0 Vote Up -0Vote Down

The latest versions of MySQL Workbench starting from version 6 have the capability to integrate with MySQL Enterprise Backup (MEB).

Though all the functionality of MEB is not supported, it is still a great tool to begin your backup and get your backup setup in minutes. Workbench also provides the additional capability to be able to schedule your backups.

The sections below illustrate the setup and usage of the Workbench UI to be able to setup and execute backup using MEB.

This is a quick start guide to setup, make a backup and do a recovery using MySQL Workbench and MEB.

The installation of WB and MEB is not discussed; it should be a simple automated install for both packages.

Setup

Workbench now comes with a new MySQL enterprise options in its navigator menu on the left. You need to click the Online

  [Read more...]
OurSQL Episode 160: More Free Hot Backup
+0 Vote Up -0Vote Down

This week we finish our 2-part series on xtrabackup. Ear Candy is changing the sorting behavior of GROUP BY and At the Movies is a talk on performance schema.

Events
DB Hangops - every other Wednesay at noon Pacific time

Upcoming MySQL events (http://www.mysql.com/news-and-events/events/)

Percona Live London 2013 is happening Monday November 11th and Tuesday November 12th, 2013 at the Millenium Gloucester Conference Center, with a community dinner on the 11th.

read more

OurSQL Episode 159: Free Hot Backup
+0 Vote Up -0Vote Down

This week, we talk about Percona's Xtrabackup. Ear Candy talks about .frm files and how many fields an InnoDB table can have, and At the Movies is a talk about indexes.

Xtrabackup
Episode 38 where we previously talked about Xtrabackup

Installing Percona's Xtrabackup

Operating system permissions and database privileges needed to use xtrabackup

read more

Backing up full server instance using MySQL Enterprise Backup
Employee +0 Vote Up -0Vote Down
Introduction:

MySQL Enterprise Backup(MEB) takes fast, consistent backups of MySQL server data, and helps in restoring the server to source server's data at the time of backup. But most of the times it is as much important to have same source server's state(server configuration like server global variables, plugins), as data. As backups become more frequent, server variables modified, plugins added or removed, it is very difficult to keep track of this changing server states for every backup. MEB 3.9.0 helps user in providing a complete backup so that the restored server can run with exactly the same state as that of the source server at the time of the backup.


  [Read more...]
TokuMX Hot Backup – Part 3
+0 Vote Up -0Vote Down

Last week I described TokuDB’s new Hot Backup feature.  This week we are going to briefly discuss the same feature, but as it was added to TokuMX, our version of MongoDB.

Since the Hot Backup library is essentially a shim between MySQL and the Linux kernel, intercepting file system calls for the life of the process, it should be easy to add this to any other system, including TokuMX.  Indeed with our addition of transactions and logging to TokuMX we can gain a consistent backup of any data set at any time.

Unlike MySQL, where system tables use the non-transactional MyISAM storage engine, TokuMX uses internal

  [Read more...]
Skip Unused Pages with MySQL Enterprise Backup 3.9.0
Employee +2 Vote Up -0Vote Down
Disclaimer
The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

Introduction

There are database usage patterns, where tables grow big at times and many rows are deleted from them later. InnoDB does never shrink a table space. In these cases we can end up with big data files, which contain a lot of unused pages. It is a waste of disk- and I/O- resources to back them up.

Users have manifold requested that MySQL Enterprise Backup does not back up unused InnoDB data pages. Some want smaller backups, some want less I/O, some want shrinked table spaces.

MySQL Enterprise Backup 3.9.0 can help with smaller backups. The effect on I/O is not that remarkable. InnoDB data files must be expanded to their original size when they are restored. Backup cannot accomplish a shrinkage of







  [Read more...]
MySQL Enterprise Backup: PITR Partial Online Recovery
Employee +3 Vote Up -0Vote Down

Here’s a look at using MySQL Enterprise Backup in a specific example:

Consider a Backup Policy

– Full Backup of the environment. – Complemental Incremental backups & online BinLogs. And the Restore: – Logical Restore. – Online, Zero impact. – Partial, single database, group of tables.

The Backup

A working environment, with 4 databases, of which 2 will require restoration. Full backup with MySQL Enterprise Backup:
mysqlbackup --user=root --socket=/tmp/mysql.sock \
  --backup-dir=/home/mysql/voju5/backup/ \
  --with-timestamp backup

Test preparation

Create 4 different databases, where the structure & content is the same.
create database v5_1; use v5_1; create table `voju5` (
  `ID` int(7) NOT NULL AUTO_INCREMENT,
  `Name` char(20) NOT NULL DEFAULT '‘,
  [Read more...]
MySQL Enterprise Backup: PITR Partial Online Recovery
Employee +0 Vote Up -0Vote Down

Here’s a look at using MySQL Enterprise Backup in a specific example:

Consider a Backup Policy

– Full Backup of the environment. – Complemental Incremental backups & online BinLogs. And the Restore: – Logical Restore. – Online, Zero impact. – Partial, single database, group of tables.

The Backup

A working environment, with 4 databases, of which 2 will require restoration. Full backup with MySQL Enterprise Backup:
mysqlbackup --user=root --socket=/tmp/mysql.sock \
  --backup-dir=/home/mysql/voju5/backup/ \
  --with-timestamp backup

Test preparation

Create 4 different databases, where the structure & content is the same.
create database v5_1; use v5_1; create table `voju5` (
  `ID` int(7) NOT NULL AUTO_INCREMENT,
  `Name` char(20) NOT NULL DEFAULT '‘,
  [Read more...]
Improved Backup Progress in MySQL Enterprise Backup(MEB) 3.8.2
Employee +0 Vote Up -0Vote Down

Nowadays huge amount of data is stored in databases and it is very inevitable for business needs. With parallel backup, MEB has best in class performance and continues to improve. With terabytes of data, it can take hours to backup the data. As a DBA you'd like to know how much of the backup is done, how much data in MB still remains and also what's currently happening in backup.

So MEB 3.8.2 has an enhanced progress indicator which will help the DBA in many ways like

  • You can analyze current and expected space requirements so that you can release some space by removing files on a file system or stop the backup if the space isn't there,to try other options like compressed backup.

      [Read more...]
    MySQL Enterprise Backup 3.8.2 has been released!
    Employee +3 Vote Up -0Vote Down

    MySQL Enterprise Backup v3.8.2, a maintenance release of online MySQL backup tool, is now available for download from My Oracle Support  (MOS) website as our latest GA release.  It will also be available via the Oracle Software Delivery Cloud in approximately 1-2 weeks. A brief summary of the changes in MySQL Enterprise Backup version 3.8.2 is given below.


      A. Functionality Added or Changed: 

    • MySQL Enterprise Backup has a new --on-disk-full command line option. mysqlbackup could hang when the disk became full, rather than detecting the low space condition. mysqlbackup now monitors disk space when running backup commands, and users can now specify the action to take at a disk-full condition with the --on-disk-full option. For more details, refer this

      [Read more...]
    mylvmbackup 0.14 has been released
    Employee +2 Vote Up -0Vote Down

    It's my great pleasure to announce the release of mylvmbackup version 0.14.

    This release includes a large number of improvements, code cleanups, and new functionality.

    I would like to thank Ask Bjørn Hansen, Ben Bonnel, Norbert Tretkowski, Neil Wilson, Klaus Ethgen and Alexandre Anriot for their feedback and contributions to this release.

    The release is available as a source tarball and generic RPM package. Packages for other distributions are available from the openSUSE Build Service.

    Some notable highlights from the ChangeLog

      [Read more...]
    Announcing TokuDB v7 Enterprise Edition: Hot Backup and Support
    +1 Vote Up -0Vote Down

    As promised, the Enterprise Edition of TokuDB®, Version 7, is ready. TokuDB Version 7, Enterprise Edition, introduces Hot Backup. You can now back up all your TokuDB tables directly from MySQL or MariaDB, with no down time. In addition, TokuDB Enterprise Edition comes with a support package.

    TokuDB v7 Enterprise Edition maintains all our established advantages: hot schema changes, excellent compression, fast trickle load, fast bulk load, fast range queries through clustering indexes, no fragmentation, and full MySQL/MariaDB compatibility for ease of installation.

    For details on pricing and supported MySQL and MariaDB versions, please see our FAQ.

    To learn more about TokuDB:

    • Download executables
      [Read more...]
    How to Back Up Selected MySQL Databases
    Employee +3 Vote Up -0Vote Down

    I recently had to do a backup of a 30 very large databases within a MySQL instance that had a total of 60+ databases.  I did NOT want to back up the other databases. In fact, the other databases had very large tables in them and I had very little disk space, so I could not afford to back them up. In this post I will share what I learned.

    I’ve simplified the problem to illustrate the idea.  My goal in this post is to backup one database when an instance has many, with mixed InnoDB and MyIsam tables.

    • I have 3 databases called dbtest1, dbtest2, and dbtest3, in one instance of MySQL .  I only want to back up  the tables in database dbtest2

      [Read more...]
    xtrabackup_51: not found & no ‘mysqld’ group in MySQL options
    +0 Vote Up -0Vote Down
    Recently I happen to setup a new MySQL instance with my tools – a standard MySQL 5.1+, xtrabackup setup and last-hotbackup.tar.gz. To restore from the backup we used xtrabackup binaries and ran into issues following standard commands (assuming no changes): To prepare the backup I used apply-log as follows: $] innobackupex-1.5.1 --defaults-file=/usr/local/mysql/data/backup-my.cnf --apply-log  /usr/local/mysql/data --ibbackup […]
    mysqldump now safe for InnoDB!
    Employee +6 Vote Up -0Vote Down

    It's now safe to backup InnoDB with mysqldump

    Before MySQL 5.6, running mysqldump to make backup of InnoDB tables could cause your backup to 'loose' some data. The problem is described in our manual  here.

    In latest MySQL 5.6 this is no longer a problem, this means you no longer risk 'loosing' data when using mysqlbackup together with DDL statements on your InnoDB tables. If you are interested in metadata-locking (MDL) you can read more about MDL  here.

    To test this we need to create a few tables and also look at which order mysqldump processes tables.

    mysql> CREATE DATABASE ted;
    mysql> USE ted;
    mysql> CREATE TABLE `a` (`i`







      [Read more...]
    Showing entries 1 to 30 of 209 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.