Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 37 Next 7 Older Entries

Displaying posts with tag: lvm (reset)

LVM read performance during snapshots
+1 Vote Up -0Vote Down

For the same customer I am exploring ZFS for backups, the twin server is using regular LVM and XFS. On this twin, I have setup mylvmbackup for a more conservative backup approach. I quickly found some odd behaviors, the backup was taking much longer than what I was expecting. It is not the first time I saw that, but here it was obvious. So I recorded some metrics, bi from vmstat and percent of cow space used from lvs during a backup. Cow space is the Copy On Write buffer used by LVM to record the modified pages like they were at the beginning of the snapshot. Upon reads, LVM must scan the list to verify that there’s no newer version. Here’s the other details about the backup:

  • Filesystem: 2TB, xfs
  • Snapsize: 60GB
  • Amount to backup: ~600GB
  • Backup tool: mylvmbackup
  • Compressor: pbzip2

  [Read more...]
Log Buffer #282, A Carnival of the Vanities for DBAs
+0 Vote Up -0Vote Down
Blogging is the way to express an idea in an informal way by a person, who has either worked with it, or planning to work with it, or has seen it at work. In databases, ideas are the foundations of everything, and these Log Buffer Edition is presenting ideas from bloggers in Log Buffer #282. [...]
Linus on Instantiation and Armadaification
+0 Vote Up -1Vote Down

I feel a sense of pride when I think that I was involved in the development and maintenance of what was probably the first piece of software accepted into Debian which then had and still has direct up-stream support from Microsoft. The world is a better place for having Microsoft in it. The first operating system I ever ran on an 08086-based CPU was MS-DOS 2.x. I remember how thrilled I was when we got to see how my friend’s 80286 system ran BBS software that would cause a modem to dial a local system and display the application as if it were running on a local machine. Totally sweet.

When we were living at 6162 NE Middle in the nine-eight 292, we got an 80386 which ran Doom. Yeah, the original one, not the fancy new one with the double barrel shotgun, but it would probably run that one, too.

  [Read more...]
The blog was down yesterday
+0 Vote Up -0Vote Down

The brief outage was due to a scheduled move of the servers to a separate rack and subnet dedicated to our work with the Center for Information Assurance & Cybersecurity (ciac) at the University of Washington Bothell (uwb), and a11y.com

I am currently exercising the new (to us) equipment and hope to winnow the less than awesome equipment over the next quarter. I spent the last six months finding the best in breed of the surplussed DL385 and DL380 chassis we (work) were going to have recycled. The team and I were able to find enough equipment to bring up one of each with eight and six gigs of memory, respectively. These will make excellent hypervisors for provisioning embedded instances of Slackware, Fedora, RHEL, CentOS, Debian, FreeBSD, OpenSolaris, OpenIndiana, FreeDOS, etc.

When I initially configured this xen paravirt environment, I failed to plan for integration with libvirt, so I am

  [Read more...]
Vote for MySQL[plus] awards 2011 !
+5 Vote Up -0Vote Down

First of all, I wish you a happy new year.
Many things happened last year, it was really exciting to be involved in the MySQL ecosystem.
I hope this enthusiasm will be increased this year, up to you !

To start the year, I propose the MySQL[plus] Awards 2011
It will only take 5 minutes to fill out these polls.
Answer with your heart first and then with your experience with some of these tools or services.

Polls will be closed January 31, so, vote now !
For “other” answers, please,  let me a comment with details.

Don’t hesitate to submit proposal for tools or services in the comments.






  [Read more...]
Join us at the OTN Sys Admin Day for Oracle Linux and Solaris on Sep. 22nd, Seattle (WA)
Employee +3 Vote Up -0Vote Down

Last week we concluded our first Oracle Technology Network Sys Admin Day in Sacramento (CA). Well, it was actually the second Sys Admin Day, but the first one that had two parallel tracks of sessions about both Oracle Linux and Oracle Solaris.

I helped preparing for the event by creating the Linux lab handbook as well as the VirtualBox appliance of Oracle Linux 6.1 that was used for the exercises. Unfortunately I could not be there in person, but it would have been pointless for me to go on an intercontinental flight just for one day.

From the feedback we've received so far, the attendees really enjoyed the event and were positively surprised about the depth and quality of the practical

  [Read more...]
On LVM: How to setup Volume Groups and Logical Volumes.
+0 Vote Up -0Vote Down
LVM (Logical Volume Management) is a very important tool to have in the toolkit of a MySQL DBA. It allows you to create and extend logical volumes on the fly. This allows me to, say, add another disk and extend a partition effortlessly. The other very important feature is the ability to take snapshots, that you can then use for backups. All in all its a must have tool. Hence, this guide will allow you to understand various terminologies associated with LVM, together with setting up LVM volumes and in a later part will also show you how to extend...
451 CAOS Links 2011.05.03
+0 Vote Up -0Vote Down

Novell sold to Attachmate. Barnes & Noble throws the book at Microsoft. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca, and daily at Paper.li/caostheory
“Tracking the open source news wires, so you don’t have to.”

# Novell closed its acquisition by Attachmate and its patent sale to CPTN.

# Attachmate’s CEO discussed the company’s plans for SUSE Linux.

# Barnes & Noble


  [Read more...]
Spreading .ibd files across multiple disks; the optimization that isn’t
+1 Vote Up -0Vote Down

Inspired by Baron's earlier post, here is one I hear quite frequently -

"If you enable innodb_file_per_table, each table is it's own .ibd file.  You can then relocate the heavy hit tables to a different location and create symlinks to the original location."

There are a few things wrong with this advice:

  • InnoDB does not support these symlinks.  If you run an ALTER TABLE command, what you will find is that a new temporary table is created (in the original location!), the symlink is destroyed, and the temporary table is renamed.  Your "optimization" is lost.
  • Striping (with RAID) is usually a far better optimization.  Striping a table across multiple disks effectively balances the  'heavy
  •   [Read more...]
    Using LVM snapshot filesystems for development database instances
    +0 Vote Up -0Vote Down

    The Problem

    Developers often need to have a development database copy of the live production system you are using in able to allow them to test their code and to test new functionality and make schema changes to the database for this new functionality to work.

    That’s normal and happens everywhere. A typical DBA task is to make a copy of the live system, sometimes to remove any confidential or sensitive information which perhaps the development database users should not be able to see, and then give them access to this development instance. The developers then “hack away”, changing their code and perhaps things in the database until they are ready to put these new changes into production when they then come along and discuss how to apply these changes into the live systems.

    Once the

      [Read more...]
    Easy Python: display LVM details in XML
    +1 Vote Up -1Vote Down

    If you need to work with LVM in your scripts but haven’t found a good method to access details about Logical Volume Groups, here’s a simple Python script that will print the details about any volumes on your system. This could be useful for writing a partition check script for your MySQL data directory (if you’re not using a standard monitoring system like Nagios).

    import sys
    import os
    import commands
    import subprocess
    import select
    
    def lvm():
        print ""
        LVM_PATH = "/sbin"
        LVM_BIN = os.path.join(LVM_PATH, 'lvm')
        argv = list()
        argv.append(LVM_BIN)
        argv.append("lvs")
        argv.append("--nosuffix")
        argv.append("--noheadings")
        argv.append("--units")
        argv.append("b")
        argv.append("--separator")
        argv.append(";")
        argv.append("-o")
        argv.append("lv_name,vg_name,lv_size")
    
        process = subprocess.Popen(argv,
      [Read more...]
    mylvmbackup 0.13 has been released
    Employee +4 Vote Up -0Vote Down

    I am happy to announce that mylvmbackup version 0.13 has now been released. This release includes a fix for a nasty bug in on of the recently added Perl hooks (precleanup.pm) and some added functionality (better support for remote rsync backups).

    From the ChangeLog: 

    • Deleted sample precleanup.pm hook as it has potential to cause harm and is too specialized on a particular use case (BUG#394668)
    • Added support for rsync via SSH (BUG#392462)
    • Fixed InnoDB recovery in case a relative path to the MySQL data directory is defined (BUG#38337), improved the documentation of relpath in the man page.

     

    Setting up LVM on Suse Linux
    Employee +0 Vote Up -0Vote Down

    You can do all sorts of magic with LVM, like backing up MySQL using file system snapshots, but in order to do so you need to set it first. When installing a new computer with Suse Linux I found that this was harder than expected, unless you know what to do. Needless to say, I didn't, but some friends gave me a hand on IRC, and now I do. :-)

    So I wrote a little article about this which you can find here, in an effort to save others some time. The article is about setting up LVM with Suse Yast, but you can probably use it for other flavors of Linux, too.

    Setting up LVM on Suse Linux
    Employee +0 Vote Up -0Vote Down

    You can do all sorts of magic with LVM, like backing up MySQL using file system snapshots, but in order to do so you need to set it first. When installing a new computer with Suse Linux I found that this was harder than expected, unless you know what to do. Needless to say, I didn't, but some friends gave me a hand on IRC, and now I do. :-)

    So I wrote a little article about this which you can find here, in an effort to save others some time. The article is about setting up LVM with Suse Yast, but you can probably use it for other flavors of Linux, too.

    mylvmbackup-0.12 has been released
    Employee +2 Vote Up -0Vote Down

    After a long hiatus, I am happy to announce that mylvmbackup version 0.12 has now been released. This release includes a large number of improvements, minor code cleanups, as well as some new functionality. In particular, I would like to thank Matthew Boehm, Tim Stoop, Baron Schwartz, Ville Skyttä and Ronald Bradford for their contributions.

    Some notable highlights from the ChangeLog:

    • Removed the absolute path names to external tools (make sure $PATH is correct)
    • Added --log-err to the startup options of the recovery instance to avoid cluttering the server's error log
    • Added support for hooks written as Perl Modules. (Matthew Boehm)
    • Added support for date/time-formatted path names for backupdir and mountdir (Matthew Boehm)

      [Read more...]
    My upcoming talks and events
    Employee +0 Vote Up -0Vote Down

    My calendar for the upcoming months is already filling up with conferences, trade fairs and other events at which I'll speak about MySQL. Here's a quick overview:

    • This coming Thursday at 15:00 CET, I'll be speaking about "Backing up MySQL using file system snapshots" at the MySQL University. The session will be hosted live using DimDim, which is a great online conferencing and presentation system (Flash required). Attendance is free, so come and join me if you want to learn more about this backup technique!
    • On Friday, 6th of March at 15:15 I'll give a presentation about "MySQL Backup and Security" in the
      [Read more...]
    MySQL University: Backing up MySQL using file system snapshots
    Employee +0 Vote Up -0Vote Down

    This Thursday (February 26th, 14:00 UTC), Lenz Grimmer will give a MySQL University session on Backing up MySQL using file system snapshots. Lenz is a member of the MySQL Community team and the maintainer of the mylvmbackup tool. mylvmbackup is a tool for quickly creating backups of a MySQL server's data files. To perform a backup, mylvmbackup obtains a read lock on all tables and flushes all server caches to disk, creates a snapshot of the volume containing the MySQL data directory, and unlocks the tables again. The snapshot process takes only a small amount of time. When it is done, the

      [Read more...]
    mylvmbackup-0.11 has been released
    Employee +0 Vote Up -0Vote Down

    Some days ago, I released version 0.11 of mylvmbackup a Perl script that performs consistent backups of a MySQL server by using LVM filesystem snapshots. The source archive as well as a generic RPM can be found on the project home page, packages for many Linux distributions are available on the openSUSE Build service.

    This release includes some new functionality as well as numerous bug fixes and improvements, most notably:

    • Added support for using rsnap as a backup backend (Matt Lohier)
    • The documentation is now maintained in POD style instead of asciidoc (Matthew Boehm)
    • Support using non-GNU tar
      [Read more...]
    mylvmbackup-0.11 has been released
    Employee +0 Vote Up -0Vote Down

    Some days ago, I released version 0.11 of mylvmbackup a Perl script that performs consistent backups of a MySQL server by using LVM filesystem snapshots. The source archive as well as a generic RPM can be found on the project home page, packages for many Linux distributions are available on the openSUSE Build service.

    This release includes some new functionality as well as numerous bug fixes and improvements, most notably:

    • Added support for using rsnap as a backup backend (Matt Lohier)
    • The documentation is now maintained in POD style instead of asciidoc (Matthew Boehm)
    • Support using non-GNU tar
      [Read more...]
    Success with OpenSolaris + ZFS + MySQL in production!
    +0 Vote Up -0Vote Down

    Pimp My Drive by Richard and Barb

    There’s remarkably little information online about using MySQL on ZFS, successfully or not, so I did what any enterprising geek would do: Built a box, threw some data on it, and tossed it into production to see if it would sink or swim.

    I’m a Linux geek, have been since 1993 (Slackware!). All of SmugMug’s datacenters (and

      [Read more...]
    mylvmbackup 0.10 has been released
    Employee +0 Vote Up -0Vote Down

    I am happy to announce that mylvmbackup version 0.10 has been released.

    You can download the updated package from the project home page or via the openSUSE Build Service.

    This version fixes some bugs and includes new functionality:

    • Applied patch from Marc Haber: added option --keep_snapshot that will skip the removal of the backup snapshot before terminating the script. Providing the option --backuptype=none will now skip creating a backup using the builtin backup modules. Both options provide more flexibility when using hooks for performing the actual backup tasks or when the snapshot is considered to be the actual backup.
    • Added two new hooks: "backupsuccess" and "backupfailure" which are called respectively upon success of failure of the backup operation (
      [Read more...]
    mylvmbackup 0.10 has been released
    Employee +0 Vote Up -0Vote Down

    I am happy to announce that mylvmbackup version 0.10 has been released.

    You can download the updated package from the project home page or via the openSUSE Build Service.

    This version fixes some bugs and includes new functionality:

    • Applied patch from Marc Haber: added option --keep_snapshot that will skip the removal of the backup snapshot before terminating the script. Providing the option --backuptype=none will now skip creating a backup using the builtin backup modules. Both options provide more flexibility when using hooks for performing the actual backup tasks or when the snapshot is considered to be the actual backup.
    • Added two new hooks: "backupsuccess" and "backupfailure" which are called respectively upon success of failure of the backup operation (
      [Read more...]
    mylvmbackup 0.9 has been released
    Employee +0 Vote Up -0Vote Down

    I am happy to announce that a new version (0.9) of mylvmbackup has been released. This is the first release since the source code has been moved from Subversion to Bazaar and is now hosted on Launchpad.net. I would like to thank Robin H. Johnson and Patrick Hahn for providing the patches that contributed to this new release!

    mylvmbackup is a tool for quickly creating backups of MySQL server's data files. To perform a backup, mylvmbackup obtains a read lock on all tables and flushes all server caches to disk, makes an LVM snapshot of the volume containing the MySQL data directory, and unlocks the tables again. The snapshot process takes only a small amount of time. When it is done, the server can continue normal operations, while the actual file backup

      [Read more...]
    The mylvmbackup source tree has moved to Bazaar/Launchpad
    Employee +0 Vote Up -0Vote Down

    JFYI: today I migrated the mylvmbackup source tree from my local Subversion repository on http://www.lenzg.org/ to a Bazaar repository on Launchpad.net.

    This will hopefully make it easier for contributors to work on the code and share their modifications with others, removing me as the bottleneck for applying and testing patches for new releases. I chose Bazaar primarily because I wanted to get some more hands-on practice with it, now that the MySQL Server source trees have been transferred to it as well (see Kaj's announcement for

      [Read more...]
    LVM Backup slides published
    Employee +0 Vote Up -0Vote Down

    JFYI, I now placed a PDF of my MySQL Conference talk slides about "Performing MySQL backups using Linux LVM Snapshots" on my MySQL talks page. Enjoy!

     

    Zumastor as an alternative for LVM/DRBD?
    Employee +0 Vote Up -0Vote Down

    While reading Colin's post about LugRadio Live, I stumbled over the Zumastor Linux Storage Project. Going through the project home page and their HOWTO got me curious - could this eventually become an alternative to using DRBD (for replicating data) and LVM snapshots (for performing backups)?

    Zumastor is Free software that adds enterprise storage features (primarily improved snapshots and remote replication) to Linux.

    Snapshots

    LVM already lets administrators create snapshots, but its design has the surprising property that every block you change on the original volume consumes one block for each snapshot.

      [Read more...]
    mylvmbackup 0.8 has been released
    Employee +0 Vote Up -0Vote Down

    I am happy to announce the release of mylvmbackup version 0.8. mylvmbackup is a tool for quickly creating backups of a MySQL server's data files. To perform a backup, mylvmbackup obtains a read lock on all tables and flushes all server caches to disk, makes an LVM snapshot of the volume containing the MySQL data directory, and unlocks the tables again. The snapshot process takes only a small amount of time. When it is done, the server can continue normal operations, while the actual file backup proceeds.

    Below is the list of changes since version 0.6. You may wonder what happened to version 0.7 - it had a rather short life cycle as I was informed about a bug that I fixed quickly before I made a wider release announcement of 0.7.

    • Fixed a bug in the InnoDB
      [Read more...]
    Book review: "Xen Virtualization" by Prabhakar Chaganti (Packt Publishing)
    Employee +0 Vote Up -0Vote Down

    I recently received a review copy of the book "Xen Virtualization" by Prabhakar Chaganti (Packt Publishing) and finished reading it a few days ago.

    The subtitle "A fast and practical guide" is a matching description - I managed to read the ~130 pages over the course of a week. The book is by no means an exhaustive reference manual, but it gives the reader a good overview about Xen and assists with performing the first steps and getting started.

     


    Continue reading "Book review: "Xen Virtualization" by Prabhakar Chaganti (Packt Publishing)"
    Speaking at the MySQL Conference & Expo 2008
    Employee +0 Vote Up -0Vote Down

    I am happy to announce that I have been selected to speak at the MySQL Conference & Expo 2008 on April 14-17, 2008 in Santa Clara, CA. The topic of my talk will be "Performing MySQL Backups using LVM Snapshots" - and I will of course include a plug for the mylvmbackup tool

    From the abstract:

    Modern Linux distributions provide a very advanced technology to maintain and administer storage devices, called “Logical Volume Manager” (LVM). Among


      [Read more...]
    mylvmbackup 0.6 has been released
    Employee +0 Vote Up -0Vote Down

    Version 0.6 of mylvmbackup, a script to perform backups of a MySQL server using Linux LVM snapshots, has now been released.

    In addition to various code cleanups and documentation improvements, many new features have been added to this version. I'd like to specially thank Robin H. Johnson from the Gentoo project for contributing many of the improvements to this release!

    • Added a new rsync backup type. This is very useful if you want to use mylvmbackup to create the initial state for your slave servers. Instead of creating a .tar.gz archive, the data directory is copied into a timestamped archive directory. (Robin)
    • Added support for a trailing argument to tar, which can be used for excluding files. (Robin)
    • Separated out the suffix of the tarball (Preperation for rsync and users that want to use bzip2 or no compression on the
      [Read more...]
    Showing entries 1 to 30 of 37 Next 7 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.