Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 269 Next 30 Older Entries

Displaying posts with tag: Tools (reset)

I'm speaking about MySQL tools on BGOUG's spring conference
Employee +0 Vote Up -0Vote Down

I will be speaking about MySQL at the Bulgarian Oracle Users Group's Spring Conference.

In furthering my mission to increase the MySQL knowledge among my countrymen I will try to give the grand tour of all the modern MySQL administrative and development tools that the MySQL team provides.

This will be the 3d BGOUG conference I'm speaking to. And I expect no less than the stellar organization and the great audience that I had the privilege to interact with during the previous 2 conferences.

We had a lot of fun too ! Parties from the last 2 conferences are

  [Read more...]
Enhancing pt-kill to Better Protect your Servers
+2 Vote Up -0Vote Down
I believe in automation as much as possible, and I'm always working to make the day to day tasks of operations as smooth as possible.  Also I try not to be afraid to take good tools and make them better.

Here in Database Ops at Box, we use pt-kill running as a service to constantly monitor our servers and help protect against long running queries.  But our thresholds are pretty generous, and in some cases it's possible for unforeseen circumstances to cause enough queries to storm the database such that we can have problems before any of them hit the threshold for "busy time."  Ditto for idle connections.

The response is that someone has to be available to manually run another copy of pt-kill with much lower thresholds to clear out these thundering herds.  But what if we could let pt-kill handle both the "normal" mode and still protect



  [Read more...]
Copy Data Between MySQL Databases with Sequel Pro
+0 Vote Up -0Vote Down

Sequel Pro

I often use Sequel Pro when I'm getting up to speed on the data model for a project or when I just want to debug in a more visual way than with the mysql command-line client. It's a free OS X application that lets you inspect and manage MySQL databases. I also find it very useful for making small changes to the data while I develop and test web apps.

Quickly Copy Data Between Databases

I recently needed a way to copy a few dozen records from one camp to another. I tried using the "SELECT...INTO OUTFILE" method but ran into a permissions issue with that approach. Using

  [Read more...]
Tool of the day: q
+1 Vote Up -0Vote Down

If you work with command line and know your SQL, q is a great tool to use:

q allows you to query your text files or standard input with SQL. You can:

SELECT c1, COUNT(*) FROM /home/shlomi/tmp/my_file.csv GROUP BY c1

And you can:

SELECT all.c2 FROM /tmp/all_engines.txt AS all LEFT JOIN /tmp/innodb_engines.txt AS inno USING (c1, c2) WHERE inno.c3 IS NULL

And you can also combine with your favourite shell commands and tools:

grep "my_term" /tmp/my_file.txt | q "SELECT c4 FROM - JOIN /home/shlomi/static.txt USING (c1)" | xargs touch

Some of q's functionality (and indeed, SQL functionality) can be found in command line tools. You can use grep for pseudo

  [Read more...]
innotop 1.9.1 released
+1 Vote Up -0Vote Down

Lefred and I spent a bit of time making innotop 1.9.1.
We’ve released a new version mainly to include MySQL 5.6 support as well as including some bugs fixed by Baron Schwartz and Frédéric Descamps.

You can download the .tar.gz and rpm’s (new!) at
http://code.google.com/p/innotop/downloads/list

Bugs fixed:

  • Issue


  [Read more...]
Foreign Data Wrappers
+0 Vote Up -0Vote Down

Original images from Flickr user jenniferwilliams

One of our clients, for various historical reasons, runs both MySQL and PostgreSQL to support their website. Information for user login lives in one database, but their customer activity lives in the other. The eventual plan is to consolidate these databases, but thus far, other concerns have been more pressing. So when they needed a report combining user account information and customer activity, the involvement of two separate databases became a significant complicating factor.

In similar situations in the past, using earlier

  [Read more...]
common_schema 2.0.0-alpha: rdebug, GPL
+1 Vote Up -0Vote Down

A new release for common_schema: an alpha version of rdebug: MySQL Debugger and Debugging API is now included with common_schema.

With a different license in mind for rdebug, common_schema changes license to GPL (2 or above).

common_schema 2.0 is ready for download. All things rdebug, it is alpha -- otherwise it's a stable release.

rdebug

I'm very happy to release this alpha version of rdebug, and urge everyone to try it out.

The idea is to have an open, free, server side debugger and debugging API for MySQL stored routines. To elaborate:

      [Read more...]
    Cheating mysqlsandbox to install MariaDB 10.0
    +3 Vote Up -0Vote Down

    mysqlsandbox is version-aware. The new 5.6 version, for example, requires special care because of the system InnoDB tables or otherwise modified system tables.

    At this moment, it will refuse to install MariaDB 10.0 (alpha):

    bash$ make_sandbox /tmp/mariadb-10.0.1-linux-x86_64.tar.gz 
    unpacking /tmp/mariadb-10.0.1-linux-x86_64.tar.gz
    unsupported version 10.0

    This is perfectly legitimate, and I have no quarrel with this fact. However, I did want to setup MariaDB 10.0 as a sandbox.

    As it turns out mysqlsandbox relies on MySQL package

      [Read more...]
    Nagios Check Calculated on Mysql Server Variables
    +0 Vote Up -0Vote Down

    Nagios Check For Calculating Based on Mysql Server Variables

    Recently we needed to make a change for a client to one of our mysql monitoring tools so I thought it would be a good opportunity to highlight the tool and discuss some of the changes that I made.

    You can access the tool on Github from our public repository here.

    Before this change if you were using either the "varcomp" or "lastrun-varcomp" modes, it would only return a WARNING if your criteria for comparison were exceeded. In the new version, both WARNING and CRITICAL states can return to nagios. Here is an example: Let's say you want to alert on maximum connections, but the number of maximum connections is different for different hosts. Instead of writing distinct

      [Read more...]
    Nagios Check Calculated on Mysql Server Variables
    +0 Vote Up -0Vote Down

    Nagios Check For Calculating Based on Mysql Server Variables

    Recently we needed to make a change for a client to one of our mysql monitoring tools so I thought it would be a good opportunity to highlight the tool and discuss some of the changes that I made.

    You can access the tool on Github from our public repository here.

    Before this change if you were using either the "varcomp" or "lastrun-varcomp" modes, it would only return a WARNING if your criteria for comparison were exceeded. In the new version, both WARNING and CRITICAL states can return to nagios. Here is an example: Let's say you want to alert on maximum connections, but the number of maximum connections is different for different hosts. Instead of writing distinct

      [Read more...]
    Percona Toolkit by example – pt-stalk
    +1 Vote Up -0Vote Down

    pt-stalk recipes: Gather forensic data about MySQL when a server problem occurs

    It happens to us all from time to time: a server issue arises that leaves you scratching your head. That’s when Percona Toolkit’s pt-stalk comes into play, helping you diagnose the problem by capturing diagnostic data that helps you pinpoint what’s causing the havoc hitting your database.

    From the documentation (http://www.percona.com/doc/percona-toolkit/pt-stalk.html):

    pt-stalk watches for a trigger condition to become true, and then collects data to help in diagnosing problems. It is designed to run as a daemon with root privileges, so that you can diagnose intermittent problems that you cannot observe directly. You can also use it

      [Read more...]
    Get Me Some Query Logs!
    +1 Vote Up -0Vote Down

    One of my favorite tools in the Percona Toolkit is pt-query-digest.  This tool is indispensable for identifying your top SQL queries, and analyzing which queries are accounting for your database load.

    But the report you get from pt-query-digest is only as good as the log of queries you give it as input.  You need a large enough sample of query logs, collected over a period of time when you have representative traffic on your database.

    You also need the log to include all the queries, not just those that take more than N seconds.  The reason is that some queries are individually quick, and would not be logged if you set the long_query_time

      [Read more...]
    How to send input to many terminals
    +0 Vote Up -0Vote Down

    Do you ever find yourself wanting to open several terminal windows and send the same commands to all of them? I’ve had this need many times, and I’ve never found a completely satisfactory solution. I’ve also known a lot of people who’ve written various sets of scripts to help them accomplish such tasks.

    In no particular order, here are a few ways I’ve done this in the past:

  • Facebook’s pmysql client
  • The dsh tool
  • Several screen windows named remoteXXX, followed by a bash for-loop: while read cmd; do screen -X at remote# stuff "$cmd"; done
  • Using many PuTTY windows and the puttycs tool
  • Opening many tabs in KDE’s Kterm tool and selecting the
  •   [Read more...]
    Here’s a quick way to Foresee if Replication Slave is ever going to catch up and When!
    +1 Vote Up -0Vote Down

    If you ever had a replication slave that is severely behind, you probably noticed that it’s not catching up with a busy master at a steady pace. Instead, the “Seconds behind master” is going up and down so you can’t really tell whether the replica is catching up or not by looking at just few samples, unless these are spread apart. And even then you can’t tell at a glance when it is going to catch up.

    Normally, the “severely behind” thing should not happen, but it does often happen in our consulting practice:

    • sometimes replication would break and then it needs to catch up after it is fixed,
    • other times new replication slave is built from a backup which is normally hours behind,
    • or, it could be that replication slave became too slow to catch
      [Read more...]
    pt-online-schema-change
    +0 Vote Up -0Vote Down
    This is one of many perl script tools from Percona that makes life easier administrating, monitoring and testing mysql, this one solves the need to alter a table (adding a key, column, partitioning, etc…) on a production system that can’t be taken down. Many of the current Percona tools came from maatkit, developed by Baron [...]
    Exploring Configuration Management with Ansible
    +0 Vote Up -0Vote Down

     

    What is Ansible?

    Ansible is a configuration management and deployment system, like Puppet, Capistrano, Fabric, and Chef. Its aim is to be radically simple and let you use your existing scripts to help with cluster configuration and software deployment whenever possible. Here are the ways that Ansible differentiates itself.

    Simplicity

    Ansible does not include a client/server architecture with pull-based clients (although in more recent versions, it does include pull-based configuration and deployment). Rather, it uses pre-existing network infrastructure: SSH. Every company has SSH installed on their cluster servers, and Ansible simply rides on top of this infrastructure to get the code and configuration out to the nodes.

    Language

      [Read more...]
    Net tools
    +0 Vote Up -3Vote Down

    A set of internet tools, is available at otala.net/tools.shtml

    • Traceroute
    • Whois & JWhois
    • NSLookup
    • Ping
    • DNSWalk
    • NetStat
    • ENUM (e164.org phone number lookup)

    More to come, as (if/when) I get time -- suggestions welcome

    Communication in a distributed company: Choose your weapons!
    +1 Vote Up -4Vote Down

    Everyone has their preferred method of communication, which they tend to overuse at the cost of other methods. At MySQL AB, email was preferred in many situations where picking up the phone would have saved loads of work, frustration and distractions. SkySQL Ab as a company is equally distributed across continents and time zones. Not surprisingly, we have inherited some of MySQL's bad habits, and created a set of new habits, good and bad. The purpose of this write-up is to take a step back and reflect upon which tool fits which purpose, focusing on a virtual company with colleagues spanning many time zones.

    read more

    pt-diskstats 2.1 may return bad results
    +2 Vote Up -0Vote Down

    Do you rely on pt-diskstats from Percona Toolkit instead of the standard iostat a lot? There appears to be a nasty bug in pt-diskstats 2.1, which makes it produce bad results.

    I noticed some of the numbers I was getting weren’t right, so I tried running iostat and two different releases of pt-diskstat side by side. Here’s what I got:

    I can understand the slight differences between the lines in iostat and pt-diskstats 2.0 as they probably weren’t reading /proc/diskstats contents in the same moments, so the values they were seeing could be a bit different. However both lines practically show the same thing.

    On the other hand, the line based on the pt-diskstats

      [Read more...]
    7 essential tools for MySQL DBA
    +2 Vote Up -0Vote Down

    1. Percona Toolkit
    Percona Toolkit (aka Maatkit and Aspersa) is must have collection of advanced command-line tools which helps in performing tasks that are too difficult or complex to perform manually.

    2. Mydumper
    Mydumper is a high-performance multi-threaded backup/restore tool for MySQL. It’s up to 10x faster compared to mysqldump, can take consistent snapshots and provides File compression on-the-fly. Though it’s still under active development but is well tested/used in production on some large installations.

    3. MySQL Master HA
    This tool helps to maintain your Master-Slave



      [Read more...]
    Chain Copying to Multiple hosts
    +1 Vote Up -0Vote Down

    This week I was given the task of repopulating our entire primary database cluster.  This was due to an alter that had to be performed on our largest table.  It was easiest to run it on one host and populate the dataset from that host everywhere.

    I recalled a while back reading a blog post from Tumblr about how to chain a copy to multiple hosts using a combination of nc, tar, and pigz.  I used this, with a few other things to greatly speed up our repopulation process.  As I was repopulating production servers, I did a combination of raw data copy and xtrabackup streams across our servers, depending on the position in our replication setup.

    For a normal straight copy, here’s what I did:

    On the last host, configure netcat to listen and then

      [Read more...]
    Taxonomy of database tools
    +1 Vote Up -0Vote Down

    Taxonomy of Database Tools

    In the MySQL ecosphere there is an ecosystem of tools.  Like real-world ecosystems, the “creatures” in the MySQL tools ecosystem can be classified and organized by a taxonomy.  There are already multiple taxonomies of software bugs (e.g. A Taxonomy of Bugs), but as far as I know this is the first Taxonomy of Database Tools.  A taxonomy of database tools serves useful purposes, as discussed in the previously linked page.  For me, the most useful purpose is the high-level ecosystem view which I


      [Read more...]
    Making MySQL comfortable for Oracle DBAs
    +0 Vote Up -1Vote Down

    I’m at Hotsos Symposium this week, and it suddenly occurred to me that a lot of Oracle DBAs who are beginning to manage MySQL servers might have some things to share with others in a similar role shift:

    • Familiar, comfortable tools and techniques, or capabilities of the Oracle Database, that you miss in MySQL
    • Equivalents or replacements for the aforementioned

    Do you have anything to share with your fellow DBAs going through ODT (Oracle Delerium Tremens)? Do you have any wishes that you haven’t satisfied yet? Post in the comments and let’s see if we can create a sort of forum for sharing and/or a wishlist in case someone gets an urge to fill in a missing piece!

    Further Reading:

      [Read more...]
    MySQL Comparison Tools
    +0 Vote Up -0Vote Down
    Here are the lists of useful tools in comparing database snapshots, files and even folders.

    1. dbForge Studio - It is a complete suite for all your needs in managing all your MySQL databases. This software can analyze your data and compare the differences and quickly create the diff statement ready to be executed. dbForge also offers Free edition the dbforge Studio for MySQL Express but of course with limited functionality

    Features | Download

    • Administration and Maintenance
    • Backup
    • Data Analysis
    • Data Comparison
    • Data Editor
    • Database Explorer
    • Debugger
    • Exporting and Importing Data
    • MySQL






      [Read more...]
    Monitoring your monitoring tools (MONyog inside) !
    +6 Vote Up -1Vote Down

    Regardless of the monitoring tool you use to monitor your databases, it can be better to monitor this tool.
    No, it’s not a joke ! Do you think you can have a benefit with a monitoring tool not connected to your servers ? ( without being alerted )

    I choose to talk about MONyog here but this can apply to all existing monitoring tools.
    I just want to share the message, the tool does not matter, so, do it !

    So, let me explain how to control if you have fresh data with MONyog.
    With MONyog it’s easy because it’s an agentless monitoring tool.

    There are two ways to check that :

    Per server general info :

     





      [Read more...]
    Cool Percona Tools
    +0 Vote Up -0Vote Down

    Part of my job at Percona is to perform customer audits.  I’ve had fun the last week getting up to speed on some of the great tools in the Percona Toolkit.  I’m sure people have talked about these before, but to be honest, I always found it hard to take the time to learn new tools, and at my previous job there were often a lot of tools in-house that made it easier to avoid learning something new.  The great thing about the Percona tools is that they aren’t environment dependent (as much as is possible), they typically only require Perl to run.

    So, with that being said, I wanted to just go through a few of the tools I learned about this last week that I found useful.

    pt-collect:

    The collect script collects information.  A lot of information.  I ran it for 10 minutes on a customer’s

      [Read more...]
    Want to hack Maatkit and Aspersa? We’re hiring
    +0 Vote Up -0Vote Down

    As announced on the Maatkit and Aspersa mailing lists, Daniel and I have created a new toolkit that represents the union of the two, and will be focusing efforts on this Percona Toolkit moving forward. The goal is to make them simpler and significantly more powerful, and to create more tools. The tools will continue to be open-source, but will be developed primarily to meet our MySQL support and consulting staff’s needs.

    If you’re interested in challenging software engineering in Perl and shell, then please apply online. You can work online from anywhere, but I strongly prefer someone in the Americas timezones.

    Further Reading:

      [Read more...]
    MySQL HA Agent Mini HowTo
    +1 Vote Up -0Vote Down

    Why This Post


    While testing Yoshinori Matsunobo's MHA agent I found that although the wiki has a very complete documentation, it was missing a some details. This article intends to close that gap and bring up some issues to keep in mind when you do your own installation. At the end of the article I added a Conclusions section, if you're not interested in the implementation details, but to read my take on the project, feel free to jump straight to the end from here.

    My Test Case


    Most of our production environments can be simplified to match the MHA's agent most simple use case: 1 master w/ 2 or more slaves and at least one more slave in an additional



      [Read more...]
    MySQL tools article
    +1 Vote Up -0Vote Down

    Although I no longer maintain hackmysql.com, I still actively develop MySQL tools. An article I wrote, 10 essential MySQL tools for admin, was published today. I hope no one’s feelings are hurt if their tool isn’t in the list, but it was rather difficult to compile the list given that so many tools are either not actively developed, not tested, or not well documented. Given lag time between writing and publishing, I was not able to write about Yoshinori’s MHA or newer tools. And given length constraints, I was not able to write about more tools. In any case, the world of MySQL tools is alive and well.

    My MySQL SNMP Agent
    +1 Vote Up -0Vote Down
    Back in February I wrote an article titled A Small Fix For mysql-agent. Since then we did a few more fixes to the agent and included a Bytes Behind Master (or BBM) chart. For those who can't wait to get their hands on the code, here's the current version: MySQL SNMP agent RPM. For those who'd like to learn about it's capabilities and issues, keep reading.

    What to Expect From this Version


    The article I quoted above pretty much describes the main differences with the original project, but we went further with the changes while still relying on Masterzen's code for the data collection piece.

    The first big change is that we transformed




      [Read more...]
    Showing entries 1 to 30 of 269 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.