Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 44 Next 14 Older Entries

Displaying posts with tag: Sys-Admin (reset)

New Maatkit tool to compute index usage
+1 Vote Up -0Vote Down

In a couple of recent consulting cases, I needed a tool to analyze how a log of queries accesses indexes and tables in the database, specifically, to find out which indexes are not used. I initially hacked together something similar to Daniel Nichter’s mysqlidxchk, but using the framework provided by Maatkit, which gave me a pretty good start right out of the box. This was useful in the very tight time constraints I was under, but was not a complete solution. Alas, I could not use anything like Percona’s enhancements for finding unused indexes.

So, in response to another consultant’s customer request (and sponsorship — thank you!) I spent more time actually writing a real tool in the Maatkit style, with

  [Read more...]
Using Aspersa to capture diagnostic data
+2 Vote Up -0Vote Down

I frequently encounter MySQL servers with intermittent problems that don’t happen when I’m watching the server. Gathering good diagnostic data when the problem happens is a must. Aspersa includes two utilities to make this easier.

The first is called ’stalk’. It would be called ‘watch’ but that’s already a name of a standard Unix utility. It simply watches for a condition to happen and fires off the second utility.

This second utility does most of the work. It is called ‘collect’ and by default, it gathers stats on a number of things for 30 seconds. It names these statistics according to the time it was started, and places them into a directory for analysis.

Here’s a sample of how to use the tools. In summary: get them and make

  [Read more...]
Aspersa, a new opensource toolkit
+0 Vote Up -0Vote Down

Some of the utilities we were adding to Maatkit really did not belong there. Yes, this included some of the functionality in the now-retired mk-audit tool. We really learned a lesson about what it’s possible to support, design, spec, code, and test in a single tool.

I’ve moved those tools to a new project, Aspersa. Some folks are revolting and calling it Asparagus, because apparently that’s easier to say. Aspersa is the name of the common garden snail, which turns out to be a fascinating creature. It is also slow. Draw your own conclusions.

This project is more of a home for simple scripts and snippets — a simple place I can grab all the little utilities I use to make my life easy. There is a “summary” tool that largely replaces

  [Read more...]
Version 1.1.7 of Better Cacti Templates released
+2 Vote Up -0Vote Down

I’ve released version 1.1.7 of the Better Cacti Templates project. This release includes a bunch of bug fixes, some new graphs for MySQL, and two new sets of graphs, for Redis graphing and for JMX graphing.

There are upgrade instructions on the project wiki for this and all releases. There is also a comprehensive tutorial on how to create your own graphs and templates with this project. Use the project issue tracker to view and report issues, and use the project mailing list to

  [Read more...]
Version 1.1.6 of Better Cacti Templates released
+1 Vote Up -0Vote Down

I’ve released version 1.1.6 of the Better Cacti Templates project. This release includes a bunch of bug fixes (but not all of them!) and two new sets of graphs. One set is for disk I/O on GNU/Linux, and the other is a new set of templates for OpenVZ. I’m looking for feedback on both of those. This release also has a bunch of code-level features: much better test coverage (hooray!), and a refactored ss_get_by_ssh.php that makes it much easier to create new graphs and templates. The SSH-based templates also take advantage of the same caching as the MySQL templates, which makes them a lot more efficient.

There are upgrade instructions on the project wiki for this and all releases. There is also

  [Read more...]
How Linux iostat computes its results
+4 Vote Up -0Vote Down

iostat is one of the most important tools for measuring disk performance, which of course is very relevant for database administrators, whether your chosen database is Postgres, MySQL, Oracle, or anything else that runs on GNU/Linux. Have you ever wondered where statistics like await (average wait for the request to complete) come from? If you look at the disk statistics the Linux kernel makes available through files such as /proc/diskstats, you won’t see await there. How does iostat compute await? For that matter, how does it compute the average queue size, service time, and utilization? This blog post will show you how that’s computed.

First, let’s look at the fields in /proc/diskstats. The order and location varies between kernels, but the following

  [Read more...]
Vote for Cacti template enhancements
+2 Vote Up -1Vote Down

If you’d like some improvements to the Cacti templates (MySQL, Apache, Memcached, and more) I’ve been maintaining, please make your voice heard — either write to the mailing list, or post a new issue (or comment on an existing one) on the issue list. I’ve got a bunch of work underway, including a test suite for ss_get_by_ssh.php.

Related posts:

  • Version 1.1.6 of Better Cacti Templates released I’ve
  • Version 1.1.2 of improved Cacti templates released I’ve
  •   [Read more...]
    Version 1.1.5 of improved Cacti templates released
    +0 Vote Up -0Vote Down

    I’ve released version 1.1.5 of my improved Cacti templates for MySQL and other components of a LAMP application. This is a pure bug-fix release. One of the bug fixes prevents spikes in graphs, but requires you to rebuild your RRD files. There are upgrade instructions on the project wiki for this and all releases. Use the project issue tracker to view and report issues, and use the project mailing list to discuss the templates and scripts.

    The full changelog follows:

    2009-12-13: version 1.1.5
      * Support for getting slave lag via
      [Read more...]
    Version 1.1.4 of improved Cacti templates released
    +2 Vote Up -0Vote Down

    I’ve released version 1.1.4 of my improved Cacti templates. Unlike the prior release, which was solely bug fixes, this one includes new graphs in the MySQL template. Some of the graphs are of data that’s exposed in standard MySQL versions, but some of it is available only in Percona’s high-performance builds of the MySQL database server. If you don’t have a Percona build, those graphs will just contain nothing, but there is no detrimental effect.

    This template release is fully backwards compatible with the previous release. The upgrade process is to copy the new PHP file into place and import the new template file. You can then add the new graphs to your hosts.

    I’ve also taken the opportunity

      [Read more...]
    A review of The Art of Capacity Planning by John Allspaw
    +2 Vote Up -0Vote Down

    The Art of Capacity Planning

    The Art of Capacity Planning. By John Allspaw, O’Reilly 2008. Page count: 130 pages. (Here’s a link to the publisher’s site.)

    This is an outstanding book. As far as I know Ewen Fortune was the first Perconian to read it, and it’s been spreading amongst us since then. I got my copy last week, and read it last night when I couldn’t sleep for some reason. It took me about 90 minutes to read.

    This book

      [Read more...]
    How to capture debugging information with mk-loadavg
    +1 Vote Up -2Vote Down

    Maatkit’s mk-loadavg tool is a helpful way to gather information about infrequent conditions on your database server (or any other server, really). We wrote it at Percona to help with those repeated cases of things like “every two weeks, my database stops processing queries for 30 seconds, but it’s not locked up and during this time there is nothing happening.” That’s pretty much impossible to catch in action, and these conditions can take months to resolve without the aid of good tools.

    In this blog post I’ll illustrate a very simple usage of mk-loadavg to help in solving a much smaller problem: find out what is happening on the database server during periods of CPU spikes that happen every

      [Read more...]
    Maatkit version 3519 released
    +2 Vote Up -0Vote Down

    Maatkit version 3519 is ready for download. There are a lot of changes in this release, many of which are incompatible with previous releases. There are also a lot of important new features. Read on for the details.

    First, thanks to everyone who contributed to this month’s release. A lot of people have jumped into Maatkit and started committing code. I attribute this to deliberately forcing a more open policy with decisions being made on the mailing list, rather than the former policy of “Percona pays for development, so they have more say than you do” — a snobby and ill-advised way to treat an open-source project. If you are interested in contributing to Maatkit, please ask. Subversion commit rights are being handed out

      [Read more...]
    The unexpected consequences of SELinux
    +0 Vote Up -0Vote Down
    I’ve been working with a client recently who has SELinux on his servers.  It has been quite a struggle sometimes. My colleages tell me that SELinux has a pretty noticeable performance impact.  I am not sure if we have benchmarks to support this; at any rate, the client said it’s OK, we’ll take the performance hit. There [...]
    Maatkit version 2325 released
    +0 Vote Up -0Vote Down
    Download Maatkit There’s a new release with a lot of goodies — speed, efficiency, user-friendliness, and new features. In particular some of Percona’s clients have sponsored features for things they need such as the ability to more frequently verify that slaves are in sync with their masters. If you need features, please ask Percona [...]
    Is agent-based or agentless monitoring best?
    +0 Vote Up -0Vote Down
    Rob Young has posted a few blog entries lately on the MySQL Enterprise monitoring software. His latest post claims that agent-based monitoring is equivalent to extensibility (MySQL Enterprise Monitor: Agent = Extensibility). I think this is conflating two completely distinct properties of a monitoring solution. Cacti is extremely extensible, with a plugin-based architecture and [...]
    Maatkit version 1972 released
    +0 Vote Up -0Vote Down

    Download Maatkit

    Before I tell you what’s new, let me tell you how cool I think it would be if Maatkit were voted Sourceforge.net project of the year. Just something to think about :-) I suggest the “Best Tool or Utility for SysAdmins” category. You can actually click the Back button and nominate it for several categories. Not that anyone would do that, of course.

    Also, if anyone wants to jump in and help out with bug fixes and new features, please, by all means. Maatkit is a true open-source project as well as being Free Software. If you can follow coding conventions and understand Perl, I’m a very benevolent dictator and would gladly grant commit rights. As it turns out, since I’ve

      [Read more...]
    MySQL Cacti templates 1.0.0 released
    +0 Vote Up -0Vote Down

    I just uploaded the 1.0.0 release of my MySQL templates for Cacti. Now there’s an actual download under the Downloads tab. I solved a number of issues in this release. The changelog:

    2008-06-01: version 1.0.0
    	* Fixed when SHOW MASTER LOGS has no File_size column.
    	* Fixed Cacti-version-specific problems with include files.
    	* Fixed when binary log is not enabled.
    	* Fixed some caching issues.
    	* Fixed make-template.pl issues when downloaded from SVN.
    	* Replication graph shows only slave_lag instead of Seconds_behind_master
    	* Generate a version for Cacti 0.8.6i.
    	* Support generating custom versions with make-template.pl.
    Cacti, monitoring, mysql
    Screenshots of improved MySQL Cacti templates
    +0 Vote Up -0Vote Down

    I finally have some images to show you what my improved Cacti templates look like.

    These aren’t a perfect demo, since for example this server doesn’t have the query cache enabled, but it should show you what I’ve done. Note, for example, that each graph is labeled with the actual values of the images drawn on it. You don’t have to guess what the values are by squinting at the graphs.

    You can click on any image to go to a larger version. Enjoy:

      [Read more...]
    I moved this blog to pairLite with zero downtime, and it was easy
    +0 Vote Up -0Vote Down

    Did you notice that I moved this blog from pair Networks to pairLite hosting?

    Probably not, unless you check the DNS of xaprb.com regularly!

    Don’t you hate it when people say “I’m moving my blog, I hope there won’t be more than a few days of downtime, blah blah…” Why is this ever necessary, I wonder? I wonder the same thing about a lot of hosting providers — recently I had a client in my consulting practice whose (very large, well-known) hosting provider tried to help them with some very simple MySQL work and ended up causing them an obscene amount of downtime, like many many days, and there was no end in sight. As I spoke on the phone with him and asked him about his

      [Read more...]
    Summary of beCamp 2008
    +0 Vote Up -0Vote Down

    Yesterday I went to beCamp 2008 along with four roomfuls of other people interested in technology (perhaps close to 100 people total). The conference was a lot of fun. Not everything went as planned, but that was as planned. This was an Open Spaces conference and I thought it worked very well. From an email Eric Pugh sent:

    Basically it all boils down to:

    Open Space is the Law of Two Feet: if anyone finds themselves in a place where they are neither learning nor contributing they should move to somewhere more productive. And from the law flow four principles:

    • Whoever comes are the right people
    • Whatever happens is the only thing that could have
    • Whenever it starts
      [Read more...]
    Improved Cacti monitoring templates for MySQL
    +0 Vote Up -0Vote Down

    Download MySQL Cacti templates

    As promised, I’ve created some improved software for monitoring MySQL via Cacti. I began using the de facto MySQL Cacti templates a while ago, but found some things I needed to improve about them. As time passed, I rewrote everything from scratch. The resulting templates are much improved.

    You can grab the templates by browsing the source repository on the project’s homepage.

    In no particular order, here are some things I improved:

    • Standard polling interval and graph size by default.
    • Full captions on every graph; you don’t have to guess at how big the values are. Each graph has current, max, and
      [Read more...]
    Introducing MySQL Parallel Restore
    +0 Vote Up -0Vote Down

    The new release of MySQL Toolkit (version 1051) updates MySQL Parallel Dump in minor ways, but more importantly, it adds MySQL Parallel Restore. Read on for details.

    How to check and optimize MySQL tables in parallel
    +0 Vote Up -0Vote Down

    I wanted to point out something that might not be obvious from the name: MySQL Parallel Dump can be used as a generic wrapper to discover tables and databases, and fork off worker processes to do something to them in parallel. That "something" can easily be invoking mysqlcheck -- or any other program. This makes it really easy for you to do multi-threaded whatever-you-need-to-do on MySQL tables. Here's how.

    Introducing MySQL Slave Delay
    +0 Vote Up -0Vote Down

    MySQL Slave Delay implements an oft-requested replication feature: the ability to make a slave lag its master, a.k.a scheduling binlog events for some time in the future. This will probably be built into MySQL replication someday, but in the meantime, you can use this tool. A delayed slave is great if disaster strikes and propagates through replication, because it lets you stop the delayed slave before it gets corrupted, restart the slave until just before the offending statement, and recover quickly. This is a lot faster than restoring last night's full backup and replaying a day's worth of binary logs on the master.

    A review of MONyog
    +0 Vote Up -0Vote Down

    This is a review of Webyog's MONyog MySQL monitoring tool. The product is still in beta and has some rough edges, but holds promise. Its design includes some interesting architectural decisions I think will help distinguish it from MySQL's own Monitoring and Advisory Service.

    MySQL Table Sync 0.9.3 released
    +0 Vote Up -0Vote Down

    This release of MySQL Table Sync adds the ability to sync only part of a table, adds minor new functionality, and fixes a bug that could crash the tool.

    MySQL Find 0.9.0 released
    +0 Vote Up -0Vote Down

    If you've used the UNIX find command for more than a trivial find-and-print, you know how powerful it is; it's almost a miniature programming environment to find and manipulate files and directories. What if you could do the same thing with MySQL tables and databases? That was the inspiration for writing this tool. I was about to write several other tools to do some MySQL administrative jobs when I realized I could generalize and make something much more useful and powerful.

    MySQL Table Checksum 1.1.0 released
    +0 Vote Up -0Vote Down

    MySQL Table Checksum 1.1.0 adds many improvements, but the most important is a new way to ensure slaves have the same data as their master. Instead of checksumming the slave and the master, it can now insert the checksum results directly on the master via an INSERT.. SELECT statement. This statement will replicate to the slave, where a simple query can find tables that differ from the master. This makes a consistent, lock-free checksum trivially easy.

    There are also many other feature improvements and bug fixes, compatibility with MySQL 3.23.2 through 6.0-alpha, and finally I've gotten the documentation finished to my satisfaction.

    innotop 1.4.2 released
    +0 Vote Up -0Vote Down

    This release of the innotop MySQL and InnoDB monitor is a major upgrade in terms of functionality, code quality, and interface consistency. It is the result of me working for over a month to get innotop into shape for the recent MySQL Conference and Expo. This article is a summary of the changes and a look at what's coming next.

    MySQL Table Sync 0.9.2 released
    +0 Vote Up -0Vote Down

    MySQL Table Sync 0.9.2 is a bug-fix release. Since the last release users have reported several bugs. I am still postponing new features until after the MySQL Conference and Expo, because I am focusing on the innotop session I'll be presenting at the conference.

    I have also created a new mailing list on sourceforge for discussing all things MySQL Toolkit.

    Showing entries 1 to 30 of 44 Next 14 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.