Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Previous 30 Newer Entries Showing entries 91 to 110

Displaying posts with tag: Maatkit (reset)

A very fast FNV hash function for MySQL
+0 Vote Up -0Vote Down

I wrote a User-Defined Function that implements the FNV (Fowler-Voll-No) hash function for MySQL. I’m not the first person to do this — in fact, I was inspired by the Google patches for MySQL. But my implementation is a little bit different from most, in a very important way that leads directly to much higher performance, especially suited for the Maatkit tools.

A bit of background: FNV hashing is a very fast hash algorithm that operates in fixed memory. It is widely used in lots of important areas in computer science. My implementation requires absolutely no malloc() calls, which is a darn good thing because I am not to be trusted with

  [Read more...]
MySQL Master-Master replication table sync
+0 Vote Up -0Vote Down
I saw a post by Baron mentioning that his tool maatkit is best for handling situations where a master-master replication setup has got out of sync. If you think Baron was blowing his own trumpet he has good reason to. I have used his mk-archiver tool as part of the Maatkit to make the problem of archiving and purging data much easier. This was much easier than rolling my own solution. Anyhow. I
How to sync tables in master-master MySQL replication
+0 Vote Up -0Vote Down

Suppose you have a master-master replication setup, and you know one of the tables has the wrong data. How do you re-sync it with the other server?

Warning: don’t just use any tool for this job! You may destroy your good copy of the data.

If your table is large, you’ll probably want to use a tool that can smartly find the differences in a very large dataset, and fix only the rows that need to be fixed. There are several tools that are either able to do this, or claim to be able to do this. However, most of them are not replication-aware, and are likely to either break replication or destroy data.

To see why this is, let’s look at a typical scenario. You have server1 and server2 set up as co-masters. On server1, your copy of sakila.film has correct data. On server2,

  [Read more...]
Maatkit version 1753 released
+0 Vote Up -0Vote Down

Download Maatkit

This release contains minor bug fixes and new features. Besides the little bug fixes, there's a fun new feature in mk-heartbeat: it can auto-discover slaves recursively, and show the replication delay on all of them, to wit:

baron@keywest ~ $ mk-heartbeat --check --host master -D rkdb --recurse 10
master 0
slave1 1
slave2 1
slave3 4

(Not actual results. Your mileage may vary. Closed course, professional driver. Do not attempt).

Nothing else in this release is very exciting. I just wanted to get the bug fixes out there.

Maatkit version 1709 released
+0 Vote Up -0Vote Down

This release contains bug fixes and new features. It also contains a new tool: my implementation of Paul Tuckfield's relay log pipelining idea. I have had quite a few responses to that blog post, and requests for the code. So I'm releasing it as part of Maatkit.

Maatkit version 1674 released
+0 Vote Up -0Vote Down

This release contains bug fixes and new features. Click through to the full article for the details. I'll also write more about the changes in a separate article.

What is new in Maatkit
+0 Vote Up -0Vote Down

My posts lately have been mostly progress reports and release notices. That's because we're in the home stretch on the book, and I don't have much spare time. However, a lot has also been changing with Maatkit, and I wanted to take some time to write about it properly.

Maatkit version 1579 released
+0 Vote Up -0Vote Down

This release contains bug fixes and new features. The biggest new feature, in my opinion, is a new sync algorithm for mk-table-sync. Now you can sync any table with an index more efficiently than previously. This is the return of the speed I promised earlier. (Though I haven't yet benchmarked it; I am very short on time these days. Your benchmarks and other contributions are welcome).

I'm finally feeling like the table sync tool is getting in good shape!

Changelog etc is in the full article.

Query Profiling Tools ? part 1, mysqlsla
+0 Vote Up -0Vote Down
The “sla” in mysqlsla stands for “statement log analyzer”. This does a much better job than mysqldumpslow of analyzing your slow query log. In fact, you can sort by many different parameters — by sheer number of times the query shows up in the slow query log, by the total or average query [...]
MySQL Query Profiling Tools ? part 0, Ma?atkit Query Profiler
+0 Vote Up -0Vote Down
Today I’ve been checking out a new client environment. My mission is to figure out (cold) some of the characteristics of the queries being run, and particularly if they’re “good” or “bad”. In my arsenal of “tools I really want to check out” has been Ma’atkit’s Query Profiler. They’re very different tools. Ma’atkit’s query [...]
Maatkit version 1508 released
+0 Vote Up -0Vote Down

This release fixes a few bugs, adds minor features, and adds some debugging support to shared code. I'm working on the Nibble sync algorithm for mk-table-sync, and someone has found a few more bugs with mk-parallel-dump, but those might take me a while to complete.

Maatkit on Ohloh
+0 Vote Up -0Vote Down

Sheeri wrote a post (now a 404 error) referring to Maatkit on Ohloh, which I have never heard of before. I took a look at what Ohloh thinks about Maatkit. It's kind of neat. Beyond just the obvious "social website" stuff that's all the rage these days, it actually looks at the project's SVN history, analyzes the codebase, and so on.

It also estimates 8 person-years of work have gone into the project, and says that at $55,000/year it would cost $450,702 to write the code as it currently exists, which is kind of funny. It took me a whole lot less than 8 years to write. (Perhaps this is why that salary strikes me as unrealistic).

It has a couple of other interesting things, like a visual timeline of source

  [Read more...]
Maatkit version 1417 released
+0 Vote Up -0Vote Down

Download Maatkit

Thanks again to all the great sponsors for my week of work on the kit!

This is the long-awaited "Baron worked on table sync" release. Hooray!

Please read the full blog post for important (very important!) information.

Progress on Maatkit bounty, part 4
+0 Vote Up -0Vote Down

... I didn't get two-way sync done, and I didn't get the Nibble algorithm done. That much I expected. But I also didn't get the current work released tonight because I'm paranoid about breaking things. I'm trying to go through all the tools and write at least a basic test for them to be sure they can do the simplest "unit of work" (such as mk-find running and printing out that it finds the mysql.columns_priv table).

It's good that I'm doing this. I found that mk-heartbeat suddenly doesn't work on my Ubuntu 7.10 laptop. It goes into infinite sleep. Can anyone repro this and/or diagnose? The same code works fine on my Gentoo servers at work.

Hopefully I'll be able to release something very soon. Release early/often is fine, but "knowingly release brokenness" isn't in my code of conduct :)

Progress on Maatkit bounty, part 3
+0 Vote Up -0Vote Down

This is the last day I'm taking off work to hack on mk-table-sync, and I thought it was time for (yet another) progress report. Here's what I have done so far. (Click through to the full article to read the details).

Progress on Maatkit bounty, part 2
+0 Vote Up -0Vote Down

Ironically, the Stream algorithm I wrote as the simplest possible syncing algorithm does what the much more efficient algorithm I wrote some time ago can't do: sync a table without a primary key, as long as there are no duplicate rows. In fact, it's so dumb, it will happily sync any table, even if there are no indexes.

The flash of inspiration I had on Friday has turned out to be good...

Progress on Maatkit bounty
+0 Vote Up -0Vote Down

My initial plans got waylaid! I didn't pull out the checksumming code first, because the code wasn't at all as I remembered it. Instead, I began writing code to handle the more abstract problem of accepting two sets of rows, finding the differences, and doing something with them. I'm ending up with a little more complicated system than I thought I would. However, it's also significantly simpler in some ways. Instead of just passing references to subroutines to use as callbacks, I'm object-ifying the entire synchronization concept...

Maatkit bounty begins tomorrow
+0 Vote Up -0Vote Down
Tomorrow is the first of five days I will spend working on mk-table-sync, the data synchronization tool I developed as part of Maatkit. The first thing I’ll do is pull the row-checksumming code out into a module and write a unit test suite for it. I’ll probably add the code to the module [...]
New Maatkit release policy
+0 Vote Up -0Vote Down

Download Maatkit

Maatkit (formerly MySQL Toolkit) has for some time been released both as a bundle, and as individual tools. It's too much work to maintain the individual packages, and I don't think it really benefits anyone much, if at all. While the tools will still be versioned separately, I'm going to discontinue releases of the individual packages, and just release the one uber-package from now on.

This will also make it easier for me to manage the name change, but that's just an extra incentive; I've been considering this for a while.

By the way, Sourceforge indicated it would take up to a couple of days to finish the project's rename, but it took only a few minutes. Lots of broken links; I've asked for a permanent redirect from the old URLs to the new.

MySQL Toolkit is now Maatkit
+0 Vote Up -0Vote Down

I am so lucky I married an archaeologist.

Choosing a new name for MySQL Toolkit has been a hassle. I wanted to avoid a literal name, such as, um, MySQL Toolkit. Short is good. And so on, and so on. All the while, the Phoenix/Firebird/Firefox naming debacle was in my thoughts. I only want to do this once.

Read on for a fun lesson in Egyptian mythology, courtesy of my wife!

Previous 30 Newer Entries Showing entries 91 to 110

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.