This release contains bug fixes and new features. The biggest new feature, in my opinion, is a new sync algorithm for mk-table-sync. Now you can sync any table with an index more efficiently than previously. This is the return of the speed I promised earlier. (Though I haven't yet benchmarked it; I am very short on time these days. Your benchmarks and other contributions are welcome).
I'm finally feeling like the table sync tool is getting in good shape!
Changelog etc is in the full article.
I've written before about how to figure out which connection is holding the InnoDB locks for which other connections are waiting. In other words, how to figure out who's blocking you from getting work done. The short and sweet: turn on the InnoDB lock monitor and use innotop to look at the locks held and waited-for. This has some disadvantages, so I made a patch to solve the issue the way I like it. The result is significantly more ability to find and solve InnoDB lock wait issues.
This release fixes a few bugs, adds minor features, and adds some debugging support to shared code. I'm working on the Nibble sync algorithm for mk-table-sync, and someone has found a few more bugs with mk-parallel-dump, but those might take me a while to complete.
Well, if my perfectionist nature were allowed to run free, and if Peter et al's encyclopedic knowledge were somehow all transferred to paper, the second edition of High Performance MySQL would end up being the perfect encyclopedia of MySQL performance. But as it is, you're apparently going to have to settle for "very good." This quote by Sheeri Kritzer Cabral, one of our tech reviewers, really made my day:
I gotta hand it to Peter, Vadim, Arjen, and Baron. They know how to write a book!
And now I must begin a solid weekend of revisions... wish me luck!
Sheeri wrote a post (now a 404 error) referring to Maatkit on Ohloh, which I have never heard of before. I took a look at what Ohloh thinks about Maatkit. It's kind of neat. Beyond just the obvious "social website" stuff that's all the rage these days, it actually looks at the project's SVN history, analyzes the codebase, and so on.
It also estimates 8 person-years of work have gone into the project, and says that at $55,000/year it would cost $450,702 to write the code as it currently exists, which is kind of funny. It took me a whole lot less than 8 years to write. (Perhaps this is why that salary strikes me as unrealistic).
It has a couple of other interesting things, like a visual timeline of source …
[Read more]Thanks again to all the great sponsors for my week of work on the kit!
This is the long-awaited "Baron worked on table sync" release. Hooray!
Please read the full blog post for important (very important!) information.
... I didn't get two-way sync done, and I didn't get the Nibble algorithm done. That much I expected. But I also didn't get the current work released tonight because I'm paranoid about breaking things. I'm trying to go through all the tools and write at least a basic test for them to be sure they can do the simplest "unit of work" (such as mk-find running and printing out that it finds the mysql.columns_priv table).
It's good that I'm doing this. I found that mk-heartbeat suddenly doesn't work on my Ubuntu 7.10 laptop. It goes into infinite sleep. Can anyone repro this and/or diagnose? The same code works fine on my Gentoo servers at work.
Hopefully I'll be able to release something very soon. Release early/often is fine, but "knowingly release brokenness" isn't in my code of conduct :)
This is the last day I'm taking off work to hack on mk-table-sync, and I thought it was time for (yet another) progress report. Here's what I have done so far. (Click through to the full article to read the details).
Dear lazyweb,
I want to mine a code repository for data to map past bugs to
sourcecode files.
I have written a small PHP script (the initial version of the script can
be found here) to import the relevant data from a
Subversion repository into the following tables of
a relational database:
bugs changes paths -------- -------- ------- bug_id path_id <--> path_id revision <--> revision path
What I need now is two queries to ask the database for
Ironically, the Stream algorithm I wrote as the simplest possible syncing algorithm does what the much more efficient algorithm I wrote some time ago can't do: sync a table without a primary key, as long as there are no duplicate rows. In fact, it's so dumb, it will happily sync any table, even if there are no indexes.
The flash of inspiration I had on Friday has turned out to be good...