Dealing with dumping and recovery of large and partially corrupted database I’ve got couple of feature ideas for mysqldump or similar tool and appropriate import tool

Dump in parallel single thread dump is not efficient of course especially on systems with multiple CPUs and disks. It is lesser issue in recovery case because import takes most of the time anyway but will be handy for some backup needs.

Dump each table in its own file This is much more convenient compared to single say 100GB sql file allowing to prioritize data load and if load fails for any reason you can easily restart it. Also it allows to prioritize data load if you’ve got to load many tables which have different priority for your application.

Safe Dump Dumping corrupted Innodb tables you will have some of the them crashing Innodb which breaks mysqldump process. It would be great for load script to …

I have not had a serious Innodb corruptions for a while, typically even if it happened it was some simple table related corruption which was easy to fix on table level. In couple of cases during last year when it was more than that we had backups and binary logs which means it was easier to recover from backup and replay binary logs.

This time I have a challenge to play it hard way because backup is in special form which will take a while to recover. It also should be nice exercise in patience because database is over 1TB in size.

One bug I already reported makes me worry. If it is global bug it should have been Innodb recovery show stopper while it goes back so many releases (5.0.33 surely still has it).

Lets see what else we run into.

One minor “practicality” I should mention is using –socket=/tmp/mysqlx.sock –port=3307 or something …

I have been very busy this past months, with both personal crisis and mega-patches. This is my MySQL blog so lets keep it to topic.
A couple of weeks ago, the plugins mega-patch was finally pushed and was released to the world in version 5.1.18. Thanks to Ingo and Sergei for keeping up the effort while I was occupied. As part of the code push, InnoDB code was stripped out of the main mysqld code and stored where it belongs, within Innobase's code. This is an exciting development as it now makes it easier than ever for plugin developers to create plugins without having to modify any existing code within mysqld. It has taken many months to get to this point but we are finally there. Anyone remember our CEO announcing the plugin initutive back when Innobase was acquired by Oracle? October 2005, I believe.

Since that push, I am back on bug-fixing... I have an annoying bug on Itanic with Intel's Itanic Compiler. (Yeah, I know the …

During JavaOne this year Rags and I did a hands on lab on DTrace. The topic was "Using DTrace on Java and other Web 2.0 Languages in Solaris". Due to some unforeseen reason the lab was scheduled from 8:45 PM to 10:45 PM (Yes PM) and that to on the day of the big party at JavaOne. So I was expecting to be speaking to an empty room but lo and behold we had close to 100 engineers ( a few pretty drunk) and most stayed until the finished the lab.It was pretty heartening and was worth all the efforts that went into …

Ssst! I'm currenty in Stockholm, Sweden, delivering the MySQL 5.1 Cluster Certification Exam on MySQL's very own Cluster summit.

During the Cluster summit, various MySQL-ers from support, consulting, training & certification, documentation, product management and …

At the OSBC conference, SnapLogic, a new open source company focused on data integration networks, will be showing their stuff.  It's pretty cool technology.  Think of it as Enterprise Application Integration (EAI) meets Yahoo! Pipes.  You have to see the demo to really appreciate how much they have simplified the complexity of integration.  The company includes some serious talent and brings a very modern network approach to solving the enterprise integration problem. 

• SnapLogic: Main, Background, …

My decision about 1 year ago to start doing training was a strange move for me. It was totally new and scary field for me, but like I said at the moment, according to my experience all change has been for good. I think I can trust my intuition.

Like Seth Godin said (more than) once:
Safe is risky

I found a short article about How we learn which summarizes my point of view after 5 courses.

10% of what we READ

20% of what we HEAR

30% of what we SEE

50% of what we SEE and HEAR

70% of what is DISCUSSED with OTHERS

80% of what …

We have certain type of the table corrupting with Innodb, as it is limited to only one particular index on one particular table type it is likely to be Innodb bug but Heikki currently could not find what could be causing it.

Happily as we have data stored in many tables of same format rather than one monster table these rare corruptions did not cause too much problems to us - as it was not clustered index we always could repair table by running ALTER TABLE with limited impact to production system (as only one table of about a hundred will be locked)

So everything kind of worked (of course we still hoped new MySQL release will have this Innodb bug fixed) until today we got MySQL to crash during recovery process right after 17% of log records were applied.

Such corruption is one of the worst onces, because you can’t really recover data on per table basics.

Also if MySQL crashes during recovery lower values of …

This thought has been floating around my head for quite some time now and I finally bit the bullet and released it from the draft state it had been sitting in for too long: there are quite many similarities between Open Source Software (OSS) projects and most of today's popular Web 2.0 sites, but there is also one odd difference that I wonder about.

For both worlds, the concept of collaboration, participation and giving more power to their users is a key component. OSS projects need contributors for patches and bug reports, but also for feedback, translations, artwork, advocacy in order to be popular and healthy. The project's developers need to be open for suggestions, listening to their user base on where the project should be heading. They also usually strive for open standards …

"That is an awfully nice bridge you got there, be a pity if something happened to it..."

So Business as Usual means protection money.

"This IP bridge enables Open Source developers to develop software free from concerns about patents."

"Our IP bridge makes lawsuits unnecessary."

Both of those statements just give me a cold feeling. Its like listening to some gangster video, where the local hood has decided to make a business of asking for protection money from the new immigrants who just opened a Kwiki Mart. Bill Hilf talks about lawsuits being unnecessary, of course they are. The local hood never wants their business practices scrutinized, they operate in a cloak of uncertainty.

When did "Business as Usual" in engineering become haggling over nickel and dime changes that came …