Not so long time ago I had task to update string column in table with 10mil+ rows, and, as the manipulation was non-trivial, I decided this task is good to try Stored Function. Function written - go ahead. Since 5 min I got totally frozen box with no free memory and giant swap.

The case was worth to look deeply - let's try simple table (experiments with MySQL 5.0.45)

PLAIN TEXT SQL:

1. CREATE TABLE `testf` (
2.   `id` int(11) NOT NULL AUTO_INCREMENT,
3.   `name` varchar(255) DEFAULT NULL,
4.   PRIMARY KEY  (`id`)
5. ) ENGINE=MyISAM AUTO_INCREMENT=20971521 DEFAULT CHARSET=latin1
6.  
7. mysql> SELECT count(*) FROM testf;
8. +----------+
9. | count(*) |
10. +----------+
11. | 20971520 |
12. +----------+
13. 1 row IN SET (0.00 sec)

and simple …

In http://jan.kneschke.de/2007/8/1/mysql-proxy-learns-r-w-splitting we did a first implementation on idea of sending non-transactional reads to the slaves and writes and transactions to the master.

While that was mostly a proof of concept it already worked pretty well. In this second round on R/W splitting several issues are taken care of:

• SELECT SQL_CALC_FOUND_ROWS ... + SELECT FOUND_ROWS()
• INSERT ... + SELECT LAST_INSERT_ID()
• default_db not in sync between client and backend

These changes should make Read/Write splitting more robust and make it a drop-in for more users.

Up to now we used the tutorial-keepalive.lua to implement the Read/Write-splitting. As the name says, it is a …

The Taso Dump

I’ve gotto go, my time has come. (maintenance window needed)
Could not get a lock. (table lock)
I needed a blocker to ensure a single thread. (–single-transaction)

Finally, got a lock. (–lock-tables)
Need to ensure no transactions lost.
Dump, Dump, Dump. (mysqldump)

I’m having flow problems.
Was the buffer to small? (key_buffer_size)
Was it the query cache? (query_cache)
My Dirty Pages may be too high? (innodb_max_dirty_pages_pct)
Or was it just Too Many Connections? (max_connections)
But it was just waiting on the flush (flush tables with read lock)

Time passes, Time Passes. No output yet.
Is it network saturation?
Is it IO bound?
Do I need a better flushing method (innodb_flush_method)
No, it was just the lag? (Seconds_behind_master)

Dump is complete (unlock tables)
Now it’s time to …

Previous post

http://consoleninja.net/code/dpm/rel/dpm-r2.tar.gz - tarball of r2
git clone http://consoleninja.net/code/dpm/dpm.git - to get the latest code, always
http://consoleninja.net/code/dpm/dpm-export.tar.gz - a tarball of the latest code, for those unwilling to git it.

I'll keep this short. Also, just noticed my tags aren't getting uploaded to the remote git. Sorry.

Bunch of commits worthy of a new release:
- new included pass-through demo.
- cleaned the included connection pooling demo a little.
- 'proxy_until' demo. Disables packet processing temporarily. …

Previous post

http://consoleninja.net/code/dpm/rel/dpm-r2.tar.gz - tarball of r2
git clone http://consoleninja.net/code/dpm/dpm.git - to get the latest code, always
http://consoleninja.net/code/dpm/dpm-export.tar.gz - a tarball of the latest code, for those unwilling to git it.

I'll keep this short. Also, just noticed my tags aren't getting uploaded to the remote git. Sorry.

Bunch of commits worthy of a new release:
- new included pass-through demo.
- cleaned the included connection pooling demo a little.
- 'proxy_until' demo. Disables packet processing temporarily. …

In MySQL Proxy learns R/W splitting we did a first implementation on idea of sending non-transactional reads to the slaves and writes and transactions to the master.

While that was mostly a proof of concept it already worked pretty well. In this second round on R/W splitting several issues are taken care of:

• SELECT SQL_CALC_FOUND_ROWS ... + SELECT FOUND_ROWS()
• INSERT ... + SELECT LAST_INSERT_ID()
• default_db not in sync between client and backend

These changes should make Read/Write splitting more robust and make it a drop-in for more users.

Up to now we used the tutorial-keepalive.lua to implement the Read/Write-splitting. As the name says, it is a tutorial for connection keepalive, …

MySQL Camp II is complete. A small group of about 18 had post dinner at Tiny Thai in New York City. Some elected to drive from Brooklyn, they arrived at least 30 minutes after those of us that the subway.

I have a lot of notes to write, if ever the time permits. For now, the following few that joined for drinks are below. I know other people took photos of the camp, for a change I actually took none. If you want to add a link in comments of photos from the camp that would be great.

Other sizes here

So last night I had to resized one of my Xen partitions. So what was the magic to make this happen?


dd if=/dev/zero count= >> /var/lib/xen/images/shiitake.dsk


Then? I need to increase the partition map for my disk:


fdisk /dev/xvda


What did I do in fdisk? I deleted the partition, and then recreated it with the new available blocks. Of course the machine was running at the time. What is life without a few risks? But I have to ask myself, why doesn't fdisk have a resize command? I swear that tool hasn't changed in well over a decade.

I am running LVM so then I had to:


pvresize /dev/xvda2
lvresize -L +5G /dev/VolGroup00/LogVol00


And finally since the filesystem is ext3:

e2fsck -f /dev/mapper/VolGroup00-LogVol00
resize2fs /dev/mapper/VolGroup00-LogVol00
…

Well, after sitting for two hours on the runway at JFK airport, I finally got home to Columbus late last night. Having such an abundance of free time (while waiting on the runway), I thought about the great three days I had had meeting with Adam and Sandro from the SKOLL Cluster on Wednesday and with the MySQL Campers on Thursday and Friday.

Camp has renewed my faith in the feeling of connection that being a MySQL community member gives to me. From newbies to old hands, MySQl Camp II brought together a great mix of inidividuals and allowed me to meet with a great group of talented and interesting folks. I'm very much looking forward to MySQL Camp III in the Washington D.C. area. A date hasn't been set yet, but likely we'll pick one in the near future. A Google Group has been set up for MySQL Campers, so if you're interested in getting notified about upcoming camps and …

You have inadvertently deleted some rows from your table and want them back. This is semi-doable with fixed row format of MyISAM. I put together a few steps, which I'll compliment with a program that does it for you, later.

• save output of SHOW TABLE STATUS LIKE 't1' and SHOW CREATE TABLE `t1`.
  
• shutdown mysql server asap!
• backup t1.frm, t1.MYI, t1.MYD immediately
• create a new table t1_recover, which has no auto-inc and no unique/pk.
• remove the files t1_recover.MYD and t1_recover.MYI
• write a C program that scans through t1.MYD reading blocks of length Avg_row_length and checking if first byte indicates a row is marked as deleted.
• if it's deleted, dump it as is into the file t1_recover.MYD
• goto mysql prompt. issue REPAIR TABLE `t1_recover` USE_FRM

Some notes.

You cannot recover entire record. You'll …