It seems that one of the frustrating things about optimization is how wrong your assumptions about the platform/environment running your code can eventually become.
At one point, programmers / compilers optimized carefully to account for 12,500 RPM rotating drum memory. Algorithms have been optimized for sequential access mass storage (tape), as well as for random access mass storage (disk). ACM Queue published an interview with Jim Gray(PDF) a couple of years ago which contains a wonderful quote illustrating how the current generation of programmers need to adjust their assumptions about data access:
So programmers have to start
thinking of the disk as a sequential device rather than a random
access device.
Dr. Gray has published …
[Read more]