High performance flv-streaming with lighttpd is possible since lighttpd 1.4.11.

With lighty you can easily handle 10000 parallel downloads of your movies including protection against hot-linking with mod_secdownload. This is basicly all you need to build the free video.google.com for yourself.

Just add this you your lighttpd.conf and restart the server:

server.modules = ( ..., "mod_flv_streaming", ... )

flv-streaming.extensions = ( ".flv" )

Players

mod_flv_streaming expects you to fetch the flv-file with a GET parameter if you want to do a seek into the file.

GET /movie.flv?start=23 HTTP/1.1
Host: ...

• Fabian Topfstedt has done this for …

jan.kneschke.de is back online again. The old server got old and rusty and it has taken while to get everything setup again.

The most important articles are online again:

• Optimizing ORDER BY RAND()
• Groupwise MAX()

The old photo-storage is disabled, but a few new pics got uploaded to flickr, e.g. from my trip to Argentina:

• http://www.flickr.com/photos/jankneschke/sets/72157594494461199/

SIGAR is a cross-platform API to gather system-specific performance-data. It is developed by Hyperic and released under the GPL. We use it at MySQL as the backend for our monitoring agent.

Sadly SIGAR requires ant for building as it is meant to be used with a JNI-wrapper in Java applications. Real men use C, so we need something else for building: cmake.

cmake

cmake is a cross-platform, autotools replacement. It works on Unix and win32 and generates native Makefiles. On Unix it is make, on win32 it is nmake, on macosx uses the XCode framework.

That keeps even the build portable. You can get a binary from …

jan.kneschke.de is back online again. The old server got old and rusty and it has taken while to get everything setup again.

The most important articles are online again:

• Optimizing ORDER BY RAND()
• Groupwise MAX()

The old photo-storage is disabled, but a few new pics got uploaded to flickr, e.g. from my trip to Argentina:

• http://www.flickr.com/photos/jankneschke/sets/72157594494461199/

If you read the MySQL manual you might have seen the ORDER BY RAND() to randomize the the rows and using the LIMIT 1 to just take one of the rows.

SELECT name
  FROM random
 ORDER BY RAND()
 LIMIT 1;

This example works fine and is fast if you only when let's say 1000 rows. As soon as you have 10000 rows the overhead for sorting the rows becomes important. Don't forget: we only sort to throw nearly all the rows away.

I never liked it. And there are better ways to do it. Without a sorting. As long as we have a numeric primary key.

For the first examples we assume the be ID is starting at 1 and we have no holes between 1 and the maximum value of the ID.

move the work into the application

First idea: We can simplify the whole job if we calculate the ID beforehand in the application.

SELECT MAX(id) FROM random;
## generate random id in application
SELECT name FROM random WHERE id = …

Some of the common conceptions of data storage seem to have been blown out of the water.

Two things I found interesting:

In their study they found that there was no correlation between disk failure rates and utilization, environmental conditions such as temperature, or age. This means that high disk utilization or age of the disk have no significant impact on the probability that it will fail.

…

They observed that older disks had a much lower failure rates then newer disks, where the newer disks in general were less expensive.

Which makes me think that buying used HDDs off Craigslist might not be a bad idea.

One could buy cheap 15k RPM low latency disks from a few years ago and forget about the storage capacity in exchange for FAST seek time.

Of course it depends on how …

Talking to a coworker at MySQL:

Coworker: "The spam is just awful."

Me: "I know, I am getting hundreds of pieces a day."

Coworker: "That bad! I am just getting a dozen a day."

Me: "I just need to unsubscribe to a number of mailing lists."

Coworker: "But all of that is important! Its company mail."

I disagree, its Spam. Sure, its not Viagra ads, but its mail I don't need to be reading. I've been at MySQL for four years, my role keeps changing, but my subscription list has not.

In a modern corporation people get signed up to new mailing list all the time. Once upon time it mattered that I read everything our connectors group did. Now? I subscribe to announcements and see what the output is. I make a point of pinging people and seeing what is floating up to the top of discussions, but the day to day I just do not need to follow.

…

In the beginner's mind there are many possibilities, in the expert's mind there are few --Shunryu Suzuki

ETL

1. Informatica - Popular ETL. Will it ever catch up to Ab Initio's parallelizing capabilities?
   
2. Ab Initio - High scalability through easy availability of all possible types of parallelisms. The pipeline parallelism is especially powerful. Popular in financial services industry. Steep learning curve to fully utilize the features along with secretive nature of the company makes availability of skilled developers a challenge
   
3. IBM/Ascential DataStage - Mixed bag of acquisitions
   

Database

1. Oracle - Popular DB. SMP King, more than enough for most situations. MPP is shared disk Grid-RAC-OPS confusion, lesser said the better.
2. DB2 - Traditionally emphasized the MPP approach. With bitmap index and range partitioning support, now …

I've just finished packaging up version 0.4 of the memcached engine for MySQL.

What is new:
* Added support for packed rows (uses less memory).
* Fixed support for IN() operations.
* Fixed bug in UPDATE when ORDER BY was given.

http://download.tangent.org/memcache_engine-0.4.tar.gz

Hurry up, submit a paper! The LinuxTag Call for Papers ends tomorrow, February 16th.

Short info about LinuxTag from the homepage:

LinuxTag 2007 opens doors from May 30 to June 2, 2007 on Berlin Expo Center under the Funkturm. We invite users and experts to learn at Europe’s leading conference and expo more about the potential of Linux, Open Source, and Free Software.