yaouh! 0.4 is out - 10x speedup hack

Helge Hafting helge.hafting at hist.no
Wed Feb 11 10:54:19 CET 2009

Robin Paulson wrote:

> this is a fine idea. it's somewhat clumsy invoking 10 scripts manually
> - i'm sure it could be coded to do this automatically, but it's a
> decent first step.

It is clumsy indeed. Just a test to see if parallel curl would help - or 
not. Turns out that it helped. :-)

Running the curl and the md5sum in parallel would help too, and it'd be 
sort of perfect: md5sum would then use the cpu while curl waits for the 
network to respond. currently, curl waits for an answer, and then md5sum 
runs after that. md5sum is quicker than curl, so the md5sum time could 
be hidden completely.  One md5sum isn't much time, but saving the time 
for 50.000 invocations will probably amount to something.

Helge Hafting

More information about the community mailing list