yaouh! 0.4 is out - 10x speedup hack
Helge Hafting
helge.hafting at hist.no
Wed Feb 11 10:54:19 CET 2009
Robin Paulson wrote:
> this is a fine idea. it's somewhat clumsy invoking 10 scripts manually
> - i'm sure it could be coded to do this automatically, but it's a
> decent first step.
It is clumsy indeed. Just a test to see if parallel curl would help - or
not. Turns out that it helped. :-)
Running the curl and the md5sum in parallel would help too, and it'd be
sort of perfect: md5sum would then use the cpu while curl waits for the
network to respond. currently, curl waits for an answer, and then md5sum
runs after that. md5sum is quicker than curl, so the md5sum time could
be hidden completely. One md5sum isn't much time, but saving the time
for 50.000 invocations will probably amount to something.
Helge Hafting
More information about the community
mailing list