yaouh! 0.4 is out - 10x speedup hack

[Robin Paulson]

2009/2/10 Helge Hafting <helge.hafting at hist.no>:
> Carlo Minucci wrote:
>>
>> download from http://wiki.openmoko.org/wiki/Yaouh!
>>
>> little bufgix and add support for multiple wget download
>> i think now is more fast
>>
>> please, test and feedback
>
> Seems to work fine, but it looks like only the downloading happens in
> parallel. A common case seems to be only 5%-10% new tiles. The rest
> just need checking. This is done with sequential use of "curl". I.e. no
> parallel checking, although this is a big part of a yaouh run.
>
>
> I am no expert in python threading, so I did a much simpler hack.
> I run 10 instances of yaouh. yaouh0.py only checks files matching
> *0.png, yaouh1.py only checks files matching *1.png, and so on.
>
> This gives a tremendous speedup because curl transfers very little data.
> Serialized curl spend most of the time waiting for an answer, while one
> request and a very small answer moves through the network to a distant
> server and back again. The bandwith of the connection is nowhere near fully
> utilized, not even when using usb networking.

this is a fine idea. it's somewhat clumsy invoking 10 scripts manually
- i'm sure it could be coded to do this automatically, but it's a
decent first step.

carlo, any chance of implementing this in a forthcoming release?