New archive file format (was: [omgps] collect feature requests)
Laszlo KREKACS
laszlo.krekacs.list at gmail.com
Thu Jul 2 19:22:04 CEST 2009
On Thu, Jul 2, 2009 at 10:24 AM, Laszlo
KREKACS<laszlo.krekacs.list at gmail.com> wrote:
> Need a serious benchmark here, if the extra overhead is true or not.
Ok, I have written the python implementation of the file archive maker.
I need to finish (ie. write the unpacking part) of it.
I compiled few benchmarks...
I compressed the whole OSM maps tiles on my laptop (I repeated it 10 times):
lol at buldergep:~/Maps/OSM$ echo -e "\noutput.kiss"; time python
../../Asztal/down/openmoko/paroli/data/kiss/kiss.py >>
../report.txt;mv output.kiss ..; echo -e "\noutput.tar"; time tar -cf
../output.tar .; echo -e "\noutput.zip"; time zip -0 -r output * >>
../report.txt; mv output.zip ..; echo -e "\noutput_comp.zip"; time zip
-r output_comp * >> ../report.txt; mv output_comp.zip ..; rm
../output*; rm ../report.txt
output.kiss
real 0m4.447s
user 0m2.748s
sys 0m1.520s
output.tar
real 0m4.039s
user 0m0.236s
sys 0m1.188s
output.zip
real 0m5.556s
user 0m1.276s
sys 0m2.632s
output_comp.zip
real 0m12.438s
user 0m8.437s
sys 0m2.620s
So the speed is about the same as in .tar file case. And it beats the zip.
File sizes:
-rw-r--r-- 1 lol lol 109M 2009-07-02 19:11 output_comp.zip
-rw-r--r-- 1 lol lol 125M 2009-07-02 19:11 output.kiss
-rw-r--r-- 1 lol lol 156M 2009-07-02 19:11 output.tar
-rw-r--r-- 1 lol lol 113M 2009-07-02 19:11 output.zip
-rw-r--r-- 1 lol lol 93M 2009-07-02 19:11 output.kiss.bz2
-rw-r--r-- 1 lol lol 94M 2009-07-02 19:11 output.tar.bz2
Total size of invidual files:
lol at buldergep:~/Maps/OSM$ du -hs .
290M .
Pretty strange, it reserves half the size ....
I think this file format worth the effort.
Best regards,
Laszlo
More information about the community
mailing list