New archive file format (was: [omgps] collect feature requests)

[Alexander Shulgin]

On Thu, Jul 2, 2009 at 10:15, Laszlo
KREKACS<laszlo.krekacs.list at gmail.com> wrote:
> On Thu, Jul 2, 2009 at 8:42 AM, Alexander Shulgin<alex.shulgin at gmail.com> wrote:
>> I fail to see how is this true for normal tar files (vs. data read
>> from pipe).  Can you elaborate please?
>
> Yepp, of course;)
>
[snip]
>
> Simplification of tar archive:
> [1. file header][1. file][2.file header][2. file][3. file header][3. file]
>
> So how you read the third file from the archive? You read the file until the
> [3. file header], your test is successfull (is it the right file?),
> and you read the
> file itself. You see? You have read the whole file, just accessing the
> last item inside.

Yes, but is lseek(2) banned on neo?  This is what I was talking about
then mentioned normal files (i.e. not pipes). :)

>> Pardon my ignorance, but wouldn't zip -0 do the trick for your purpose? :)
>
> It will do more or less, however there are three main problems with it:
>
> 1. you can only obtain the whole file from the archive. So you cant
>  read a part of the file. So if you packed lets say a 700MB file to zip,
>  you run out of memory on neo.
>  At least this is the case on standard python zipfile module.
>
> 2. There is no random access feature, at
>    least not in standard python modules.
> 3. There are significant processor time wasted when accessing to a file
>   (many computation required). Btw, it needs to benchmark on the neo, how
>   worse is it.

OK, I see now.  Thanks for explanation.

--
Cheers,
Alex