[rb-general] lzip/plzip alternatives

[Sylvain]

Hi,

On Wed, Apr 19, 2017 at 06:53:24PM +0000, Peter Stuge wrote:
> On Tue, Apr 18, 2017 at 08:43:05PM +0000, Daniel Shahaf wrote:
> > Sylvain wrote on Tue, Apr 18, 2017 at 21:50:23 +0200:
> > > This means that if I produce my tarball using 'tar -c --lzip' or
> > > 'tar -c | lzip', depending on whether plzip is installed or not, I
> > > will silently get a different output.
> > >
> > > Should I file a bug report against the plzip package?
> > > What severity would you recommend?
> >
> > It's perfectly fine for lzip and plzip not to produce identical output
> > to each other; whichever of them was *actually used* by the build should
> > be recorded in the .buildinfo file, which would enable reproducing that
> > particular build.  (That's exactly analogous to a "gzip 1.3 and gzip 1.4
> > produce different outputs for identical inputs" situation.)

.buildinfo is nice but this is specific to Debian packaging AFAICS.
I personally welcome features that allow reproducibility by default
- I mean, the front-page example at https://reproducible-builds.org/
is Volkswagen firmware.

About gzip:

$ ls -lh a
-rw-r--r-- 1 me me 81M Apr 19 21:23 a
$ gzip -c a > a.gz16
$ gzip-1.4/gzip -c a > a.gz14
$ gzip-1.3.13/gzip -c a > a.gz13
$ diff a.gz13 a.gz14
[no differences]
$ diff a.gz13 a.gz16
[no differences]

I didn't see any reference to versioning in pristine-gz(1).

It seems gzip's output has been pretty stable, or maybe did you have a
precise example in mind?

Also gzip's pigz alternative do not replace /bin/gzip.
I'm used to stable output from base compressors.


> > So, I don't think a bug against plzip would be warranted.  However, the
> > .buildinfo format should record not only the versions of lzip and plzip,
> > but also the value of the /usr/bin/lzip alternative.  If it doesn't,
> > then _this_ would be worth a wishlist bug (against dpkg, I think?).
> 
> Keep in mind that alternative is a debian concept which isn't
> neccessarily equivalent elsewhere.

I'm not sure I understand, Peter -- so is .buildinfo, isn't it?


> > Another thing that _would_ be worth a bug report is if the output of
> > lzip was non-deterministic; that is: if the value of "`echo foo | lzip
> > -c`" was different when computed twice, or depended on the username
> > / hostname / etc (see https://tests.reproducible-builds.org/debian/index_variations.html).

It is deterministic - I wouldn't be bitching about plzip otherwise :P


> > > (Bonus question: is it theoretically possible to have such tools pairs
> > >  (gz/pigz, xz/pgz, lzip/plzip) produce the same output? :))
> >
> > Theoretically?  Yes.  Whether that'd be a good idea is another question.

Do you have elements about why this would be possible?
I was wondering if that was an unavoidable consequence of parallism or
just a suboptimal implementation.

Cheers!
Sylvain