unreproducible zlib/deflate compression in ZIP/APK files

[Fay Stegerman]

* "Bernhard M. Wiedemann" <bernhardout at lsmod.de> [2024-09-07 17:46]:
> If I read it correctly, there are different zlib implementations that will
> turn identical uncompressed data into different compressed versions.
> As long as the uncompressed data matches, that is fine security-wise.

Yes.  But since the resulting artefact is not identical, it breaks RB (and
invalidates signatures), even if it is not a security issue.

> This relates to #12 of our
>  https://reproducible-builds.org/docs/commandments/
> 
> > 12. If Thou publishst binaries, Thou shall take note of your build inputs
> 
> If you knew exactly, which zlib version was used, it should be possible to
> reproduce compressed results.

Indeed.  Unfortunately, the RB situation on Android -- reproducing APKs built
and created by developers on their own machines/CI (which can be Windows, mac
OS, Linux, using various JDK versions etc.) -- makes things a bit more
complicated than e.g. building distro packages.

It's already impossible to recreate some APKs built on Windows because of
embedded build paths that we have no way to reproduce in the Linux container
used by our rebuilder even if we know exactly what they are.  And e.g. we
already need workarounds to handle newline differences between Windows and Unix
builds (a bug we reported to Google but have not seen any progress on since).
And this tooling needs to recompress the data, so it would also run into issues
with different zlib implementations being used.

But yes, my hope is we can identify and catalogue these different zlib
implementations causing different uncompressed data so we can use identical
versions for all builds.

- Fay