Introducing: Semantically reproducible builds

[FC Stegerman]

* Vagrant Cascadian <vagrant at reproducible-builds.org> [2023-05-29 06:10]:
[...]
> I still expect it will be harder to actually do "semantically
> reproducible builds" than "fully reproducible builds".
> 
> To be honest, it sounds like a lot of extra work to avoid fixing things
> properly...

+1

> I find it hard to believe it could so close that you can programatically
> determine something is (probably!) mostly harmless and yet still have it
> be implausible to go all the way to make a properly reproducible build.
> 
> That flys in the face of the thousands of packages I have personally
> reviewed, and submitted patches for hundreds of them... sometimes only
> partially successful, but in the vast majority of cases I end up staring
> at megabytes of leftover gibberish... or something bit-for-bit
> reproducible.

+1

> The main type of issue you have mentioned, timestamps, is usually the
> easiest one to identify and actually remove from resulting builds.
> 
> The other one, arbitrary files that end up in the package? That also
> sounds like it ought to be something easy to fix; part of the build
> process should remove files known to arbitrarily appear if they are
> truely non-functional and not intended to be part of the built
> artifacts. Although why these files appear arbitrarily and unpredictibly
> is a bit of a concerning situation...
> 
> The only solid case I can think of off the top of my head would be
> embedded cryptographic signatures, but as I understand it, there are
> projects (android apks used by f-droid? rpm?) that have some ways to
> deal with this, even if they are less than ideal.

F-Droid treats the APK signature as a build input: apksigcopier [1]
allows us to extract the signature from the reference APK and copy it
to another (unsigned) APK.

The last step in creating the reference build is to sign the unsigned
APK using the developer's private key.  The last step in creating the
verification build is doing the same by copying the extracted
signature, thus removing the need to have access to the private key.

We can then verify whether the build was reproducible by checking
whether both APKs are bit-by-bit identical as usual, the simplest
method providing the strongest guarantee.

Admittedly, detached signatures would be much easier to handle than
embedded ones.  But it's much easier to make sure that the signature
copying does not copy "too much" (i.e. anything not part of the
signature) than it is to be certain that a comparison tool that skips
(or strips) signatures is not comparing "too little" (as many such
tools have been found to do [2]) and thus falsely reporting a build as
reproducible.

> If there are still outstanding issues on a given piece of software (or
> toolchain), it should be tracked in some sort of issue tracker...
> 
> >> - That the naming terminology conflates with true reproducible builds,
> >> therefore creating the potential for misunderstanding to consumers.
> >
> > Naming is hard. As long as the term is carefully defined I think it works.
> > You can use "fully reproducible build" when you want to contrast, and
> > that makes it clear that a normal "reproducible build" is the stronger test
> > (at the cost of being sometimes harder to achieve).
> 
> I do not like the idea of calling this thing using the Reproducible
> Builds name... it is hard enough to get across the bit-for-bit identical
> meaning of Reproducible Builds, and using it to mean something else will
> inevitibly dilute that meaning, no matter how many other very clearly
> and appropriately selected words you prefix it with.
> 
> I very much worry that the meaning of Reproducible Builds may gradually
> get whittled down by adding more exceptions up to the point of being
> little more than a checkbox in a compliance checklist... with little
> actual benefit to the world at large.
> 
> The primary benefit of Reproducible Builds is that it is easy to
> actually verify the result is in fact reproducible... or not.

+1

- FC

[1] https://github.com/obfusk/apksigcopier
[2] https://gist.github.com/obfusk/c51ebbf571e04ddf29e21146096675f8