Evaluation of bundling .buildinfo in .deb proposal

[Vagrant Cascadian]

On 2020-08-31, kpcyrd wrote:
> I'm a bit short on time, sorry in advance if the email is a little short/blunt:
>
> - What was the original motivation of putting the size and checksum of the
>   package into the buildinfo file? We aren't tracking this info in Arch Linux
>   and it turned out we didn't need those fields to implement a rebuilder.
>   Please consider simply dropping those fields instead of trying to build a
>   tool to work around this.

It's an attestation of of "with source X and build environment Y I
produced artifacts Z".

So I guess you're saying if the .buildinfo is embedded in the binary
package, you can get the hashes of the artifacts from the binaries
themselves as, obviously, you already have the binaries.


>   In Arch Linux we consider the buildinfo file a build parameter to ensure the
>   build environment is always identical, but strictly speaking it's not a build
>   output (even though it's generated during the package build, but you can
>   generate it without actually running the whole build). Having access to the build
>   outputs is not necessary and out of scope of "recording the build
>   environment". In my opinion everything in the buildinfo file that goes beyond
>   "a collection of parameters for the build" is feature-creep at the cost of
>   complexity.
>
>   This also solves the .changes problem (if I understood it correctly). The
>   buildinfo file is available very early (as long as you stop referencing build
>   outputs) and you can simply include it when creating the deb in the first
>   place instead of manipulating it afterwards.

I'm definitely coming around to this idea.


> - The current debian reproducible builds effort is very focused on debian.org,
>   but virtually none of that can be downstreamed by debian derivates. Having
>   externally hosted buildinfo files is an effort that every downstream would
>   need to repeat and every rebuilder need to know about. All Arch Linux
>   downstreams I've checked ship buildinfo files, while zero debian downstreams
>   do. This is an advantage that's currently not mentioned yet.

Nice point!


> - The "having the buildinfo file in the binary package is wasteful" argument is
>   a micro optimization that pushes a non-trivial amount of extra complexity on
>   the debian r-b developers. Considering that debian rebuilder tooling is still
>   very sparse due to the lack of developer resources I'm not sure that's a
>   smart trade-off.

It is largely very compressable, though with source packages that
produce multiple binaries, it's a bit more resources; shipping in each
binary instead of once per source package.

If you then extract the .buildinfo when installing the package is
another resource question; Arch Linux does not, it might make it easier
to get that information out of the installed system than re-fetch
it. Might not be worth it, though.

That said, embedding the .buildinfo seems like a much more workable
approach to a problem we've had for so long...


> - I don't understand the concern about source-only uploads. The uploader can't
>   know the build environment that buildd is going to setup, therefore the
>   buildinfo file needs to be generated by buildd anyway.

The .buildinfo is also generated by the buildd. This is part of the
"multiple signers" property that .buildinfo files were attempting to
address. For the initial upload, you typically have one .buildinfo from
the uploader, and one from each of the architectures it was built
on. The buildd on the relevent architecture in this case becomes a
(somewhat sloppy) rebuilder.

If in fact the .buildinfo is embedded in the binary package, and the
.buildinfo itself is made effectively reproducible, it's been pointed
out to me on IRC that you could then just sign the binary packge itself.


Even if we do manage to start embedding .buildinfo information inside
the .deb files, we could still produce external .buildinfo files with
checksums just as is done now, though minor differences in build
environment will always result in differing builds then even if the
packages are otherwise reproducible...


> Sorry for being rather Arch centric in this email, but I think it's a good idea
> to ensure you're familiar with how other distros solved the problem that
> debian is facing since a few years.

Thank you very much!


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20200831/708262b7/attachment.sig>