[rb-general] [jvm] How to share rebuilder attestations

Thu Jan 3 09:21:49 CET 2019

Le mercredi 2 janvier 2019, 13:11:43 CET Arnout Engelen a écrit :
> Hello,
> 
> I didn't have the chance to join in Paris, so I'm not sure whether
> mechanisms for rebuilders to share their attestations were discussed
> there.
I was there on the first day only, and we didn't discuss that *for the JVM*: 
if you look at the "preliminary Meeting report by Aspiration" linked from [1], 
in parallel to the "Java reproducibility" strategic workshop (from where I'm 
focusing first on the buildinfo for the JVM), there was a "Rebuilders" 
workshop
I just read the report of this WS, and see as I expected: it is yet very 
"Linux distribution" centric, and even there, it's yet in the early phases.

IMHO, we need to learn from what's done there and extend/update to the JVM 
ecosystem (with public repositories in the Maven format, or perhaps even other 
formats)

[1] https://reproducible-builds.org/events/paris2018/

> 
> Having each successful rebuilder append his signature to a shared .asc
> would indeed be elegant
+1

> when we can expect the .buildinfo for the
> original build to always be identical to the .buildinfo of any
> rebuilder.
yes, what to sign? buildinfo?
if buildinfo was a pure specification of the instructions to rebuild and only 
that, signing them would be ok.
But buildinfo is currently also a recording of an environment used to build = 
something that I don't expect to be reproducible in the JVM world: we're not 
in a Linux distribution case, I'd like to keep some flexibility to rebuild

Why not signing the output artefact?
= attesting: "I was able to produce such a binary" (and I don't tell precisely 
how my build environment was near from the official one)?

> As recorded at the Debian wiki, though, I don't think we
> should expect this:
> 
> At https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles:
> > We thought that .buildinfo files would be all the data required to do a
> > rebuild, as opposed to a description of the state of the build system.
> > However, it's likely that we don't know the actual requirements,
> > and it's likely that the description will be more detailed than is
> > necessary in some cases. This means that two different buildinfo files
> > could attest to the same exact binary artifact.
> I think it would be useful to allow rebuilders to share buildinfo's
> that conflict with the 'official' ones, showing any differences in the
> build system and potentially unreproducibilities.
> 
> Indeed this is what happens on the Debian PoC .buildinfo server at
> https://buildinfo.debian.net, for example
> https://buildinfo.debian.net/sources/ca-certificates/1.0 .
> 
> This server achieves a unique signed-buildinfo URL by including the
> hash of the buildinfo in the path. Alternatively, we could name the
> file as "${artifactId}-${version}-${string}.buildinfo" where "string"
> is an arbitrary disambiguating string. This could be the hash of the
> buildinfo file, or some other information. Something similar was
> proposed for the Debian .buildinfo earlier as well
> (https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles?action=diff&rev2=
> 35&rev1=34).
> 
> You mentioned you expect scalability problems when each rebuilder adds
> 1 or 2 files - indeed this could quickly add up. On the other hand,
> what we're specifying is just the API. Since we could choose a more
> efficient way to store this information 'internally' if this turned
> out to be a bottleneck I don't think we should worry about this while
> designing the API.
> 
> For the JVM, I think staying close to the Maven repository layout/API,
> and naming buildinfo's according to
> "${artifactId}-${version}-${string}.buildinfo" (and signatures as
> "${artifactId}-${version}-${string}.buildinfo.asc") is both feasible
> and relatively elegant.
+1 when original artifacts are stored in a repository in the Maven repository 
format
perhaps this does not cover every case: I know Debian rebuilds every JVM 
artifact, but I don't know where the Debian artifacts are stored to be reused

> 
> You mentioned enumerating the attestations as one of the challenges
> when not including them all in the same file. One way to do this might
> be like http://repo.bzzt.net:8000/net/bzzt/simple_2.12/0.1.0-SNAPSHOT
a few comments when seeing this example:
1. if we have multiple buildinfo, we need to have a way to have a chance that 
people reuse an existing buildinfo instead of creating a new one (in the 
example, recording the date of the build means that nobody will add his 
signature to an existing buildinfo...)
2. if 2 buildinfos get the same output hashes, why recording everything?
and if the output hashes are not the same, should we accept the buildinfo?

this discussion is very interesting and is a good next step after our first 
JVM buildinfo spec (once I'll have implemented the spec for Maven and you for 
SBT :) )
But I feel we're still not many people in the discussion: and this rebuilds 
attestation sharing is exactly where the general Reproducible Builds community 
feedback would be useful

Or perhaps we're going just a little bit too fast given we don't have yet 
feedback on the first steps?

Regards,

Hervé

> .
> 
> 
> Kind regards,
> 
> Arnout
> 
> On Sun, Dec 23, 2018 at 3:08 PM Hervé Boutemy <hboutemy at apache.org> wrote:
> > Le dimanche 23 décembre 2018, 14:01:47 CET Arnout Engelen a écrit :
> > > I think it would make sense to upload your own uniquely-named
> </hboutemy at apache.org>
> 
> > > buildinfo and accompanying signature to a separate
> > > "certification/attestation repository", to which anyone can upload
> > > (append only) additional buildinfo's and signatures.
> > 
> > "uniquely-named buildinfo": good catch, I didn't even think at this
> > detail.
> > To me, this is the additional detail that makes this scenario not viable:
> > what algorithm to create uniquely named files? and how would people just
> > wanting to benefit from the rebuild signature list the files?
> > I already did not think that additional file was a good scenario, since
> > this would mean 1 or 2 files per rebuilder (pgp signature + eventual
> > separate buildinfo), then thousands of files (let's be optimistic and
> > think many many people will rebuild  )
> > 
> > that's why I thought at appending personal signature appended to existing
> > .asc: this does not add new files, just grows the existing files
> > and makes the discovery of others signatures quite easy
> > 
> > Do you know if some other strategy for rebuilders has been discussed
> > during
> > Reproducible Builds day in Paris (be it for any other type of repository)?
> > 
> > 
> > Regards,
> > 
> > Hervé
> > 
> > > Kind regards,
> > > 
> > > Arnout
> > > _______________________________________________
> > > rb-general at lists.reproducible-builds.org mailing list
> > > 
> > > To change your subscription options, visit
> > > https://lists.reproducible-builds.org/listinfo/rb-general.
> > > 
> > > To unsubscribe, send an email to
> > > rb-general-unsubscribe at lists.reproducible-builds.org.
> > 
> > _______________________________________________
> > rb-general at lists.reproducible-builds.org mailing list
> > 
> > To change your subscription options, visit
> > https://lists.reproducible-builds.org/listinfo/rb-general.
> > 
> > To unsubscribe, send an email to
> > rb-general-unsubscribe at lists.reproducible-builds.org.
> _______________________________________________
> rb-general at lists.reproducible-builds.org mailing list
> 
> To change your subscription options, visit
> https://lists.reproducible-builds.org/listinfo/rb-general.
> 
> To unsubscribe, send an email to
> rb-general-unsubscribe at lists.reproducible-builds.org.