[rb-general] Reproducible Java builds with Maven

Tue Nov 27 12:26:33 CET 2018

On Tue, Nov 27, 2018 at 9:58 AM Hervé BOUTEMY <herve.boutemy at free.fr> wrote:
> Yes, the Buildinfo seems an interesting part to work together.
>
> I'm quite a noob on that, I'll need some pointers on basic info first.

Buildinfo is more of a general concept rather than a well-defined specifciation,
what it looks like exactly will probably vary per 'ecosystem'.

Some words on its purpose and the conventions adopted by Debian, Arch and Tails
can be found at https://reproducible-builds.org/docs/recording/ . For
Scala for now
I generally followed the layout from the Debian approach.

> Then it seems the way we look at this topic is quite different when you think
> as a Linux distribution manager or as a Java/Maven user publishing to Maven
> Central = what I'm looking first as Maven developer.
>
> Should we start by defining a convention for anybody to publish a Buildinfo

That might make sense to me. For sbt-reproducible-builds, I kept it
simple so far:

sbt-reproducible-builds_2.12_0.16_all_aengelen-1543317396000.buildinfo.asc:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Architecture: all
JavaVersion: 1.8.0_171
Format: 1.8
ScalaVersion: 2.12.6
SbtVersion: 1.2.1
Version: 0.16
ScalaBinaryVersion: 2.12
Build-Architecture: all
Checksums-Sha256:
  cd577828ea6ca50c4b1256c57360a391a4904e7e2f4d189a20f735c619f468a3
24627 sbt-reproducible-builds-0.16.jar
Source: sbt-reproducible-builds_2.12
Package: sbt-reproducible-builds_2.12
Binary: sbt-reproducible-builds_2.12
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEQBwUQgsoMY3fOK1ZxrluhORJLnYFAlv9J5QACgkQxrluhORJ
Lnbrng/+P4va6mb6Z6NA0a/YzybcmaInhcoMhGCGSSNvPJrHrzUlhA6ItscHpvRf
dNle7FeqUPjBSTktR/ohOxlO4LIxzwOw8ZDn+5rMYOyonA5+wqUnSBCXySaYAch6
EagmDz6JYSAkHVarfkhEboHopoBVwIlWnCj5YAGKt6cMpFGqLgbd0DK6jf6ObLbu
wJh5CPM9vjT1QS+cJBwT35cKRejLmx6OAdn4F7R43JPdWBio8u16oaVu3DZ7gC6P
vYU6egU5I2e36cLkepti2OqBJ+iIum7AdXcsT1WA4AlqoCnbcVnx90Rb4a3wlNf7
sV8cQE/yTS0LQ8rFTDR6VTkPipqUUTBQMpCMUg85NeQGnmcyrPC8ITrtSF//ayXn
YkwON6BEGK4bEYiojPLacuk+AGs37kpGYb0K3n/j9Q8GKZQppp8LfYI9KcuFyntc
iZZGI6w8jPsGogiQZr3fqO8rIRRZGo7SsvvgDGqhjvcBsLVnXjPrAmoQxRQS5gce
/rA7r0SwWW8LpC2NdKOkyfxPTiZfdH3kdA4Xah+ahXItkXwfs2dqPkPIenc0hHz5
q1eU663FDX/Ec+Gqlh6ZEJRdchEaYBtVGZ3vCXHxhKQZtthuG8rdoVA4kEysq7+P
F9pYUaHcmsaIKv1RKonvbGDW7b+kHLAgwVjVMhvpEXXRSSUeRTs=
=a9ab
-----END PGP SIGNATURE-----

While this format allows multiple checksums in one attestation, for
our ecosystem I think it would make sense to have an attestation
per jar, even for multi-project builds.

We should probably include things like groupId, artifactId, and 'canonical'
upstream mvn/ivy repository?

> alongside his binary artifacts while publishing to Central?

Where to publish those is actually an interesting question: an important part of
reproducible builds, IMO, is that it's not just the package maintainer
that shares
an attestation, but third parties can share their independent
attestations as well,
adding to the confidence that the artifact indeed corresponds to the sources.
Those third parties of course cannot publish those to Central alongside the
'official' package.

For that reason I kept the attestations separate for now - also that makes it
easier to iterate on the format a bit more without feeling we're 'setting things
in stone' just yet.

Kind regards,

Arnout

> Le lundi 26 novembre 2018, 09:40:44 CET Arnout Engelen a écrit :
> > On Mon, Nov 26, 2018 at 9:08 AM Hervé Boutemy <hboutemy at apache.org> wrote:
> > > A few years ago, the work on this started and I created a Wiki page [1] at
> > > Maven to try to consolidate efforts from many isolated people I met who
> > > were interested in the topic: this Wiki page did not attract many
> > > contributions nor even discussions on Maven mailing lists, I hope this
> > > thread at reproducible- builds will help convergence between efforts.
> >
> > Thanks, I wasn't aware of this page.
> >
> > > And one thing that worries me is the variability introduced by the JDK
> > > version used: this one is quite generic to Java, I don't know if there is
> > > currently a global strategy that we could reuse.
> >
> > I don't think there is much to do except including the JDK version in
> > the Buildinfo.
> >
> > > Anybody interested in working together?
> >
> > Quite possibly! I do a lot of programming in Scala (another language
> > targeting the JVM),
> > and have been working on improving reproducibility there by
> > introducing a r-b plugin for
> > its sbt build system, sbt-reproducible-builds[1].
> >
> > That uses the maven plugin you mentioned[2] as a basis for
> > post-processing the artifact
> > (though I'm planning to extract the logic to a separate library). I
> > agree it would be good to
> > fix more things 'at the source', but (as you mentioned above) I
> > suspect some aspects such
> > as jar file generation will probably need post-processing for the
> > foreseeable future.
> >
> > It also has some (crude, very incomplete) features for uploading
> > signed Buildinfo attestations
> > and comparing them with Buildinfo's uploaded by others.
> >
> > This might be an area we could work together on: putting together the
> > conventions and
> > infrastructure to share Buildinfo attestations for JVM library
> > projects. In the JVM world
> > it is common to distribute libraries independently through
> > repositories such as Maven
> > Central, which might be a bit different from how Linux distributions
> > work. Starting on
> > that would be interesting. So far I've been using
> > sbt-reproducible-builds with a (very)
> > simple web service to collect Buildinfo's,
> > reproducible-builds-certification-repository[3].
> > Unfortunately my example server is currently not running so I can't
> > point to that
> > right now.
> >
> >
> > Kind regards,
> >
> > Arnout
> >
> > [1]: https://github.com/raboof/sbt-reproducible-builds
> > [2]: https://github.com/Zlika/reproducible-build-maven-plugin
> > [3]: http://github.com/raboof/reproducible-builds-certification-repository
> > _______________________________________________
> > rb-general at lists.reproducible-builds.org mailing list
> >
> > To change your subscription options, visit
> > https://lists.reproducible-builds.org/listinfo/rb-general.
> >
> > To unsubscribe, send an email to
> > rb-general-unsubscribe at lists.reproducible-builds.org.
>
>
>
>
> _______________________________________________
> rb-general at lists.reproducible-builds.org mailing list
>
> To change your subscription options, visit https://lists.reproducible-builds.org/listinfo/rb-general.
>
> To unsubscribe, send an email to rb-general-unsubscribe at lists.reproducible-builds.org.