[rb-general] Reproducible Java builds with Maven

Tue Nov 27 16:02:35 CET 2018

ok, I'll have a look at the formats.

On the question "where to publish", I think we have no choice when artifacts 
go to Maven Central: there is one "official" build that goes into Central, 
then this build requires its build descriptor created during the "official" 
build. And this "official" build descriptor will be signed with the same pgp 
key as the "official" artifact.

For checks done by users, that may reinforce the value for the info, this is 
another question: these won't go to Central, since adding new files is not 
something expected by Central. I think this reinforcement is something 
optional in our case: whoever wants to rebuild the artifact and check it 
against the official one, using info from the buildinfo to know how far the 
build environment was from the original "official" build info.

Doesn't it make sense?

Regards,

Hervé

Le mardi 27 novembre 2018, 12:26:33 CET Arnout Engelen a écrit :
> On Tue, Nov 27, 2018 at 9:58 AM Hervé BOUTEMY <herve.boutemy at free.fr> wrote:
> > Yes, the Buildinfo seems an interesting part to work together.
> > 
> > I'm quite a noob on that, I'll need some pointers on basic info first.
> 
> Buildinfo is more of a general concept rather than a well-defined
> specifciation, what it looks like exactly will probably vary per
> 'ecosystem'.
> 
> Some words on its purpose and the conventions adopted by Debian, Arch and
> Tails can be found at https://reproducible-builds.org/docs/recording/ . For
> Scala for now
> I generally followed the layout from the Debian approach.
> 
> > Then it seems the way we look at this topic is quite different when you
> > think as a Linux distribution manager or as a Java/Maven user publishing
> > to Maven Central = what I'm looking first as Maven developer.
> > 
> > Should we start by defining a convention for anybody to publish a
> > Buildinfo
> 
> That might make sense to me. For sbt-reproducible-builds, I kept it
> simple so far:
> 
> sbt-reproducible-builds_2.12_0.16_all_aengelen-1543317396000.buildinfo.asc:
> 
> Architecture: all
> JavaVersion: 1.8.0_171
> Format: 1.8
> ScalaVersion: 2.12.6
> SbtVersion: 1.2.1
> Version: 0.16
> ScalaBinaryVersion: 2.12
> Build-Architecture: all
> Checksums-Sha256:
>   cd577828ea6ca50c4b1256c57360a391a4904e7e2f4d189a20f735c619f468a3
> 24627 sbt-reproducible-builds-0.16.jar
> Source: sbt-reproducible-builds_2.12
> Package: sbt-reproducible-builds_2.12
> Binary: sbt-reproducible-builds_2.12
> 
> While this format allows multiple checksums in one attestation, for
> our ecosystem I think it would make sense to have an attestation
> per jar, even for multi-project builds.
> 
> We should probably include things like groupId, artifactId, and 'canonical'
> upstream mvn/ivy repository?
> 
> > alongside his binary artifacts while publishing to Central?
> 
> Where to publish those is actually an interesting question: an important
> part of reproducible builds, IMO, is that it's not just the package
> maintainer that shares
> an attestation, but third parties can share their independent
> attestations as well,
> adding to the confidence that the artifact indeed corresponds to the
> sources. Those third parties of course cannot publish those to Central
> alongside the 'official' package.
> 
> For that reason I kept the attestations separate for now - also that makes
> it easier to iterate on the format a bit more without feeling we're
> 'setting things in stone' just yet.
> 
> 
> Kind regards,
> 
> Arnout
> 
> > Le lundi 26 novembre 2018, 09:40:44 CET Arnout Engelen a écrit :
> > > On Mon, Nov 26, 2018 at 9:08 AM Hervé Boutemy <hboutemy at apache.org> 
wrote:
> > > > A few years ago, the work on this started and I created a Wiki page
> > > > [1] at
> > > > Maven to try to consolidate efforts from many isolated people I met
> > > > who
> > > > were interested in the topic: this Wiki page did not attract many
> > > > contributions nor even discussions on Maven mailing lists, I hope this
> > > > thread at reproducible- builds will help convergence between efforts.
> > > 
> > > Thanks, I wasn't aware of this page.
> > > 
> > > > And one thing that worries me is the variability introduced by the JDK
> > > > version used: this one is quite generic to Java, I don't know if there
> > > > is
> > > > currently a global strategy that we could reuse.
> > > 
> > > I don't think there is much to do except including the JDK version in
> > > the Buildinfo.
> > > 
> > > > Anybody interested in working together?
> > > 
> > > Quite possibly! I do a lot of programming in Scala (another language
> > > targeting the JVM),
> > > and have been working on improving reproducibility there by
> > > introducing a r-b plugin for
> > > its sbt build system, sbt-reproducible-builds[1].
> > > 
> > > That uses the maven plugin you mentioned[2] as a basis for
> > > post-processing the artifact
> > > (though I'm planning to extract the logic to a separate library). I
> > > agree it would be good to
> > > fix more things 'at the source', but (as you mentioned above) I
> > > suspect some aspects such
> > > as jar file generation will probably need post-processing for the
> > > foreseeable future.
> > > 
> > > It also has some (crude, very incomplete) features for uploading
> > > signed Buildinfo attestations
> > > and comparing them with Buildinfo's uploaded by others.
> > > 
> > > This might be an area we could work together on: putting together the
> > > conventions and
> > > infrastructure to share Buildinfo attestations for JVM library
> > > projects. In the JVM world
> > > it is common to distribute libraries independently through
> > > repositories such as Maven
> > > Central, which might be a bit different from how Linux distributions
> > > work. Starting on
> > > that would be interesting. So far I've been using
> > > sbt-reproducible-builds with a (very)
> > > simple web service to collect Buildinfo's,
> > > reproducible-builds-certification-repository[3].
> > > Unfortunately my example server is currently not running so I can't
> > > point to that
> > > right now.
> > > 
> > > 
> > > Kind regards,
> > > 
> > > Arnout
> > > 
> > > [1]: https://github.com/raboof/sbt-reproducible-builds
> > > [2]: https://github.com/Zlika/reproducible-build-maven-plugin
> > > [3]:
> > > http://github.com/raboof/reproducible-builds-certification-repository
> > > _______________________________________________
> > > rb-general at lists.reproducible-builds.org mailing list
> > > 
> > > To change your subscription options, visit
> > > https://lists.reproducible-builds.org/listinfo/rb-general.
> > > 
> > > To unsubscribe, send an email to
> > > rb-general-unsubscribe at lists.reproducible-builds.org.
> > 
> > _______________________________________________
> > rb-general at lists.reproducible-builds.org mailing list
> > 
> > To change your subscription options, visit
> > https://lists.reproducible-builds.org/listinfo/rb-general.
> > 
> > To unsubscribe, send an email to
> > rb-general-unsubscribe at lists.reproducible-builds.org.
> _______________________________________________
> rb-general at lists.reproducible-builds.org mailing list
> 
> To change your subscription options, visit
> https://lists.reproducible-builds.org/listinfo/rb-general.
> 
> To unsubscribe, send an email to
> rb-general-unsubscribe at lists.reproducible-builds.org.