[rb-general] buildinfo content for JVM based build

[Arnout Engelen]

On Sat, Dec 29, 2018 at 6:50 PM Hervé Boutemy <hboutemy at apache.org> wrote:
> I just published https://reproducible-builds.org/docs/jvm/ with a definition of .buildinfo after the little discussion we had (see the commit [1])

Ah, I had already started by adding a JVM section to
https://reproducible-builds.org/docs/recording/ last week. The full
description would be unwieldy there, though, so I agree it makes sense
to introduce a dedicated page. I'll condense the JVM section and link
to the page. Let's collaborate on the JVM page through Merge Requests?

> I'll now work on implementing this in reproducible-build-maven-plugin.

Great! I'm sure you are already aware of
https://github.com/Zlika/reproducible-build-maven-plugin , which deals
only with stripping jar files, not with buildinfo metadata.


Kind regards,

Arnout

> Le dimanche 23 décembre 2018, 15:41:54 CET Hervé Boutemy a écrit :
> > Le dimanche 23 décembre 2018, 13:57:16 CET Arnout Engelen a écrit :
> > > On Sat, Dec 22, 2018 at 7:17 PM Hervé BOUTEMY <herve.boutemy at free.fr> wrote:
> > > > > I do think we should include the
> > > > > 'classifier' field, if any, though.
> > > >
> > > > what do you call "classifier"?
> > >
> > > The field as described at https://maven.apache.org/pom.html
> >
> > oh, I see
> > we need both a classifier and an extension, ie we need a type (also called
> > artifact handler...)
> > but there is no default type currently for source tarballs...
> > see https://maven.apache.org/ref/3.6.0/maven-core/artifact-handlers.html
> >
> > or we can simply define a convention in buildinfo: by default, source
> > tarball has "source-release" classifier and "zip" extension
> > this is the convention used by most Apache published releases, and reused by
> > many: see 62000 results on
> > https://search.maven.org/search?q=l:source-release
> >
> > we can add a new default type in a future Maven release to make this
> > convention more visible than the current habit-only
> > Any idea for the type id?
> >
> > > > > I agree it would be useful to include those: they shouldn't affect the
> > > > > build, but including them in the buildinfo may make it easier to spot
> > > > > when they accidentally do (combined with comparing diffoscope output
> > > > > of course).
> > > >
> > > > notice that we could imagine a verbose buildinfo for investigation,
> > > > while
> > > > normal buildinfo would not contain too much
> > >
> > > I think the buildinfo has 2 purposes: both to be able to recreate the
> > > environment
> > > to be able to reproduce the build, but also to describe aspects of the
> > > environment that may help explain reproducibility problems.
> >
> > I'm not sure the second purpose is really intended.
> >
> > > For this reason
> > > I think it is useful to always produce the "verbose" buildinfo. I don't
> > > see
> > > a strong need for having both a "verbose" and a "succinct" variation of
> > > the
> > > buildinfo.
> >
> > I see multiple reasons to have buildinfo as minimal as possible:
> > - don't add noise on what elements of environment are real prerequisites
> > - not publish too much details on the machine I used for the release
> >
> > I think we should have 2 separate terms for the 2 puproses:
> > - buildinfo is the info to rebuilt, as minimal as possible, and published
> > alongside the official releases, with infinite lifetime
> > - environment info dump, or something like this, that are just temporary
> > dumps for debugging between people having found an unexpected issue during
> > rebuild: either the fix will be to make the build process insensitive to
> > the unexpected aspect of the environment, or to add something key to the
> > official buildinfo
> >
> > adding something to the official buildinfo should not happen once we have
> > sufficient experience in reproducible builds on the JVM
> >
> > > > source is useful, since it's the location where to get sources
> > > > should support many options: artifact coordinates, url of tarball, vcs
> > > > tag
> > > > coordinates...
> > >
> > > Hmm. Perhaps it might also be useful to be able to record both the uri of
> > > the source tarball and the vcs location and tag. Perhaps we should group
> > > those under various "source.xxx" fields?
> >
> > good idea
> >
> > > > > > > java.version=1.8.0_191
> > > > > >
> > > > > > ok, this is the exact version of JVM used to run the build tool, and
> > > > > > we
> > > > > > expect that any 1.8.0 JDK will permit same rebuild
> > > > >
> > > > > I'm not sure that is safe to expect, but in any case we should record
> > > > > it
> > > > > ;).>
> > > >
> > > > we'll need to share experience on how build result is sensitive to the
> > > > JDK
> > > > version: I hope patch version is not something important, nor JDK
> > > > distribution, or having effective rebuild will require complex specific
> > > > setup on each case
> > >
> > > This would indeed be great, I'm just less optimistic I guess ;).
> >
> > yes, we need to transform individual guesses into concrete experience
> > I'll ask Java gurus like Remi Forax when I meet him if there is some
> > documented expectations on compiler's output stability.
> > And we'll need to write down concrete examples when we find some issues.
> >
> > I started to create an account on Salsa to publish docs to reproducible-
> > builds: geven where we are now, I think we should start defining our
> > buildinfo format for SBT and Maven and any other tool that wants to join
> > the effort.
> >
> > Regards,
> >
> > Hervé
> >
> > > Kind regards,
> > >
> > > Arnout
> > > _______________________________________________
> > > rb-general at lists.reproducible-builds.org mailing list
> > >
> > > To change your subscription options, visit
> > > https://lists.reproducible-builds.org/listinfo/rb-general.
> > >
> > > To unsubscribe, send an email to
> > > rb-general-unsubscribe at lists.reproducible-builds.org.
> >
> > _______________________________________________
> > rb-general at lists.reproducible-builds.org mailing list
> >
> > To change your subscription options, visit
> > https://lists.reproducible-builds.org/listinfo/rb-general.
> >
> > To unsubscribe, send an email to
> > rb-general-unsubscribe at lists.reproducible-builds.org.
>
>
>
>
> _______________________________________________
> rb-general at lists.reproducible-builds.org mailing list
>
> To change your subscription options, visit https://lists.reproducible-builds.org/listinfo/rb-general.
>
> To unsubscribe, send an email to rb-general-unsubscribe at lists.reproducible-builds.org.