[rb-general] buildinfo content for JVM based build

Mon Dec 31 00:23:08 CET 2018

Le dimanche 30 décembre 2018, 15:10:08 CET Arnout Engelen a écrit :
> On Sat, Dec 29, 2018 at 6:50 PM Hervé Boutemy <hboutemy at apache.org> wrote:
> > I just published https://reproducible-builds.org/docs/jvm/ with a
> > definition of .buildinfo after the little discussion we had (see the
> > commit [1])
> Ah, I had already started by adding a JVM section to
> https://reproducible-builds.org/docs/recording/ last week. The full
> description would be unwieldy there, though, so I agree it makes sense
> to introduce a dedicated page. I'll condense the JVM section and link
> to the page. Let's collaborate on the JVM page through Merge Requests?
sure, let's use Merge Requests

> 
> > I'll now work on implementing this in reproducible-build-maven-plugin.
> 
> Great! I'm sure you are already aware of
> https://github.com/Zlika/reproducible-build-maven-plugin , which deals
> only with stripping jar files, not with buildinfo metadata.
yes, I'm in contact with Zlika, he knows I'll provide him a Pull Request for a 
new goal :)

Regards,

Hervé

> 
> 
> Kind regards,
> 
> Arnout
> 
> > Le dimanche 23 décembre 2018, 15:41:54 CET Hervé Boutemy a écrit :
> > > Le dimanche 23 décembre 2018, 13:57:16 CET Arnout Engelen a écrit :
> > > > On Sat, Dec 22, 2018 at 7:17 PM Hervé BOUTEMY <herve.boutemy at free.fr> 
wrote:
> > > > > > I do think we should include the
> > > > > > 'classifier' field, if any, though.
> > > > > 
> > > > > what do you call "classifier"?
> > > > 
> > > > The field as described at https://maven.apache.org/pom.html
> > > 
> > > oh, I see
> > > we need both a classifier and an extension, ie we need a type (also
> > > called
> > > artifact handler...)
> > > but there is no default type currently for source tarballs...
> > > see https://maven.apache.org/ref/3.6.0/maven-core/artifact-handlers.html
> > > 
> > > or we can simply define a convention in buildinfo: by default, source
> > > tarball has "source-release" classifier and "zip" extension
> > > this is the convention used by most Apache published releases, and
> > > reused by many: see 62000 results on
> > > https://search.maven.org/search?q=l:source-release
> > > 
> > > we can add a new default type in a future Maven release to make this
> > > convention more visible than the current habit-only
> > > Any idea for the type id?
> > > 
> > > > > > I agree it would be useful to include those: they shouldn't affect
> > > > > > the
> > > > > > build, but including them in the buildinfo may make it easier to
> > > > > > spot
> > > > > > when they accidentally do (combined with comparing diffoscope
> > > > > > output
> > > > > > of course).
> > > > > 
> > > > > notice that we could imagine a verbose buildinfo for investigation,
> > > > > while
> > > > > normal buildinfo would not contain too much
> > > > 
> > > > I think the buildinfo has 2 purposes: both to be able to recreate the
> > > > environment
> > > > to be able to reproduce the build, but also to describe aspects of the
> > > > environment that may help explain reproducibility problems.
> > > 
> > > I'm not sure the second purpose is really intended.
> > > 
> > > > For this reason
> > > > I think it is useful to always produce the "verbose" buildinfo. I
> > > > don't
> > > > see
> > > > a strong need for having both a "verbose" and a "succinct" variation
> > > > of
> > > > the
> > > > buildinfo.
> > > 
> > > I see multiple reasons to have buildinfo as minimal as possible:
> > > - don't add noise on what elements of environment are real prerequisites
> > > - not publish too much details on the machine I used for the release
> > > 
> > > I think we should have 2 separate terms for the 2 puproses:
> > > - buildinfo is the info to rebuilt, as minimal as possible, and
> > > published
> > > alongside the official releases, with infinite lifetime
> > > - environment info dump, or something like this, that are just temporary
> > > dumps for debugging between people having found an unexpected issue
> > > during
> > > rebuild: either the fix will be to make the build process insensitive to
> > > the unexpected aspect of the environment, or to add something key to the
> > > official buildinfo
> > > 
> > > adding something to the official buildinfo should not happen once we
> > > have
> > > sufficient experience in reproducible builds on the JVM
> > > 
> > > > > source is useful, since it's the location where to get sources
> > > > > should support many options: artifact coordinates, url of tarball,
> > > > > vcs
> > > > > tag
> > > > > coordinates...
> > > > 
> > > > Hmm. Perhaps it might also be useful to be able to record both the uri
> > > > of
> > > > the source tarball and the vcs location and tag. Perhaps we should
> > > > group
> > > > those under various "source.xxx" fields?
> > > 
> > > good idea
> > > 
> > > > > > > > java.version=1.8.0_191
> > > > > > > 
> > > > > > > ok, this is the exact version of JVM used to run the build tool,
> > > > > > > and
> > > > > > > we
> > > > > > > expect that any 1.8.0 JDK will permit same rebuild
> > > > > > 
> > > > > > I'm not sure that is safe to expect, but in any case we should
> > > > > > record
> > > > > > it
> > > > > > ;).>
> > > > > 
> > > > > we'll need to share experience on how build result is sensitive to
> > > > > the
> > > > > JDK
> > > > > version: I hope patch version is not something important, nor JDK
> > > > > distribution, or having effective rebuild will require complex
> > > > > specific
> > > > > setup on each case
> > > > 
> > > > This would indeed be great, I'm just less optimistic I guess ;).
> > > 
> > > yes, we need to transform individual guesses into concrete experience
> > > I'll ask Java gurus like Remi Forax when I meet him if there is some
> > > documented expectations on compiler's output stability.
> > > And we'll need to write down concrete examples when we find some issues.
> > > 
> > > I started to create an account on Salsa to publish docs to reproducible-
> > > builds: geven where we are now, I think we should start defining our
> > > buildinfo format for SBT and Maven and any other tool that wants to join
> > > the effort.
> > > 
> > > Regards,
> > > 
> > > Hervé
> > > 
> > > > Kind regards,
> > > > 
> > > > Arnout
> > > > _______________________________________________
> > > > rb-general at lists.reproducible-builds.org mailing list
> > > > 
> > > > To change your subscription options, visit
> > > > https://lists.reproducible-builds.org/listinfo/rb-general.
> > > > 
> > > > To unsubscribe, send an email to
> > > > rb-general-unsubscribe at lists.reproducible-builds.org.
> > > 
> > > _______________________________________________
> > > rb-general at lists.reproducible-builds.org mailing list
> > > 
> > > To change your subscription options, visit
> > > https://lists.reproducible-builds.org/listinfo/rb-general.
> > > 
> > > To unsubscribe, send an email to
> > > rb-general-unsubscribe at lists.reproducible-builds.org.
> > 
> > _______________________________________________
> > rb-general at lists.reproducible-builds.org mailing list
> > 
> > To change your subscription options, visit
> > https://lists.reproducible-builds.org/listinfo/rb-general.
> > 
> > To unsubscribe, send an email to
> > rb-general-unsubscribe at lists.reproducible-builds.org.
> _______________________________________________
> rb-general at lists.reproducible-builds.org mailing list
> 
> To change your subscription options, visit
> https://lists.reproducible-builds.org/listinfo/rb-general.
> 
> To unsubscribe, send an email to
> rb-general-unsubscribe at lists.reproducible-builds.org.