[rb-general] buildinfo content for JVM based build

[Hervé Boutemy]

Le dimanche 23 décembre 2018, 13:57:16 CET Arnout Engelen a écrit :
> On Sat, Dec 22, 2018 at 7:17 PM Hervé BOUTEMY <herve.boutemy at free.fr> wrote:
> > > I do think we should include the
> > > 'classifier' field, if any, though.
> > 
> > what do you call "classifier"?
> 
> The field as described at https://maven.apache.org/pom.html
oh, I see
we need both a classifier and an extension, ie we need a type (also called 
artifact handler...)
but there is no default type currently for source tarballs...
see https://maven.apache.org/ref/3.6.0/maven-core/artifact-handlers.html

or we can simply define a convention in buildinfo: by default, source tarball 
has "source-release" classifier and "zip" extension
this is the convention used by most Apache published releases, and reused by 
many: see 62000 results on https://search.maven.org/search?q=l:source-release

we can add a new default type in a future Maven release to make this 
convention more visible than the current habit-only
Any idea for the type id?

> 
> > > I agree it would be useful to include those: they shouldn't affect the
> > > build, but including them in the buildinfo may make it easier to spot
> > > when they accidentally do (combined with comparing diffoscope output
> > > of course).
> > 
> > notice that we could imagine a verbose buildinfo for investigation, while
> > normal buildinfo would not contain too much
> 
> I think the buildinfo has 2 purposes: both to be able to recreate the
> environment
> to be able to reproduce the build, but also to describe aspects of the
> environment that may help explain reproducibility problems.
I'm not sure the second purpose is really intended.

> For this reason
> I think it is useful to always produce the "verbose" buildinfo. I don't see
> a strong need for having both a "verbose" and a "succinct" variation of the
> buildinfo.
I see multiple reasons to have buildinfo as minimal as possible:
- don't add noise on what elements of environment are real prerequisites
- not publish too much details on the machine I used for the release

I think we should have 2 separate terms for the 2 puproses:
- buildinfo is the info to rebuilt, as minimal as possible, and published 
alongside the official releases, with infinite lifetime
- environment info dump, or something like this, that are just temporary dumps 
for debugging between people having found an unexpected issue during rebuild: 
either the fix will be to make the build process insensitive to the unexpected 
aspect of the environment, or to add something key to the official buildinfo

adding something to the official buildinfo should not happen once we have 
sufficient experience in reproducible builds on the JVM

> 
> > source is useful, since it's the location where to get sources
> > should support many options: artifact coordinates, url of tarball, vcs tag
> > coordinates...
> 
> Hmm. Perhaps it might also be useful to be able to record both the uri of
> the source tarball and the vcs location and tag. Perhaps we should group
> those under various "source.xxx" fields?
good idea

> 
> > > > > java.version=1.8.0_191
> > > > 
> > > > ok, this is the exact version of JVM used to run the build tool, and
> > > > we
> > > > expect that any 1.8.0 JDK will permit same rebuild
> > > 
> > > I'm not sure that is safe to expect, but in any case we should record it
> > > ;).> 
> > we'll need to share experience on how build result is sensitive to the JDK
> > version: I hope patch version is not something important, nor JDK
> > distribution, or having effective rebuild will require complex specific
> > setup on each case
> 
> This would indeed be great, I'm just less optimistic I guess ;).
yes, we need to transform individual guesses into concrete experience
I'll ask Java gurus like Remi Forax when I meet him if there is some 
documented expectations on compiler's output stability.
And we'll need to write down concrete examples when we find some issues.

I started to create an account on Salsa to publish docs to reproducible-
builds: geven where we are now, I think we should start defining our buildinfo 
format for SBT and Maven and any other tool that wants to join the effort.

Regards,

Hervé

> 
> 
> Kind regards,
> 
> Arnout
> _______________________________________________
> rb-general at lists.reproducible-builds.org mailing list
> 
> To change your subscription options, visit
> https://lists.reproducible-builds.org/listinfo/rb-general.
> 
> To unsubscribe, send an email to
> rb-general-unsubscribe at lists.reproducible-builds.org.