[rb-general] Reproducible Java builds with Maven

[Bernhard M. Wiedemann]

Hi Hervé,

thanks for raising this topic.

On 26/11/2018 09.08, Hervé Boutemy wrote:
> Currently, there is a plugin that does the job as archive post-processing, but 
> one intent would be to have native reproducible builds when that does not hurt 
> other objectives, like performance or anything else: we did already change 
> many plugins that for sure improve the situation, but I don't have a real view 
> on how far we are from native reproducible builds.

I think, one major consideration is how many code-places touch .jar
files, because if we want to get rid of post-processing, we might need
to patch all of them to have normalized mtime values in the .jar.

> I know that one key step will be to have reproducible archives creation (with 
> reproducible timestamps and order), which won't be easy without sacrificing 
> performance (ConcurrentJarCreator was created to improve performance, but adds 
> even more variability in order...)

Just because you use parallelism does not mean, your output has to vary.
E.g. you can play with 'pigz' (parallel gzip):
dd if=/dev/urandom bs=1M count=100 of=input
time pigz -p 2 -nc1 input | md5sum

notice how it produces bit-identical results, independent of the number
of threads (given with -p).

> Anybody interested in working together?

I'm interested, but so far kept my fingers off Java as much as I could,
because I felt it is hard to work with.

With Maven, there are also major issues in a related domain, that is
called "bootstrappable builds".
With openSUSE we are doing all builds offline to ensure that we can
repeat builds later (without worry about offline or hacked servers), but
for maven this often meant we had to download 300 MB of someone else's
binaries to use in the build.

https://build.opensuse.org/project/show/home:fstrba:maven
is an ongoing effort to build more things from source.

But bootstrapping would be much easier if maven and its plugins could be
built with ant, make or some other pre-existing bootstrappable build system.

Do you know if such is possible or who would be interested in making it
possible?

Ciao
Bernhard M.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20181126/0332ed54/attachment.sig>