Two questions about build-path reproducibility in Debian

[Vagrant Cascadian]

On 2024-03-05, John Neffenger wrote:
> On 3/5/24 8:08 AM, John Gilmore wrote:
>> Our instructions for reproducing any package would have to identify what
>> container/chroot/namespace/whatever the end-user must set up to be able
>> to successfully reproduce a package.

The build instructions always have to identify what defines the build
environment and exactly what that includes may be different from project
to project.


> And even then, it won't always work.
>
> I need to verify the JavaFX builds done by Launchpad, for example, where 
> its LXD container uses a build path as follows:
>
> /build/openjfx/parts/jfx/build/
>
> When I run the same build locally using the same command and a local LXD 
> container, it uses a build path as follows and fails to be reproducible:
>
> /root/parts/jfx/build/
>
> I have no way to change these choices.

Then clearly you have not been provided sufficient information,
configuration, software, etc. in order to reproduce the build!


> I intend to fix this reproducibility bug, and I shouldn't get away
> with not fixing it!

> JDK-8307082: Build path is recorded in JavaFX Controls module
> https://bugs.openjdk.org/browse/JDK-8307082

Great, please do, we will all be better off for it having been fixed!


>> If we move the goal posts in order to claim victory, who are we fooling
>> but ourselves?

There are no moving goalposts, as the goal has always been to be able to
be able to independently verify bit-for-bit results.

Maybe we take the bike path to get there, maybe we go by train or
hovercraft or jetpack. Some ways might be easier or more expensive or
have other consequences or downsides, but as long as the destination is
bit-for-bit reproducible... so be it, if it does the job.

Normalized build environments have been a technique to achieve
reproducible builds, even going back to the early work in bitcoin and
tor over a decade ago, and is used by various projects today to achieve
bit-for-bit identical reproducible builds.

While making software immune to various forms of non-determinism in the
build environment is certainly preferred, it is not the one and only
true way to achieve bit-for-bit identical results that are able to be
independently verified.

  https://reproducible-builds.org/docs/definition/

  "A build is reproducible if given the same source code, build
  environment and build instructions, any party can recreate bit-by-bit
  identical copies of all specified artifacts.

  The relevant attributes of the build environment, the build
  instructions and the source code as well as the expected reproducible
  artifacts are defined by the authors or distributors. The artifacts of
  a build are the parts of the build results that are the desired
  primary output."

The authors or distributors may choose to include build paths or various
other things (e.g. LANG=C, LC_ALL=C.UTF-8) as part of their build
instructions.

So yes, the fewer rube-goldbergian contraptions you need to set up in
order to produce the build environment, the better, surely!

It is technically possible to get an exact matching clock across two
builds for matching timestamps, but it is non-trivial and I would say
unreasonable (but very interesting academic work!). Given current
technologies, it would be unreasonable to expect setting the quark spin
state of particles involved the build process... some things are
relatively easy to normalize.

Builds that build independent of the build path are better for
reproducible builds and even other reasons (e.g. storing the build path
takes a few extra "useless" bits). With most software, it is possible to
get it to build idependent of the build path. With some software, it is
unfortunately more difficult.


> I agree completely. In fact, almost all of us agreed completely on this 
> issue in October 2022:
>
> Give guidance on reproducible builds #1865
> https://github.com/coreinfrastructure/best-practices-badge/issues/1865
>
> Why is this coming up again as if we've forgotten all those arguments 
> against it?

That was a fundamentally different issue about having builds not produce
bit-for-bit identical results still meeting some sort of reproducible
criterion, as opposed to this discussion is, as I see it, about
normalizing the path in which the build is performed in order to get
bit-for-bit identical results.


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240305/bded996b/attachment.sig>