[rb-general] Definition of "reproducible build"

[John Gilmore]

=?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo at gnu.org> wrote:
> I agree that insisting on provenance is crucial.  Dockerfiles (andsimilar) are often viewed as “source”, but they really aren’t source:the actual source would come with the distros they refer to (Debian,pip, etc.)
> Those distros might in turn refer to external pre-built binaries,though, such as “bootstrap binaries” for compilers (Rust, OpenJDK, andso on.)

I propose a definition for whether a bootable OS distro is reproducible.
(If what you're building is not a whole distro that can self-compile,
this definition doesn't apply.)

Our initial goal would be to produce a bootable binary release (DVD or
USB stick) and a source release (ditto).  The source release would
include the script that allows the binary release to recompile the
source release to a new binary release that ends up bit-for-bit
identical.  Such a binary/source release pair would be called
"reproducible".

That's useful: If you have to fix a bug in it, you can make the mods you
need in the source tree, rebuild the world, and out will come a release
with just that one change in the binaries, verifiably identical except
where it matters.  And developers can use such a release to detect what
changes matter to whom, such as: when you alter a system include file,
which binaries change?

During development, the code would be built by some earlier release's
tools, built piecemeal, etc, like current build processes do.  Anytime
before release, the developers can test whether a draft source release
builds into a binary release that itself can build the sources into the
same binary release.  And fix any discrepancies, ideally long before
release.

This is similar to what GCC does to test itself, or what Cygnus did to
test the whole toolchain for cross-compiling.  But applied to the
entire OS release.

Such a paired source/binary release doesn't require a chain of
provenance of earlier binary software, particularly if people can
demonstrate bootstrapping it using several different earlier compiler
toolchains, still producing the same binaries.  You can bootstrap
it with itself.

The separate efforts to minimize the amount of binary code we have to
trust to do a rebuild are laudable and fascinating.  Keep going!  But we
shouldn't require whole distros to do that yet.  We haven't even
accomplished a basic paired binary/source reproducible release yet, for
any major release -- or have we?

	John

PS: For extra points, the binary release should be able to cross-compile
its source release into a binary release for each other supported
platform, reproducibly.  And those other-platform binary releases should
cross-compile the source release back bit-for-bit into the same binary
release you started with.