Irregular status update about reproducible Debian live ISO images

[Vagrant Cascadian]

On 2025-03-27, Simon Josefsson wrote:
> Vagrant Cascadian <vagrant at reproducible-builds.org> writes:
>
>> So yes, the live images may not be reproducible builds all the way
>> down... but the live images are reproducible, in the sense that you can
>> build them from all of the various inputs which are .deb files.
>
> It is great that live images are reproducible from binary *.deb's!
>
> I agree with Ian that claiming that the live images are 100%
> reproducible is not describing the entire picture, as long as they
> contain non-free firmware that we cannot rebuild from source code.

Sure, I admit there needs to be clarity on exactly what the claim is.

The source code is a project which downloads a bunch of binary blobs
(ALL of the .deb files, not just the non-free-firmware) and assembles
them into artifacts (the live image .iso files) in a way that can be
done 100% reproducibly.

I admit signficant discomfort and concern in using the .deb files as the
input to the source code... and the risks if this logic was misapplied
in some more egregious context. But in the context of building an
operating system image out of the very same operating system's packages,
it is not, in my opinion... inherrantly wrong.

For the case of live images for Debian, this work is essential to be
able to eventually build it from packages that are 100% reproducible,
and the fact that some small fraction of the .deb files are not
themselves reproducible now should not block that work... otherwise we
have a problem of never improving anything because you cannot improve
everything all at once.


> I believe the definition of Reproducible Builds has changed over time, I
> believe it initially was about rebuilding bit-by-bit identical outputs
> from freely licensed source code.

I think it still is:

  https://reproducible-builds.org/docs/definition/


> My perception is that some doesn't care about that property any more,
> and are content with rebuilding from source-like artifacts which may
> themselves be binaries.

I see the concern, treating the .deb files as inputs to the build
process...

At least in Debian, so far, I believe we only even attempt to rebuild
Debian packages from main to check for reproducibility, which does not
include the non-free, non-free-firmware and contrib components.

There are *some* packages in Debian contrib, non-free and possibly
non-free-firmware which could actually have meaningful reproducibility
testing and include full source code, but have other licensing
restrictions or dependencies that prevent them from being fully free
software (e.g. GNU Documentation under GFDL without the litany of
license exceptions). But that is a truely tiny fraction of packages in
Debian...


> Otherwise I cannot explain why people go through such effort to
> rebuild binary Debian packages based on other older binary packages
> [that nobody check if they can be built reproducibly].

I am not even sure what you are getting at here...

You have to rebuild the packages with the same toolchain that was used
to build them in order to have any expectation that you get the same
artifact outputs. You cannot expect to get the same garbage out if you
do not put the same garbage in, even if it occasionally happens by luck
and chance.


>> Obviously, if all the .deb files themselves were built fully from source
>> and reproducible, this would be ideal. A much stronger foundation would
>> be to actually independently rebuild each and every package from source,
>> compare the checksums against the package in Debian proper, and then
>> build a live image from those packages.
...
>> Someone can check out the source code to build the live images, create a
>> sufficiently similar build environment(maybe needing to use
>> snapshot.debian.org to avoid package version drift), and follow the
>> instructions to get bit-for-bit identical live images, without
>> independently verifing every .deb contained in the image itself. Huzzah!
>
> Yay!  I believe we need two steps:
>
> 1) Create an environment suitable for dpkg-buildpackage based on Guix
> binaries.
>
> 2) Build Debian packages incrementally until we have rebuilt all of
> them.

Sounds like an interesting project! I have toyed with aspects of this in
the past, but mostly as a thought experiment...

I would be a herculean project to actually attempt (re)bootstrapping all
of Debian from a known bootstrappable base.

Though even guix still has a few binary seeds outside of the full-source
bootstrap (a bootstrap guile ... some language-specific
bootstraps... haskell?).


> Compare each built Debian package with what's in the archive, publishing
> diffoscope outputs along the way.

I predict a huge pile of inscrutible binary diff output! :)


> Because doing that takes time, I would divide it into two parallel
> efforts:
>
> 1) Make sure the Debian archive can be idempotently rebuilt in a
> bit-by-bit identical way.

The idempotently rebuilding concept seems far-fetched. For example, if
you change a compiler version, I would expect the new compiler (at least
potentially) results in different outputs. Unless the compiler is making
a promise to never improve the resulting binary code with current
knowledge... my guess is that will be a hard sell to compiler
developers. Though declaring a compiler finished in this way would be
interesting, for sure!

It is definitely possible, in very limited and targeted special-purpose
software (e.g. Mes) where the fundamental design goal aligns with such
aims.  I think it is bordering on unreasonable to expect that of the
whole corpus of all (free) software for the forseeable future.


> 2) Cross-build the smallest set of Debian packages (build-essential?)
> needed to rebuild the rest, using binaries from Guix which I believe is
> bootstrappable all the way down.

The smallest set of build-essential in Debian has a very large
dependency set (well over 5000 packages):

  https://tests.reproducible-builds.org/debian/unstable/amd64/pkg_set_build-essential-depends.html

And I am not even sure that 5000+ packages includes all of the
build-essential-depends dependencies... bootstrapping today is quite the
mess.

I would truely love to hear results from such a huge effort!


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20250327/4a2f342b/attachment.sig>