[rb-general] Quick reproducible test for GNU Guix

[Vagrant Cascadian]

I did some quick reproducibility testing running GNU Guix, and so far
got pretty good results:

Using guix (and packages) built from commit:

  f83d07f7778b699d46741a5667113342f5f0a737

$ guix challenge --verbose --diff=diffoscope ...
2,463 store items were analyzed:
  - 2,016 (81.9%) were identical
  - 37 (1.5%) differed
  - 410 (16.6%) were inconclusive

The log file produced was 951MB compressed with lz4, including
diffoscope output.

On an x86_64 virtual machine with 8-12 cores and 16-32 GB of ram (I
upped the specs later in the build process), running Debian and some WIP
packages from:

  https://salsa.debian.org/vagrant/guix

The packages tested were from a minimal system using
gnu/system/examples/bare-bones.tmpl, diffoscope, a couple guix pulls,
all without using any substitutes; all the builds for this system should
be locally built.


At ~82% identical, that's not too bad. The ~17% inconclusive are likely
substitutes that haven't yet built or failed to build on the substitute
server, or some local builds that are not yet in guix (e.g. WIP builds
of reprotest!!). I wonder if "guix challenge" may compare some things
that are normally only built locally (e.g. grub.cfg or similar ?).


There are about 10,000 packages I haven't built and compared yet,
presuming "guix package -A | wc -l" is a reasonable guess, though there
are some hidden packages used as transitive build dependencies, if I
recall.


I did notice that when doing multiple builds in parallel, it uses a
different user for each build, though I don't know if that's normalized
within the build environment container. If not, that could fix quite a
few issues! See issues mentioning "user" at:

  https://salsa.debian.org/reproducible-builds/reproducible-notes/blob/master/issues.yml


Eventually, I'd like to do more systematic test of guix packages, with
published logs per-package, rather than whatever I happened to build on
the system so far, but this was a quick start to help flesh out ideas
for feature requests to "guix challenge" to make this all easier... more
on that soon!


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20200207/e2804e43/attachment.sig>