[rb-general] auto-analyzing indeterminism

[Bernhard M. Wiedemann]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

during the r-b summit in Berlin I heard of an idea to automatically
'bisect' sources of indeterminism to make it easier to fix software.

Yesterday (when I could not sleep), I did a quick proof of that
concept in 60 lines of code in
https://github.com/bmwiedemann/reproducibleopensuse/blob/devel/autoclass
ify

This will get further refined, but already seems useful enough so that
I'm currently running it with the ~500 smaller unreproducible packages
in openSUSE.


The basic idea is that there is only a limited number of sources of
indeterminism. In openSUSE we already have a rather normalized build
environment with constant user, path, locale, timezone

And now it is possible to not vary some of the others.
Which of them gets more or less indeterminism is a small number of
bits in my script:
1. date
2. hostname
3. filesystem readdir order (using disorderfs sort mode)
4. date+time (when called via 'date' command)
5. apply strip-nondeterminism to all files after build
6. use some experimental r-b-patched package versions


I was thinking to add some more tweaks, e.g.

one bit for building with a 1-core VM instead of a 4-core VM to reduce
indeterminism from races.

And one bit for replacing /dev/{random,urandom} with /dev/zero (or
some PRNG seeded by S_D_E) during build which might catch cases such
as xrdp and nrpe including newly generated cryptographic keys or dhparam
s.



The raw outcome of those tests then look like
==> fio/.rb.autoclassify <==
0 0 1 0 0 0

This has only bit 3 set, meaning it only built reproducibly when
readdir gave ordered results, making it easier to find the proper patch
https://github.com/axboe/fio/commit/785e49c659023df1735bff195ad4ba133ebd
23a7


Here are some of the more meaningful results so far:

==> antlr/.rb.autoclassify <==
0 0 0 0 1 0

==> ant/.rb.autoclassify <==
1 0 0 0 0 0

==> apache-commons-beanutils/.rb.autoclassify <==
0 0 1 0 1 0

==> apache-commons-cli/.rb.autoclassify <==
0 0 1 0 0 1

==> automake/.rb.autoclassify <==
1 0 0 0 0 0

==> avalon-logkit/.rb.autoclassify <==
0 0 1 0 1 0

==> bcel5_3/.rb.autoclassify <==
0 0 1 0 1 0

==> bcel/.rb.autoclassify <==
0 0 0 0 1 0

==> beansbinding/.rb.autoclassify <==
0 0 1 0 1 0

==> bea-stax/.rb.autoclassify <==
0 0 0 0 1 0

==> berkeleydb/.rb.autoclassify <==
0 0 1 0 1 0

==> blobwars/.rb.autoclassify <==
0 0 1 0 0 0

==> bouncycastle/.rb.autoclassify <==
0 0 0 0 1 1



==> werken-xpath/.rb.autoclassify <==
0 0 1 0 1 1

==> xalan-j2/.rb.autoclassify <==
0 0 0 0 1 0

==> xerces-j2/.rb.autoclassify <==
0 0 0 0 1 0

==> xmlbeans/.rb.autoclassify <==
0 0 1 0 1 0

==> xml-commons-apis-bootstrap/.rb.autoclassify <==
0 0 0 0 1 0

==> xml-commons/.rb.autoclassify <==
0 0 0 0 1 1

==> xorg-x11-server/.rb.autoclassify <==
0 0 0 1 0 0

==> xpp2/.rb.autoclassify <==
0 0 1 0 1 0

==> xpp3/.rb.autoclassify <==
0 0 1 0 1 0


Ciao
Bernhard M.
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQRk4KvQEtfG32NHprVJNgs7HfuhZAUCWWwurwAKCRBJNgs7Hfuh
ZMuqAKCBtLyOkT7zeY60+LVRgyXwzHjCDwCg0vnWe3NVQ0W+1sKHQ0lqZMAutTI=
=RIIS
-----END PGP SIGNATURE-----