Second status update about reproducible live-build ISO images in Jenkins

[Roland Clobus]

Hello lists,

here is a second update of the status for reproducible live-build ISO
images [1].

* All major configurations are now built on a daily basis using
live-build [2]
* All major configurations (except for one) are reproducible
* Only the cinnamon image has shown non-reproducible builds, but...
** Diffoscope has issues when comparing these ISO files [3]
   The ISO images were captured, but on my computer diffoscope is able
to finish without a crash (taking more than 2 hours and a lot of space
on /tmp though)
   So this crash might be 'unique' to the current Jenkins setup
** The Perl script /usr/share/perl5/XML/SAX/Debian.pm of libxml-sax-perl
contains a foreach on a hash, which _occasionally_ results in a
different sort order [4] (a patch is pending, to be added to the Jenkins
script, [1] and a new bug report)
* While generating the artifacts for later retrieval, I missed a cleanup
step, which resulted in /tmp on the Jenkins master node to fill up.
Sorry about that...
** As an emergency step, the generation of artifacts is disabled
** A merge request (containing several modifications) is planned, which
prevents such possible scenarios

You can stop reading here if you want...

Future plans:
* The building of the live-build images will be spread more evenly, to
avoid heavy spikes [5]
* The new snapshot service will be used [6]
* Reprotest might be used instead of just 2 builds without a short time
frame, to capture more variations
* Reprotest does not appear to set PERL_HASH_SEED, which might trigger
some more non-reproducible cases
* The reporting page of the Jenkins job is still rather minimal
* The generated ISO files will be stored again (for 24 hours), when it
can be assured that Jenkins will not be filled up again
* I would like to test the functionality of the generated ISO image.
** I've read about the approach by Tails, that looks really promising
(and cool) [7]
** There is also OpenQA, which already tests the current daily images [8]
** Running tests of the functionality of the installer images would
reduce a lot of stress during release times
* When live-build images are working fine, the work could be extended to
other images, e.g. the live-wrapper images, the netinst images or
perhaps even Docker images

With kind regards,
Roland Clobus

[1] https://wiki.debian.org/ReproducibleInstalls/LiveImages
[2] https://jenkins.debian.net/view/live/
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=991059
[4] https://reproducible-builds.org/docs/stable-outputs/
[5]
https://jenkins.debian.net/munin/debian.net/osuosl173-amd64.debian.net/index.html
[6] https://debian.notset.fr/snapshot
[7] https://tails.boum.org/contribute/release_process/test/automated_tests/
[8] https://openqa.debian.net/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20210725/cfd635f8/attachment.sig>