Tenth status update about reproducible live-build ISO images in Jenkins

[Roland Clobus]

Hello lists,

here is the tenth update of the status for reproducible live-build ISO 
images [1].

Reproducible status:
* All major desktops build reproducibly with bullseye, bookworm and sid ...
** ... including Cinnamon on bookworm and sid, but at a small 
functionality cost [9][PS1]
* Number of patches in the live-build script that are not yet in sid:
** non-Cinnamon: 0 Cinnamon: 3 [7][8][10]

Functionality tests with openQA:
* A category for live images is present [16]
* Several images have been tested with the 'walk-boot' test [4]
** Issue discovered: kernel option 'nomodeset' hangs in BIOS mode too in 
qemu [17] (fixed for UEFI in [5])
* The initrd for the Debian installer is regenerated, to make it 
reproducible [18]
* Work in progress:
** Testing the installer (both debian-installer and calamares)
** Booting with secure boot
** Automatic starting of the openQA tests, e.g. by Jenkins or the image 
builder script

New and changed:
* Closed: libxmlb2: Used a pointer address (%p) for a hash value [6] was 
uploaded as 0.3.8-1 (2022-04-10)
* Presentation of the status at the Debian Reunion in Hamburg [13][14]
* Many discussions and a brainstorm session during the Debian Reunion in 
Hamburg
** Outcome: many ideas that still need to be organised
* Video chat about re-building the weekly sid/bookworm live images
** Outcome: can be done. I'll need to prepare a patch
* The Cinnamon issues should disappear when #1006472 will be uploaded [15]
** The fixes for the texlive packages will get a lower priority

Work to be done:
* Jenkins does currently not create ISO files that are tested by openQA
* OpenQA does not have sufficient tests
** Everyone can wrote tests for openQA, primary contact: Philip Hands
* Live images are not generated officially by Debian
** Needs some changes in 'live-setup'
* Reproducible live images can only be generated with the help of a 
snapshot server
* The Debian snapshot server at snapshot.debian.org cannot handle high 
network traffic
** Who can help with this?
* The snapshot server at snapshot.notset.fr should be replaced by 
snapshot.reproducible-builds.org
** The files from s.notset.fr are already copied to s.r-b.o
** The REST-API needs to be installed on s.r-b.o
*** h01ger pointed out that mapreri has created a puppet configuration 
for mail.r-b.o
*** Could this be an appointed volunteer? :-)

Options for getting 'the best' live configuration:
* Reconfigure the default settings for live-build
** Needs investigation of differences between current live-wrapper 
images and live-build images
* Re-activate live-wrapper
** Needs porting from Python2 to Python3 and re-introduction in Debian
* Use alternative tool: kiwi-ng [19]
** Is used by openSUSE, can create Ubuntu images
** Uses dracut, not initrd
* Use python3-dmm [20]
** New, under heavy development
* Use FAI [21]
** Uses dracut, not initrd
* All options need development time
* For now: continuing live-build seems the best option (there are also 
many users of live-build)

Unchanged: Patch available but not released yet:
* texlive-base: Reported differences in the generated ls-R [7]
* texlive-binaries: Randomness in .fmt files due to Lua hash seeds [8]
* texlive-binaries: updmap creates a logfile with the timestamps of 
files that it just has generated [10]

Future plans/ideas:
* Reprotest might be used instead of just 2 builds without a short time 
frame, to capture more variations
* Use disorderfs
* Long term: When live-build images are working fine, the work could be 
extended to other images, e.g. the netinst images or perhaps even Docker 
images
* Transfer the special features of the (now disabled) live-wrapper live 
images to live-build
* Start building official live-images again [11]

With kind regards,
Roland Clobus

[1] https://wiki.debian.org/ReproducibleInstalls/LiveImages
[4] 
https://salsa.debian.org/qa/openqa/openqa-tests-debian/-/merge_requests/2
[5] https://salsa.debian.org/live-team/live-build/-/merge_requests/278
[6] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1006358
[7] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1003449
[8] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1009196
[9] 
https://salsa.debian.org/live-team/live-build/-/commit/f1a98e4da62c3551f523553c6e23774aaf5e41b4
[10] Unreported, patch is in [9]
[11] https://lists.debian.org/debian-live/2022/03/msg00012.html
[13] https://wiki.debian.org/DebianEvents/de/2022/DebianReunionHamburg
[14] 
https://meetings-archive.debian.net/pub/debian-meetings/2022/Debian-Reunion-Hamburg/
[15] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1006472
[16] https://openqa.debian.net/group_overview/14
[17] https://salsa.debian.org/live-team/live-build/-/merge_requests/282
[18] https://salsa.debian.org/live-team/live-build/-/merge_requests/281
[19] https://tracker.debian.org/pkg/kiwi
[20] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1011631
[21] https://wiki.fai-project.org/index.php/Use_nfsroot_for_diskless_clients

[PS1] 14 words will be incorrectly abbreviated
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20220628/a509e9a9/attachment.sig>