Reproducible Builds in February 2024

Chris Lamb chris at reproducible-builds.org
Tue Mar 12 12:30:58 UTC 2024 

--------------------------------------------------------------------
        o
      ⬋   ⬊      February 2024 in Reproducible Builds
     o     o
      ⬊   ⬋      https://reproducible-builds.org/reports/2024-02/
        o
--------------------------------------------------------------------

Welcome to the February 2024 report from the Reproducible Builds
project. In our reports, we try to outline what we have been up to over
the past month as well as mentioning some of the important things
happening in software supply-chain security.

                                    §


Reproducible Builds at FOSDEM 2024
----------------------------------

Core Reproducible Builds developer Holger Levsen presented at the main
track at FOSDEM [2] on Saturday 3rd February this year in Brussels,
Belgium. However, that wasn't the only talk related to
Reproducible Builds.

However, please see our comprehensive FOSDEM 2024 news post [3] for the
full details and links.

 [2] https://fosdem.org/2024/
 [3] https://reproducible-builds.org/news/2024/02/08/reproducible-builds-at-fosdem-2024/

                                    §


Maintainer Perspectives on Open Source Software Security
--------------------------------------------------------

Bernhard M. Wiedemann spotted that a recent report entitled "Maintainer
Perspectives on Open Source Software Security" [5] written by Stephen
Hendrick and Ashwin Ramaswami of the Linux Foundation [6] sports an
infographic which mentions that "56% of [polled] projects support
reproducible builds" [4].

 [4] https://www.linuxfoundation.org/hubfs/LF%20Research/MaintainerSecurityBPs_Infographic.pdf
 [5] https://www.linuxfoundation.org/research/maintainer-perspectives-on-security?hsLang=en
 [6] https://www.linuxfoundation.org/

                                    §


Three new reproducibility-related academic papers
-------------------------------------------------

A total of three separate scholarly papers related to Reproducible
Builds have appeared this month:

"Signing in Four Public Software Package Registries: Quantity, Quality,
and Influencing Factors" [7] by Taylor R. Schorlemmer, Kelechi G. Kalu,
Luke Chigges, Kyung Myung Ko, Eman Abdul-Muhd, Abu Ishgair, Saurabh
Bagchi, Santiago Torres-Arias and James C. Davis (Purdue University [8],
Indiana, USA) is concerned with the problem that:

> Package maintainers can guarantee package authorship through
> software signing [but] it is unclear how common this practice is,
> and whether the resulting signatures are created properly. Prior
> work has provided raw data on signing practices, but measured single
> platforms, did not consider time, and did not provide insight on
> factors that may influence signing. We lack a comprehensive,
> multi-platform understanding of signing adoption and relevant
> factors. This study addresses this gap.

(arXiv [9], full PDF [10])

 [ 7] https://arxiv.org/abs/2401.14635
 [ 8] https://www.purdue.edu/
 [ 9] https://arxiv.org/abs/2401.14635
 [10] https://arxiv.org/pdf/2401.14635.pdf

"Reproducibility of Build Environments through Space and Time" [11] by
Julien Malka, Stefano Zacchiroli and Théo Zimmermann (Institut
Polytechnique de Paris, France [12]) addresses:

> [The] principle of reusability […] makes it harder to reproduce
> projects’ build environments, even though reproducibility of build
> environments is essential for collaboration, maintenance and
> component lifetime. In this work, we argue that functional package
> managers provide the tooling to make build environments reproducible
> in space and time, and we produce a preliminary evaluation to
> justify this claim.

The abstract continues with the claim that "Using historical data, we
show that we are able to reproduce build environments of about 7
million Nix [13] packages, and to rebuild 99.94% of the 14 thousand
packages from a 6-year-old Nixpkgs revision. (arXiv [14], full PDF
[15])

 [11] https://arxiv.org/abs/2402.00424
 [12] https://www.ip-paris.fr/
 [13] https://nixos.org/
 [14] https://arxiv.org/abs/2402.00424
 [15] https://arxiv.org/pdf/2402.00424.pdf

"Options Matter: Documenting and Fixing Non-Reproducible Builds in
Highly-Configurable Systems" [16] by Georges Aaron Randrianaina, Djamel
Eddine Khelladi, Olivier Zendra and Mathieu Acher (Inria centre at
Rennes University, France [17]):

> This paper thus proposes an approach to automatically identify
> configuration options causing non-reproducibility of builds. It
> begins by building a set of builds in order to detect
> non-reproducible ones through binary comparison. We then develop
> automated techniques that combine statistical learning with
> symbolic reasoning to analyze over 20,000 configuration options.
> Our methods are designed to both detect options causing
> non-reproducibility, and remedy non-reproducible configurations,
> two tasks that are challenging and costly to perform manually.
> (HAL Portal [18], full PDF [19])

 [16] https://inria.hal.science/hal-04441579v2
 [17] https://www.inria.fr/en/inria-centre-rennes-university
 [18] https://inria.hal.science/hal-04441579v2
 [19] https://inria.hal.science/hal-04441579/file/msr24.pdf

                                    §

Mailing list highlights
-----------------------

>From our mailing list [20] this month:

* User "cen" posted a query asking "How to verify a package by
  rebuilding it locally on Debian [21]" which received a followup from
  Vagrant Cascadian [22].

* James Addison asked "Two questions about build-path reproducibility
  in Debian [23]" regarding the differences in the testing performed by
  Debian's GitLab continuous integration (CI) pipeline [24] and the
  Debian-specific testing performed by the Reproducible Builds project
  itself [25], and followed this with a separate but related question
  regarding misconfigured *reprotest* [26] configurations.

 [20] https://lists.reproducible-builds.org/listinfo/rb-general/
 [21] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003238.html
 [22] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003240.html
 [23] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003246.html
 [24] https://salsa.debian.org/salsa-ci-team/pipeline
 [25] https://tests.reproducible-builds.org/debian/reproducible.html
 [26] https://salsa.debian.org/reproducible-builds/reprotest

                                    §

Distribution work
-----------------

In Debian this month, 5 reviews of Debian packages were added, 22 were
updated and 8 were removed this month adding to Debian's knowledge about
identified issues [27]. A number of issue types were updated as well.

In addition, Roland Clobus posted his 23rd update of the status of
reproducible ISO images [28] on our mailing list. In particular,
Roland helpfully summarised that "all major desktops build
reproducibly with "bullseye", "bookworm", "trixie" and "sid" provided
they are built for a second time within the same DAK run (i.e.
[within] 6 hours)" and that there will likely be further work at a
MiniDebCamp in Hamburg [29]. Furthermore, Roland also responded in-
depth [30] to a query about a previous report [31].

 [27] https://tests.reproducible-builds.org/debian/index_issues.html
 [28] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003251.html
 [29] https://wiki.debian.org/DebianEvents/de/2024/MiniDebCampHamburg
 [30] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003233.html
 [31] https://lists.reproducible-builds.org/pipermail/rb-general/2024-January/003217.html

Fedora [32] developer Zbigniew Jędrzejewski-Szmek [33] announced a work-
in-progress script called fedora-repro-build [34] that attempts to
reproduce an existing package within a koji [35] build environment.
Although the projects' README file [36] lists a number of "fields will
always or almost always vary" and there is a non-zero list of other
known issues [37], this is an excellent first step towards full
Fedora reproducibility.

 [32] https://fedoraproject.org/
 [33] https://github.com/keszybz
 [34] https://github.com/keszybz/fedora-repro-build
 [35] https://pagure.io/koji/
 [36] https://github.com/keszybz/fedora-repro-build#readme
 [37] https://pagure.io/fedora-reproducible-builds/project/issues?tags=irreproducibility

Jelle van der Waa introduced a new linter rule [38] for Arch Linux [39]
packages in order to detect cache files leftover by the Sphinx
documentation generator [40] which are unreproducible by nature and
should not be packaged. At the time of writing, 7 packages in the Arch
repository are affected by this.

 [38] https://gitlab.archlinux.org/pacman/namcap/-/merge_requests/64
 [39] https://archlinux.org/
 [40] https://www.sphinx-doc.org/en/master/

Elsewhere, Bernhard M. Wiedemann posted another monthly update [41] for
his work elsewhere in openSUSE.

 [41] https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/I66U56F5R3TR4ZTLYGPSGWINNOLZ7XP4/

                                    §

diffoscope
----------

diffoscope [43] is our in-depth and content-aware diff utility that can
locate and diagnose reproducibility issues. This month, Chris Lamb made
a number of changes such as uploading versions 256, 257 and 258 to
Debian and made the following additional changes:

* Use a deterministic name instead of trusting gpg's --use-embedded-
  filenames. Many thanks to Daniel Kahn Gillmor (dkg) for
  reporting this issue and providing feedback. [44][45]
* Don't error-out with a traceback if we encounter struct.unpack-
  related errors when parsing Python .pyc files. (#1064973). [47]
* Don't try and compare rdb_expected_diff on non-GNU systems as %p
  formatting can vary, especially with respect to MacOS. [48]
* Fix compatibility with pytest [49] 8.0. [50]
* Temporarily fix support for Python 3.11.8. [51]
* Use the 7zip package (over p7zip-full) after a Debian package
  transition. (#1063559 [52]). [53]
* Bump the minimum Black source code reformatter [54] requirement to
  24.1.1+. [55]
* Expand an older changelog entry with a CVE reference. [56]
* Make test_zip black clean. [57]

 [43] https://diffoscope.org
 [44] https://salsa.debian.org/reproducible-builds/diffoscope/commit/458f7f04
 [45] https://salsa.debian.org/reproducible-builds/diffoscope/commit/18d69030
 [47] https://salsa.debian.org/reproducible-builds/diffoscope/commit/466523ac
 [48] https://salsa.debian.org/reproducible-builds/diffoscope/commit/c09d0a9e
 [49] https://docs.pytest.org/en/8.0.x/
 [50] https://salsa.debian.org/reproducible-builds/diffoscope/commit/ce04e0dd
 [51] https://salsa.debian.org/reproducible-builds/diffoscope/commit/5e6cfbf0
 [52] https://bugs.debian.org/1063559
 [53] https://salsa.debian.org/reproducible-builds/diffoscope/commit/43ee3684
 [54] https://black.readthedocs.io/en/stable/
 [55] https://salsa.debian.org/reproducible-builds/diffoscope/commit/00418fb4
 [56] https://salsa.debian.org/reproducible-builds/diffoscope/commit/86645633
 [57] https://salsa.debian.org/reproducible-builds/diffoscope/commit/10c0c6fc

In addition, James Addison contributed a patch to parse the headers from
the diff(1) correctly [58][59] — thanks! And lastly, Vagrant Cascadian
pushed updates in GNU Guix [60] for diffoscope to version 255 [61], 256
[62], and 258 [63], and updated trydiffoscope to 67.0.6 [64].

 [58] https://salsa.debian.org/reproducible-builds/diffoscope/commit/4648dcfa
 [59] https://salsa.debian.org/reproducible-builds/diffoscope/commit/fa73fc2b
 [60] https://guix.gnu.org/
 [61] https://git.savannah.gnu.org/cgit/guix.git/commit/?id=9d52585ebd4d759607eacfef31144676b08edc81
 [62] https://git.savannah.gnu.org/cgit/guix.git/commit/?id=30196aec07dab8cc0f4a614b160f1857377a6a84
 [63] https://git.savannah.gnu.org/cgit/guix.git/commit/?id=16ab67182bc1e5b046caee9a2e38b71159703f34
 [64] https://git.savannah.gnu.org/cgit/guix.git/commit/?id=f45d05133472a9da13eae20ba4a676c696682c90

                                    §

reprotest
---------

reprotest [66] is our tool for building the same source code twice in
different environments and then checking the binaries produced by each
build for any differences. This month, Vagrant Cascadian made a number
of changes, including:

* Create a (working) proof of concept for enabling a specific number of
  CPUs. [67][68]
* Consistently use 398 days for time variation rather than choosing
  randomly and update README.rst to match. [69][70]
* Support a new --vary=build_path.path option. [71][72][73][74]

 [66] https://salsa.debian.org/reproducible-builds/reprotest
 [67] https://salsa.debian.org/reproducible-builds/reprotest/commit/cab6270
 [68] https://salsa.debian.org/reproducible-builds/reprotest/commit/9d0562d
 [69] https://salsa.debian.org/reproducible-builds/reprotest/commit/86365b5
 [70] https://salsa.debian.org/reproducible-builds/reprotest/commit/57ab249
 [71] https://salsa.debian.org/reproducible-builds/reprotest/commit/f94904b
 [72] https://salsa.debian.org/reproducible-builds/reprotest/commit/9ea2e4b
 [73] https://salsa.debian.org/reproducible-builds/reprotest/commit/9b0f5dc
 [74] https://salsa.debian.org/reproducible-builds/reprotest/commit/94e66c4

                                    §

Website updates
---------------

There were made a number of improvements to our website this
month, including:

* Chris Lamb:

    * Improve the relative sizing of headers. [75]
    * Re-order and "punch" up the introduction and documentation on the
      SOURCE_DATE_EPOCH [76] page. [77]
    * Update SOURCE_DATE_EPOCH [78] documentation re.
      datetime.datetime.fromtimestamp. Thanks, James Addison. [79]
    * Add a post about Reproducible Builds at FOSDEM 2024 [80]. [81]

* Holger Levsen:

    * Update the GNU Guix [82] page to include their reproducibility QA
      page [83]. [84]
    * Add Sune Vuorela and Jan-Benedict Glaw to our contributors
      list. [85][86]

* Mattia Rizzolo:

    * Add Sovereign Tech Fund [87]'s logo to our sponsors. [88]
    * Update our sponsors list. [89]

 [75] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/3243e14b
 [76] https://reproducible-builds.org/docs/source-date-epoch/
 [77] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/05a76405
 [78] https://reproducible-builds.org/docs/source-date-epoch/
 [79] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/502769f1
 [80] https://reproducible-builds.org/news/2024/02/08/reproducible-builds-at-fosdem-2024/
 [81] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/b09d3c22
 [82] https://reproducible-builds.org/projects/guix
 [83] https://qa.guix.gnu.org/reproducible-builds
 [84] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/d33582dc
 [85] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/3bed935a
 [86] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/8bf556b5
 [87] https://www.sovereigntechfund.de/
 [88] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a54f6e20
 [89] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/de187090

                                    §

Reproducibility testing framework
---------------------------------

The Reproducible Builds project operates a comprehensive testing
framework (available at "tests.reproducible-builds.org" [90]) in order
to check packages and other artifacts for reproducibility. In February,
a number of changes were made by Holger Levsen:

* Debian [91]-related changes:

    * Temporarily disable upgrading/bootstrapping Debian "unstable" and
      "experimental" as they are currently broken. [92][93]
    * Use the 64-bit amd64 kernel on all i386 nodes; no more 686 PAE
      [94] kernels. [95]
    * Add an Erlang [96] package set. [97]

* Other changes:

    * Grant Jan-Benedict Glaw shell access to the Jenkins node. [98]
    * Enable debugging for NetBSD [99] reproducibility testing. [100]
    * Use "/usr/bin/du --apparent-size" in the Jenkins shell
      monitor. [101]
    * Thanks again to Codethink [103], for they have doubled the RAM on
      our arm64 nodes. [104]
    * Only set "/proc/$pid/oom_score_adj" to -1000 if it has not already
      been done. [105]
    * Add the "opemwrt-target-tegra" and "jtx" task to the list of zombie
      jobs. [106][107]

 [90] https://tests.reproducible-builds.org
 [91] https://debian.org/
 [92] https://salsa.debian.org/qa/jenkins.debian.net/commit/ef88cc3ae
 [93] https://salsa.debian.org/qa/jenkins.debian.net/commit/7ed553444
 [94] https://en.wikipedia.org/wiki/Physical_Address_Extension
 [95] https://salsa.debian.org/qa/jenkins.debian.net/commit/53c3c39bd
 [96] https://www.erlang.org/
 [97] https://salsa.debian.org/qa/jenkins.debian.net/commit/d29d41e3b
 [98] https://salsa.debian.org/qa/jenkins.debian.net/commit/252598e99
 [99] https://www.netbsd.org/
 [100] https://salsa.debian.org/qa/jenkins.debian.net/commit/091fa73f1
 [101] https://salsa.debian.org/qa/jenkins.debian.net/commit/fd54c037d
 [103] https://www.codethink.co.uk/
 [104] https://salsa.debian.org/qa/jenkins.debian.net/commit/640c38126
 [105] https://salsa.debian.org/qa/jenkins.debian.net/commit/c99da2ef3
 [106] https://salsa.debian.org/qa/jenkins.debian.net/commit/e3b188dff
 [107] https://salsa.debian.org/qa/jenkins.debian.net/commit/7fbed0735

Vagrant Cascadian also made the following changes:

* Overhaul the handling of OpenSSH [108] configuration files after
  updating from Debian *bookworm*. [109][110][111]
* Add two new armhf architecture build nodes, virt32z and virt64z, and
  insert them into the Munin monitoring [112]. [113][114] [115][116]

In addition, Alexander Couzens updated the OpenWrt [117] configuration
in order to replace the tegra target with mpc85xx [118], Jan-Benedict
Glaw updated the NetBSD [119] build script to use a separate $TMPDIR to
mitigate out of space issues on a tmpfs [120]-backed /tmp [121] and
Zheng Junjie added a link to the GNU Guix [122] tests [123].

 [108] https://www.openssh.com/
 [109] https://salsa.debian.org/qa/jenkins.debian.net/commit/3e58ee08c
 [110] https://salsa.debian.org/qa/jenkins.debian.net/commit/7d8a99cb5
 [111] https://salsa.debian.org/qa/jenkins.debian.net/commit/5484a9db0
 [112] https://munin-monitoring.org/
 [113] https://salsa.debian.org/qa/jenkins.debian.net/commit/8700924ae
 [114] https://salsa.debian.org/qa/jenkins.debian.net/commit/2c462cc3c
 [115] https://salsa.debian.org/qa/jenkins.debian.net/commit/7feece465
 [116] https://salsa.debian.org/qa/jenkins.debian.net/commit/6159ad4f9
 [117] https://openwrt.org/
 [118] https://salsa.debian.org/qa/jenkins.debian.net/commit/b5b63be56
 [119] https://www.netbsd.org/
 [120] https://en.wikipedia.org/wiki/Tmpfs
 [121] https://salsa.debian.org/qa/jenkins.debian.net/commit/910b83f88
 [122] https://guix.gnu.org/
 [123] https://salsa.debian.org/qa/jenkins.debian.net/commit/57b21155e

Lastly, node maintenance was performed by Holger Levsen
[124][125][126][127][128][129] and Vagrant Cascadian [130][131][132][133].

 [124] https://salsa.debian.org/qa/jenkins.debian.net/commit/01ecc9495
 [125] https://salsa.debian.org/qa/jenkins.debian.net/commit/2f650ed98
 [126] https://salsa.debian.org/qa/jenkins.debian.net/commit/20e9e5c64
 [127] https://salsa.debian.org/qa/jenkins.debian.net/commit/9ce43116c
 [128] https://salsa.debian.org/qa/jenkins.debian.net/commit/9a37e768d
 [129] https://salsa.debian.org/qa/jenkins.debian.net/commit/b7417a2f8
 [130] https://salsa.debian.org/qa/jenkins.debian.net/commit/a2315e19f
 [131] https://salsa.debian.org/qa/jenkins.debian.net/commit/aa7579a92
 [132] https://salsa.debian.org/qa/jenkins.debian.net/commit/c78087b27
 [133] https://salsa.debian.org/qa/jenkins.debian.net/commit/5b9d95648

                                    §

Upstream patches
----------------

The Reproducible Builds project detects, dissects and attempts to fix as
many currently-unreproducible packages as possible. We endeavour to send
all of our patches upstream where appropriate. This month, we wrote a
large number of such patches, including:

* Philip Rinn:

    * gimagereader [134] (date)

* Bernhard M. Wiedemann:

    * grass [135] (date-related issue)
    * grub2 [136] (filesystem ordering issue)
    * latex2html [137] (drop a non-deterministic log)
    * mhvtl [138] (tar)
    * obs [139] (build-tool issue)
    * ollama [140] (GZip embedding the modification time)
    * presenterm [141] (filesystem-ordering issue)
    * qt6-quick3d [142] (parallelism)

* Chris Lamb:

    * #1064506 [143] filed against geophar [144].
    * #1064891 [145] filed against pytest-repeat [146].
    * #1064892 [147] filed against klepto [148].

* James Addison:

    * #1064519 [149] filed against flask-limiter [150].
    * python-parsl-doc [151] (disable dynamic argument evaluation by
      Sphinx autodoc extension)
    * python3-pytest-repeat [152] (remove entry_points.txt creation
      that varied by shell)
    * python3-selinux [153] (remove packaged direct_url.json file that
      embeds build path)
    * python3-sepolicy [154] (remove packaged direct_url.json file that
      embeds build path)
    * #1064575 [155] filed against pyswarms [156].
    * #1064638 [157] filed against python-x2go [158].
    * snapd [159] (fix timestamp header in packaged manual-page)
    * zzzeeksphinx [160] (existing RB patch forwarded and merged
      (with modifications))

* Johannes Schauer Marin Rodrigues:

    * #1063939 [161] filed against fop [162].

 [134] https://github.com/manisandro/gImageReader/pull/667
 [135] https://github.com/OSGeo/grass/pull/3417
 [136] https://build.opensuse.org/request/show/1144993
 [137] https://build.opensuse.org/request/show/1150775
 [138] https://github.com/markh794/mhvtl/pull/128
 [139] https://github.com/openSUSE/obs-build/issues/980
 [140] https://github.com/ollama/ollama/pull/2836
 [141] https://github.com/mfontanini/presenterm/pull/202
 [142] https://bugreports.qt.io/browse/QTBUG-122722
 [143] https://bugs.debian.org/1064506
 [144] https://tracker.debian.org/pkg/geophar
 [145] https://bugs.debian.org/1064891
 [146] https://tracker.debian.org/pkg/pytest-repeat
 [147] https://bugs.debian.org/1064892
 [148] https://tracker.debian.org/pkg/klepto
 [149] https://bugs.debian.org/1064519
 [150] https://tracker.debian.org/pkg/flask-limiter
 [151] https://bugs.debian.org/1063542
 [152] https://bugs.debian.org/1064891
 [153] https://bugs.debian.org/1064894
 [154] https://bugs.debian.org/1064895
 [155] https://bugs.debian.org/1064575
 [156] https://tracker.debian.org/pkg/pyswarms
 [157] https://bugs.debian.org/1064638
 [158] https://tracker.debian.org/pkg/python-x2go
 [159] https://bugs.debian.org/1064404
 [160] https://bugs.debian.org/1042955
 [161] https://bugs.debian.org/1063939
 [162] https://tracker.debian.org/pkg/fop

… and finally
-------------

If you are interested in contributing to the Reproducible Builds
project, please visit our "Contribute" [163] page on our website.
However, you can get in touch with us via:

 * IRC: #reproducible-builds on irc.oftc.net.

 * Twitter: @ReproBuilds [164]

 * Mastodon: @reproducible_builds at fosstodon.org [165]

 * Mailing list: rb-general at lists.reproducible-builds.org [166]

 [163] https://reproducible-builds.org/contribute/
 [164] https://twitter.com/ReproBuilds
 [165] https://fosstodon.org/@reproducible_builds
 [166] https://lists.reproducible-builds.org/listinfo/rb-general



-- 
      o
    ⬋   ⬊      Chris Lamb
   o     o     reproducible-builds.org 💠
    ⬊   ⬋
      o

More information about the rb-general mailing list