Reproducible Builds in August 2024
Chris Lamb
chris at reproducible-builds.org
Wed Sep 4 13:41:47 UTC 2024
--------------------------------------------------------------------
o
⬋ ⬊ August 2024 in Reproducible Builds
o o
⬊ ⬋ https://reproducible-builds.org/reports/2024-08/
o
--------------------------------------------------------------------
Welcome to the August 2024 report from the Reproducible Builds project!
In our reports, we outline what we've been up to over the past month
and highlight news items in software supply-chain security more
broadly. As always, if you are interested in contributing to the
project, please visit our Contribute [1] page on our website.
[0] https://reproducible-builds.org
[1] https://reproducible-builds.org/contribute/
§
Table of contents:
* LWN: The history, status, and plans for reproducible builds
* Intermediate Autotools build artifacts removed from PostgreSQL distribution tarballs
* Distribution news
* Mailing list news
* "diffoscope"
* Website updates
* Upstream patches
* Reproducibility testing framework
§
"LWN: The history, status, and plans for reproducible builds"
-------------------------------------------------------------
The free software newspaper of record, Linux Weekly News [3],
published an in-depth article based on Holger Levsen's talk,
"Reproducible Builds: The First Eleven Years" [4] which was presented at
the recent DebConf24 [5] conference in Busan, South Korea.
Titled "The history, status, and plans for reproducible builds" [6] and
written by Jake Edge, LWN's article not only summarises Holger's talk
and clarifies its message but it links to external information as well.
Holger's original talk can also be watched on the DebConf24 webpage [7]
(direct .webm link [8] and his HTML slides [9] are available as well
There are also a significant number of comments on LWN's page, too.
Holger Levsen also headed a scheduled discussion session at DebConf24 on
"Preserving *other* build artifacts" [10] addressing a topic where a
number of Debian packages are (or would like to) produce results that
are neither the .deb files, the build logs nor the logs of CI tests
[11]. This is an issue for reproducible builds as this "4th type" of
build artifact are typically shipped within the binary .deb packages,
and are invariably non-deterministic; thus making the .deb files
unreproducible. (A direct .webm link [12] and HTML slides [13]
are available).
[ 2] https://lwn.net/SubscriberLink/985739/19ce503ee4e83da9/
[ 3] https://lwn.net/
[ 4] https://debconf24.debconf.org/talks/18-reproducible-builds-the-first-eleven-years/
[ 5] https://debconf24.debconf.org/
[ 6] https://lwn.net/Articles/985739/
[ 7] https://debconf24.debconf.org/talks/18-reproducible-builds-the-first-eleven-years/
[ 8] https://meetings-archive.debian.net/pub/debian-meetings/2024/DebConf24/debconf24-103-reproducible-builds-the-first-eleven-years.av1.webm
[ 9] https://reproducible-builds.org/_lfs/presentations/2024-08-02-preserving-other-build-artifacts/
[10] https://debconf24.debconf.org/talks/17-preserving-other-build-artifacts/
[11] https://wiki.debian.org/BuildArtifacts
[12] https://meetings-archive.debian.net/pub/debian-meetings/2024/DebConf24/debconf24-304-preserving-other-build-artifacts.av1.webm
[13] https://reproducible-builds.org/_lfs/presentations/2024-08-02-preserving-other-build-artifacts/
§
Intermediate Autotools build artifacts removed from PostgreSQL distribution tarballs
------------------------------------------------------------------------------------
Peter Eisentraut [14] wrote a detailed blog post on the subject of "The
new PostgreSQL 17 make dist" [15]. Like many projects, the PostgreSQL
[16] database has previously pre-built parts of its GNU Autotools [17]
build system: "the reason for this is a mix of convenience and
traditional practice [18]". Peter astutely notes that this arrangement
in the build system is "quite tricky" as:
> You need to carefully maintain the different states of “clean source
> code”, “partially built source code”, and “fully built source code”,
> and the commands to transition between them.
However, Peter goes on to mention that:
> … a lot more attention is nowadays paid to the software supply chain.
> There are security and legal reasons for this. When users install
> software, they want to know where it came from, and they want to be sure
> that they got the right thing, not some fake version or some version of
> dubious legal provenance.
And cites the XZ Utils backdoor [19] as a reason to care about
transparent and reproducible ways of distributing and communicating a
source tarball and provenance. Because of this, intermediate build
artifacts are now henceforth essentially disallowed from PostgreSQL
distribution tarballs.
[14] https://peter.eisentraut.org/
[15] https://peter.eisentraut.org/blog/2024/08/13/the-new-postgresql-17-make-dist
[16] https://www.postgresql.org/
[17] https://en.wikipedia.org/wiki/GNU_Autotools
[18] https://www.gnu.org/prep/standards/html_node/Releases.html
[19] https://en.wikipedia.org/wiki/XZ_Utils_backdoor
§
Distribution news
-----------------
In Debian this month, 30 reviews of Debian packages were added, 17 were
updated and 10 were removed this month adding to our knowledge about
identified issues [20]. One issue type was added by Chris Lamb,
too. [21]
In addition, an issue was filed [22] to update the Salsa CI pipeline
[23] (used by 1,000s of Debian packages) to no longer test for
reproducibility with reprotest's build_path variation. Holger Levsen
provided a rationale [24] for this change in the issue, which has
already been made to the tests being performed by tests.reproducible-
builds.org [25].
[20] https://tests.reproducible-builds.org/debian/index_issues.html
[21] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/40348f23
[22] https://salsa.debian.org/salsa-ci-team/pipeline/-/issues/368
[23] https://salsa.debian.org/salsa-ci-team/pipeline
[24] https://salsa.debian.org/salsa-ci-team/pipeline/-/issues/368#note_520933
[25] https://tests.reproducible-builds.org
In Arch Linux [26] this month, Jelle van der Waa published a short blog
post on the topic of "Investigating creating reproducible images with
mkosi" [27], motivated by the desire to make it possible for anyone to
"re-recreate the official Arch cloud image bit-by-bit identical on their
own machine as per [the] reproducible builds definition." In addition,
Jelle filed a patch for pacman [28], the Arch Linux package manager,
to respect the SOURCE_DATE_EPOCH environment variable [29] when
installing a package.
[26] https://archlinux.org/
[27] https://vdwaa.nl/mkosi-reproducible-images.html#mkosi-reproducible-images
[28] https://gitlab.archlinux.org/pacman/pacman
[29] https://gitlab.archlinux.org/pacman/pacman/-/merge_requests/213
In openSUSE news, Bernhard M. Wiedemann published another report [30]
for that distribution.
[30] https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/2XMBAI6G4ZFAS7LFWM2XPYHZYMG5C2GF/
In Android news, the IzzyOnDroid [31] project added 49 new rebuilder
recipes and now features 256 total reproducible applications
representing 21% of the total offerings in the repository. IzzyOnDroid
is "an F-Droid style repository for Android apps[:] applications in this
repository are official binaries built by the original application
developers, taken from their resp. repositories (mostly GitHub)."
[31] https://apt.izzysoft.de/fdroid/
§
Mailing list news
-----------------
>From our mailing list [32] this month:
* Bernhard M. Wiedemann posted a brief message to the list with some
helpful information regarding nondeterminism within Rust [33]
binaries, positing the use of the codegen-units = 16 default and
resulting in a bug being filed in the Rust issue tracker [34]. [35]
[32] https://lists.reproducible-builds.org/listinfo/rb-general/
[33] https://www.rust-lang.org/
[34] https://github.com/rust-lang/rust/issues/128675
[35] https://lists.reproducible-builds.org/pipermail/rb-general/2024-August/003488.html
* Bernhard also wrote to the list, following up to a thread in November
2023 [36], on attempts to make the LibreOffice [37] suite of office
applications build reproducibly. In the thread from this month [38],
Bernhard could announce that the four patches previously mentioned
have landed in LibreOffice upstream.
[36] https://lists.reproducible-builds.org/pipermail/rb-general/2023-November/thread.html#3121
[37] https://www.libreoffice.org/
[38] https://lists.reproducible-builds.org/pipermail/rb-general/2024-August/003489.html
* Fay Stegerman linked the mailing list to a thread she made on the
Signal issue tracker regarding whether "[device-specific binaries
[can] ever be considered meaningfully
reproducible". In particular: "the whole part
about 'allow[ing] multiple third parties to come to a consensus on a
“correct” result' breaks down completely when 'correct' is device-
specific and not something everyone can agree on." [39]
[39] https://lists.reproducible-builds.org/pipermail/rb-general/2024-August/003495.html
* Developer kpcyrd posted an update [40] for source code indexing
project, whatsrc.org [41]. Announcing that it now importing
packages from live-bootstrap [42] ("a usable Linux system [that is]
created with only human-auditable, and wherever possible, human-
written, source code") into its database of provenance data.
[40] https://lists.reproducible-builds.org/pipermail/rb-general/2024-August/003517.html
[41] https://whatsrc.org/
[42] https://github.com/fosslinux/live-bootstrap
* Lastly, Mechtilde Stehmann posted an update to an earlier thread
about how Java builds are not reproducible on the armhf architecture,
enquiring how they might gain temporary access to such a machine in
order to perform some deeper testing. [43]
[43] https://lists.reproducible-builds.org/pipermail/rb-general/2024-August/003511.html
§
diffoscope
----------
diffoscope [45] is our in-depth and content-aware diff utility that can
locate and diagnose reproducibility issues. This month, Chris Lamb
released versions 274, 275, 276 and 277, uploaded these to Debian, and
made the following changes as well:
[45] https://diffoscope.org
* New features:
* Strip ANSI escapes—usually colour codes—from the output of the
Procyon [46] Java decompiler. [47]
* Factor out a method for stripping ANSI escapes. [48]
* Append output from dumppdf(1) in more cases, avoiding situations
where we fallback to a binary diff. [49]
* Add support for versions of Perl's IO::Compress::Zip [50] version
2.212. [51]
[46] https://github.com/mstrobel/procyon
[47] https://salsa.debian.org/reproducible-builds/diffoscope/commit/12e34398
[48] https://salsa.debian.org/reproducible-builds/diffoscope/commit/d16faf7c
[49] https://salsa.debian.org/reproducible-builds/diffoscope/commit/956114dd
[50] https://metacpan.org/pod/IO::Compress::Zip
[51] https://salsa.debian.org/reproducible-builds/diffoscope/commit/c195a4e1
* Bug fixes:
* Also catch RuntimeError exceptions when importing the PyPDF [52]
library so that it, or, crucially, its transitive dependencies,
cannot not cause diffoscope to traceback at runtime *and* build
time. [53]
* Do not call marshal.load(…) of precompiled Python bytecode as it,
alas, inherently unsafe. Replace for now with a brief summary of
the code section of .pyc. [54][55]
* Don't include excessive debug output when calling
dumppdf(1). [56]
[52] https://pypdf.readthedocs.io/en/stable/
[53] https://salsa.debian.org/reproducible-builds/diffoscope/commit/c1aa6259
[54] https://salsa.debian.org/reproducible-builds/diffoscope/commit/c9c69fcd
[55] https://salsa.debian.org/reproducible-builds/diffoscope/commit/e75871b0
[56] https://salsa.debian.org/reproducible-builds/diffoscope/commit/9382ba74
* Testsuite-related changes:
* Don't bother to check version number in test_python.py: the
fixture for this test is fixed. [57][58]
* Update test_zip text fixtures and definitions to support new
changes to the Perl IO::Compress [59] library. [60]
[57] https://salsa.debian.org/reproducible-builds/diffoscope/commit/8052ceb5
[58] https://salsa.debian.org/reproducible-builds/diffoscope/commit/288c65c1
[59] https://metacpan.org/dist/IO-Compress
[60] https://salsa.debian.org/reproducible-builds/diffoscope/commit/e6ef1100
In addition, Mattia Rizzolo updated the available architectures for a
number of test dependencies [61] and Sergei Trofimovich fixed an issue
to avoid diffoscope crashing when hashing directory symlinks [62] and
Vagrant Cascadian proposed GNU Guix updates for diffoscope versions 275
and 276 [63] and 277 [64].
[61] https://salsa.debian.org/reproducible-builds/diffoscope/commit/22bfca88
[62] https://salsa.debian.org/reproducible-builds/diffoscope/commit/f545e70a
[63] https://issues.guix.gnu.org/72679
[64] https://issues.guix.gnu.org/72894
§
Website updates
---------------
There were a rather substantial number of improvements made to our
website this month, including:
* Alba Herrerias:
* Substantially extend the guidance on the "Contribute" [65]
page. [66]
[65] https://reproducible-builds.org/contribute/
[66] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/7e1ebc6c
* Chris Lamb:
* Set the future: true configuration value so we render *all* files
and documents in the website, regardless of whether they have a
date property in the future. After all, we don't re-generate the
website on a timer, and have other ways of making unpublished,
draft posts. [67][68]
[67] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/c852eafe
[68] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a6a695c9
* Fay Stegerman:
* Add IzzyOnDroid [69] (IoD) to the *Projects* [70] page. [71]
[69] https://apt.izzysoft.de/fdroid/
[70] https://reproducible-builds.org/who/projects/
[71] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a939f2ec
* *hulkoba*:
* Considerably overhaul the "History" [72] page in the
documentation, linking strip-nondeterminism and
SOURCE_DATE_EPOCH [73], fixing the test statistics link [74],
adjusting the Google Summer of Code application link [75], a
link to a Debian bug [76], and removed a dead link to the
debhelper utility [77].
* Use the jekyll-sitemap plugin to create a sitemap for the
website. [78]
* Use raw HTML to avoid a literal { .lead } directive appearing in
the page. [79]
* Fix a number of issues on the "Virtual machine drivers" [80]
page, such as keeping the Gitian [81] info, linking (and then
removing) an issue on the Bitcoin issue tracker [82] [83] and
fixing a link to the Bazel [84] website [85].
* Address a broken footnote link on the "Timestamps" [86]
page. [87]
* Unify the style on the "Commandments of Reproducible Builds" [88]
page in order to match other documentation entries. [89]
* Add a table of contents to the main "Documentation" [90]
page. [91]
* Avoid a number of so-called "here" links on the "Variations in
the build environment" [92] page. [93]
* Fix a link to the man2html patch on the SOURCE_DATE_EPOCH
documentation [94] page. [95]
* Fix a link to sources.debian.org [96] on the "Randomness" [97]
page. [98]
[72] https://reproducible-builds.org/docs/history/
[73] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/443a5514
[74] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/17329510
[75] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/b8e560fc
[76] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/f0b529c3
[77] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/8b4afd53
[78] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/2e9dc869
[79] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/e06876b3
[80] https://reproducible-builds.org/docs/virtual-machine-drivers/
[81] https://gitian.org/
[82] https://github.com/bitcoin/bitcoin/issues/21145
[83] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/159a16ed
[84] https://bazel.build
[85] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/4575a1fa
[86] https://reproducible-builds.org/docs/timestamps/
[87] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/eb9e7c0e
[88] https://reproducible-builds.org/docs/commandments/
[89] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/6f78b6d0
[90] https://reproducible-builds.org/docs/
[91] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/2550843a
[92] https://reproducible-builds.org/docs/env-variations/
[93] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/3cbd4902
[94] https://reproducible-builds.org/docs/source-date-epoch/
[95] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/0beb3595
[96] https://sources.debian.org/
[97] https://reproducible-builds.org/docs/randomness/
[98] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/9880b6a8
* kpcyrd:
* Fix a typo on the "Variations in the build environment" [99]
page. [100]
[99] https://reproducible-builds.org/docs/env-variations/
[100] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/e16062bf
* Mattia Rizzolo:
* Add all the sponsors to the Hamburg 2024 summit [101]
page. [102][103][104][105][106]
* Fix a link in the summit sponsor prospectus PDF. [107]
[101] https://reproducible-builds.org/events/hamburg2024/
[102] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/12222b23
[103] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/47caf995
[104] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/012a89b3
[105] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/b8f4bc62
[106] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/8390a494
[107] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/6a479e94
* Pol Dellaiera:
* Fix the DoI [108] for their thesis on the "Publications" [109]
page. [110]
[108] https://en.wikipedia.org/wiki/Digital_object_identifier
[109] https://reproducible-builds.org/docs/publications/
[110] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a7b2a581
§
Upstream patches
----------------
The Reproducible Builds project detects, dissects and attempts to fix as
many currently-unreproducible packages as possible. We endeavour to send
all of our patches upstream where appropriate. This month, we wrote a
large number of such patches, including:
* Bernhard M. Wiedemann:
* agama-integration-tests [111] (uses a random TCP-port number in
.lock file)
* ca-certificates-mozilla:ca-certificates-mozilla-prebuilt [112]
* cosmic [113] (hash order issue)
* openSUSE [114] (meta-issue to test reproducibility in the
openSUSE Build Service [115])
* pop-launcher [116] (parallelism-related issue)
* post [117] (toolchain-issue, avoiding Rust [118] parallelism)
* rpm-config-SUSE [119] (date-related issue)
* rust [120] (Rust [121] toolchain issue)
* weblate [122] (build gets stuck)
* Chris Lamb:
* #1078048 [123] filed against scikit-optimize [124].
* #1078153 [125] filed against pan [126].
* #1078687 [127] filed against tetzle [128].
* #1079041 [129] filed against apg [130].
* #1079312 [131] filed against receptor [132].
* James Addison:
* #1064782 [133] forwarded and merged in bind9-doc [134]
* #1066083 [135] forwarded and merged in gnome-maps [136]
[111] https://build.opensuse.org/request/show/1194375
[112] https://build.opensuse.org/request/show/1192626
[113] https://github.com/pop-os/cosmic-edit/issues/221
[114] https://github.com/openSUSE/openSUSE-release-tools/pull/3129
[115] https://build.opensuse.org/
[116] https://build.opensuse.org/request/show/1190933
[117] https://github.com/openSUSE/post-build-checks/pull/65
[118] https://www.rust-lang.org/
[119] https://build.opensuse.org/request/show/1192491
[120] https://github.com/rust-lang/rust/issues/128675
[121] https://www.rust-lang.org/
[122] https://build.opensuse.org/request/show/1194546
[123] https://bugs.debian.org/1078048
[124] https://tracker.debian.org/pkg/scikit-optimize
[125] https://bugs.debian.org/1078153
[126] https://tracker.debian.org/pkg/pan
[127] https://bugs.debian.org/1078687
[128] https://tracker.debian.org/pkg/tetzle
[129] https://bugs.debian.org/1079041
[130] https://tracker.debian.org/pkg/apg
[131] https://bugs.debian.org/1079312
[132] https://tracker.debian.org/pkg/receptor
[133] https://bugs.debian.org/1064782
[134] https://tracker.debian.org/pkg/bind9
[135] https://bugs.debian.org/1066083
[136] https://tracker.debian.org/pkg/gnome-maps
§
Reproducibility testing framework
---------------------------------
The Reproducible Builds project operates a comprehensive testing
framework running primarily at tests.reproducible-builds.org [137] in
order to check packages and other artifacts for reproducibility. In
August, a number of changes were made by Holger Levsen, including:
[137] https://tests.reproducible-builds.org
* Temporarily install the openssl-provider-legacy package for the
Debian unstable environments for running diffoscope due to
Debian bug #1078944 [138]. [139][140][141][142]
* Mark Debian armhf architecture nodes as being down due to proxy
down. [143][144]
* Detect proxy failures. [145][146][147]
* Run the index-buildinfo for the "builtin-pho" [148] script with
the -q switch. [149]
* Disable all Arch Linux [150] reproducible jobs. [151]
[138] https://bugs.debian.org/1078944
[139] https://salsa.debian.org/qa/jenkins.debian.net/commit/9e8be37aa
[140] https://salsa.debian.org/qa/jenkins.debian.net/commit/4f2779c2c
[141] https://salsa.debian.org/qa/jenkins.debian.net/commit/28ba5e52d
[142] https://salsa.debian.org/qa/jenkins.debian.net/commit/7763f181e
[143] https://salsa.debian.org/qa/jenkins.debian.net/commit/8587849f6
[144] https://salsa.debian.org/qa/jenkins.debian.net/commit/b27eecedf
[145] https://salsa.debian.org/qa/jenkins.debian.net/commit/494df0898
[146] https://salsa.debian.org/qa/jenkins.debian.net/commit/57d47b8c0
[147] https://salsa.debian.org/qa/jenkins.debian.net/commit/12d067d0e
[148] https://salsa.debian.org/bremner/builtin-pho
[149] https://salsa.debian.org/qa/jenkins.debian.net/commit/ac82812e0
[150] https://archlinux.org/
[151] https://salsa.debian.org/qa/jenkins.debian.net/commit/0a6b6152d
In addition, Mattia Rizzolo updated the website configuration to install
the ruby-jekyll-sitemap package as it is now used in the website [152],
Roland Clobus updated the script to build Debian 'live' images to treat
openQA [153] issues as warnings [154], and Vagrant Cascadian marked the
cbxi4b node as down [155].
[152] https://salsa.debian.org/qa/jenkins.debian.net/commit/cd734305d
[153] https://open.qa/
[154] https://salsa.debian.org/qa/jenkins.debian.net/commit/c0f0465fb
[155] https://salsa.debian.org/qa/jenkins.debian.net/commit/86c09e677
§
If you are interested in contributing to the Reproducible Builds
project, please visit our "Contribute" [156] page on our website.
However, you can get in touch with us via:
* IRC: #reproducible-builds on irc.oftc.net.
* Mastodon: @reproducible_builds at fosstodon.org [157]
* Mailing list: rb-general at lists.reproducible-builds.org [158]
* Twitter: @ReproBuilds [159]
[156] https://reproducible-builds.org/contribute/
[157] https://fosstodon.org/@reproducible_builds
[158] https://lists.reproducible-builds.org/listinfo/rb-general
[159] https://twitter.com/ReproBuilds
--
o
⬋ ⬊
o o reproducible-builds.org 💠
⬊ ⬋
o
More information about the rb-general
mailing list