From simon at josefsson.org  Thu Aug  1 03:36:26 2024
From: simon at josefsson.org (Simon Josefsson)
Date: Thu, 01 Aug 2024 05:36:26 +0200
Subject: Announcing Android Reproducible Builds at IzzyOnDroid with rbtlog
In-Reply-To: <Zqq0ojfIqPGp2Z2n@nihonium> (Fay Stegerman's message of "Thu, 1
 Aug 2024 00:03:14 +0200")
References: <Zqq0ojfIqPGp2Z2n@nihonium>
Message-ID: <87jzh0ewxx.fsf@kaka.sjd.se>

Fay Stegerman <flx at obfusk.net> writes:

> rbtlog [3] is a Reproducible Builds transparency log for Android APKs.  Its git
> repository contains scripts forming a rebuilder framework, recipes to build
> various apps, rebuild logs forming a transparency log of reproduction attempts,
> and CI workflows to automate everything.  It allows anyone to easily run a
> rebuilder for any apps available from a git repository with release tags plus
> accompanying APKs built and signed by the developer.

Nice!  Are the build dependencies (e.g., Android SDK) built from source
these days, or are they used as a untrusted binary blob during these
builds?  I recall rebuilding Android SDK from source used to be tricky.

/Simon

From flx at obfusk.net  Fri Aug  2 16:11:29 2024
From: flx at obfusk.net (Fay Stegerman)
Date: Fri, 2 Aug 2024 18:11:29 +0200
Subject: Announcing Android Reproducible Builds at IzzyOnDroid with rbtlog
In-Reply-To: <87jzh0ewxx.fsf@kaka.sjd.se>
References: <Zqq0ojfIqPGp2Z2n@nihonium>
 <87jzh0ewxx.fsf@kaka.sjd.se>
Message-ID: <Zq0FMdQElf9c3gFy@nihonium>

* Simon Josefsson <simon at josefsson.org> [2024-08-01 05:36]:
> Fay Stegerman <flx at obfusk.net> writes:
> 
> > rbtlog [3] is a Reproducible Builds transparency log for Android APKs.  Its git
> > repository contains scripts forming a rebuilder framework, recipes to build
> > various apps, rebuild logs forming a transparency log of reproduction attempts,
> > and CI workflows to automate everything.  It allows anyone to easily run a
> > rebuilder for any apps available from a git repository with release tags plus
> > accompanying APKs built and signed by the developer.
> 
> Nice!  Are the build dependencies (e.g., Android SDK) built from source
> these days, or are they used as a untrusted binary blob during these
> builds?  I recall rebuilding Android SDK from source used to be tricky.

Thanks!  Sadly not.  Looking at using SDK rebuilds [1] is on the roadmap, but
would require a lot of work.  Sadly, the Android ecosystem relies heavily on
downloading toolchains and dependencies as binary blobs.  Building all of that
from source would be an immense task.  And many tools like gradle are very hard
to bootstrap from source.

AFAIK no one has made much progress on doing full source builds for Android apps
that include dependencies and toolchains (that aren't available in Debian like
e.g. OpenJDK is).

- Fay

[1] https://codeberg.org/Starfish/SDK-Rebuilds

From bernhardout at lsmod.de  Sun Aug  4 14:00:36 2024
From: bernhardout at lsmod.de (Bernhard M. Wiedemann)
Date: Sun, 4 Aug 2024 16:00:36 +0200
Subject: rust non-determinism
Message-ID: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>

Hello fellow R-B-ings,

In previous months I had seen several rust-builds that were 
non-deterministic in similar ways.

Via https://github.com/pop-os/launcher/issues/230
I finally found one big clue that points to rust's default of
  codegen-units = 16 as the cause.

I'll advocate [1] to add to our (openSUSE) build envs:
export RUSTFLAGS="-C codegen-units=1"
that helps for many of these cases.

In my measurements, it only needs 5-10% longer to compile on a 4-core-VM 
and some builds are even faster on 1-core VMs.
Plus the docs say, it optimizes resulting binaries better, which makes 
it a worthy trade-off.


Ciao
Bernhard M.

[1] https://github.com/openSUSE/post-build-checks/pull/65
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240804/4f632016/attachment.sig>

From bernhardout at lsmod.de  Sun Aug  4 14:29:39 2024
From: bernhardout at lsmod.de (Bernhard M. Wiedemann)
Date: Sun, 4 Aug 2024 16:29:39 +0200
Subject: LibreOffice success story
In-Reply-To: <641a31ad-b5da-4c38-b35b-b787e4f6e5e4@lsmod.de>
References: <641a31ad-b5da-4c38-b35b-b787e4f6e5e4@lsmod.de>
Message-ID: <918bf253-5041-4d31-bb3d-57fbd4211815@lsmod.de>



On 07/11/2023 10.11, Bernhard M. Wiedemann via rb-general wrote:
> Dear fellow R-B-ings
> 
> Just 2 weeks ago, when I re-reviewed the remaining ~120 major issues in 
> openSUSE, I pretty much skipped over LibreOffice (and only this one), 
> noting it down as "various issues", because some years ago, when I had 
> previously taken a closer look, there had been so many issues in files 
> of various formats and additionally the build of that large package took 
> hours, so that I had quickly stopped debugging and left it "for later".
> 
> Then on the r-b-summit I met Thorsten from LibreOffice upstream and 
> together we reviewed the old diff I had from 2022 [1]. We noticed that 
> there was an issue with docs, but I had recently made
> https://code.opensuse.org/package/doxygen/blob/master/f/reproducible.patch
> that turned out to solve this exact issue. It still needs upstreaming.
> 
> There was a timestamp-issue from long-orphaned clucene and we found 3 
> others that Thorsten quickly fixed:
> https://gerrit.libreoffice.org/q/topic:reprobuild
> 
> Now there were only mtimes left in .jar and .zip files that were easily 
> normalized with strip-nondeterminism.
> 
> So today I hold in my hands the first two bit-identical LibreOffice rpm 
> packages.
> And this is the success I wanted to share with you all today.
> 
> It makes me feel as if we can solve anything.
> 
> Ciao
> Bernhard M.
> 
> 
> [1] 
> https://rb.zq1.de/compare.factory-20230109/diffs/libreoffice-compare.out

As a follow up on this: all 4 patches have landed in libreoffice 
upstream, but since openSUSE uses its own clucene-core package, it still 
needed some more work:
https://build.opensuse.org/request/show/1188998
https://build.opensuse.org/request/show/1191150

So after 9 months, the story finally reaches its conclusion...

Ciao
Bernhard M.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240804/27f1e9cb/attachment.sig>

From gnu at toad.com  Sun Aug  4 16:42:23 2024
From: gnu at toad.com (John Gilmore)
Date: Sun, 04 Aug 2024 09:42:23 -0700
Subject: rust non-determinism 
In-Reply-To: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de> 
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
Message-ID: <17232.1722789743@hop.toad.com>

"Bernhard M. Wiedemann via rb-general" wrote:
> Via https://github.com/pop-os/launcher/issues/230
> I finally found one big clue that points to rust's default of
>  codegen-units = 16 as the cause.

Should the Rust documentation be improved with a patch pointing out that
the default codegen-units setting mnakes the result not reproducible,
(and how to fix it)?

And should a bug be filed against Rustc that their compiler defaults to
irreproducible results, while its default should be reproducibility
rather than slightly faster compiler runtimes?

(They could fix it by making reproducible symbols even when codegen-units
is greater than 1, if they want both reproducibility and faster compiles.)

	       John

From bernhardout at lsmod.de  Sun Aug  4 17:02:26 2024
From: bernhardout at lsmod.de (Bernhard M. Wiedemann)
Date: Sun, 4 Aug 2024 19:02:26 +0200
Subject: rust non-determinism
In-Reply-To: <17232.1722789743@hop.toad.com>
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
 <17232.1722789743@hop.toad.com>
Message-ID: <60860778-62c2-4077-9006-db0e65b0ab18@lsmod.de>



On 04/08/2024 18.42, John Gilmore wrote:
> "Bernhard M. Wiedemann via rb-general" wrote:
>> Via https://github.com/pop-os/launcher/issues/230
>> I finally found one big clue that points to rust's default of
>>   codegen-units = 16 as the cause.
> 
> Should the Rust documentation be improved with a patch pointing out that
> the default codegen-units setting mnakes the result not reproducible,
> (and how to fix it)?
> 
> And should a bug be filed against Rustc that their compiler defaults to
> irreproducible results, while its default should be reproducibility
> rather than slightly faster compiler runtimes?
> 
> (They could fix it by making reproducible symbols even when codegen-units
> is greater than 1, if they want both reproducibility and faster compiles.)

Yes, I can file a rust issue tomorrow.
I hope, it goes better than with sphinx, ghc and guile that only produce 
deterministic results without parallelism.

Ciao
Bernhard M.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240804/95cf179b/attachment.sig>

From andrius at stikonas.eu  Sun Aug  4 17:13:53 2024
From: andrius at stikonas.eu (Andrius =?utf-8?B?xaB0aWtvbmFz?=)
Date: Sun, 04 Aug 2024 18:13:53 +0100
Subject: rust non-determinism
In-Reply-To: <60860778-62c2-4077-9006-db0e65b0ab18@lsmod.de>
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
 <17232.1722789743@hop.toad.com>
 <60860778-62c2-4077-9006-db0e65b0ab18@lsmod.de>
Message-ID: <2740816.mvXUDI8C0e@starbook>

Hi,

guile 3.0.8+ works fine with parallelism now. Live-bootstrap package [1] can be 
build with -j enabled.

Andrius

[1] https://github.com/fosslinux/live-bootstrap/blob/master/steps/guile-3.0.9/
pass1.sh

2024 m. rugpj??io 4 d., sekmadienis 18:02:26 GMT+1 Bernhard M. Wiedemann via 
rb-general ra??:
> On 04/08/2024 18.42, John Gilmore wrote:
> 
> > "Bernhard M. Wiedemann via rb-general" wrote:
> > 
> >> Via https://github.com/pop-os/launcher/issues/230
> >> I finally found one big clue that points to rust's default of
> >> 
> >>   codegen-units = 16 as the cause.
> > 
> > 
> > Should the Rust documentation be improved with a patch pointing out that
> > the default codegen-units setting mnakes the result not reproducible,
> > (and how to fix it)?
> > 
> > And should a bug be filed against Rustc that their compiler defaults to
> > irreproducible results, while its default should be reproducibility
> > rather than slightly faster compiler runtimes?
> > 
> > (They could fix it by making reproducible symbols even when codegen-units
> > is greater than 1, if they want both reproducibility and faster
> > compiles.)
> 
> 
> Yes, I can file a rust issue tomorrow.
> I hope, it goes better than with sphinx, ghc and guile that only produce 
> deterministic results without parallelism.
> 
> Ciao
> Bernhard M.





From bernhardout at lsmod.de  Sun Aug  4 17:16:19 2024
From: bernhardout at lsmod.de (Bernhard M. Wiedemann)
Date: Sun, 4 Aug 2024 19:16:19 +0200
Subject: rust non-determinism
In-Reply-To: <2740816.mvXUDI8C0e@starbook>
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
 <17232.1722789743@hop.toad.com>
 <60860778-62c2-4077-9006-db0e65b0ab18@lsmod.de> <2740816.mvXUDI8C0e@starbook>
Message-ID: <58ca9b48-236c-4a04-b4ac-32ac3890ad41@lsmod.de>



On 04/08/2024 19.13, Andrius ?tikonas wrote:
> 
> guile 3.0.8+ works fine with parallelism now. Live-bootstrap package [1] can be
> build with -j enabled.

Oh, that is good news, indeed.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240804/67d77abb/attachment.sig>

From thb at libreoffice.org  Sun Aug  4 19:44:42 2024
From: thb at libreoffice.org (Thorsten Behrens)
Date: Sun, 4 Aug 2024 21:44:42 +0200
Subject: LibreOffice success story
In-Reply-To: <918bf253-5041-4d31-bb3d-57fbd4211815@lsmod.de>
References: <641a31ad-b5da-4c38-b35b-b787e4f6e5e4@lsmod.de>
 <918bf253-5041-4d31-bb3d-57fbd4211815@lsmod.de>
Message-ID: <2lswbewte2f7zcay4dlnbzmmkmomsncrrde3zuq2jldw5b7djc@itjqgk6tjqdw>

Bernhard M. Wiedemann via rb-general wrote:
> So after 9 months, the story finally reaches its conclusion...
> 
:high-fives: :)

Cheers,

-- Thorsten
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 427 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240804/034665ee/attachment.sig>

From flx at obfusk.net  Mon Aug  5 00:41:02 2024
From: flx at obfusk.net (Fay Stegerman)
Date: Mon, 5 Aug 2024 02:41:02 +0200
Subject: can device-specific binaries ever be considered meaningfully
 reproducible?
Message-ID: <ZrAfngAbTg9TKJ8m@nihonium>

Hi!

As I wrote in [1] (see [2] for details):

> Split APKs certainly complicate Reproducible Builds.  Having only
> device-specific APKs instead of a single reference APK (or a handful of
> reference APKs when using split ABI) makes consensus on a "correct" result
> much harder, if not impossible, as "correct" is no longer universal but
> device-specific.

I personally don't think these device-specific APKs can be considered
meaningfully reproducible even if building from source for a specific device
gives me the same APKs installed on that specific device.

Because the whole part about "allow[ing] multiple third parties to come to a
consensus on a ?correct? result" breaks down completely when "correct" is
device-specific and not something everyone can agree on.

For example, I would not be able to rebuild and compare results with a friend as
-- unless we have (sufficiently) identical devices -- we would never get the
same bitwise identical artefacts.

I'm wondering what y'all think?

- Fay

[1] https://github.com/signalapp/Signal-Android/issues/13565#issuecomment-2254784638
[2] https://github.com/signalapp/Signal-Android/blob/main/reproducible-builds/README.md

From kpcyrd at archlinux.org  Mon Aug  5 01:48:49 2024
From: kpcyrd at archlinux.org (kpcyrd)
Date: Mon, 5 Aug 2024 03:48:49 +0200
Subject: can device-specific binaries ever be considered meaningfully
 reproducible?
In-Reply-To: <ZrAfngAbTg9TKJ8m@nihonium>
References: <ZrAfngAbTg9TKJ8m@nihonium>
Message-ID: <cd54060a-ffdd-4cbc-ae63-f3da0e16086b@archlinux.org>

On 8/5/24 2:41 AM, Fay Stegerman wrote:
> I personally don't think these device-specific APKs can be considered
> meaningfully reproducible even if building from source for a specific device
> gives me the same APKs installed on that specific device.
> 
> Because the whole part about "allow[ing] multiple third parties to come to a
> consensus on a ?correct? result" breaks down completely when "correct" is
> device-specific and not something everyone can agree on.
> 
> For example, I would not be able to rebuild and compare results with a friend as
> -- unless we have (sufficiently) identical devices -- we would never get the
> same bitwise identical artefacts.
> 
> I'm wondering what y'all think?

I know very little about android (especially about split APKs), but I 
think if "device specific" is sufficiently documented/specified it can 
be considered reproducible no problem.

It's just that there's now more permutations/artifacts that need to be 
reproduced instead of a canonical

Signal-Android-website-prod-universal-release-7.12.3.apk

I think it can be compared (somewhat) to how Debian has different .deb 
files for different CPUs.

cheers,
kpcyrd

From flx at obfusk.net  Mon Aug  5 02:56:29 2024
From: flx at obfusk.net (Fay Stegerman)
Date: Mon, 5 Aug 2024 04:56:29 +0200
Subject: can device-specific binaries ever be considered meaningfully
 reproducible?
In-Reply-To: <cd54060a-ffdd-4cbc-ae63-f3da0e16086b@archlinux.org>
References: <ZrAfngAbTg9TKJ8m@nihonium>
 <cd54060a-ffdd-4cbc-ae63-f3da0e16086b@archlinux.org>
Message-ID: <ZrA_XWblH1374s0J@nihonium>

* kpcyrd <kpcyrd at archlinux.org> [2024-08-05 03:48]:
> On 8/5/24 2:41 AM, Fay Stegerman wrote:
> > I personally don't think these device-specific APKs can be considered
> > meaningfully reproducible even if building from source for a specific device
> > gives me the same APKs installed on that specific device.
> > 
> > Because the whole part about "allow[ing] multiple third parties to come to a
> > consensus on a ?correct? result" breaks down completely when "correct" is
> > device-specific and not something everyone can agree on.
> > 
> > For example, I would not be able to rebuild and compare results with a friend as
> > -- unless we have (sufficiently) identical devices -- we would never get the
> > same bitwise identical artefacts.
> > 
> > I'm wondering what y'all think?
> 
> I know very little about android (especially about split APKs), but I think
> if "device specific" is sufficiently documented/specified it can be
> considered reproducible no problem.
> 
> It's just that there's now more permutations/artifacts that need to be
> reproduced instead of a canonical
> 
> Signal-Android-website-prod-universal-release-7.12.3.apk
> 
> I think it can be compared (somewhat) to how Debian has different .deb files
> for different CPUs.

Thanks!  I agree it's similar.  But there are a lot more permutations/artefacts
here than one .deb per CPU architecture (one .apk per CPU architecture is
already common).

Now you have a different APK (or actually several APKs that get combined when
installed) depending on a specific combination of at least CPU architecture,
installed locales, screen density, and android version [1].  As well as dynamic
features loaded on demand [2].

Debian still has canonical .deb files per CPU architecture.  Reaching consensus
on the "correct" .deb is still easy.  Though even then e.g. arm64 and amd64
users don't get the same .deb and thus can't really have any guarantees that
they are both running the same code (just that they are both running the
canonical code for their CPU architectures, generated from identical sources).

But here, when you install/update an app, Google Play generates the right
combination specifically for your device, based on its characteristics; and may
add additional components on-demand later.

Is it still theoretically possible to have this be reproducible?  Yes.  My worry
is that it is simply no longer *practical* with this many variables.

- Fay

[1] https://developer.android.com/tools/bundletool#device_specific_apks
[2] https://medium.com/googleplaydev/what-a-new-publishing-format-means-for-the-future-of-android-2e34981793a

From bernhardout at lsmod.de  Mon Aug  5 08:41:30 2024
From: bernhardout at lsmod.de (Bernhard M. Wiedemann)
Date: Mon, 5 Aug 2024 10:41:30 +0200
Subject: rust non-determinism
In-Reply-To: <17232.1722789743@hop.toad.com>
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
 <17232.1722789743@hop.toad.com>
Message-ID: <6a5b54d7-c4fe-42c5-b122-5ac46fb8295e@lsmod.de>

On 04/08/2024 18.42, John Gilmore wrote:
> And should a bug be filed against Rustc that their compiler defaults to
> irreproducible results, while its default should be reproducibility
> rather than slightly faster compiler runtimes?

=> https://github.com/rust-lang/rust/issues/128675

From gnu at toad.com  Mon Aug  5 16:37:10 2024
From: gnu at toad.com (John Gilmore)
Date: Mon, 05 Aug 2024 09:37:10 -0700
Subject: rust non-determinism 
In-Reply-To: <6a5b54d7-c4fe-42c5-b122-5ac46fb8295e@lsmod.de> 
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
 <17232.1722789743@hop.toad.com>
 <6a5b54d7-c4fe-42c5-b122-5ac46fb8295e@lsmod.de>
Message-ID: <8941.1722875830@hop.toad.com>

Bernhard M. Wiedemann <bernhardout at lsmod.de> wrote:
> => https://github.com/rust-lang/rust/issues/128675

Two Rustc developers closed it within 8 hours as "already completed",
even though it isn't.

They also said "CGU partitioning is very deliberately designed to be
deterministic."  Implying that therefore there is no bug, because design
and implementation are the same thing?

Rust has 36 open reproducibility bugs (not including this closed one).
It'd be worth seeing what other ones they closed unfixed.

  https://github.com/rust-lang/rust/labels/A-reproducibility

A bug with similar effects is still open since Nov 10, 2023 here:

   Parallel frontend (-Z threads=8) results in a different output than single-threaded #117776 
  https://github.com/rust-lang/rust/issues/117776

	John


From bernhardout at lsmod.de  Tue Aug  6 07:06:22 2024
From: bernhardout at lsmod.de (Bernhard M. Wiedemann)
Date: Tue, 6 Aug 2024 09:06:22 +0200
Subject: rust non-determinism
In-Reply-To: <8941.1722875830@hop.toad.com>
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
 <17232.1722789743@hop.toad.com>
 <6a5b54d7-c4fe-42c5-b122-5ac46fb8295e@lsmod.de>
 <8941.1722875830@hop.toad.com>
Message-ID: <fcafe0ab-4264-4805-a41d-83666f849cbb@lsmod.de>

On 05/08/2024 18.37, John Gilmore wrote:
> Bernhard M. Wiedemann <bernhardout at lsmod.de> wrote:
>> => https://github.com/rust-lang/rust/issues/128675
> 
> Two Rustc developers closed it within 8 hours as "already completed",
> even though it isn't.
> 
> They also said "CGU partitioning is very deliberately designed to be
> deterministic."  Implying that therefore there is no bug, because design
> and implementation are the same thing?
> 
> Rust has 36 open reproducibility bugs (not including this closed one).
> It'd be worth seeing what other ones they closed unfixed.

my workaround for openSUSE at
https://github.com/Firstyear/cargo-packaging/pull/11 was also rejected.

It is a complex ecosystem. pop-launcher pulls in 258 modules and many 
have their own build.rs that could break reproducibility. There is 
`crate` and `just` as build tools that can cause issues, too.
In the past there were also LLVM bugs that caused non-determinism.

So it is often hard to pin-point the source of non-determinism with 
precision and confidence.

And it does not help that rust HashMaps have non-deterministic order by 
default.


I'm getting closer to think that llvm's LTO and rust's codegen-units=16 
might be determistic by themselves, but (similar to PGO[1]) amplify 
other sources of non-determinism which makes it harder to debug.

If that is indeed the case, we can disable them during debugging r-b but 
leave them on when the package-specific issue is fixed.


Ciao
Bernhard M.

[1] https://github.com/bmwiedemann/theunreproduciblepackage/tree/master/pgo

From richard.purdie at linuxfoundation.org  Tue Aug  6 08:03:38 2024
From: richard.purdie at linuxfoundation.org (Richard Purdie)
Date: Tue, 06 Aug 2024 09:03:38 +0100
Subject: rust non-determinism
In-Reply-To: <fcafe0ab-4264-4805-a41d-83666f849cbb@lsmod.de>
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
 <17232.1722789743@hop.toad.com>
 <6a5b54d7-c4fe-42c5-b122-5ac46fb8295e@lsmod.de>
 <8941.1722875830@hop.toad.com>
 <fcafe0ab-4264-4805-a41d-83666f849cbb@lsmod.de>
Message-ID: <48e30e3b96d261e7e4fc3b559e077322ae234748.camel@linuxfoundation.org>

On Tue, 2024-08-06 at 09:06 +0200, Bernhard M. Wiedemann via rb-general wrote:
> On 05/08/2024 18.37, John Gilmore wrote:
> > Bernhard M. Wiedemann <bernhardout at lsmod.de> wrote:
> > > => https://github.com/rust-lang/rust/issues/128675
> > 
> > Two Rustc developers closed it within 8 hours as "already completed",
> > even though it isn't.
> > 
> > They also said "CGU partitioning is very deliberately designed to be
> > deterministic."? Implying that therefore there is no bug, because design
> > and implementation are the same thing?
> > 
> > Rust has 36 open reproducibility bugs (not including this closed one).
> > It'd be worth seeing what other ones they closed unfixed.
> 
> my workaround for openSUSE at
> https://github.com/Firstyear/cargo-packaging/pull/11?was also rejected.
> 
> It is a complex ecosystem. pop-launcher pulls in 258 modules and many
> have their own build.rs that could break reproducibility. There is 
> `crate` and `just` as build tools that can cause issues, too.
> In the past there were also LLVM bugs that caused non-determinism.
> 
> So it is often hard to pin-point the source of non-determinism with 
> precision and confidence.
> 
> And it does not help that rust HashMaps have non-deterministic order by 
> default.
> 
> 
> I'm getting closer to think that llvm's LTO and rust's codegen-units=16 
> might be determistic by themselves, but (similar to PGO[1]) amplify 
> other sources of non-determinism which makes it harder to debug.
> 
> If that is indeed the case, we can disable them during debugging r-b but 
> leave them on when the package-specific issue is fixed.

You might find this interesting:

https://git.yoctoproject.org/poky/commit/?id=e2e7017350d0b5324811fef3b841f98b00273887

Yocto Project has been chasing a problem where rust itself was
reproducible but rustdoc was not. We found it only happened if the
build was in paths of a different length. We had a pid in the buildpath
so this sometimes happened and sometimes did not, we've fixed things so
they're always different lengths now.

We found we had to disable both lto and use codegen-units=1 to make
things always reproduce. It took us months to work that out!

Exactly where the bug is, we're not sure but it does mean we have a
working workaround for now.

Cheers,

Richard




From bernhardout at lsmod.de  Tue Aug  6 09:48:55 2024
From: bernhardout at lsmod.de (Bernhard M. Wiedemann)
Date: Tue, 6 Aug 2024 11:48:55 +0200
Subject: rust non-determinism
In-Reply-To: <48e30e3b96d261e7e4fc3b559e077322ae234748.camel@linuxfoundation.org>
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
 <17232.1722789743@hop.toad.com>
 <6a5b54d7-c4fe-42c5-b122-5ac46fb8295e@lsmod.de>
 <8941.1722875830@hop.toad.com>
 <fcafe0ab-4264-4805-a41d-83666f849cbb@lsmod.de>
 <48e30e3b96d261e7e4fc3b559e077322ae234748.camel@linuxfoundation.org>
Message-ID: <17ef5a46-a49f-4030-b179-b3601c79d0c9@lsmod.de>

On 06/08/2024 10.03, Richard Purdie wrote:
> 
> We found we had to disable both lto and use codegen-units=1 to make
> things always reproduce. It took us months to work that out!

This sounds similar to the issue I see.
Do you have a simple reproducer?
When I try rustc with a hello-world.rs the binary only varies from path 
when I add -g and no -C strip=debuginfo
but that is independent of lto and codegen-units.

Maybe it needs some library?


Ciao
Bernhard M.

From richard.purdie at linuxfoundation.org  Tue Aug  6 10:34:34 2024
From: richard.purdie at linuxfoundation.org (Richard Purdie)
Date: Tue, 06 Aug 2024 11:34:34 +0100
Subject: rust non-determinism
In-Reply-To: <17ef5a46-a49f-4030-b179-b3601c79d0c9@lsmod.de>
References: <ed87a9eb-76d6-4bb1-bfa2-0365cbc5ba20@lsmod.de>
 <17232.1722789743@hop.toad.com>
 <6a5b54d7-c4fe-42c5-b122-5ac46fb8295e@lsmod.de>
 <8941.1722875830@hop.toad.com>
 <fcafe0ab-4264-4805-a41d-83666f849cbb@lsmod.de>
 <48e30e3b96d261e7e4fc3b559e077322ae234748.camel@linuxfoundation.org>
 <17ef5a46-a49f-4030-b179-b3601c79d0c9@lsmod.de>
Message-ID: <697691b8e14fa1ee0c9697e74873e0b31d8ec73a.camel@linuxfoundation.org>

On Tue, 2024-08-06 at 11:48 +0200, Bernhard M. Wiedemann wrote:
> On 06/08/2024 10.03, Richard Purdie wrote:
> > 
> > We found we had to disable both lto and use codegen-units=1 to make
> > things always reproduce. It took us months to work that out!
> 
> This sounds similar to the issue I see.
> Do you have a simple reproducer?
> When I try rustc with a hello-world.rs the binary only varies from
> path 
> when I add -g and no -C strip=debuginfo
> but that is independent of lto and codegen-units.
> 
> Maybe it needs some library?

We've struggled to reduce it down outside of the rust build environment
so sadly we don't have a simple reproducer, no.

Cheers,

Richard

From chris at reproducible-builds.org  Tue Aug  6 16:00:58 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Tue, 06 Aug 2024 17:00:58 +0100
Subject: Please review the draft for July's report
Message-ID: <172295998136.175978.11122273791068021167@copycat>

Hi all,

Please review the draft for July's Reproducible Builds report:

  https://reproducible-builds.org/reports/2024-07/?draft

? or, via the Git repository itself:

  https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2024-07.md

I intend to publish it no earlier than:

  $ date -d 'Thu, 08 Aug 2024 15:00:00 +0100'

  https://time.is/compare/1500_08_Aug_2024_in_BST

?

Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2024-07.md" file in the "reproducible-website" repository:

  $ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
  $ cd reproducible-website
  $ sensible-editor _reports/2024-07.md

I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:

  https://reproducible-builds.org/contribute/salsa/


Regards,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From chris at reproducible-builds.org  Thu Aug  8 15:30:25 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Thu, 08 Aug 2024 16:30:25 +0100
Subject: Reproducible Builds in July 2024
Message-ID: <3cbfe34a-cc7e-4195-8f46-994f91ce8d70@app.fastmail.com>

--------------------------------------------------------------------
        o
      ?   ?      July 2024 in Reproducible Builds
     o     o
      ?   ?      https://reproducible-builds.org/reports/2024-07/
        o
--------------------------------------------------------------------


Welcome to the July 2024 report from the Reproducible Builds project!

In our reports, we outline what we've been up to over the past month
and highlight news items in software supply-chain security more
broadly. As always, if you are interested in contributing to the
project, please visit our Contribute [1] page on our website.

 [0] https://reproducible-builds.org
 [1] https://reproducible-builds.org/contribute/

                                    ?


Reproducible Builds Summit 2024 [2]
-----------------------------------

Last month, we were very pleased to announce the upcoming Reproducible
Builds Summit [3], set to take place from September 17th ? 19th 2024
in Hamburg, Germany.

We are thrilled to host the seventh edition of this exciting event,
following the success of previous summits in various iconic locations
around the world, including Venice, Marrakesh, Paris, Berlin and
Athens. Our summits are a unique gathering that brings together
attendees from diverse projects, united by a shared vision of
advancing the Reproducible Builds effort. During this enriching event,
participants will have the opportunity to engage in discussions,
establish connections and exchange ideas to drive progress in this
vital field. Our aim is to create an inclusive space that fosters
collaboration, innovation and problem-solving.

If you're interesting in joining us this year, please make sure to read
the event page [4], which has more details about the event and location.
We are very much looking forward to seeing many readers of these
reports there.

 [2] https://reproducible-builds.org/events/hamburg2024/
 [3] https://reproducible-builds.org/events/hamburg2024/
 [4] https://reproducible-builds.org/events/hamburg2024/

                                    ?


"Pulling Linux up by its bootstraps" (LWN)
------------------------------------------

In a recent edition of Linux Weekly News [6], Daroc Alden [7] has
written an article on "bootstrappable" builds. Starting with a brief
introduction that:

> [?] a bootstrappable build [8] is one that builds existing software
> from scratch ? for example, building GCC without relying on an
> existing copy of GCC. In 2023, the Guix project announced [9] that
> the project had reduced the size of the binary bootstrap seed needed
> to build its operating system to just 357-bytes ? not counting the
> Linux kernel required to run the build process.

The article goes onto to describe that "now, the live-bootstrap [10]
project has gone a step further and removed the need for an existing
kernel at all." and concludes:

> The real benefit of bootstrappable builds comes from a few things.
> Like reproducible builds, they can make users more confident that
> the binary packages downloaded from a package mirror really do
> correspond to the open-source project whose source code they can
> inspect. Bootstrappable builds have also had positive effects on the
> complexity of building a Linux distribution from scratch [?]. But
> most of all, bootstrappable builds are a boon to the longevity of
> our software ecosystem. It's easy for old software to become
> unbuildable. By having a well-known, self-contained chain of
> software that can build itself from a small seed, in a variety of
> environments, bootstrappable builds can help ensure that today's
> software is not lost, no matter where the open-source community goes
> from here.

 [5] https://lwn.net/Articles/983340/
 [6] https://lwn.net/
 [7] https://setupminimal.github.io/
 [8] https://lwn.net/Articles/841797/
 [9] https://lwn.net/Articles/930650/
 [10] https://github.com/fosslinux/live-bootstrap

                                    ?


"Towards Idempotent Rebuilds?" [11]
-----------------------------------

Trisquel [12] developer Simon Josefsson [13] wrote an interesting blog
post [14] comparing the output of the .deb files from our
tests.reproducible-builds.org [15] testing framework and the ones in
the official Debian archive.

Following up from a previous post on the reproducibility of Trisquel
[16], Simon notes that "typically [the] rebuilds do not match the
official packages, even when they say the package is reproducible",
Simon correctly identifies that "the purpose of [these] rebuilds are
not to say anything about the official binary build, instead the
purpose is to offer a QA service to maintainers by performing two
builds of a package and declaring success if both builds match."

However, Simon's post swiftly moves on to announce a new tool called
debdistrebuild [17] that performs rebuilds of the difference between
two distributions in a GitLab pipeline [18] and displays diffoscope
[19] output for further analysis.

 [11] https://blog.josefsson.org/2024/07/10/towards-idempotent-rebuilds/
 [12] https://trisquel.info/
 [13] https://josefsson.org/
 [14] https://blog.josefsson.org/2024/07/10/towards-idempotent-rebuilds/
 [15] https://tests.reproducible-builds.org/debian/reproducible.html
 [16] https://blog.josefsson.org/2023/04/10/trisquel-is-42-reproducible/
 [17] https://gitlab.com/debdistutils/debdistrebuild
 [18] https://docs.gitlab.com/ee/ci/pipelines/
 [19] https://diffoscope.org

                                    ?


"AROMA: Automatic Reproduction of Maven Artifacts" [20]
-------------------------------------------------------

Mehdi Keshani, Tudor-Gabriel Velican, Gideon Bot and Sebastian Proksch
of the Delft University of Technology [21], Netherlands, have
published a new paper in the ACM Software Engineering on a new tool to
automatically reproduce Apache Maven [22] artifacts:

> Reproducible Central [23] is an initiative that curates a list of
> reproducible Maven libraries, but the list is limited and
> challenging to maintain due to manual efforts. [We] investigate the
> feasibility of automatically finding the source code of a library
> from its Maven release and recovering information about the original
> release environment. Our tool, AROMA, can obtain this critical
> information from the artifact and the source repository through
> several heuristics and we use the results for reproduction attempts
> of Maven packages. Overall, our approach achieves an accuracy of up
> to 99.5% when compared field-by-field to the existing manual
> approach [and] we reveal that automatic reproducibility is feasible
> for 23.4% of the Maven packages using AROMA, and 8% of these
> packages are fully reproducible.

 [20] https://dl.acm.org/doi/pdf/10.1145/3643764
 [21] https://www.tudelft.nl/en/
 [22] https://maven.apache.org/
 [23] https://github.com/jvm-repo-rebuild/reproducible-central

                                    ?


Community updates
-----------------

On our mailing list [24] this month:

* Nichita Morcotilo reached out to the community, first to share their
  efforts "to build reproducible packages cross-platform with a new
  build tool called rattler-build [25], noting that "as you can
  imagine, building packages reproducibly on Windows is the hardest
  challenge (so far!)". Nichita goes onto mention that the Apple
  ecosystem appears to be using ZERO_AR_DATE over SOURCE_DATE_EPOCH
  [26]. [27]

     [24] https://lists.reproducible-builds.org/listinfo/rb-general/
     [25] https://github.com/prefix-dev/rattler-build
     [26] https://reproducible-builds.org/docs/source-date-epoch/
     [27] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/003442.html

* Roland Clobus announced that the Debian bookworm 12.6 live images
  are "nearly reproducible" [28], with more detail in the post [29]
  itself and input in the thread [30] from other contributors.

     [28] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/003443.html
     [29] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/003443.html
     [30] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/thread.html#3443

* As reported in last month's report [31], Pol Dellaiera [32] completed
  his master thesis on "Reproducibility in Software Engineering" [33]
  at the University of Mons [34], Belgium. This month, Pol announced
  this on the list with more background info [35]. Since the master
  thesis sources have been available, it has received some feedback and
  contributions. As a result, an updated version of the thesis has been
  published [36] containing those community fixes.

     [31] https://reproducible-builds.org/reports/2024-06/
     [32] https://orcid.org/0009-0008-7972-7160
     [33] https://doi.org/10.5281/zenodo.12666898
     [34] https://web.umons.ac.be/
     [35] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/003449.html
     [36] https://doi.org/10.5281/zenodo.12666898

* Daniel Gr?ber asked for help [37] in getting the Yosys [38]
  documentation to build reproducibly, citing issues in inter alia
  the PDF generation causing differing CreationDate metadata values.

     [37] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/003452.html
     [38] https://yosyshq.net/yosys/

* James Addison continued his long journey [39] towards getting the
  Sphinx [40] documentation generator to build reproducible
  documentation. In this thread, James concerns himself with the
  problem that even "when SOURCE_DATE_EPOCH [41] is configured, Sphinx
  projects that have configured their copyright notices using dynamic
  elements can produce nonsensical output under some circumstances."
  James' query ended up generating a number of replies [42].

     [39] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/003458.html
     [40] https://www.sphinx-doc.org/en/master/
     [41] https://reproducible-builds.org/docs/source-date-epoch/
     [42] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/thread.html#3458

* Allen 'gunner' Gunner posted a brief update on the progress the
  core team is making towards introducing a Code of Conduct (CoC) such
  that it is "in place in time for the RB Summit in Hamburg in
  September". In particular, gunner asks "if you are interested in
  helping with CoC design and development in the weeks ahead, simply
  email rb-core at lists.reproducible-builds.org and let us know". [43]

     [43] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/003475.html

                                    ?


Android Reproducible Builds at IzzyOnDroid with rbtlog [44]
-----------------------------------------------------------

On our mailing list, Fay Stegerman announced [45] a new Reproducible
Builds collaboration in the Android ecosystem:

> We are pleased to announce "Reproducible Builds, special client
> support and more in our repo" [46]: a collaboration between various
> independent interoperable projects: the IzzyOnDroid [47] team,
> 3rd-party clients Droid-ify [48] & Neo Store [49], and rbtlog [50]
> (part of my collection of tools for Android Reproducible Builds) to
> bring Reproducible Builds to IzzyOnDroid and the wider Android
> ecosystem.

 [44] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/003485.html
 [45] https://lists.reproducible-builds.org/pipermail/rb-general/2024-July/003485.html
 [46] https://android.izzysoft.de/articles/named/iod-rbs-mirrors-clients
 [47] https://apt.izzysoft.de/fdroid/
 [48] https://github.com/Droid-ify/client
 [49] https://github.com/NeoApplications/Neo-Store
 [50] https://github.com/obfusk/rbtlog

                                    ?


"Extending the Scalability, Flexibility and Responsiveness of Secure Software Update Systems" [51]
--------------------------------------------------------------------------------------------------

Congratulations to Marina Moore of the New York Tandon School of
Engineering [52] who has submitted her PhD thesis on "Extending the
Scalability, Flexibility and Responsiveness of Secure Software Update
Systems" [53]. The introduction outlines its contributions to the field:

> [S]oftware repositories are a vital component of software
> development and release, with packages downloaded both for direct
> use and to use as dependencies for other software. Further, when
> software is updated due to patched vulnerabilities or new features,
> it is vital that users are able to see and install this patched
> version of the software. However, this process of updating software
> can also be the source of attack. To address these attacks, secure
> software update systems have been proposed. However, these secure
> software update systems have seen barriers to widespread adoption.
> The Update Framework (TUF) was introduced in 2010 to address several
> attacks on software update systems including repository compromise,
> rollback attacks, and arbitrary software installation. Despite this,
> compromises continue to occur, with millions of users impacted by
> such compromises. My work has addressed substantial challenges to
> adoption of secure software update systems grounded in an
> understanding of practical concerns. Work with industry and academic
> communities provided opportunities to discover challenges, expand
> adoption, and raise awareness about secure software updates. [?]

 [51] https://www.proquest.com/openview/07eb1454d3e506cd39b43ee0961bdabb/1?pq-origsite=gscholar&cbl=18750&diss=y
 [52] https://engineering.nyu.edu/
 [53] https://www.proquest.com/openview/07eb1454d3e506cd39b43ee0961bdabb/1?pq-origsite=gscholar&cbl=18750&diss=y

                                    ?


Development news
----------------

In Debian this month, 12 reviews of Debian packages were added, 13 were
updated and 6 were removed this month adding to our knowledge about
identified issues [54]. A new toolchain issue type was identified as
well, specifically ordering_differences_in_pkg_info [55].

Colin Percival filed a bug against the LLVM [56] compiler noting that
building i386 binaries on the i386 architecture is different when
building i386 binaries under amd64 [57]. The fix was narrowed down to
"x87 excess precision, which can result in slightly different register
choices when the compiler is hosted on x86_64 or i386" and a fix
committed. [58]

Fay Stegerman performed some in-depth research [59] surrounding her
"apksigcopier" [60] tool, after some Android .apk files signed with the
latest apksigner could no longer be verified as reproducible. Fay
identified the issue as follows:

> Since build-tools >= 35.0.0-rc1, backwards-incompatible changes to
apksigner break apksigcopier as it now by default forcibly replaces
existing alignment padding and changed the default page alignment from
4k to 16k (same as Android Gradle Plugin >= 8.3, so the latter is only
an issue when using older AGP). [61]

She documented multiple available workarounds and filed a bug in
Google's issue tracker [62].

Lastly, diffoscope [63] is our in-depth and content-aware diff utility
that can locate and diagnose reproducibility issues. This month, Chris
Lamb uploaded version 272 and Mattia Rizzolo uploaded version 273 to
Debian, and the following changes were made as well:

* Chris Lamb:

    * Ensure that the convert utility is from ImageMagick version 6.x.
      The command-line interface has seemingly changed with the 7.x
      series of ImageMagick. [64]
    * Factor out version detection in test_jpeg_image. [65]
    * Correct the import of the identify_version method after a
      refactoring change in a previous commit [66]. [67]
    * Move away from using DSA OpenSSH keys in tests as support has
      been deprecated and removed [68] in OpenSSH version 9.8p1. [69]
    * Move to assert_diff in the test_openssh_pub_key package. [70]
    * Update copyright years. [71]

* Mattia Rizzolo:

    * Add support for ffmpeg version 7.x which adds some extra context
      to the diff. [72]
    * Rework the handling of OpenSSH testing of DSA keys if OpenSSH is
      strictly 9.7, and add an OpenSSH key test with a ed25519-format
      key [73][74][75]
    * Temporarily disable a few packages that are not available in
      Debian "testing". [76][77]
    * Stop ignoring the results of Debian "testing" in the continuous
      integration system. [78]
    * Adjust options in debian/source to make sure not to pack the
      Python sdist directory into the binary Debian package. [79]
    * Adjust Lintian overrides. [80]

 [54] https://tests.reproducible-builds.org/debian/index_issues.html
 [55] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/af496924
 [56] https://llvm.org/
 [57] https://github.com/llvm/llvm-project/issues/99396
 [58] https://github.com/llvm/llvm-project/pull/100165/commits/28997387abf874345e9583c53739d2acbfedf761
 [59] https://github.com/obfusk/apksigcopier/issues/105
 [60] https://github.com/obfusk/apksigcopier
 [61] https://github.com/obfusk/apksigcopier/issues/105#issuecomment-2206799316
 [62] https://issuetracker.google.com/issues/351408623
 [63] https://diffoscope.org
 [64] https://salsa.debian.org/reproducible-builds/diffoscope/commit/bbcf367c
 [65] https://salsa.debian.org/reproducible-builds/diffoscope/commit/037bdcbb
 [66] https://salsa.debian.org/reproducible-builds/diffoscope/commit/037bdcbb
 [67] https://salsa.debian.org/reproducible-builds/diffoscope/commit/38f76379
 [68] https://lwn.net/Articles/958048/
 [69] https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/382
 [70] https://salsa.debian.org/reproducible-builds/diffoscope/commit/e8c5dc10
 [71] https://salsa.debian.org/reproducible-builds/diffoscope/commit/5b5c8c62
 [72] https://salsa.debian.org/reproducible-builds/diffoscope/commit/cadfc73a
 [73] https://salsa.debian.org/reproducible-builds/diffoscope/commit/7917b746
 [74] https://salsa.debian.org/reproducible-builds/diffoscope/commit/f3f72b9f
 [75] https://salsa.debian.org/reproducible-builds/diffoscope/commit/652b5793
 [76] https://salsa.debian.org/reproducible-builds/diffoscope/commit/c6bba336
 [77] https://salsa.debian.org/reproducible-builds/diffoscope/commit/f8a1e142
 [78] https://salsa.debian.org/reproducible-builds/diffoscope/commit/bc2229c7
 [79] https://salsa.debian.org/reproducible-builds/diffoscope/commit/40b63b71
 [80] https://salsa.debian.org/reproducible-builds/diffoscope/commit/633654ee

                                    ?


Website updates
---------------

There were a number of improvements made to our website this
month, including:

* Bernhard M. Wiedemann updated the SOURCE_DATE_EPOCH [81] page to
  include instructions on how to create reproducible .zip files from
  within Python using the "zipfile" [82] module. [83]

   [81] https://reproducible-builds.org/docs/source-date-epoch/
   [82] https://docs.python.org/3/library/zipfile.html
   [83] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/7678a3e8

* Chris Lamb fixed a potential duplicate heading on the Projects [84]
  page. [85]

   [84] https://reproducible-builds.org/who/projects/
   [85] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/3a701087

* Fay Stegerman added rbtlog [86] to the Tools [87] page [88] and
  IzzyOnDroid [89] to the Projects [90] page [91], also ensuring that
  the latter page was always sorted regardless of the ordering within
  the input data files. [92]

   [86] https://github.com/obfusk/rbtlog
   [87] https://reproducible-builds.org/tools/
   [88] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/6882b92f
   [89] https://apt.izzysoft.de/fdroid/
   [90] https://reproducible-builds.org/who/projects/
   [91] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a939f2ec
   [92] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/b3e7154b

* Holger Levsen added Linus Nordberg to our global list of contributors
  [93][94] as well as made a number of changes to the page for the
  upcoming Reproducible Builds summit later this year
  [95][96][97][98][99].

   [93] https://reproducible-builds.org/who/people/
   [94] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/595ccb28
   [95] https://reproducible-builds.org/events/hamburg2024/
   [96] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/de398031
   [97] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/ed5eb6f4
   [98] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/d69d7503
   [99] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/e048075e

* Mattia Rizzolo updated the Civil Infrastructure Platform [100] logo
  [101] and also updated the 2024 summit page [102][103][104].

   [100] https://www.cip-project.org/
   [101] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/13058e9e
   [102] https://reproducible-builds.org/events/hamburg2024/
   [103] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/99054850
   [104] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/fc371a3f

* Nichita Morcotilo added rattler-build [105] to the Projects [106]
  page. [107][108][109]

   [105]  https://github.com/prefix-dev/rattler-build
   [106] https://reproducible-builds.org/who/projects/
   [107] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a5130fea
   [108] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a9d32515
   [109] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/5d1b5fc7

* Pol Dellaiera updated the Academic Publications [110] page, adding
  two publications. [111][112]

   [110] https://reproducible-builds.org/docs/publications/
   [111] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/b98049f2
   [112] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/36da2c24

                                    ?


Upstream patches
----------------

The Reproducible Builds project detects, dissects and attempts to fix as
many currently-unreproducible packages as possible. We endeavour to send
all of our patches upstream where appropriate. This month, we wrote a
large number of such patches, including:

* Bernhard M. Wiedemann:

    * armagetron [113] (date)
    * blaspp [114] (hostname)
    * cligen [115] (GnuTLSs date)
    * cloudflared [116] (date)
    * dpdk [117] (Sphinx doctrees)
    * fonttosfnt/xorg-x11-fonts [118] (toolchain, date)
    * gegl [119] (build machine details)
    * gettext-runtime [120] (jar mtime)
    * kf6-kirigami+kf6-qqc2-desktop-style [121] (race-condition)
    * kubernetes1.26 [122] (backport upstream fix for random path)
    * lapackpp [123] (hostname)
    * latex2html [124] (nochecks)
    * libdb-4_8 [125] (.jar modification time)
    * librcc [126] (already merged upstream)
    * libreoffice [127] (strip .jar mtimes + clucene-core
      [128] toolchain)
    * maliit-keyboard [129] (nocheck)
    * nautilus [130] (date)
    * openblas [131] (CPU type, fixed [132])
    * openssl-3 [133] (random-related issue)
    * python-ruff [134] (ASLR)
    * python3 [135] (date [136], parallelism/race [137])
    * reproducible-faketools [138] (0.5.2)
    * sphinx [139] (GZip modification time)
    * sphinxcontrib [140] (gzip mtime)

     [113] https://build.opensuse.org/request/show/1188202
     [114] https://github.com/icl-utk-edu/blaspp/pull/87
     [115] https://gitlab.com/gnutls/cligen/-/merge_requests/5
     [116] https://github.com/cloudflare/cloudflared/pull/1289
     [117] https://build.opensuse.org/request/show/1185443
     [118] https://gitlab.freedesktop.org/xorg/app/fonttosfnt/-/merge_requests/22
     [119] https://build.opensuse.org/request/show/1188550
     [120] https://build.opensuse.org/request/show/1188059
     [121] https://bugzilla.opensuse.org/show_bug.cgi?id=1228131
     [122] https://build.opensuse.org/request/show/1190449
     [123] https://github.com/icl-utk-edu/lapackpp/pull/68
     [124] https://build.opensuse.org/request/show/1188512
     [125] https://build.opensuse.org/request/show/1190247
     [126] https://build.opensuse.org/request/show/1188204
     [127] https://build.opensuse.org/request/show/1189287
     [128] https://build.opensuse.org/request/show/1188447
     [129] https://build.opensuse.org/request/show/1185254
     [130] https://gitlab.gnome.org/GNOME/nautilus/-/merge_requests/1555
     [131] https://bugzilla.opensuse.org/show_bug.cgi?id=1228177
     [132] https://build.opensuse.org/request/show/1190320
     [133] https://build.opensuse.org/request/show/1187438
     [134] https://github.com/astral-sh/ruff/issues/12169
     [135] https://bugzilla.opensuse.org/show_bug.cgi?id=1227999
     [136] https://github.com/python/cpython/pull/121872
     [137] https://github.com/python/cpython/pull/121883
     [138] https://build.opensuse.org/request/show/1186763
     [139] https://github.com/sphinx-doc/sphinx/pull/12606
     [140] https://github.com/sphinx-doc/sphinxcontrib-devhelp/pull/13

* Chris Lamb:

    * #1076368 [141] filed against nautilus [142].
    * #1076507 [143] filed against mccode [144].
    * #1076806 [145] filed against meson-python [146].
    * #1077479 [147] filed against debcraft [148].
    * #1077485 [149] filed against pytest [150].
    * #1077601 [151] filed against setuptools [152].

     [141] https://bugs.debian.org/1076368
     [142] https://tracker.debian.org/pkg/nautilus
     [143] https://bugs.debian.org/1076507
     [144] https://tracker.debian.org/pkg/mccode
     [145] https://bugs.debian.org/1076806
     [146] https://tracker.debian.org/pkg/meson-python
     [147] https://bugs.debian.org/1077479
     [148] https://tracker.debian.org/pkg/debcraft
     [149] https://bugs.debian.org/1077485
     [150] https://tracker.debian.org/pkg/pytest
     [151] https://bugs.debian.org/1077601
     [152] https://tracker.debian.org/pkg/setuptools

* Fridrich Strba:

    * javapackages-tools [153]
    * ant [154]
    * java-21-openjdk [155]

     [153] https://build.opensuse.org/package/show/Java:packages/javapackages-tools
     [154] https://build.opensuse.org/package/show/Java:packages/ant
     [155] https://build.opensuse.org/package/show/Java:Factory/java-21-openjdk

* Evangelos Ribeiro Tzaras:

    * buffybox [156]

     [156] https://gitlab.com/postmarketOS/buffybox/-/merge_requests/24

                                    ?


Reproducibility testing framework
---------------------------------

The Reproducible Builds project operates a comprehensive testing
framework running primarily at "tests.reproducible-builds.org" [157] in
order to check packages and other artifacts for reproducibility. In
July, a number of changes were made by Holger Levsen, including:

* Grant bremner access to the ionos7 node. [158][159]
* Perform a dummy change to force update of all jobs. [160][161]

In addition, Vagrant Cascadian performed some necessary node maintenance
of the underlying build hosts. [162]

 [157] https://tests.reproducible-builds.org
 [158] https://salsa.debian.org/qa/jenkins.debian.net/commit/bf4a10cc0
 [159] https://salsa.debian.org/qa/jenkins.debian.net/commit/fb5ac9533
 [160] https://salsa.debian.org/qa/jenkins.debian.net/commit/d30ab7185
 [161] https://salsa.debian.org/qa/jenkins.debian.net/commit/94a1e8367
 [162] https://salsa.debian.org/qa/jenkins.debian.net/commit/bc50a42a1

                                    ?

If you are interested in contributing to the Reproducible Builds
project, please visit our "Contribute" [163] page on our website.
However, you can get in touch with us via:

 * IRC: #reproducible-builds on irc.oftc.net.

 * Mastodon: @reproducible_builds at fosstodon.org [164]

 * Mailing list: rb-general at lists.reproducible-builds.org [165]

 * Twitter: @ReproBuilds [166]

 [163] https://reproducible-builds.org/contribute/
 [164] https://fosstodon.org/@reproducible_builds
 [165] https://lists.reproducible-builds.org/listinfo/rb-general
 [166] https://twitter.com/ReproBuilds



-- 
      o
    ?   ?
   o     o     reproducible-builds.org ?
    ?   ?
      o

From chris at reproducible-builds.org  Thu Aug  8 15:21:42 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Thu, 08 Aug 2024 16:21:42 +0100
Subject: Please review the draft for July's report
In-Reply-To: <172295998136.175978.11122273791068021167@copycat>
References: <172295998136.175978.11122273791068021167@copycat>
Message-ID: <a59d3212-5bb9-4ded-84a5-64555cd28fa1@app.fastmail.com>

Chris Lamb wrote:

> Please review the draft for July's Reproducible Builds report:

This has now been published ? thanks to all who contributed.

If possible, please share the following link:

  https://reproducible-builds.org/reports/2024-07/

.. and also consider retweeting:

  https://x.com/ReproBuilds/status/1821567357346537977


Regards,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From chris at reproducible-builds.org  Fri Aug  9 15:52:14 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Fri, 09 Aug 2024 16:52:14 +0100
Subject: =?UTF-8?Q?diffoscope_274_released_=F0=9F=92=A0?=
Message-ID: <172320468946.362174.7778175905264562840@copycat>

Hi,

The diffoscope maintainers are pleased to announce the release of
version 274 of diffoscope.

diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.

Version 274 includes the following changes:

  [ Chris Lamb ]
  * Add support for IO::Compress::Zip >= 2.212. (Closes: #1078050)
  * Don't include debug output when calling dumppdf(1).
  * Append output from dumppdf(1) in more cases.
    (Closes: reproducible-builds/diffoscope#387)
  * Update copyright years.

  [ Mattia Rizzolo ]
  * Update the available architectures for test dependencies.

## Download

Version 274 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:

   https://diffoscope.org/

? but source tarballs may be located here:

  https://diffoscope.org/archive/

The corresponding Docker image may be run via (for example):

  $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
      registry.salsa.debian.org/reproducible-builds/diffoscope a b


## Contribute

diffoscope is developed within the "Reproducible builds" effort.

  - Git repository
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Docker image, eg.
    registry.salsa.debian.org/reproducible-builds/diffoscope
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Issues and feature requests
    https://salsa.debian.org/reproducible-builds/diffoscope/issues

  - Contribution instructions (eg. to file an issue)
    https://reproducible-builds.org/contribute/salsa/


Regards,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From jay at jp-hosting.net  Sun Aug 11 10:37:28 2024
From: jay at jp-hosting.net (James Addison)
Date: Sun, 11 Aug 2024 11:37:28 +0100
Subject: Nearly reproducible Bookworm 12.6 live images
In-Reply-To: <c0b25026-52b5-4c15-879b-30326eeebab5@rclobus.nl>
References: <501cc7bc-e95f-45fd-970b-6166ac796870@rclobus.nl>
 <CALDQ5Nyi1WjYJcybGx+UBdWki1tjyTCeV1COOKCDrMR1Ygipcg@mail.gmail.com>
 <c0b25026-52b5-4c15-879b-30326eeebab5@rclobus.nl>
Message-ID: <CALDQ5NyhQHP1U5PtHACvKsjpDL6eZWSTCjYiBCgX3yb28SW47Q@mail.gmail.com>

Thanks Roland - I've found some time to read the relevant rebuild.sh script
from the Debian live-build.git repository, to reassure myself about the
implementation you've described.

Generally it does seem reliable to me - I'll add one or two comments inline
below:

On Tue, 16 Jul 2024 at 08:58, Roland Clobus <rclobus at rclobus.nl> wrote:
> [ ... snip ... ]
> On 16/07/2024 00:21, James Addison wrote:
> > This sounds great - I have one specific question / concern:
>
> > Limiting the timestamp to a maximum of SOURCE_DATE_EPOCH makes sense so that
> > the modification-time is consistent for equivalent rebuilds.
> >
> > However: if the contents of the txt file / other content in the git repository
> > change, would that prevent an independent rebuilder from recreating the
> > identical CD image as output?
>
> First the rebuild script determines the latest modification date of the
> archive (as found in InRelease) and sets SOURCE_DATE_EPOCH accordingly.
> Then the rebuild script uses the git repository of live-build and finds
> the commit that matches SOURCE_DATE_EPOCH. Thus it ensures that for the
> Debian point releases you get exactly what was configured at that time.
> During this 'git checkout HASH' command, the timestamps of the modified
> files will be updated to 'now', which is the intended behaviour, as the
> final ISO image will have all dates truncated to SOURCE_DATE_EPOCH.
>
> So an independent rebuilder should call the rebuild.sh script without
> setting S_D_E and will get identical output.

>From the script contents: yes, this is accurate.  A rebuilder that builds from
the same InRelease version of the archive (deb.debian.org), or, subsequently,
from a snapshot of the archive, will (unless configured to use a local copy)
retrieve live-build.git and check out the most recent single commit found
_after_ the release timestamp that is reachable from the HEAD commit ref.

A nitpick: should we clone a specific named branch to retrieve the HEAD
reference?  Rationale: the default branch name for a repo can change.


Initially I was worried about the possibility that subsequent git commits with
amended dates could break reproducibility, but on inspection, although git
commit _author date_ can be modified and adjusted by commits, the min-age
filter used by the rebuild.sh script filters by _commit date_, which cannot be
amended or overridden by subsequently pushed commits.


The general angle that I'm considering here is: what resources should rebuilder
sites download in advance if they want to perform a rebuild, and how can they
reasonably assure that they are using the same sources as others, and that
those will not diverge at a later date?

> > (note: given that the package archive is regularly updated, I think that there
> > are other reasons why an identical point-in-time rebuild may require sharing
> > of some build-time/archive metadata -- the reason I'm asking is that I'd like
> > to check whether this git repo could require similar co-ordination during
> > rebuilds.  fixed commit ID / signed tag, or other mechanism for example)
>
> I'm using the timestamp from InRelease, which was confirmed to be 'the'
> timestamp of the archive [1]. Therefore I don't need tags.

Provided that sites receive the same live-build.git checkout (and they should),
I agree with this.  It may seem like I'm over-challenging the design but I
would like to determine whether we could provide any additional guarantees that
the install media and live-build correspond to each other (perhaps it is not
possible, there may be a circular dependency problem).

(that may be steering back towards a Debian-specific conversation, but some
of this source-trust-consensus is no doubt relevant to other distros also)

Thanks again,
James

From chris at reproducible-builds.org  Fri Aug 16 13:33:23 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Fri, 16 Aug 2024 14:33:23 +0100
Subject: =?UTF-8?Q?diffoscope_275_released_=F0=9F=92=A0?=
Message-ID: <172380327959.858792.8022232951365586808@copycat>

Hi,

The diffoscope maintainers are pleased to announce the release of
version 275 of diffoscope.

diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.

Version 275 includes the following changes:

  [ Chris Lamb ]
  * Update the test_zip.py text fixtures and definitions to support new changes
    to IO::Compress. (Closes: #1078050)
  * Do not call marshal.loads(...) of precompiled Python bytecode as it is
    inherently unsafe. Replace, at least for now, with a brief summary of the
    code section of .pyc files. (Re: reproducible-builds/diffoscope#371)
  * Don't bother to check the Python version number in test_python.py: the
    fixture for this test is deterministic/fixed.
  * Update copyright years.

## Download

Version 275 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:

   https://diffoscope.org/

? but source tarballs may be located here:

  https://diffoscope.org/archive/

The corresponding Docker image may be run via (for example):

  $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
      registry.salsa.debian.org/reproducible-builds/diffoscope a b


## Contribute

diffoscope is developed within the "Reproducible builds" effort.

  - Git repository
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Docker image, eg.
    registry.salsa.debian.org/reproducible-builds/diffoscope
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Issues and feature requests
    https://salsa.debian.org/reproducible-builds/diffoscope/issues

  - Contribution instructions (eg. to file an issue)
    https://reproducible-builds.org/contribute/salsa/


Regards,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From chris at reproducible-builds.org  Sun Aug 18 17:45:07 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Sun, 18 Aug 2024 13:45:07 -0400 (EDT)
Subject: =?utf-8?q?diffoscope_276_released_=F0=9F=92=A0?=
Message-ID: <172400302571.1061048.9184344417231566223@copycat>

Hi,

The diffoscope maintainers are pleased to announce the release of
version 276 of diffoscope.

diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.

Version 276 includes the following changes:

  [ Chris Lamb ]
  * Also catch RuntimeError when importing PyPDF so that PyPDF or, crucially,
    its transitive dependencies do not cause diffoscope to traceback at runtime
    and build time. (Closes: #1078944, reproducible-builds/diffoscope#389)
  * Factor out a method for stripping ANSI escapes.
  * Strip ANSI escapes from the output of Procyon. Thanks, Aman Sharma!
  * Update copyright years.

## Download

Version 276 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:

   https://diffoscope.org/

??? but source tarballs may be located here:

  https://diffoscope.org/archive/

The corresponding Docker image may be run via (for example):

  $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
      registry.salsa.debian.org/reproducible-builds/diffoscope a b


## Contribute

diffoscope is developed within the "Reproducible builds" effort.

  - Git repository
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Docker image, eg.
    registry.salsa.debian.org/reproducible-builds/diffoscope
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Issues and feature requests
    https://salsa.debian.org/reproducible-builds/diffoscope/issues

  - Contribution instructions (eg. to file an issue)
    https://reproducible-builds.org/contribute/salsa/


Regards,

-- 
      o
    ???   ???      Chris Lamb
   o     o     reproducible-builds.org ????
    ???   ???
      o

From mechtilde at debian.org  Sat Aug 24 18:30:34 2024
From: mechtilde at debian.org (Mechtilde Stehmann)
Date: Sat, 24 Aug 2024 20:30:34 +0200
Subject: (java) Builds not reproducible on armhf
In-Reply-To: <87y183oze3.fsf@wireframe>
References: <b658fe97-78d3-4327-a96c-019d8ef59a02@debian.org>
 <87y183oze3.fsf@wireframe>
Message-ID: <89506054-db95-44f5-a57b-06c45699bb2d@debian.org>

Hello,

in the meantime I did some more tests to find the reason for it.

I build it again with pbuilder specifying ARCH=armhf and get the right 
date for the pom file (same date as the amd64 build)

Now I think I need access to an armhf box to test it natively.

What should I do to get temporary.

Kind regards

Mechtilde

Am 21.05.24 um 19:20 schrieb Vagrant Cascadian:
> On 2024-05-20, Mechtilde Stehmann wrote:
>> I want to clean up my Java packages.
>>
>> There are several with FTBR. I found that the day of the *.poms s a date
>> from 1970.
>>
>> for example they are the packages
>>
>> vinnie
> 
> Looking at the history for vinnie:
> 
>    https://tests.reproducible-builds.org/debian/history/armhf/vinnie.html
> 
> It is only very recently that this started happening (2024-05-04)
> without source changes in vinnie itself, so I would suspect some change
> in the toolchain used to produce the .pom files?
> 
> commons-email is similar, although starting 2024-04-04:
> 
>    https://tests.reproducible-builds.org/debian/history/armhf/commons-email.html
> 
> ez-vcard is similar too, starting 2024-04-20:
> 
>    https://tests.reproducible-builds.org/debian/rb-pkg/trixie/armhf/diffoscope-results/ez-vcard.html
> 
> Although some of those builds also have differences in some xz
> contents... might just be related to the timestamp differences.
> 
> 
> Wild hunch is one build is run on a 64-bit kernel (without a linux32
> personality) and one build on a 32-bit kernel... that is one of the main
> differences between these armhf test builds and builds on other
> architectures, where this does not seem to happen...
> 
> 
> live well,
>    vagrant

-- 
Mechtilde Stehmann
## Debian Developer
## PGP encryption welcome
## F0E3 7F3D C87A 4998 2899  39E7 F287 7BBA 141A AD7F

From mechtilde at debian.org  Sun Aug 25 11:26:03 2024
From: mechtilde at debian.org (Mechtilde Stehmann)
Date: Sun, 25 Aug 2024 13:26:03 +0200
Subject: (java) Builds not reproducible on armhf
In-Reply-To: <87y183oze3.fsf@wireframe>
References: <b658fe97-78d3-4327-a96c-019d8ef59a02@debian.org>
 <87y183oze3.fsf@wireframe>
Message-ID: <9c03f7c4-410b-42c8-9606-85caa18d8398@debian.org>

Hello Vagrant,

in the meantime I did a build at amdahl.debian.org and can't reproduce 
the problem too.

i did it with this package 
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/armhf/csvjdbc.html

Dates are 2022-06-08 as expected and not 1970-01-03 as described at 
tests.rb.o.

There are many other packages with this problem in the java-team.

Do you have an idea, how I can help to fix it. Can I have a (temporaly) 
access to reproduce the problem and then maybe to fix it.

Kind regards

Mechtilde

Am 21.05.24 um 19:20 schrieb Vagrant Cascadian:
> On 2024-05-20, Mechtilde Stehmann wrote:
>> I want to clean up my Java packages.
>>
>> There are several with FTBR. I found that the day of the *.poms s a date
>> from 1970.
>>
>> for example they are the packages
>>
>> vinnie
> 
> Looking at the history for vinnie:
> 
>    https://tests.reproducible-builds.org/debian/history/armhf/vinnie.html
> 
> It is only very recently that this started happening (2024-05-04)
> without source changes in vinnie itself, so I would suspect some change
> in the toolchain used to produce the .pom files?
> 
> commons-email is similar, although starting 2024-04-04:
> 
>    https://tests.reproducible-builds.org/debian/history/armhf/commons-email.html
> 
> ez-vcard is similar too, starting 2024-04-20:
> 
>    https://tests.reproducible-builds.org/debian/rb-pkg/trixie/armhf/diffoscope-results/ez-vcard.html
> 
> Although some of those builds also have differences in some xz
> contents... might just be related to the timestamp differences.
> 
> 
> Wild hunch is one build is run on a 64-bit kernel (without a linux32
> personality) and one build on a 32-bit kernel... that is one of the main
> differences between these armhf test builds and builds on other
> architectures, where this does not seem to happen...
> 
> 
> live well,
>    vagrant

-- 
Mechtilde Stehmann
## Debian Developer
## PGP encryption welcome
## F0E3 7F3D C87A 4998 2899  39E7 F287 7BBA 141A AD7F
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240825/b822d780/attachment.sig>

From zeha at debian.org  Sun Aug 25 22:05:56 2024
From: zeha at debian.org (Chris Hofstaedtler)
Date: Mon, 26 Aug 2024 00:05:56 +0200
Subject: (java) Builds not reproducible on armhf
In-Reply-To: <9c03f7c4-410b-42c8-9606-85caa18d8398@debian.org>
Message-ID: <4o6fsmm22y3j5kvbsgu5jbq3pujrmqr5eqengehifgv67mdzms@wzjqclzmtois>

[..]
> Dates are 2022-06-08 as expected and not 1970-01-03 as described
> at tests.rb.o.
[..]
> >   https://tests.reproducible-builds.org/debian/history/armhf/vinnie.html

I briefly looked at vinnie. Given it is arch:all, I kinda expect it
to produce the "same" deb regardless of the arch it is built on.
libvinnie-java_2.0.2-3_all.deb in the archive was built on
x86-grnet-03, so on amd64. It does -not- contain a 1970-x-y
timestamp.

I'll note that fakeroot was probably broken on armel, armhf since
the t64 migration until mid-August.
Is fakeroot involved in any way? If so, it might make sense to
discard any results from that period.

Chris


From mechtilde at debian.org  Mon Aug 26 04:46:35 2024
From: mechtilde at debian.org (Mechtilde Stehmann)
Date: Mon, 26 Aug 2024 06:46:35 +0200
Subject: (java) Builds not reproducible on armhf
In-Reply-To: <4o6fsmm22y3j5kvbsgu5jbq3pujrmqr5eqengehifgv67mdzms@wzjqclzmtois>
References: <4o6fsmm22y3j5kvbsgu5jbq3pujrmqr5eqengehifgv67mdzms@wzjqclzmtois>
Message-ID: <ddcc127e-54fc-420a-8780-8813284c62a0@debian.org>

Hello Chris,

Am 26.08.24 um 00:05 schrieb Chris Hofstaedtler:
> [..]
>> Dates are 2022-06-08 as expected and not 1970-01-03 as described
>> at tests.rb.o.
> [..]
>>>    https://tests.reproducible-builds.org/debian/history/armhf/vinnie.html
> 
> I briefly looked at vinnie. Given it is arch:all, I kinda expect it
> to produce the "same" deb regardless of the arch it is built on.
> libvinnie-java_2.0.2-3_all.deb in the archive was built on
> x86-grnet-03, so on amd64. It does -not- contain a 1970-x-y
> timestamp.
> 
> I'll note that fakeroot was probably broken on armel, armhf since
> the t64 migration until mid-August.
> Is fakeroot involved in any way? If so, it might make sense to
> discard any results from that period.

Thanks for your explanations.
Are there plans to ran the tests again?

> 
> Chris
> 

-- 
Mechtilde Stehmann
## Debian Developer
## PGP encryption welcome
## F0E3 7F3D C87A 4998 2899  39E7 F287 7BBA 141A AD7F
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240826/2c9be7ca/attachment.sig>

From holger at layer-acht.org  Mon Aug 26 06:51:32 2024
From: holger at layer-acht.org (Holger Levsen)
Date: Mon, 26 Aug 2024 06:51:32 +0000
Subject: (java) Builds not reproducible on armhf
In-Reply-To: <ddcc127e-54fc-420a-8780-8813284c62a0@debian.org>
References: <4o6fsmm22y3j5kvbsgu5jbq3pujrmqr5eqengehifgv67mdzms@wzjqclzmtois>
 <ddcc127e-54fc-420a-8780-8813284c62a0@debian.org>
Message-ID: <Zswl9P0gLMeueAKE@layer-acht.org>

On Mon, Aug 26, 2024 at 06:46:35AM +0200, Mechtilde Stehmann wrote:
> > I'll note that fakeroot was probably broken on armel, armhf since
> > the t64 migration until mid-August.
> > Is fakeroot involved in any way? If so, it might make sense to
> > discard any results from that period.
> Thanks for your explanations.
> Are there plans to ran the tests again?

normally that happens automatically, but the armhf builders have been very 
sad for at least a week, but rather more like 4 weeks, and mid August ist
like a week ago.

https://tests.reproducible-builds.org/debian/index_performance.html shows
zero armhf builds in the last week and 245 per day in the last 4 weeks.


-- 
cheers,
	Holger

 ???????
 ???????  holger@(debian|reproducible-builds|layer-acht).org
 ???????  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ???

These heat waves aren?t tragedies, they?re crimes. The fossil fuel industry
knew decades ago that this is what their pollution was causing, so they
spent billions to lie to the public and block climate action.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240826/9162164e/attachment.sig>

From chris at reproducible-builds.org  Fri Aug 30 09:03:50 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Fri, 30 Aug 2024 10:03:50 +0100
Subject: =?UTF-8?Q?diffoscope_277_released_=F0=9F=92=A0?=
Message-ID: <172500827332.159604.9890513071593225252@copycat>

Hi,

The diffoscope maintainers are pleased to announce the release of
version 277 of diffoscope.

diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.

Version 277 includes the following changes:

  [ Sergei Trofimovich ]
  * Don't crash when attempting to hashing symlinks with targets that point to
    a directory.

## Download

Version 277 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:

   https://diffoscope.org/

? but source tarballs may be located here:

  https://diffoscope.org/archive/

The corresponding Docker image may be run via (for example):

  $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
      registry.salsa.debian.org/reproducible-builds/diffoscope a b


## Contribute

diffoscope is developed within the "Reproducible builds" effort.

  - Git repository
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Docker image, eg.
    registry.salsa.debian.org/reproducible-builds/diffoscope
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Issues and feature requests
    https://salsa.debian.org/reproducible-builds/diffoscope/issues

  - Contribution instructions (eg. to file an issue)
    https://reproducible-builds.org/contribute/salsa/


Regards,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From kpcyrd at archlinux.org  Sat Aug 31 18:51:26 2024
From: kpcyrd at archlinux.org (kpcyrd)
Date: Sat, 31 Aug 2024 20:51:26 +0200
Subject: whatsrc: Added live-bootstrap provenance data
Message-ID: <270a0687-4eff-4971-ab9e-f917d4dd1eff@archlinux.org>

Dear list,

I've added live-bootstrap <https://github.com/fosslinux/live-bootstrap> 
to the list of distros I import. On the following page:

https://whatsrc.org/artifact/sha256:b3a24de97a8fdbc835b9833169501030b8977031bcb54b3b3ac13740f846ab30

It now lists this source code is considered `zlib 1.2.13` in 
live-bootstrap, it was also seen in Gentoo, Guix, openSUSE and Wolfi OS.

 From their readme, live-bootstrap's objective is:

 > How can a usable Linux system be created with only human-auditable, 
and wherever possible, human-written, source code?

They also have a note about pre-processed source code archives:

 > GNU Guix is currently the furthest along project to automate 
bootstrapping. However, there are a number of non-auditable files used 
in many of their packages. Here is a list of file types that we deem 
unsuitable for bootstrapping.
 > [...]
 > 2. Any pre-generated configure scripts, or Makefile.in?s from autotools.

I did find instances of source code inputs that seem autotools 
pre-processed:

libffi 3.3: 
https://whatsrc.org/artifact/sha256:72fba7922703ddfa7a028d513ac15a85c8d54c8d67f55fa5a4802885dc652056
curl 8.5.0: 
https://whatsrc.org/artifact/sha256:42ab8db9e20d8290a3b633e7fbb3cec15db34df65fd1015ef8ac1e4723750eeb

But I strongly agree with the overall stance.

In total the following vendors are currently present in the database:

- alpine
- archlinux
- crates.io (partial)
- debian
- fedora
- gentoo
- guix
- homebrew
- kali
- live-bootstrap
- opensuse
- registry.yarnpkg.com (partial)
- ubuntu
- void
- wolfi
- yocto

cheers,
kpcyrd

From kpcyrd at archlinux.org  Sat Aug 31 19:01:59 2024
From: kpcyrd at archlinux.org (kpcyrd)
Date: Sat, 31 Aug 2024 21:01:59 +0200
Subject: postgres 17 bans intermediate build artifacts from their dist tarballs
Message-ID: <e3d232b6-184d-4e2d-9f56-1c6f608e3e52@archlinux.org>

Dear list,

I brought this up on irc but figured it's worth mentioning this on the 
mailing list too:

https://peter.eisentraut.org/blog/2024/08/13/the-new-postgresql-17-make-dist

They explicitly mention the XZ backdoor incident, and I'd like to highlight:

 > Anyway, with PostgreSQL 17, this is changed. The tarball generation is
 > still invoked by calling make dist, but that internally now calls git
 > archive. git archive packs the files belonging to a given Git commit
 > into a tar (or other) archive in a reproducible and verifiable way.
 > Therefore, if I now run make dist on a given commit (such as a release
 > tag), then I will get the exact same (bit-identical) tarball as the
 > next person.

Postgres 17 is currently in beta so I don't have any tarballs in 
whatsrc.org yet, but there's likely going to be a consensus around them 
among the different distros.

cheers,
kpcyrd

From andrius at stikonas.eu  Sat Aug 31 19:13:31 2024
From: andrius at stikonas.eu (Andrius =?utf-8?B?xaB0aWtvbmFz?=)
Date: Sat, 31 Aug 2024 20:13:31 +0100
Subject: whatsrc: Added live-bootstrap provenance data
In-Reply-To: <270a0687-4eff-4971-ab9e-f917d4dd1eff@archlinux.org>
References: <270a0687-4eff-4971-ab9e-f917d4dd1eff@archlinux.org>
Message-ID: <12502822.O9o76ZdvQC@starbook>

> Dear list,
> 
> I've added live-bootstrap <https://github.com/fosslinux/live-bootstrap>
> to the list of distros I import. On the following page:

Thanks!

> 
> https://whatsrc.org/artifact/sha256:b3a24de97a8fdbc835b9833169501030b8977031
> bcb54b3b3ac13740f846ab30
> 
> It now lists this source code is considered `zlib 1.2.13` in
> live-bootstrap, it was also seen in Gentoo, Guix, openSUSE and Wolfi OS.
> 
>  From their readme, live-bootstrap's objective is:
>  > How can a usable Linux system be created with only human-auditable,
> 
> and wherever possible, human-written, source code?
> 
> They also have a note about pre-processed source code archives:
>  > GNU Guix is currently the furthest along project to automate
> 
> bootstrapping. However, there are a number of non-auditable files used
> in many of their packages. Here is a list of file types that we deem
> unsuitable for bootstrapping.
> 
>  > [...]
>  > 2. Any pre-generated configure scripts, or Makefile.in?s from autotools.
> 
> I did find instances of source code inputs that seem autotools
> pre-processed:

Yes, at the moment we use upstream tarballs, so they do come with preprocessd 
autotools files. But we try to rebuild them, e.g. here
https://github.com/fosslinux/live-bootstrap/blob/
71ff0a0481992c79347a57f622f3f091a985f67a/steps/libffi-3.3/pass1.sh#L8
we run autoreconf -fi. Though I've also seen some thread recently that 
autoreconf -fi is not guaranteed to rebuild absolutely everything...

> 
> libffi 3.3:
> https://whatsrc.org/artifact/sha256:72fba7922703ddfa7a028d513ac15a85c8d54c8d
> 67f55fa5a4802885dc652056 curl 8.5.0:
> https://whatsrc.org/artifact/sha256:42ab8db9e20d8290a3b633e7fbb3cec15db34df6
> 5fd1015ef8ac1e4723750eeb
> 
> But I strongly agree with the overall stance.
> 
> In total the following vendors are currently present in the database:
> 
> - alpine
> - archlinux
> - crates.io (partial)
> - debian
> - fedora
> - gentoo
> - guix
> - homebrew
> - kali
> - live-bootstrap
> - opensuse
> - registry.yarnpkg.com (partial)
> - ubuntu
> - void
> - wolfi
> - yocto
> 
> cheers,
> kpcyrd

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20240831/c9216b6b/attachment.sig>