From herve.boutemy at free.fr  Fri Nov  1 06:58:56 2024
From: herve.boutemy at free.fr (=?ISO-8859-1?Q?Herv=E9?= Boutemy)
Date: Fri, 01 Nov 2024 07:58:56 +0100
Subject: Reproducible Builds by default in Maven 4.0.0-beta-5
Message-ID: <3542951.QJadu78ljV@orchid>

I'm happy to announce that we just released Maven 4.0..0-beta-5 with a very 
nice improvement: Reproducible Builds is enabled by default!!!

The story started during our Reproducible Builds Summit in Hamburg, where I 
created the objective in the issue tracker:
   https://issues.apache.org/jira/browse/MNG-8258

Back from the event, the important step was the discussion with the wider 
Maven community:
this change was very welcome by everybody, as we have a multi-year experience 
on having configured our builds for reproducibility, checking on every release 
of every component, and fixing our build when something weird is detected (yes, 
this happens a few times per year)

We're now pushing for Maven 4 to leave beta phase, as it is when this feature 
will be used at larger scale: but here we are, Reproducible Builds is becoming 
the default

Regards,

Herv?



From holger at layer-acht.org  Fri Nov  1 11:55:40 2024
From: holger at layer-acht.org (Holger Levsen)
Date: Fri, 1 Nov 2024 11:55:40 +0000
Subject: Reproducible Builds by default in Maven 4.0.0-beta-5
In-Reply-To: <3542951.QJadu78ljV@orchid>
References: <3542951.QJadu78ljV@orchid>
Message-ID: <ZyTBvDCSgwv_a3H_@layer-acht.org>

On Fri, Nov 01, 2024 at 07:58:56AM +0100, Herv? Boutemy wrote:
> I'm happy to announce that we just released Maven 4.0..0-beta-5 with a very 
> nice improvement: Reproducible Builds is enabled by default!!!

wheeeehooo, very nice!

& thanks for sharing this and the details! :)


-- 
cheers,
	Holger

 ???????
 ???????  holger@(debian|reproducible-builds|layer-acht).org
 ???????  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ???

The greatest danger in times of turbulence is not the turbulence;
it is to act with yesterdays logic. (Peter Drucker)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241101/084b847c/attachment.sig>

From olimpiu.pop at gmail.com  Mon Nov  4 13:16:52 2024
From: olimpiu.pop at gmail.com (olimpiu pop)
Date: Mon, 4 Nov 2024 15:16:52 +0200
Subject: Reproducible Builds by default in Maven 4.0.0-beta-5
In-Reply-To: <ZyTBvDCSgwv_a3H_@layer-acht.org>
References: <3542951.QJadu78ljV@orchid> <ZyTBvDCSgwv_a3H_@layer-acht.org>
Message-ID: <CAEoay9XFNyQVwcAurj+k9fB4whUwaROX88=ZNLZPnNNKmMw0Jw@mail.gmail.com>

Huh! Nice job, Herve!

I hope next year the planets align, and I will attend the RB Summit again.
Otherwise, you have all the fun without me :D.

Have a great week, everyone.

On Fri, Nov 1, 2024 at 1:55?PM Holger Levsen <holger at layer-acht.org> wrote:

> On Fri, Nov 01, 2024 at 07:58:56AM +0100, Herv? Boutemy wrote:
> > I'm happy to announce that we just released Maven 4.0..0-beta-5 with a
> very
> > nice improvement: Reproducible Builds is enabled by default!!!
>
> wheeeehooo, very nice!
>
> & thanks for sharing this and the details! :)
>
>
> --
> cheers,
>         Holger
>
>  ???????
>  ???????  holger@(debian|reproducible-builds|layer-acht).org
>  ???????  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
>  ???
>
> The greatest danger in times of turbulence is not the turbulence;
> it is to act with yesterdays logic. (Peter Drucker)
>


-- 
Olimpiu  Gh. Pop, MSc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241104/75713306/attachment.htm>

From dwheeler at dwheeler.com  Mon Nov  4 17:15:34 2024
From: dwheeler at dwheeler.com (David A. Wheeler)
Date: Mon, 4 Nov 2024 12:15:34 -0500
Subject: Reproducible Builds by default in Maven 4.0.0-beta-5
In-Reply-To: <3542951.QJadu78ljV@orchid>
References: <3542951.QJadu78ljV@orchid>
Message-ID: <FF9D2146-3BDE-4704-A2F2-7A06386776A2@dwheeler.com>



> On Nov 1, 2024, at 2:58 AM, Herv? Boutemy <herve.boutemy at free.fr> wrote:
> 
> I'm happy to announce that we just released Maven 4.0..0-beta-5 with a very 
> nice improvement: Reproducible Builds is enabled by default!!!

HUGE congrats! Nice job.

> ... this change was very welcome by everybody, as we have a multi-year experience 
> on having configured our builds for reproducibility, checking on every release 
> of every component, and fixing our build when something weird is detected (yes, 
> this happens a few times per year)

I'm impressed with not just the configuration, but the systemic *verification*.
That is fantastic.

Again, congrats, this is great to see.

--- David A. Wheeler


From gunner at aspirationtech.org  Mon Nov  4 17:57:33 2024
From: gunner at aspirationtech.org (Allen Gunn)
Date: Mon, 4 Nov 2024 09:57:33 -0800
Subject: Reproducible Builds Code of Conduct - Update
Message-ID: <fd11d875-6502-498b-89e4-1db297b52d0a@aspirationtech.org>

Hello RB friends,

We apologize it's been longer than we would have hoped since our last 
CoC update.

After much research and consultation with a range of RB and other folks, 
we are leaning toward adopting a text based on the Django code of 
conduct for the RB project.

Based on input from a number of you, and after reviewing more than 20 
other Codes of Conduct, we felt this the Django code of conduct struck 
the best balance of clarity and brevity, along with conveying a positive 
and constructive overall tone.

Here are the relevant links:

https://www.djangoproject.com/conduct/
https://www.djangoproject.com/conduct/faq/
https://www.djangoproject.com/conduct/reporting/
https://www.djangoproject.com/conduct/enforcement-manual/

Our plan is to draft an RB-specific version of the above pages, pending 
feedback from this list, and we'll share those drafts for review when 
they are committed to git.

We are deeply grateful to those of you have actively participated in 
reviewing the above, and offering other input and guidance along the way.

We request any comments and feedback by Friday 15 November.

We thank you in advance for any feedback you have time to provide.

peace,
gunner, for the Reproducible Builds Core Team

-- 

Allen Gunn
Executive Director, Aspiration
www.aspirationtech.org

Aspiration: "Better Tools for a Better World"

Read our Manifesto: https://aspirationtech.org/publications/manifesto

Follow us on...
Mastodon: https://mastodon.social/@aspirationtech
Bluesky:  https://bsky.app/profile/aspirationtech.bsky.social


From chris at reproducible-builds.org  Thu Nov  7 22:01:33 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Thu, 07 Nov 2024 14:01:33 -0800
Subject: Please review the draft for October's report
Message-ID: <173101663986.173575.17461195322518773399@copycat>

Hi all,

Please review the draft for October's Reproducible Builds report:

  https://reproducible-builds.org/reports/2024-10/?draft

? or, via the Git repository itself:

  https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2024-10.md

I intend to publish it no earlier than:

  $ date -d 'Sun, 09 Nov 2024 16:00:00 -0000'

  https://time.is/compare/1600_09_Nov_2024_in_UTC

?

Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2024-10.md" file in the "reproducible-website" repository:

  $ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
  $ cd reproducible-website
  $ sensible-editor _reports/2024-10.md

I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:

  https://reproducible-builds.org/contribute/salsa/


Regards,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From rclobus at rclobus.nl  Fri Nov  8 16:40:03 2024
From: rclobus at rclobus.nl (Roland Clobus)
Date: Fri, 8 Nov 2024 17:40:03 +0100
Subject: Reproducibility for Java
Message-ID: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>

Hello list,

I'm tracing a non-reproducibility issue in the Debian package 
ca-certificates-java for the Debian Junior live image.

It embeds timestamps for 'now' in /etc/ssl/certs/java/cacerts.
The proposed solution at [1] does not work, the deeper code has 'new 
Date()' in several places.

To solve the issue I have several options:
* Remove the offending file from the live image and generate it again at 
runtime -> makes the ISO image reproducible, but does not improve the 
infrastructure
* Run the command that generates this file in a faketime environment 
(the value for SOURCE_DATE_EPOCH is ignored)
* Propose changes to Java at higher level, i.e. fixing 
sun.security.provider.JavaKeyStore [2] at several places
* Propose changes to Java in JavaKeyStore to stop using 'now' but 
instead the timestamp of the files of the certificate
* Propose changes to Java at a deep level, i.e. fixing the class 
java.util.Date [3]

There are already some SOURCE_DATE_EPOCH uses in Java, but mainly for 
build Java and Javadoc [4].

What strategy would you propose? Should production runtime environments 
be sensitive to SOURCE_DATE_EPOCH (instead of during building)?

With kind regards,
Roland Clobus

[1] 
https://stackoverflow.com/questions/2001671/override-java-system-currenttimemillis-for-testing-time-sensitive-code
[2] 
https://sources.debian.org/src/openjdk-23/23.0.1+11-1/src/java.base/share/classes/sun/security/provider/JavaKeyStore.java/?hl=381#L381
[3] 
https://sources.debian.org/src/openjdk-23/23.0.1+11-1/src/java.base/share/classes/java/util/Date.java/?hl=162#L162
[4] 
https://codesearch.debian.net/search?q=package%3Aopenjdk-23+SOURCE_DATE_EPOCH

From chris at reproducible-builds.org  Fri Nov  8 19:50:48 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Fri, 08 Nov 2024 11:50:48 -0800
Subject: Reproducibility for Java
In-Reply-To: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
Message-ID: <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>

Roland Clobus wrote:

> Should production runtime environments 
> be sensitive to SOURCE_DATE_EPOCH (instead of during building)?

Starting with this bit first: given the context of your question is in
relation to building live images, I assume that installing the
ca-certificates-java package is the source of the non-reproducibility?
presumably via the postinst script doing some processing.

Indeed, yes, this seems to be the case:

  https://salsa.debian.org/java-team/ca-certificates-java/-/blob/master/debian/ca-certificates-java.postinst

If so, then I agree in general and specific terms: I believe that
preinst and postinst scripts should be deterministic, and I've filed
many such bugs in the past to make them so, chiefly in the process of
getting Tails reproducible.

> What strategy would you propose? [The package] embeds timestamps for
> 'now' in /etc/ssl/certs/java/cacerts.

Hm. Given that the codebase calls 'new Date()' in a bunch of places,
then I think that any of the options that propose changes to Java are
not going to be visible in the short- or medium-term because of the
time it would take for those changes to filter down to Debian. They
are, of course, worth pursuing; but I suspect you would also like a
stopgap situation as well.

Using faketime would of course 'work', but are you proposing that the
maintainer of the ca-certificates-java patch their postinst to always
use faketime? Otherwise, I am not sure how you would ensure that this
bit was called within the faketime environment only when building your
live image.

Now I do like your idea of not shipping the unreproducible file, and
it would be especially elegant if the package worked with or without
the file(s) being present. But I don't think that is the case: the
very point is that it generates these files in a known place on the
filesystem so that other programs can access them.

Similarly, I don't think this package has any broader concept of a
'first run' in which it could be generated if it doesn't exist. You
can't even be 100% sure that these files will only be accessed by a
Debian-shipped Java runtime.

But I do note that the update_cacerts method is called in that
postinst when a new Java runtime is installed. The very fact that this
is abstracted out is promising. I wonder if you could: (a) remove the
offending file as you outline; and then (b) call this very method
during the live script's boot, perhaps by manually invoking the dpkg
trigger that is meant to be for when a new JRE is installed?

Lastly: the package's maintainers may have a more elegant solution,
so it might be worth looping them in.


Best wishes,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From chris at reproducible-builds.org  Sat Nov  9 00:17:41 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Fri, 08 Nov 2024 16:17:41 -0800
Subject: =?UTF-8?Q?diffoscope_283_released_=F0=9F=92=A0?=
Message-ID: <173109673153.354494.5231095842914423578@copycat>

Hi,

The diffoscope maintainers are pleased to announce the release of
version 283 of diffoscope.

diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.

Version 283 includes the following changes:

  [ Martin Abente Lahaye ]
  * Fix crash when objdump is missing when checking .EFI files.

## Download

Version 283 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:

   https://diffoscope.org/

? but source tarballs may be located here:

  https://diffoscope.org/archive/

The corresponding Docker image may be run via (for example):

  $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
      registry.salsa.debian.org/reproducible-builds/diffoscope a b


## Contribute

diffoscope is developed within the "Reproducible builds" effort.

  - Git repository
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Docker image, eg.
    registry.salsa.debian.org/reproducible-builds/diffoscope
    https://salsa.debian.org/reproducible-builds/diffoscope

  - Issues and feature requests
    https://salsa.debian.org/reproducible-builds/diffoscope/issues

  - Contribution instructions (eg. to file an issue)
    https://reproducible-builds.org/contribute/salsa/


Regards,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From chris at reproducible-builds.org  Sun Nov 10 21:16:17 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Sun, 10 Nov 2024 13:16:17 -0800
Subject: Please review the draft for October's report
In-Reply-To: <173101663986.173575.17461195322518773399@copycat>
References: <173101663986.173575.17461195322518773399@copycat>
Message-ID: <20dd8385-2c8c-4fcf-951a-47955f84c217@app.fastmail.com>

Chris Lamb wrote:

> Please review the draft for October's Reproducible Builds report:

This has now been published ? thanks to all who contributed.

If possible, please share the following link:

  https://reproducible-builds.org/reports/2024-10/

.. and also consider retweeting:

  https://x.com/ReproBuilds/status/1855721046655328642


Regards,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From chris at reproducible-builds.org  Mon Nov 11 18:55:38 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Mon, 11 Nov 2024 10:55:38 -0800
Subject: Reproducible Builds in October 2024
Message-ID: <66f01be8-7adf-47c2-a695-826f9fa3a12b@app.fastmail.com>

--------------------------------------------------------------------
        o
      ?   ?      October 2024 in Reproducible Builds
     o     o
      ?   ?      https://reproducible-builds.org/reports/2024-10/
        o
--------------------------------------------------------------------


Welcome to the October 2024 report from the Reproducible Builds
project!

Our reports attempt to outline what we've been up to over the past
month, highlighting news items from elsewhere in tech where they are
related.

As ever, if you are interested in contributing to the project,
please visit our Contribute [1] page on our website.

 [0] https://reproducible-builds.org
 [1] https://reproducible-builds.org/contribute/

                                    ?

Table of contents:

 * Beyond bitwise equality for Reproducible Builds?
 * ?Two Ways to Trustworthy? at SeaGL 2024
 * Number of cores affected Android compiler output
 * On our mailing list
 * diffoscope
 * IzzyOnDroid passed 25% of reproducible apps
 * Distribution work
 * Website updates
 * Reproducibility testing framework
 * Supply-chain security at Open Source Summit EU
 * Upstream patches


                                    ?


Beyond bitwise equality for Reproducible Builds?
------------------------------------------------

Jens Dietrich, Tim White, of Victoria University of Wellington, New
Zealand along with Behnaz Hassanshahi and Paddy Krishnan of Oracle Labs
Australia published a paper entitled "Levels of Binary Equivalence for
the Comparison of Binaries from Alternative Builds [3]":

> The availability of multiple binaries built from the same sources
> creates new challenges and opportunities, and raises questions such
> as: ?Does build A confirm the integrity of build B?? or ?Can build A
> reveal a compromised build B??. To answer such questions requires a
> notion of equivalence between binaries. We demonstrate that the
> obvious approach based on bitwise equality has significant
> shortcomings in practice, and that there is value in opting for
> alternative notions. We conceptualise this by introducing levels of
> equivalence, inspired by clone detection types.

A PDF [4] of the paper is freely available.

 [3] https://doi.org/10.48550/arXiv.2410.08427
 [4] https://arxiv.org/pdf/2410.08427v1

                                    ?


"Two Ways to Trustworthy" at SeaGL 2024
---------------------------------------

On Friday 8th November, Vagrant Cascadian will present a talk entitled
"Two Ways to Trustworthy" [6] at SeaGL [5] in Seattle, WA.

Founded in 2013, SeaGL is a free, grassroots technical summit dedicated
to spreading awareness and knowledge about free source software,
hardware and culture. Vagrant's talk:

> [?] delves into how two project[s] approaches fundamental security
> features through Reproducible Builds, Bootstrappable Builds, code
> auditability, etc. to improve trustworthiness, allowing independent
> verification; trustworthy projects require little to no trust.
>
> Exploring the challenges that each project faces due to very
> different technical architectures, but also contextually relevant
> social structure, adoption patterns, and organizational history
> should provide a good backdrop to understand how different
> approaches to security might evolve, with real-world merits and
> downsides.

 [5] https://seagl.org/
 [6] https://pretalx.seagl.org/2024/talk/W73ACM/

                                    ?


Number of cores affected Android compiler output
------------------------------------------------

Fay Stegerman wrote [8] that the cause of the Android toolchain bug from
September's report [9] that she reported to the Android issue tracker
[10] has been found and the bug has been fixed.

> the D8 Java to DEX [11] compiler (part of the Android toolchain)
> eliminated a redundant field load if running the class's static
> initialiser was known to be free of side effects, which ended up
> accidentally depending on the sharding of the input, which is
> dependent on the number of CPU cores used during the build.

To make it easier to understand the bug and the patch, Fay also made a
small example [12] to illustrate when and why the optimisation
involved is valid.

 [8] https://tech.lgbt/@obfusk/113403959098151861
 [9] https://reproducible-builds.org/reports/2024-09/#android-toolchain-core-count-issue-reported
 [10] https://issuetracker.google.com/issues/366412380
 [11] https://source.android.com/docs/core/runtime/dex-format
 [12] https://gist.github.com/obfusk/83822140509dad4148b14bba41adf008

                                    ?


On our mailing list?
--------------------

On our mailing list [13] this month:

* Following-up to previous work, James Addison informed the list that
  the recently-released Sphinx [14] documentation generator includes
  improvements to the next copyright notice substitutions [15].

   [13] https://lists.reproducible-builds.org/listinfo/rb-general/
   [14] https://www.sphinx-doc.org/en/master/
   [15] https://lists.reproducible-builds.org/pipermail/rb-general/2024-October/003562.html

* Pol Dellaiera wrote to the list in order to seek advice around
  introducing the concept of reproducibility [16] to computer science
  Masters students at the University of Mons, Belgium [17].

   [16] https://lists.reproducible-builds.org/pipermail/rb-general/2024-October/003560.html
   [17] https://web.umons.ac.be/

* James Addison also followed up to a previous thread on
  "CONFIG_MODULE_SIG and the unreproducible Linux Kernel" [18] to add:
  "I wonder whether it would be possible to use the Linux kernel's
  Integrity Policy Enforcement [19] to deploy a policy that would
  prevent loading of anything except a set of expected kernel
  modules." [20]

   [18] https://lists.reproducible-builds.org/pipermail/rb-general/2024-September/003530.html
   [19] https://docs.kernel.org/admin-guide/LSM/ipe.html
   [20] https://lists.reproducible-builds.org/pipermail/rb-general/2024-October/003553.html

* There were also two informative replies from David Wheeler [21] to a
  broad-based discussion on Reproducible Builds being defined in
  various standards. [22][23]

 [21] https://dwheeler.com/
 [22] https://lists.reproducible-builds.org/pipermail/rb-general/2024-October/003550.html
 [23] https://lists.reproducible-builds.org/pipermail/rb-general/2024-October/003551.html

                                    ?


diffoscope
----------

diffoscope [25] is our in-depth and content-aware diff utility that can
locate and diagnose reproducibility issues. This month, Chris Lamb made
the following changes, including preparing and uploading versions 279,
280, 281 and 282 to Debian:

* Ignore errors when listing .ar archives (#1085257 [26]). [27]
* Don't try and test with systemd-ukify in the Debian stable
  distribution. [28]
* Drop Depends on the deprecated python3-pkg-resources (#1083362
  [29]). [30]

 [25] https://diffoscope.org
 [26] https://bugs.debian.org/1085257
 [27] https://salsa.debian.org/reproducible-builds/diffoscope/commit/e0e10c41
 [28] https://salsa.debian.org/reproducible-builds/diffoscope/commit/0736b361
 [29] https://bugs.debian.org/1083362
 [30] https://salsa.debian.org/reproducible-builds/diffoscope/commit/997c6adb

In addition, Jelle van der Waa added support for Unified Kernel Image
[31] (UKI) files. [32][33][34] Furthermore, Vagrant Cascadian updated
diffoscope in GNU Guix [35] to version 282. [36][37]

 [31] https://wiki.archlinux.org/title/Unified_kernel_image
 [32] https://salsa.debian.org/reproducible-builds/diffoscope/commit/9d5b5d32
 [33] https://salsa.debian.org/reproducible-builds/diffoscope/commit/2d7f54bf
 [34] https://salsa.debian.org/reproducible-builds/diffoscope/commit/153d6185
 [35] https://guix.gnu.org/
 [36] https://debbugs.gnu.org/74072
 [37] https://git.savannah.gnu.org/cgit/guix.git/commit/?id=d6f775c30c6f47e174f6110d1089edc6315600e4

                                    ?


IzzyOnDroid passed 25% reproducible apps
----------------------------------------

The IzzyOnDroid [39] project has reached a good milestone by reaching
over 25% [40] of the ~1,200 Android apps provided by their repository
(of official APKs built by the original application developers) having
been confirmed to be reproducible by a rebuilder [41].

 [39] https://apt.izzysoft.de/fdroid/
 [40] https://floss.social/@IzzyOnDroid/113350034406251501
 [41] https://codeberg.org/IzzyOnDroid/rbtlog

                                    ?


Distribution work
-----------------

In Debian this month:

* Holger Levsen uploaded devscripts version 2.24.2 [42], including many
  changes to the debootsnap, debrebuild and reproducible-check scripts.
  This is the first time that debrebuild actually works (using sbuild's
  unshare backend). As part of this, Holger also fixed an issue in the
  reproducible-check script where a typo in the code led to incorrect
  results [43]

   [42] https://tracker.debian.org/news/1581399/accepted-devscripts-2242-source-into-unstable/
   [43] https://salsa.debian.org/debian/devscripts/-/commit/4b3cf6bfbb3940700aab407879bf411c58b97847

* Recently, a news entry was added to snapshot.debian.org [44]'s
  homepage, describing the recent changes that made the system
  stable again:

    > The new server has no problems keeping up with importing the
    > full archives on every update, as each run finishes comfortably
    > in time before it's time to run again. [While] the new server is
    > the one doing all the importing of updated archives, the HTTP
    > interface [45] is being served by both the new server and one of
    > the VM's at LeaseWeb [46].

    The entry list a number of specific updates surrounding the API
    endpoints and rate limiting.

   [44] http://snapshot.debian.org/
   [45] https://snapshot.debian.org/
   [46] https://www.leaseweb.com/

* Lastly, 12 reviews of Debian packages were added, 3 were updated and
  18 were removed this month adding to our knowledge about identified
  issues [47].

   [47] https://tests.reproducible-builds.org/debian/index_issues.html

Elsewhere in distribution news, Zbigniew J?drzejewski-Szmek performed
another rebuild of Fedora [48] 42 packages, with the headline result
being that 91% of the packages are reproducible [49]. Zbigniew also
reported a reproducibility problem with QImage [50].

 [48] https://fedoraproject.org
 [49] https://in.waw.pl/~zbyszek/fedora/builds-f42-with-add-det-4.x.summary.txt
 [50] https://lists.fedoraproject.org/archives/list/devel at lists.fedoraproject.org/thread/67ICWGGE3TUPG5RH32GZAXICO4T5BXFG/

Finally, in openSUSE, Bernhard M. Wiedemann published another report
[51] for that distribution.

 [51] https://lists.opensuse.org/archives/list/factory at lists.opensuse.org/thread/NRT3XWO4ZRSIMAPSHD7HVSD5Z62WQWAA/

                                    ?


Website updates
---------------

There were an *enormous* number of improvements made to our website this
month, including:

* Alba Herrerias:

    * Improve consistency across distribution-specific guides. [52]
    * Fix a number of links on the "Contribute" page. [53]

     [52] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/12826f09
     [53] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/1274584d

* Chris Lamb:

    * Correct the name of Civil Infrastructure Platform [54] name and
      update image on the "Projects" [55] page. [56]
    * Update broken link on the "Value Initialization" [57] page. [58]
    * Try and make pipeline/branch builds of the website easier to
      browse. [59][60][61][62]

     [54] https://www.cip-project.org/
     [55] https://reproducible-builds.org/who/projects/
     [56] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/e53ecbc7
     [57] https://reproducible-builds.org/docs/value-initialization/
     [58] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/56f708d7
     [59] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/8bff7574
     [60] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/df01bf5f
     [61] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a3faf5be
     [62] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/0c735ea6

* hulkoba

    * Contribute to the new 'Success stories [63]' page. [64]

     [63] https://reproducible-builds.org/success-stories/
     [64] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/4ca4e410

* James Addison:

    * Huge and significant work on a (as-yet-merged) quickstart guide
      to be linked from the homepage [65][66][67][68][69]
    * On the homepage [70], link directly to the Projects [71]
      subpage. [72]
    * Relocate "dependency-drift" notes to the Volatile inputs [73]
      page. [74]

     [65] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/30d226e0
     [66] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/6ccad0f4
     [67] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/aeb73a4a
     [68] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/5ee3ac46
     [69] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/8e8f7d55
     [70] https://reproducible-builds.org/
     [71] https://reproducible-builds.org/who/projects
     [72] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/ca047d2e
     [73] https://reproducible-builds.org/docs/volatile-inputs/
     [74] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/63d58e09

* Ninette Adhikari:

    * Add a brand new 'Success stories [75]' page that "highlights the
      success stories of Reproducible Builds, showcasing real-world
      examples of projects shipping with verifiable, reproducible
      builds". [76][77][78][79][80][81]

     [75] https://reproducible-builds.org/success-stories/
     [76] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/12f4df01
     [77] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/ddc6df7c
     [78] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/6b3dba82
     [79] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/19a17974
     [80] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/28d82a04
     [81] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/115ed658

* Pol Dellaiera:

    * Update the website's README page for building the website under
      NixOS [82]. [83][84][85][86][87]
    * Add a new academic paper citation. [88]

     [82] https://nixos.org/
     [83] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/5428366d
     [84] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/f90aba5c
     [85] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/13a338ab
     [86] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/b9e51c38
     [87] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/39598567
     [88] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/62aa449b

Lastly, Holger Levsen filed an extensive issue detailing a request to
create an overview of recommendations and standards [89] in relation to
reproducible builds.

 [89] https://salsa.debian.org/reproducible-builds/reproducible-website/-/issues/59

                                    ?


Reproducibility testing framework
---------------------------------

The Reproducible Builds project operates a comprehensive testing
framework running primarily at tests.reproducible-builds.org [90] in
order to check packages and other artifacts for reproducibility. In
October, a number of changes were made by Holger Levsen, including:

* Add a basic index.html for rebuilderd. [91]
* Update the nginx.conf configuration file for rebuilderd. [92]
* Document how to use a rescue system for Infomaniak's OpenStack
  cloud. [93]
* Update usage info for two particular nodes. [94]
* Fix up a version skew check to fix the name of the riscv64
  architecture. [95]
* Update the rebuilderd-related TODO. [96]

   [90] https://tests.reproducible-builds.org
   [91] https://salsa.debian.org/qa/jenkins.debian.net/commit/2fdfe56c1
   [92] https://salsa.debian.org/qa/jenkins.debian.net/commit/e43de52e2
   [93] https://salsa.debian.org/qa/jenkins.debian.net/commit/254f86399
   [94] https://salsa.debian.org/qa/jenkins.debian.net/commit/fbf8d89a4
   [95] https://salsa.debian.org/qa/jenkins.debian.net/commit/8eae72e56
   [96] https://salsa.debian.org/qa/jenkins.debian.net/commit/6a383397b

In addition, Mattia Rizzolo added a new IP address for the inos5 node
[97] and Vagrant Cascadian brought 4 virt nodes back online [98].

 [97] https://salsa.debian.org/qa/jenkins.debian.net/commit/5cea8f4a8
 [98] https://salsa.debian.org/qa/jenkins.debian.net/commit/114838df4

                                    ?


Supply-chain security at Open Source Summit EU
----------------------------------------------

The Open Source Summit EU [100] took place recently, and covered plenty
of topics related to supply-chain security, including:

* Public Sector & OpenSSF: Principles for Package Repository
  Security [101]
* The Model Openness Framework: Promoting Completeness and Openness for
  Reproducibility, Transparency and Usability in AI [102]
* Structured Scorecard Results: Tailor Your Own Supply-Chain Security
  Policies [103]
* Lightning Talk: Elephant in the Room: How Supply Chain Security
  Standards Are Not Standard and What to Do About It [104]
* Lightning Talk: Charting the Course for Secure Software Supply Chain
  with Guac-AI-Mole! [105]
* TPMs, Merkle Trees and TEEs: Enhancing SLSA with Hardware-Assisted
  Build Environment Verification [106]
* Accountability Taxonomy for AI Software Bill of Materials [107]
* Securing Your Supply Chain with an Open Source Ecosystem [108]
* OSS Supply Chain Threats and Why You Need a Holistic Security
  Strategy [109]
* A Step Closer to in-Toto?lly Secure: Using in-Toto and OPA Gatekeeper
  to Verify Artifact Integrity [110]
* Panel Discussion: Improving Supply Chain Integrity with OpenSSF
  Technologies [111]
* Case Study: 10+ Years of Developing an SBOM System and the Dos and
  Don?ts [112]
* SBOM in SaaS Environments: An Update [113]
* Securing Git Repositories with Gittuf [114]

 [100] https://events.linuxfoundation.org/open-source-summit-europe/
 [101] https://www.youtube.com/watch?v=EyzFZYeSj5g&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=124
 [102] https://www.youtube.com/watch?v=-GFcUgT77oE&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=114
 [103] https://www.youtube.com/watch?v=ZT3XdMF6U5A&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=106
 [104] https://www.youtube.com/watch?v=ICrlIlWAiGA&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=103
 [105] https://www.youtube.com/watch?v=mHjsaDDkbKo&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=102
 [106] https://www.youtube.com/watch?v=Gk0LDi05KRg&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=100
 [107] https://www.youtube.com/watch?v=nSQ3rsaqpaQ&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=47
 [108] https://www.youtube.com/watch?v=154gKafXhnc&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=33
 [109] https://www.youtube.com/watch?v=cLPZ7dYndH0&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=30
 [110] https://www.youtube.com/watch?v=b_ImE70Vhd8&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=28
 [111] https://www.youtube.com/watch?v=6EPROzPfqD8&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=26
 [112] https://www.youtube.com/watch?v=1LTqB4czzEs&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=142
 [113] https://www.youtube.com/watch?v=4rA9JOESvL8&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=182
 [114] https://www.youtube.com/watch?v=eCSeIEdMbCw&list=PLbzoR-pLrL6poagnac0dQuTXcmNvUHVOj&index=179

                                    ?


Upstream patches
----------------

The Reproducible Builds project detects, dissects and attempts to fix as
many currently-unreproducible packages as possible. We endeavour to send
all of our patches upstream where appropriate. This month, we wrote a
large number of such patches, including:

* Bernhard M. Wiedemann

    * apache-ivy [115] (.zip modification time)
    * ccache [116] (build failure)
    * colord [117] (CPU)
    * efivar [118] (CPU/march=native)
    * gsl [119] (no check)
    * libcamera [120] (date/copyright year)
    * libreoffice [121] (possible rpm/build toolchain corruption bug)
    * moto [122] (.gz modification time)
    * openssl-1_1 [123] (date-related issue)
    * python-pygraphviz [124] (benchmark)
    * sphinx/python-pygraphviz [125] (benchmark)
    * python-panel [126] (package.lock has random port)
    * python-propcache [127] (random temporary path)
    * python314 [128] (.gz-related modification time)
    * rusty_v8 [129] (random .o files)
    * scapy [130] (date)
    * wine [131] (parallelism)
    * ibmtss [132] (FTBFS-2026 [133])
    * pymol [134] (date)
    * pandas [135] (ASLR)
    * linutil [136] (drop date)
    * lsof [137] (also filed in openSUSE [138]: uname -r in LSOF_VSTR)
    * schily [139] (also filed in openSUSE [140]: uname -r)
    * superlu [141] (nocheck)
    * util [142] (random test failure)
    * ceph [143] (year-2038 variation from embedded boost)

     [115] https://build.opensuse.org/request/show/1206032
     [116] https://github.com/ccache/ccache/pull/1525
     [117] https://github.com/hughsie/colord/issues/174
     [118] https://bugzilla.opensuse.org/show_bug.cgi?id=1231368
     [119] https://build.opensuse.org/request/show/1206278
     [120] https://lists.libcamera.org/pipermail/libcamera-devel/2024-October/045731.html
     [121] https://bugzilla.opensuse.org/show_bug.cgi?id=1231580
     [122] https://github.com/getmoto/moto/pull/8218
     [123] https://bugzilla.opensuse.org/show_bug.cgi?id=1231667
     [124] https://github.com/pygraphviz/pygraphviz/pull/544
     [125] https://github.com/sphinx-gallery/sphinx-gallery/pull/1385
     [126] https://bugzilla.opensuse.org/show_bug.cgi?id=1231254
     [127] https://build.opensuse.org/request/show/1207574
     [128] https://github.com/python/cpython/pull/125261
     [129] https://bugzilla.opensuse.org/show_bug.cgi?id=1231548
     [130] https://build.opensuse.org/request/show/1205217
     [131] https://bugzilla.opensuse.org/show_bug.cgi?id=1231620
     [132] https://github.com/kgoldman/ibmtss/commit/3a17ac01bea73d3568272d61b895a16a0bd85440
     [133] https://sourceforge.net/p/ibmtpm20tss/tickets/49/
     [134] https://github.com/schrodinger/pymol-open-source/pull/404
     [135] https://github.com/pandas-dev/pandas/issues/60078
     [136] https://github.com/ChrisTitusTech/linutil/pull/869
     [137] https://build.opensuse.org/request/show/1218747
     [138] https://bugzilla.opensuse.org/show_bug.cgi?id=1232425
     [139] https://codeberg.org/schilytools/schilytools/pulls/81
     [140] https://bugzilla.opensuse.org/show_bug.cgi?id=1232434
     [141] https://bugzilla.opensuse.org/show_bug.cgi?id=1232550
     [142] https://github.com/util-linux/util-linux/issues/3259
     [143] https://tracker.ceph.com/issues/68778

* Chris Lamb:

    * #1085097 [144] filed against python-roborock [145].
    * #1085280 [146] filed against pywayland [147].
    * #1085283 [148] filed against readsb [149].
    * #1085381 [150] filed against xraylarch [151].

     [144] https://bugs.debian.org/1085097
     [145] https://tracker.debian.org/pkg/python-roborock
     [146] https://bugs.debian.org/1085280
     [147] https://tracker.debian.org/pkg/pywayland
     [148] https://bugs.debian.org/1085283
     [149] https://tracker.debian.org/pkg/readsb
     [150] https://bugs.debian.org/1085381
     [151] https://tracker.debian.org/pkg/xraylarch

* James Addison:

    * #1085112 [152] filed against distro-info [153].

     [152] https://bugs.debian.org/1085112
     [153] https://tracker.debian.org/pkg/distro-info

* Zbigniew J?drzejewski-Szmek:

    * calibre [154] (two sort issues) [155][156]

     [154] https://github.com/kovidgoyal/calibre
     [155] https://github.com/kovidgoyal/calibre/pull/2483
     [156] https://github.com/kovidgoyal/calibre/pull/2484


?

Finally, If you are interested in contributing to the Reproducible
Builds project, please visit our "Contribute" [157] page on our website.
However, you can get in touch with us via:

 * IRC: #reproducible-builds on irc.oftc.net.

 * Mastodon: @reproducible_builds at fosstodon.org [158]

 * Mailing list: rb-general at lists.reproducible-builds.org [159]

 * Twitter: @ReproBuilds [160]

 [157] https://reproducible-builds.org/contribute/
 [158] https://fosstodon.org/@reproducible_builds
 [159] https://lists.reproducible-builds.org/listinfo/rb-general
 [160] https://twitter.com/ReproBuilds


-- 
      o
    ?   ?
   o     o     reproducible-builds.org ?
    ?   ?
      o


From jbglaw at lug-owl.de  Tue Nov 12 08:26:13 2024
From: jbglaw at lug-owl.de (Jan-Benedict Glaw)
Date: Tue, 12 Nov 2024 09:26:13 +0100
Subject: NetBSD Reproducibility Report #5
Message-ID: <20241112082613.GA29897@lug-owl.de>

Hi!

As the building-on-modern-Linux-systems issues should now all be solved, I was
running a new round of reproducibility tests for NetBSD, resulting in [0].
Indeed, those issues are all fixed, just one new problem showed up
(cross-building from Linux) after some MOP rework for the VAX port.

  On Linux, of 82 of all tested 94 port/arch combinations built successfully,
with 78 building reproducible on two consecutive builds. Exceptions are the
alpha/alpha, mac68k/m68k, macppc/powerpc and zaurus/earm ports.

  Building on NetBSD current, 83 (of 94) combinations build successfully, of
those 68 were reproducible. Issues were seen with algor/mipsel,
dreamcast/sh3el, evbarm/earmv4, evbarm/earmv4eb, evbarm/earmv6eb,
evbmips/mips64eb, evbmips/mips64el, evbmips/mipseb, evbmips/mipsel,
evbppc/powerpc, evbsh3/sh3el, iyonix/earm, mac68k/m68k, macppc/powerpc,
riscv/riscv32 and zaurus/earm.

  44 (of 94) port/arch combinations are totally reproducible, creating
bit-identical output on NetBSD and Linux. (The biggest impact here are CTF
debug infos, which are generated differently right now, so there seems to
be a remaining host dependency. The remaining issues are usually things like
permission bits on symlinks or other individual issues within bootloaders or
early boot code resp. embedded filesystem images.)

MfG, JBG

[0] http://toolchain.lug-owl.de/reports/netbsd-reproducibility-overview-5.html

-- 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241112/ff0907d6/attachment.sig>

From rclobus at rclobus.nl  Tue Nov 12 11:41:43 2024
From: rclobus at rclobus.nl (Roland Clobus)
Date: Tue, 12 Nov 2024 12:41:43 +0100
Subject: Reproducibility for Java
In-Reply-To: <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
 <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
Message-ID: <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>

Hello Chris, list,

On 08/11/2024 20:50, Chris Lamb wrote:
> Roland Clobus wrote:
> 
>> Should production runtime environments
>> be sensitive to SOURCE_DATE_EPOCH (instead of during building)?
> 
> Starting with this bit first: given the context of your question is in
> relation to building live images, I assume that installing the
> ca-certificates-java package is the source of the non-reproducibility?
> presumably via the postinst script doing some processing.
> 
> Indeed, yes, this seems to be the case:
> 
>    https://salsa.debian.org/java-team/ca-certificates-java/-/blob/master/debian/ca-certificates-java.postinst
> 
> If so, then I agree in general and specific terms: I believe that
> preinst and postinst scripts should be deterministic, and I've filed
> many such bugs in the past to make them so, chiefly in the process of
> getting Tails reproducible.

Indeed, the source of the non-reproducible file is coming from the 
postinst step of ca-certificates-java. However, none of the code from 
that package needs adjustment, the fix needs to be applied to openJDK 
(as the 'new Date()' calls are in openJDK itself.

>> What strategy would you propose? [The package] embeds timestamps for
>> 'now' in /etc/ssl/certs/java/cacerts.
> 
> Hm. Given that the codebase calls 'new Date()' in a bunch of places,
> then I think that any of the options that propose changes to Java are
> not going to be visible in the short- or medium-term because of the
> time it would take for those changes to filter down to Debian. They
> are, of course, worth pursuing; but I suspect you would also like a
> stopgap situation as well.

Changing Java (just for Debian) will possibly have undesired, 
unmaintainable side-effects, which I cannot oversee, so those need to be 
tested and approved by upstream.

But a stopgap is required too, to get a reproducible live image sooner.

> Using faketime would of course 'work', but are you proposing that the
> maintainer of the ca-certificates-java patch their postinst to always
> use faketime? Otherwise, I am not sure how you would ensure that this
> bit was called within the faketime environment only when building your
> live image.

After the regular postinst has run, I can run the postinst step again 
but then with faketime active.
There is sufficient control over the order of the installation of 
packages, that it is possible to ensure that the faketime-based version 
will be embedded in the live image.
This will fulfil the need in the live image, but will not be a general 
solution.

Generally, it would be possible to use faketime in the postinst, with 
the same value as SOURCE_DATE_EPOCH.
I would propose faketime as a 'Suggests:' dependency for the package. 
Then only when _both_ faketime is installed _and_ SOURCE_DATE_EPOCH is 
set, the faketime will be applied.

> Now I do like your idea of not shipping the unreproducible file, and
> it would be especially elegant if the package worked with or without
> the file(s) being present. But I don't think that is the case: the
> very point is that it generates these files in a known place on the
> filesystem so that other programs can access them.
  >
> Similarly, I don't think this package has any broader concept of a
> 'first run' in which it could be generated if it doesn't exist. You
> can't even be 100% sure that these files will only be accessed by a
> Debian-shipped Java runtime.
  >
> But I do note that the update_cacerts method is called in that
> postinst when a new Java runtime is installed. The very fact that this
> is abstracted out is promising. I wonder if you could: (a) remove the
> offending file as you outline; and then (b) call this very method
> during the live script's boot, perhaps by manually invoking the dpkg
> trigger that is meant to be for when a new JRE is installed?

The package live-config is the package to use for such first-run steps 
as (re)generating files.

As the stopgap method, I'll go with the faketime version of the postinst 
step, as that will cause no additional startup-delay for the live image. [1]

> Lastly: the package's maintainers may have a more elegant solution,
> so it might be worth looping them in.

I'll send a mail to the maintainers and debian-java later.

Thanks for thinking along,
With kind regards,
Roland

[1] https://salsa.debian.org/live-team/live-build/-/merge_requests/385
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241112/e540362a/attachment.sig>

From magnus.ihse.bursie at oracle.com  Tue Nov 12 14:33:26 2024
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 12 Nov 2024 15:33:26 +0100
Subject: Reproducibility for Java
In-Reply-To: <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
 <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
 <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
Message-ID: <0a70fcf0-f4b7-46a1-a20e-947cc62832ed@oracle.com>

Hi,

I'm working with the Build Group on OpenJDK. We have tried to make the 
build of the JDK to be reproducible, as a long-term project spanning 
several years. This has, at time, included changes to Java itself (like 
a way to make .properties files without a timestamp comment).

The builds we produce in the OpenJDK project, on Linux, is fully 
reproducible as far as I am aware. I must admit I did not follow all 
links in your mail to fully understand the problem you are seeing, but 
it seems that the gist is "the problem is in our distribution, but the 
fix must lie within Java itself".

If indeed a fix is needed in Java to enable reproducibility, I can 
probably help to try and drive such a change. However, I need to 
understand the problem better first, to determine if I agree that a fix 
in the JDK itself is in place.

Can you indulge me by writing down a clear problem statement in a mail 
to this list, on why reproducibility fails in your build, what change 
you want to make in the JDK, and why you think this will solve the 
problem? I'm sorry to put the burden on you, but the Build Group is 
unfortunately understaffed and I do not have much time to work with 
"non-essential" stuff (like reproducibility) as I'd like.

/Magnus



On 2024-11-12 12:41, Roland Clobus wrote:
> Hello Chris, list,
>
> On 08/11/2024 20:50, Chris Lamb wrote:
>> Roland Clobus wrote:
>>
>>> Should production runtime environments
>>> be sensitive to SOURCE_DATE_EPOCH (instead of during building)?
>>
>> Starting with this bit first: given the context of your question is in
>> relation to building live images, I assume that installing the
>> ca-certificates-java package is the source of the non-reproducibility?
>> presumably via the postinst script doing some processing.
>>
>> Indeed, yes, this seems to be the case:
>>
>> https://salsa.debian.org/java-team/ca-certificates-java/-/blob/master/debian/ca-certificates-java.postinst
>>
>> If so, then I agree in general and specific terms: I believe that
>> preinst and postinst scripts should be deterministic, and I've filed
>> many such bugs in the past to make them so, chiefly in the process of
>> getting Tails reproducible.
>
> Indeed, the source of the non-reproducible file is coming from the 
> postinst step of ca-certificates-java. However, none of the code from 
> that package needs adjustment, the fix needs to be applied to openJDK 
> (as the 'new Date()' calls are in openJDK itself.
>
>>> What strategy would you propose? [The package] embeds timestamps for
>>> 'now' in /etc/ssl/certs/java/cacerts.
>>
>> Hm. Given that the codebase calls 'new Date()' in a bunch of places,
>> then I think that any of the options that propose changes to Java are
>> not going to be visible in the short- or medium-term because of the
>> time it would take for those changes to filter down to Debian. They
>> are, of course, worth pursuing; but I suspect you would also like a
>> stopgap situation as well.
>
> Changing Java (just for Debian) will possibly have undesired, 
> unmaintainable side-effects, which I cannot oversee, so those need to 
> be tested and approved by upstream.
>
> But a stopgap is required too, to get a reproducible live image sooner.
>
>> Using faketime would of course 'work', but are you proposing that the
>> maintainer of the ca-certificates-java patch their postinst to always
>> use faketime? Otherwise, I am not sure how you would ensure that this
>> bit was called within the faketime environment only when building your
>> live image.
>
> After the regular postinst has run, I can run the postinst step again 
> but then with faketime active.
> There is sufficient control over the order of the installation of 
> packages, that it is possible to ensure that the faketime-based 
> version will be embedded in the live image.
> This will fulfil the need in the live image, but will not be a general 
> solution.
>
> Generally, it would be possible to use faketime in the postinst, with 
> the same value as SOURCE_DATE_EPOCH.
> I would propose faketime as a 'Suggests:' dependency for the package. 
> Then only when _both_ faketime is installed _and_ SOURCE_DATE_EPOCH is 
> set, the faketime will be applied.
>
>> Now I do like your idea of not shipping the unreproducible file, and
>> it would be especially elegant if the package worked with or without
>> the file(s) being present. But I don't think that is the case: the
>> very point is that it generates these files in a known place on the
>> filesystem so that other programs can access them.
> ?>
>> Similarly, I don't think this package has any broader concept of a
>> 'first run' in which it could be generated if it doesn't exist. You
>> can't even be 100% sure that these files will only be accessed by a
>> Debian-shipped Java runtime.
> ?>
>> But I do note that the update_cacerts method is called in that
>> postinst when a new Java runtime is installed. The very fact that this
>> is abstracted out is promising. I wonder if you could: (a) remove the
>> offending file as you outline; and then (b) call this very method
>> during the live script's boot, perhaps by manually invoking the dpkg
>> trigger that is meant to be for when a new JRE is installed?
>
> The package live-config is the package to use for such first-run steps 
> as (re)generating files.
>
> As the stopgap method, I'll go with the faketime version of the 
> postinst step, as that will cause no additional startup-delay for the 
> live image. [1]
>
>> Lastly: the package's maintainers may have a more elegant solution,
>> so it might be worth looping them in.
>
> I'll send a mail to the maintainers and debian-java later.
>
> Thanks for thinking along,
> With kind regards,
> Roland
>
> [1] https://salsa.debian.org/live-team/live-build/-/merge_requests/385

From rclobus at rclobus.nl  Tue Nov 12 16:55:07 2024
From: rclobus at rclobus.nl (Roland Clobus)
Date: Tue, 12 Nov 2024 17:55:07 +0100
Subject: Reproducibility for Java -> technical details
In-Reply-To: <0a70fcf0-f4b7-46a1-a20e-947cc62832ed@oracle.com>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
 <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
 <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
 <0a70fcf0-f4b7-46a1-a20e-947cc62832ed@oracle.com>
Message-ID: <d70fc9df-c27e-45c5-a7f0-ba5a374fcc0d@rclobus.nl>

Hello Magnus,

Thanks for your interest.

On 12/11/2024 15:33, Magnus Ihse Bursie wrote:
> I'm working with the Build Group on OpenJDK. We have tried to make the 
> build of the JDK to be reproducible, as a long-term project spanning 
> several years. This has, at time, included changes to Java itself (like 
> a way to make .properties files without a timestamp comment).
> 
> The builds we produce in the OpenJDK project, on Linux, is fully 
> reproducible as far as I am aware. I must admit I did not follow all 
> links in your mail to fully understand the problem you are seeing, but 
> it seems that the gist is "the problem is in our distribution, but the 
> fix must lie within Java itself".

Building Java (the binary program that executes Java code) is 
reproducible (for amd64 on Debian) [5], but running programs written in 
Java might not be, as is the case here for the Debian 
ca-certificates-java package.

> If indeed a fix is needed in Java to enable reproducibility, I can 
> probably help to try and drive such a change. However, I need to 
> understand the problem better first, to determine if I agree that a fix 
> in the JDK itself is in place.
> 
> Can you indulge me by writing down a clear problem statement in a mail 
> to this list, on why reproducibility fails in your build, what change 
> you want to make in the JDK, and why you think this will solve the 
> problem? I'm sorry to put the burden on you, but the Build Group is 
> unfortunately understaffed and I do not have much time to work with 
> "non-essential" stuff (like reproducibility) as I'd like.

The package ca-certificates-java is a tool that (re)generates 
/etc/ssl/cert/java/cacerts. It uses java.security.KeyStore [1], which in 
turn uses sun.security.provider.JavaKeyStore [2], which at lines 290, 
353 and 394 calls 'new Date()'.
The default constructor of java.util.Date [3] uses 
System.currentTimeMillis().
System.currentTimeMillis() is marked as @IntrinsicCandidate (as well as 
the more precise System.nanoTime().

My proposed change would be:
* At startup of Java, check if SOURCE_DATE_EPOCH is set. If so, create a 
Clock.fixed with the timestamp from SOURCE_DATE_EPOCH
* In java.util.Date replace System.currentTimeMillis() with 
Clock.getInstance().currentTimeMillis() (as suggested at [4] and other 
sources)
* Something similar might be required for System.nanoTime, but I didn't 
need that for this specific case.

It should be noted that regular user _will not_ and _should not_ set 
SOURCE_DATE_EPOCH [6]. That environment variable it typically used for 
rebuilds.

With kind regards,
Roland Clobus

[1] 
https://sources.debian.org/src/ca-certificates-java/20240118/src/main/java/org/debian/security/KeyStoreHandler.java/#L43
[2] 
https://sources.debian.org/src/openjdk-23/23.0.1+11-1/src/java.base/share/classes/sun/security/provider/JavaKeyStore.java/?hl=381#L381
[3] 
https://sources.debian.org/src/openjdk-23/23.0.1+11-1/src/java.base/share/classes/java/util/Date.java/?hl=128#L128
[4] 
https://stackoverflow.com/questions/2001671/override-java-system-currenttimemillis-for-testing-time-sensitive-code
[5] 
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/openjdk-23.html
[6] https://reproducible-builds.org/docs/source-date-epoch/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241112/d8301eb0/attachment.sig>

From vivia at kth.se  Tue Nov 12 11:32:11 2024
From: vivia at kth.se (Vivi Andersson)
Date: Tue, 12 Nov 2024 18:32:11 +0700
Subject: Geth Rebuild: Reproducibility Case Study in Go
Message-ID: <6a85b5db-c8eb-4e69-8606-2e676621f00f@kth.se>

Hi All,

I recently worked on reproducibility for the Ethereum client Geth 
<https://github.com/ethereum/go-ethereum/> and wanted to share some 
insights from this work.


Although the Go toolchain provides reproducibility primitives, achieving 
fully reproducible builds in practice is still challenging.

In our work with Geth, we found three primary reasons for this:

 1.

    *CGO Complexity*
    Integrating C code in Go introduces complexity, here resulting in
    unreproducible builds. Specifically, we?ve observed inconsistent
    build IDs (|.note.go.buildid| and |.note.gnu.build-id|), likely due
    to system-specific paths.

 2.

    *Project-Specific Build Configurations*
    Custom build scripts and environment settings can create subtle
    issues. For instance, inconsistencies arise when embedding metadata
    with |-ldflag -X|, or when shared build states affect the |go
    buildinfo| across jobs.

 3.

    *Software Bugs*
    As known, reproducibility efforts also reveal software bugs. We
    identified two ones: an issue
    <https://github.com/golang/go/issues/67011> with Go?s |trimpath|
    command and a bug
    <https://travis-ci.community/t/travis-uses-unexpected-ubuntu-distribution>
    in the Travis CI client leading to inconsistent GCC versions when
    building.


For details, see *Chapter 5* in Geth Rebuild: Verifiable Builds for Go 
Ethereum 
<https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1908608&dswid=445>.


*Takeaway:*

The reproducibility effort in Go still requires manual intervention and 
community support to succeed, despite the existing primitives.

In my experience, ease of adoption for developers is essential for 
reproducible builds to be implemented as a security mechanism. 
Therefore, it would be valuable to explore the broader reproducibility 
state and remaining challenges to reproducible builds in Go, eg. through 
a Go-specific rebuilder project.


Any thoughts are welcome!

-- 
Vivi Andersson
PhD Student, Department of Theoretical Computer Science
KTH Royal Institute of Technology
Stockholm, Sweden
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241112/40895d5d/attachment.htm>

From gnu at toad.com  Tue Nov 12 19:57:06 2024
From: gnu at toad.com (John Gilmore)
Date: Tue, 12 Nov 2024 11:57:06 -0800
Subject: Reproducibility for Java -> technical details 
In-Reply-To: <d70fc9df-c27e-45c5-a7f0-ba5a374fcc0d@rclobus.nl> 
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
 <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
 <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
 <0a70fcf0-f4b7-46a1-a20e-947cc62832ed@oracle.com>
 <d70fc9df-c27e-45c5-a7f0-ba5a374fcc0d@rclobus.nl>
Message-ID: <24944.1731441426@hop.toad.com>

Roland Clobus <rclobus at rclobus.nl> wrote:
> My proposed change would be:
> * At startup of Java, check if SOURCE_DATE_EPOCH is set. If so, create
> a Clock.fixed with the timestamp from SOURCE_DATE_EPOCH
> * In java.util.Date replace System.currentTimeMillis() with
> Clock.getInstance().currentTimeMillis() (as suggested at [4] and other
> sources)

The clock functions in Java shouldn't stop working because somebody set
a common environment variable.  This is hitting a fly with a
sledgehammer.

> It should be noted that regular user _will not_ and _should not_ set
> SOURCE_DATE_EPOCH [6]. That environment variable it typically used for
> rebuilds.

Certainly many programmers will set the variable, particularly during
development or debugging sessions.  And they will and should expect the
ordinary programs that they run in that shell to keep working -- whether
they are written in Java or not.

A much less intrusive fix would be to provide a date option on
ca-certificates-java that would override the current date with
a date specified on the command line.

	John
	

From vagrant at reproducible-builds.org  Tue Nov 12 23:19:39 2024
From: vagrant at reproducible-builds.org (Vagrant Cascadian)
Date: Tue, 12 Nov 2024 15:19:39 -0800
Subject: Reproducibility for Java -> technical details
In-Reply-To: <24944.1731441426@hop.toad.com>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
 <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
 <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
 <0a70fcf0-f4b7-46a1-a20e-947cc62832ed@oracle.com>
 <d70fc9df-c27e-45c5-a7f0-ba5a374fcc0d@rclobus.nl>
 <24944.1731441426@hop.toad.com>
Message-ID: <87cyj02g6s.fsf@yucca>

On 2024-11-12, John Gilmore wrote:
> Roland Clobus <rclobus at rclobus.nl> wrote:
>> My proposed change would be:
>> * At startup of Java, check if SOURCE_DATE_EPOCH is set. If so, create
>> a Clock.fixed with the timestamp from SOURCE_DATE_EPOCH
>> * In java.util.Date replace System.currentTimeMillis() with
>> Clock.getInstance().currentTimeMillis() (as suggested at [4] and other
>> sources)
>
> The clock functions in Java shouldn't stop working because somebody set
> a common environment variable.  This is hitting a fly with a
> sledgehammer.

We've been hitting those flies with this particular mechanism for nearly
a decade now and it is largely working as intended:

  https://reproducible-builds.org/docs/source-date-epoch/

SOURCE_DATE_EPOCH is not pretty or ideal, but it sure is pragmatic. I
would rather people not embed timestamps, there are no timestamps like
no timestamps! Moving forward sometimes requires making imperfect
compromises...


>> It should be noted that regular user _will not_ and _should not_ set
>> SOURCE_DATE_EPOCH [6]. That environment variable it typically used for
>> rebuilds.
>
> Certainly many programmers will set the variable, particularly during
> development or debugging sessions.  And they will and should expect the
> ordinary programs that they run in that shell to keep working -- whether
> they are written in Java or not.

What about C? It has been the default behavior in GCC and CLANG for
years. Numerous other programming environments as well.

The vast majority of reproducible builds fixes are from various
toolchains respecting SOURCE_DATE_EPOCH.

It is out there. It is widely supported. It does the job, however ugly.


> A much less intrusive fix would be to provide a date option on
> ca-certificates-java that would override the current date with
> a date specified on the command line.

Having to patch each and every package to specify a bespoke build flag
is really not feasible for tens of thousands (millions?) of packages...
that sounds pretty intrusive to me.

Would I rather there be no timestamps embedded anywhere? Definitely!

Do I want to rehash that argument with each and every upstream project
in a conversation re-explaining the reasons why? Definitely not!


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241112/cd7498ef/attachment.sig>

From arnout at bzzt.net  Wed Nov 13 00:23:36 2024
From: arnout at bzzt.net (Arnout Engelen)
Date: Wed, 13 Nov 2024 01:23:36 +0100
Subject: Reproducibility for Java -> technical details
In-Reply-To: <87cyj02g6s.fsf@yucca>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
 <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
 <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
 <0a70fcf0-f4b7-46a1-a20e-947cc62832ed@oracle.com>
 <d70fc9df-c27e-45c5-a7f0-ba5a374fcc0d@rclobus.nl>
 <24944.1731441426@hop.toad.com> <87cyj02g6s.fsf@yucca>
Message-ID: <b71eb879-5c97-4fb4-b18b-084a4ff276c7@app.fastmail.com>

On Wed, Nov 13, 2024, at 00:19, Vagrant Cascadian wrote:
> On 2024-11-12, John Gilmore wrote:
> > Roland Clobus <rclobus at rclobus.nl> wrote:
> >> It should be noted that regular user _will not_ and _should not_ set
> >> SOURCE_DATE_EPOCH [6]. That environment variable it typically used for
> >> rebuilds.
> >
> > Certainly many programmers will set the variable, particularly during
> > development or debugging sessions.  And they will and should expect the
> > ordinary programs that they run in that shell to keep working -- whether
> > they are written in Java or not.
> 
> What about C? It has been the default behavior in GCC and CLANG for
> years. Numerous other programming environments as well.
> The vast majority of reproducible builds fixes are from various
> toolchains respecting SOURCE_DATE_EPOCH.

We're not talking about the Java compiler here, but about the Java runtime environment - so the equivalent would be to recommend the libc date and time functions to honor SOURCE_DATE_EPOCH. While there's something to be said for that as well, it's a bit more extreme.

ca-certificates-java has some Debian-specific Java code that uses the JDK KeyStore APIs to create a keystore (https://sources.debian.org/src/ca-certificates-java/20240118/). As Roland mentions above, the JDK JavaKeyStore implementation uses 'new Date()' to put the 'current' date into the 'date' field of the KeyEntry ('the creation date of this entry'). The KeyStore APIs don't seem to provide a way to influence that value, and it ends up in the end result.

This means AFAICT adding a flag to ca-certificates-java like John suggests as an alternative is not really possible without changes to the KeyStore API in the JRE. Perhaps a nice middle ground would be for the JavaKeyStore implementation to honor SOURCE_DATE_EPOCH for the date of the key entries? https://reproducible-builds.org/docs/source-date-epoch/#java--gradle


Kind regards,

-- 
Arnout Engelen
Engelen Open Source
https://engelen.eu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241113/b3737f4b/attachment.htm>

From rclobus at rclobus.nl  Wed Nov 13 08:10:24 2024
From: rclobus at rclobus.nl (Roland Clobus)
Date: Wed, 13 Nov 2024 09:10:24 +0100
Subject: Reproducibility for Java -> technical details
In-Reply-To: <87cyj02g6s.fsf@yucca>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
 <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
 <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
 <0a70fcf0-f4b7-46a1-a20e-947cc62832ed@oracle.com>
 <d70fc9df-c27e-45c5-a7f0-ba5a374fcc0d@rclobus.nl>
 <24944.1731441426@hop.toad.com> <87cyj02g6s.fsf@yucca>
Message-ID: <e7215aff-3892-429a-8c4f-6afc1149452d@rclobus.nl>

On 13/11/2024 00:19, Vagrant Cascadian wrote:
> On 2024-11-12, John Gilmore wrote:
>> Roland Clobus <rclobus at rclobus.nl> wrote:
>>> My proposed change would be:
>>> * At startup of Java, check if SOURCE_DATE_EPOCH is set. If so, create
>>> a Clock.fixed with the timestamp from SOURCE_DATE_EPOCH
>>> * In java.util.Date replace System.currentTimeMillis() with
>>> Clock.getInstance().currentTimeMillis() (as suggested at [4] and other
>>> sources)
>>
>> The clock functions in Java shouldn't stop working because somebody set
>> a common environment variable.  This is hitting a fly with a
>> sledgehammer.
> 
> We've been hitting those flies with this particular mechanism for nearly
> a decade now and it is largely working as intended:
> 
>    https://reproducible-builds.org/docs/source-date-epoch/
> 
> SOURCE_DATE_EPOCH is not pretty or ideal, but it sure is pragmatic. I
> would rather people not embed timestamps, there are no timestamps like
> no timestamps! Moving forward sometimes requires making imperfect
> compromises...

Indeed, SOURCE_DATE_EPOCH overrides the value of 'now' with a value that 
the person that set the environment variable deemed more suitable.

>>> It should be noted that regular user _will not_ and _should not_ set
>>> SOURCE_DATE_EPOCH [6]. That environment variable it typically used for
>>> rebuilds.
>>
>> Certainly many programmers will set the variable, particularly during
>> development or debugging sessions.  And they will and should expect the
>> ordinary programs that they run in that shell to keep working -- whether
>> they are written in Java or not.

Many programs will keep on working, they just have a different value for 
'now'. If you need to measure a duration, you should not use 'now-1' 
minus 'now-2', but use a monotonic clock, which is relative and 
therefore will (and should) not be affected by SOURCE_DATE_EPOCH.

>> A much less intrusive fix would be to provide a date option on
>> ca-certificates-java that would override the current date with
>> a date specified on the command line.
> 
> Having to patch each and every package to specify a bespoke build flag
> is really not feasible for tens of thousands (millions?) of packages...
> that sounds pretty intrusive to me.
> 
> Would I rather there be no timestamps embedded anywhere? Definitely!

However, the KeyStore class does not offer an API to explicitly set the 
timestamp, it always uses 'now'.
Even if such an API would have been available, what would the suitable 
value be? The timestamp of the file of the certificate, the timestamp of 
the symlink to the certificate, 'now' or something else?

You can see the timestamps:
keytool -list -cacerts -storepass changeit

It will break the file format, and possibly some other tools as well, 
but perhaps the file `cacerts` could work very fine without this timestamp.
However, as recently in a non-related discussion on debian-devel: 
"Chesterton's fence" -> someone might actually need this timestamp and 
assign something meaningful to its value.

With kind regards,
Roland Clobus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241113/37ee2c13/attachment.sig>

From chris at reproducible-builds.org  Wed Nov 13 14:25:24 2024
From: chris at reproducible-builds.org (Chris Lamb)
Date: Wed, 13 Nov 2024 14:25:24 +0000
Subject: Reproducibility for Java
In-Reply-To: <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
 <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
 <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
Message-ID: <a91a6d95-1074-47fe-a617-b6f5dec78400@app.fastmail.com>

Roland Clobus wrote:

> After the regular postinst has run, I can run the postinst step again 
> but then with faketime active.

Mm, that's likely the most elegant solution available. Even if it is,
alas, a solution specific to building live images. :(

Separate to that, I would file a bug against keytool and/or KeyStore
class so that the command-line keytool utility either:

 a) obeys SOURCE_DATE_EPOCH internally to the tool
 b) accepts a date on the command-line (as suggested explicitly by John)
 c) there is some kind of -nodate option

As you mention, at least (a) and (b) would require a bunch of the
new Date() calls in the KeyStore class to be checked over, and likely
the KeyStore API needs to change as you imply so that a date can be
poked through to the right place. That's probably a design decision
best left to the maintainers of the KeyStore class and keytool utility,
however.

I don't think we need to propose that the entire JRE/JDK starts to
obey SOURCE_DATE_EPOCH ?




Regards,

-- 
      o
    ?   ?      Chris Lamb
   o     o     reproducible-builds.org ?
    ?   ?
      o

From anleonar at redhat.com  Wed Nov 13 14:57:55 2024
From: anleonar at redhat.com (Andrew Leonard)
Date: Wed, 13 Nov 2024 14:57:55 +0000
Subject: Reproducibility for Java
In-Reply-To: <a91a6d95-1074-47fe-a617-b6f5dec78400@app.fastmail.com>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
 <ee6f2374-d619-435d-91fd-048ba3f5b7ff@app.fastmail.com>
 <ed605bf8-70df-42bc-8e3c-3128ac2acc8f@rclobus.nl>
 <a91a6d95-1074-47fe-a617-b6f5dec78400@app.fastmail.com>
Message-ID: <CAFFhbGMP1Ov6fgEKGGmOvz5FxNn=7V-mJAdkRs9SegWuw9moVQ@mail.gmail.com>

Hi,

Fyi, OpenJDK build itself does not actually use "keytool" itself, as it has
some determinism issues, but also
doesn't quite do exactly what is required for the cacerts store creation.
OpenJDK uses it's own GenerateCacerts
tool:
https://github.com/openjdk/jdk21u/blob/master/make/jdk/src/classes/build/tools/generatecacerts/GenerateCacerts.java
I did open a bug a while back to report the keytool issues, but is was
closed as not an issue... https://bugs.openjdk.org/browse/JDK-8278157

With regards the build of OpenJDK as a whole, it is fully Reproducible, we
produce identical Temurin JDKs at Eclipse Adoptium.

Cheers
Andrew


On Wed, Nov 13, 2024 at 2:25?PM Chris Lamb <chris at reproducible-builds.org>
wrote:

> Roland Clobus wrote:
>
> > After the regular postinst has run, I can run the postinst step again
> > but then with faketime active.
>
> Mm, that's likely the most elegant solution available. Even if it is,
> alas, a solution specific to building live images. :(
>
> Separate to that, I would file a bug against keytool and/or KeyStore
> class so that the command-line keytool utility either:
>
>  a) obeys SOURCE_DATE_EPOCH internally to the tool
>  b) accepts a date on the command-line (as suggested explicitly by John)
>  c) there is some kind of -nodate option
>
> As you mention, at least (a) and (b) would require a bunch of the
> new Date() calls in the KeyStore class to be checked over, and likely
> the KeyStore API needs to change as you imply so that a date can be
> poked through to the right place. That's probably a design decision
> best left to the maintainers of the KeyStore class and keytool utility,
> however.
>
> I don't think we need to propose that the entire JRE/JDK starts to
> obey SOURCE_DATE_EPOCH ?
>
>
>
>
> Regards,
>
> --
>       o
>     ?   ?      Chris Lamb
>    o     o     reproducible-builds.org ?
>     ?   ?
>       o
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241113/7c039329/attachment.htm>

From holger at layer-acht.org  Sat Nov 16 13:32:45 2024
From: holger at layer-acht.org (Holger Levsen)
Date: Sat, 16 Nov 2024 13:32:45 +0000
Subject: NetBSD Reproducibility Report #5
In-Reply-To: <20241112082613.GA29897@lug-owl.de>
References: <20241112082613.GA29897@lug-owl.de>
Message-ID: <Zzie_Z9Lw2KdPtGK@layer-acht.org>

On Tue, Nov 12, 2024 at 09:26:13AM +0100, Jan-Benedict Glaw wrote:
>   On Linux, of 82 of all tested 94 port/arch combinations built successfully,
> with 78 building reproducible on two consecutive builds. [...]
>   Building on NetBSD current, 83 (of 94) combinations build successfully, of
> those 68 were reproducible. [...]
and
>   44 (of 94) port/arch combinations are totally reproducible, creating
> bit-identical output on NetBSD and Linux.

whooohooo, thats awesome, thanks for sharing!

> (The biggest impact here are CTF
> debug infos, which are generated differently right now, so there seems to
> be a remaining host dependency. The remaining issues are usually things like
> permission bits on symlinks or other individual issues within bootloaders or
> early boot code resp. embedded filesystem images.)

the devel is in the details! :)


-- 
cheers,
	Holger

 ???????
 ???????  holger@(debian|reproducible-builds|layer-acht).org
 ???????  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ???

Home is where the climate crisis is.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241116/bdb0655a/attachment.sig>

From bernhardout at lsmod.de  Mon Nov 18 07:38:05 2024
From: bernhardout at lsmod.de (Bernhard M. Wiedemann)
Date: Mon, 18 Nov 2024 08:38:05 +0100
Subject: Reproducibility for Java
In-Reply-To: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
References: <872d4c76-9605-4eb2-87e7-2cf759a26789@rclobus.nl>
Message-ID: <52e5b755-5b92-450a-943e-64c8b45aaad6@lsmod.de>

On 08/11/2024 17.40, Roland Clobus wrote:

In openSUSE, we support SOURCE_DATE_EPOCH in the creation of the java 
certs DB:
https://github.com/p11-glue/p11-kit/blob/fa9ec0e450ce4d34b8245874121b1086cfa97928/trust/extract-jks.c#L263

We don't have SOURCE_DATE_EPOCH set during %post scripts, so I had to 
duplicate some code:
https://build.opensuse.org/requests/1198030


Ciao
Bernhard M.

From ellundel at kth.se  Wed Nov 20 14:00:58 2024
From: ellundel at kth.se (Elias Lundell)
Date: Wed, 20 Nov 2024 14:00:58 +0000
Subject: release of maven-lockfile
In-Reply-To: <f2ff044b0f2246e0bc5e99e238b8a7be@kth.se>
References: <5aeccc4e-f01f-488d-8cdc-0a8966296e28@gnieh.org>,
 <822f2ce6-ffa2-4a2f-951c-bd013a89f79b@status6.com>,
 <f2ff044b0f2246e0bc5e99e238b8a7be@kth.se>
Message-ID: <8b31ad2bde514050b07ab39c20092040@kth.se>

Thank you for highlighting this feature!

It is related to maven-lockfile<https://github.com/chains-project/maven-lockfile/>. Here is a summary of my comparison of the two tools:

>From my testing I found two main advantages for trusted checksums:
+ It is builtin to maven and require no additional downloads.
+ It captures the checksums of maven plugins that are run (e.g. maven-compiler-plugin, etc).

The main uniques features of maven-lockfile:
+ Creates a pom containing specific version of both dependencies and transitive dependencies to download locked versions. This enables re-building the exact same version
+ Includes essential environment information such as java and maven version

I think it would be a good idea to combine the tools to enable automatic verification on build/tests. The integrity part of the maven-lockfile is handled very well by trusted checksums. However, the :freeze function of maven-lockfile is missing in trusted-checksum. If the `pom.xml` contains a version-range and a new version is downloaded the trusted checksums would fail, and there would be quite a lot of manual work to get the project running, especially if it is a transitive dependency that has changed.

Best,
Elias Lundell

________________________________
From: rb-general <rb-general-bounces at lists.reproducible-builds.org> on behalf of John Neffenger <john at status6.com>
Sent: Friday, September 27, 2024 5:29:36 PM
To: Reproducible Builds List
Subject: Re: release of maven-lockfile

On 9/27/24 1:20 AM, Martin Monperrus wrote:
> To generate a lock file, run the following command:
> $ mvn io.github.chains-project:maven-lockfile:generate

I just discovered yesterday that Maven 3.9.2 or later now has support
for dependency and plugin verification built in! It's called "Trusted
Checksums," and it's rather poorly documented for the moment. It's very
similar to how the Gradle dependency verification works.

There's a demonstration project here:

Trusted Checksums Demo
https://github.com/cstamas/tc-demo

Some unhelpful Maven documentation is here:

Trusted Checksums
https://maven.apache.org/resolver/expected-checksums.html#trusted-checksums

There's a more helpful Stack Overflow answer here:

How to use Maven Resolver "Trusted Checksums" to ensure artifact integrity?
https://stackoverflow.com/q/78746427

I added it to my Maven project by setting the following command-line
options in my project's '.mvn/maven.config' file:

------------------------------------------------------------------------
$ cat .mvn/maven.config
--strict-checksums
-Daether.trustedChecksumsSource.summaryFile=true
-Daether.trustedChecksumsSource.summaryFile.basedir=${session.rootDirectory}/.mvn/checksums/
-Daether.artifactResolver.postProcessor.trustedChecksums=true
-Daether.artifactResolver.postProcessor.trustedChecksums.checksumAlgorithms=SHA-512
-Daether.artifactResolver.postProcessor.trustedChecksums.failIfMissing=true
------------------------------------------------------------------------

Then I recorded the SHA-512 checksums just once with:

------------------------------------------------------------------------
$ mvn clean verify
-Daether.artifactResolver.postProcessor.trustedChecksums.record
------------------------------------------------------------------------

After that, when anyone builds my project, the checksums for all 379
dependencies and plugins are verified:

------------------------------------------------------------------------
$ mvn clean package
...
[INFO] Loaded 379 trusted checksums from
/home/john/src/pub/hello-java/.mvn/checksums/checksums-central.sha512
...
[INFO] BUILD SUCCESS
------------------------------------------------------------------------

John

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241120/cc3ed186/attachment.htm>

From dwheeler at dwheeler.com  Thu Nov 21 01:44:56 2024
From: dwheeler at dwheeler.com (David A. Wheeler)
Date: Wed, 20 Nov 2024 20:44:56 -0500
Subject: =?utf-8?Q?J=C3=A9r=C3=A9my_Bobbio_=28Lunar=29_on_Friday=2C_Novemb?=
 =?utf-8?Q?er_8=2C_2024=2E?=
Message-ID: <03D75A76-2C24-4784-8399-BDD7448A3AC4@dwheeler.com>

I'm sad to report that J?r?my Bobbio (Lunar) died on Friday, November 8, 2024,
according to: https://www.debian.org/News/2024/20241119

I know he made important contributions to Tor, Debian & Reproducible Build.

I suspect a number of people on this list already knew, but I suspect others didn't.
I also thought it'd be important to acknowledge his work here.

With a sad heart,

--- David A. Wheeler


From holger at layer-acht.org  Thu Nov 21 09:51:05 2024
From: holger at layer-acht.org (Holger Levsen)
Date: Thu, 21 Nov 2024 09:51:05 +0000
Subject: =?utf-8?B?SsOpcsOpbQ==?= =?utf-8?Q?y?= Bobbio (Lunar) on Friday, 
 November 8, 2024.
In-Reply-To: <03D75A76-2C24-4784-8399-BDD7448A3AC4@dwheeler.com>
References: <03D75A76-2C24-4784-8399-BDD7448A3AC4@dwheeler.com>
Message-ID: <Zz8CiQoDubN2UWSP@layer-acht.org>

On Wed, Nov 20, 2024 at 08:44:56PM -0500, David A. Wheeler via rb-general wrote:
> I'm sad to report that J?r?my Bobbio (Lunar) died on Friday, November 8, 2024,
> according to: https://www.debian.org/News/2024/20241119
> I know he made important contributions to Tor, Debian & Reproducible Build.
> I suspect a number of people on this list already knew, but I suspect others didn't.
> I also thought it'd be important to acknowledge his work here.
 
Indeed, we forgot to update this mailing list but the info was
on our irc channels and on 
https://reproducible-builds.org/news/2024/11/14/reproducible-builds-mourns-the-passing-of-lunar/
and elsewhere already.

Thank you for informing this list as well!


-- 
cheers,
	Holger

 ???????
 ???????  holger@(debian|reproducible-builds|layer-acht).org
 ???????  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ???

war is peace. freedom is slavery. ignorance is strength. infection is health.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20241121/3d1ae1c9/attachment.sig>