Verified vs. Verifiable Reproducible Builds

[David A. Wheeler]

> 
> On 2025-04-29, Larry Doolittle wrote:
>> On Tue, Apr 29, 2025 at 11:55:40AM -0400, Matthew Suozzo via rb-general wrote:
>>> * A single example of a reproduction does not convey the ability for anyone
>>> to do the same, even given the same exact environment. Non-determinism in
>>> some input or in the build itself could spuriously cause a success in spite
>>> of the build not being reproducible to another party. ...
>> 
> On Apr 29, 2025, at 3:16 PM, Vagrant Cascadian <vagrant at reproducible-builds.org> wrote:
> 
> Simplest example that I have seen in the wild is ... a randomized list
> of two elements ... will appear reproducible about 50% of the time!
> 
> There are some things that come out reproducible 99% of the time ... is
> that verifiably reproducible?

I think "verified reproducible build" simply means that someone *else*
(ideally more than one) successfully reproduced the build artifacts.

If there's an occasional failure but you eventually succeeded,
then it should count:

1. Remember, the *point* is to counter risks that the
build result doesn't match the build input (e.g., because someone maliciously
subverted the build process itself). If you eventually replicated, then you eliminated
that risk. With an eventual duplication, you successfully met the *purpose* of reproducible builds.

2. Somethings things just go wrong. E.g., a storage device has
decided to Have a Bad Day ("TM"), a gamma ray decided to flip
a bit you were counting on, whatever.

That said, if it doesn't normally repeat, you have a *practical* problem,
because having to re-run executions many times for many builds can
start to cost real money. So at that point you have a problem that needs fixing.

Similarly, if the build requires the datetime to be set before SPECIALDAY
(e.g., because of expired certs),
you could require the build environment datetime to be set before SPECIALDAY.
That's okay as a stopgap, but a lousy long-term process, because
again you're creating cost/time problems that will bite you long term.

This is all true for "normal" software operations also.
If a program can technically do "important task X", but it's absurdly
costly or difficult to do or unreliable, we'll say it does a bad job at it, and either
fix it or replacement. You might concede that it
"technically does task X" yet still find it unacceptable.
Merely doing a task in a legalistic sense isn't
usually enough; it needs to be *practical* or people will do something else.

--- David A. Wheeler