Best Practices badge criterion build_reproducible - proposed tweak

[David A. Wheeler]

As many of you know, the OpenSSF Best Practices badge for
 Free/Libre and Open Source Software (FLOSS) projects
<https://bestpractices.coreinfrastructure.org/>
has three levels: passing, silver, & gold. The "gold" level
includes criterion "build_reproducible".

However, some projects have argued that the "build_reproducible" criterion
should be relaxed slightly. The details are in this GitHub issue:
 https://github.com/coreinfrastructure/best-practices-badge/issues/1865

Basically, the claim is that this doesn't make sense for projects that don't release built
software, and that timestamp differences by *themselves* don't indicate malicious changes.

If you have opinions (pro or con) please post on that GitHub issue.
For your convenience, I've copied the main text of the issue below.

--- David A. Wheeler

========
 
Some projects have raised concerns about challenges meeting the build_reproducible gold criterion. The purpose of this criterion is to counter malicious builds, as happened in SolarWinds' Orion, by enabling verifiable reproducible builds. We still want to counter the attack, but we may be able to relax the requirement slightly while still countering the attack:

	• Many projects don't release built software at all (Linux kernel, Apache Software Foundation). In those cases this can be marked as N/A. The project may still explain how to do verified reproducible builds in those cases (and get credit for it), but they should be allowed to say N/A in those cases.
	• Many projects struggle only because of timestamps being different when there's a rebuild. Those produce differences in bit-for-bit comparisons, but I don't see how such date/timestamp differences by themselves lead to subverted software (there would have to be something else to act as a trigger). Timestamp differences are one of the most common causes for non-reproducibility, and since those differences by themselves aren't security-relevant, it seems overly harsh to demand them. I think it's good to do because it makes later comparison easier, but as I said, it's probably overly harsh.
So under the `build_reproducible' gold criterion, modify:

        description: >-
          The project MUST have a <a href="https://reproducible-builds.org/">reproducible
          build</a>. If no building occurs (e.g., scripting languages
          where the source code is used directly instead of being
          compiled), select "not applicable" (N/A).

Change the second sentence to read:

If the project does not release built results (such as an executable, package, or container), but instead only releases unbuilt source code, the project MAY select "not applicable" (N/A).
        details: >-
          A reproducible build means that multiple parties can
          independently redo the process of generating information
          from source files and get exactly the same bit-for-bit
          result.  In some cases, this can be resolved by forcing
          some sort order. JavaScript developers may consider
          using npm shrinkwrap and webpack OccurenceOrderPlugin.
          GCC and clang users may find the -frandom-seed option
          useful. The build environment (including the toolset)
          can often be defined for external parties by specifying
          the cryptographic hash of a specific container or virtual
          machine that they can use for rebuilding. The <a href="https://reproducible-builds.org/docs/">reproducible
          builds project has documentation on how to do this</a>.

Change "result" to "built result", and replace the final period with:

, for example, by adding sorts to enforce deterministic input ordering and setting date/timestamp values. For purposes of this badge, a project MAY consider a result a reproducible build if it produces the same bit-for-bit results except for timestamps. That is because it can be difficult to force consistent timestamps in some build environments and such differences typically cannot be used for attack (without some other additional subversion).