<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><br><div><br><blockquote type="cite"><div><meta charset="UTF-8"><blockquote type="cite" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">On Thu, Sep 26, 2024 at 3:24 AM Bernhard M. Wiedemann via rb-general <<a href="mailto:rb-general@lists.reproducible-builds.org">rb-general@lists.reproducible-builds.org</a>> wrote:<br>Hi,<br><br>On our summit in Hamburg we discussed that r-b should be listed as a<span class="Apple-converted-space"> </span><br>recommendation or requirement in new standards to encourage people to<span class="Apple-converted-space"> </span><br>ensure builds are reproducible.<br><br>Via [1] I found 3 relevant standards:<br><br>* NIST Secure Software Development Framework =<span class="Apple-converted-space"> </span><br><a href="https://csrc.nist.gov/Projects/ssdf">https://csrc.nist.gov/Projects/ssdf</a><br>* OpenSSF Scorecard =<span class="Apple-converted-space"> </span><a href="https://openssf.org/resources/guides/">https://openssf.org/resources/guides/</a><br>* SLSA (Supply Chain Levels for Software Artifacts Framework)<br><br><br>SLSA level4 already lists reproducible builds as optional/recommended<br>=<span class="Apple-converted-space"> </span><a href="https://slsa.dev/spec/v1.0/faq#q-what-about-reproducible-builds">https://slsa.dev/spec/v1.0/faq#q-what-about-reproducible-builds</a></blockquote></div></blockquote><div><br></div><div><div>Not really. Reproducible builds are *mentioned*, but they are not really "recommended" in SLSA.</div><div>Here is what its FAQ says:</div><div><br></div><div>"[Verified reproducible] is one option to secure build steps of a supply chain ...</div><div>That said, verified reproducible builds are not a complete solution to supply chain integrity, nor are they practical in all cases:</div><div>... Some builds cannot easily be made reproducible, as noted above. ...</div><div>Therefore, SLSA does not require verified reproducible builds directly. Instead, verified reproducible builds are one option for implementing the requirements."</div><div><br></div><div>I have earlier proposed creating a separate track in SLSA to specifically</div><div>focus on reproducible builds. Since sometimes reproducible builds</div><div>are hard to do, my idea was to create some intermediate levels that would</div><div>help people gain partial credit for steps moving towards reproducibility.</div><div>However, lack of time has meant I haven't</div><div>had any time to devote to making progress. </div></div><div><br></div><br><blockquote type="cite"><div>NIST <br><blockquote type="cite" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><a href="https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-218.pdf">https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-218.pdf</a><span class="Apple-converted-space"> </span><br>has on page 16:<br><blockquote type="cite">PO.3.2: Follow recommended security practices to<br>deploy, operate, and maintain tools and toolchains.<br></blockquote><br><blockquote type="cite">Example 4: Implement the technologies and processes needed for reproducible<br>builds.</blockquote></blockquote></div></blockquote><div><br></div><div><div>That's simply an example of how one might implement supporting toolchains.</div><div><br></div><div>The overall practice says you should use automation to "improve... reproducibility</div><div>[and many other characteristics]", but I don't think you could claim that</div><div>reproducibility is a *requirement* of the SSDF. You can easily *NOT*</div><div>have reproducible builds, yet claim conformance to SSDF with a straight face.</div></div><br><blockquote type="cite"><div>In the OpenSSF docs, I found<br><blockquote type="cite" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><a href="https://github.com/ossf/scorecard/blob/main/docs/checks.md">https://github.com/ossf/scorecard/blob/main/docs/checks.md</a><br>but I think, it should be promoted in other contexts there, too.</blockquote></div></blockquote><div><br></div><div><div>Yes, that's where the Scorecard checks are described.</div><div>There's no check for reproducible builds in Scorecard.</div><div><br></div><div>Most of the Scorecard checks focus on the source code repo, but</div><div>"Signed-Releases" does indeed have a check on release artifacts</div><div>(in this case, are they signed). So checks on releases *are* already being done</div><div>in certain cases.</div><div><br></div><div>One problem is the effort to do so. Scorecard is run weekly on over 1 million OSS</div><div>projects. Adding a test for reproducibility would take a LOT more CPU power</div><div>(and therefore real money), unless it can depend on another data source</div><div>like <https://reproducible-builds.org/citests/>. Most of those are system-level</div><div>packages (though Go is language-level). In contrast, a vast number of the Scorecard</div><div>projects are packaged at the language-level (e.g., JavaScript, Python, etc.),</div><div>so we currently have a data mismatch.</div><div><br></div><div>I can imagine having such an integration, but that'd take effort. If someone is</div><div>willing to re-run reproducible build checks for many language-level ecosystems, and</div><div>we can work out an API to query & can trust those results, I can *totally* see</div><div>it being a Scorecard measure. If LF or OpenSSF has to run many tests every week,</div><div>I won't say it *can't* be done, but someone would have to argue for its funding.</div></div><div><br></div><br><blockquote type="cite"><div>On Sep 26, 2024, at 10:21 AM, Justin Cappos <jc4946@nyu.edu> wrote:<br><blockquote type="cite" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br>You could also suggest this for addition to the OpenSSF Badge Program. https://www.bestpractices.dev/en<br><br>It likely would need to be at a new, higher level rather than gold. Other things in this ecosystem like in-toto attestations, etc. could likely also be added there. </blockquote></div></blockquote><br></div><div><br></div><div><div>Reproducible builds are a gold-level requirement, and have been for years.</div><div><br></div><div>Here's more info.</div><div><br></div><div>You can see the OpenSSF Best Practices Badge criteria at all levels here:</div><div>https://www.bestpractices.dev/criteria</div><div>To force display of the English version (instead of your preferred natural language) use:</div><div>https://www.bestpractices.dev/en/criteria</div><div>There are 3 badge levels: Passing, Silver, and Gold.</div><div><br></div><div>I lead that project & I'm a big fan of reproducible builds. When they're available</div><div>they can counter many attacks. HOWEVER, while</div><div>in some cases they're already available or relatively easy, in other cases</div><div>reproducible builds are challenging. So they're not at the Passing level.</div><div><br></div><div>Instead, at the Passing level we simply require an automated build system:</div><div>• If the software produced by the project requires building for use, the project MUST provide a working build system that can automatically rebuild the software from source code. {N/A allowed} [build]</div><div><br></div><div>At the Silver level there is this requirement which is "on the way":</div><div> • The project MUST be able to repeat the process of generating information from source files and get exactly the same bit-for-bit result. If no building occurs (e.g., scripting languages where the source code is used directly instead of being compiled), select "not applicable" (N/A). {N/A justification} {Met justification} [build_repeatable]</div><div>That's *almost* reproducible builds, but there's no requirement that someone outside be able to do or verify it, which obviously is a weaker form.</div><div><br></div><div>Reproducible builds *ARE* already a gold-level requirement:</div><div>Gold:</div><div>The project MUST have a reproducible build. If no building occurs (e.g., scripting languages where the source code is used directly instead of being compiled), select "not applicable" (N/A). {N/A justification} {Met URL} [build_reproducible]</div><div><br></div><div>--- David A. Wheeler</div><div><br></div></div><br></body></html>