<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, "EmojiFont", "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p>Hi all,</p>
<p><br>
</p>
<p>First of all, thank you for the encouraging reaction :) I'll have a look at the <span>ASF</span>'s work and probably add it to the related work section of the docs. Below are some responses and thoughts.</p>
<p><br>
</p>
<p><br>
</p>
<p>> <span>I found quite a difference between JS actions and Docker actions. [...]</span></p>
<p><br>
</p>
<p>This is indeed true and I find it a bit hard to address since they're so different. Basically, as Aman already elaborated, for Docker we're just deferring the "what is equivalent" to diffoci (if we went with bit-wise equal I don't think we'd find any Docker-based
actions that are reproducible). I am not particularly interested in aligning the definitions between the two, but it might be worth being more explicit about this aspect in the docs.</p>
<p><br>
</p>
<p>For JavaScript actions some prerequisite knowledge may be necessary. GitHub requires that JavaScript-based actions are runnable Node.js applications *from just the source*. In practice this means one of two things: JavaScript GitHub Action repositories contain
either node_modules or the build output. In case of the latter, the goal of my project is to reproduce that build output.</p>
<p><br>
</p>
<p>The definition of reproducibility here is roughly as follows (would be happy to hear what is missing from the README and what is still missing here to improve the docs): the build is reproducible if I can</p>
<ol style="margin-bottom:0px; margin-top:0px">
<li>take the git repository</li><li>check out the version (which includes the build output!)</li><li>(re)install the dependencies using npm/yarn/pnpm/deno/bun or w/e the project wishes to use (ideally matching their version of the CLI and/or runtime, if I can derive it from the source)</li><li>run their build command they have (e.g. `npm run build`)</li><li>remove all files that git ignores</li><li>normalize line endings</li><li>end up with no diff</li></ol>
<div><br>
</div>
<div>So basically, I try to match the build environment as closely as possible, except for the OS (under the assumption that the runtime binaries are entirely statically linked and the build doesn't shell out to any other system binaries).</div>
<div><span>and it could be</span><br>
</div>
<div><br>
</div>
<p>> <span>From my understanding, this means that the build process of the underlying package manager is reproducible, not the action itself. </span></p>
<p><br>
</p>
<p>It is not clear to me what you mean by this, I hope the above clarifies. Otherwise, please elaborate.</p>
<p><br>
</p>
<p><br>
</p>
<p>> <span>I also see many unpinned actions in that file. You may want to pin them :)</span></p>
<p><br>
</p>
<p>As a side, but still related note, I'm also monitoring if the releases of actions are immutable using <a href="https://github.com/ericcornelissen/reproducing-actions/blob/da6c6a11838790356e8442eeefb93e9cae0d1b16/.github/workflows/0-ci.yml#L49" class="OWAAutoLink">https://github.com/ericcornelissen/reproducing-actions/blob/da6c6a11838790356e8442eeefb93e9cae0d1b16/.github/workflows/0-ci.yml#L49</a></p>
<p>If a release is immutable it should be fine to use unpinned actions.</p>
<p><br>
</p>
<p><br>
</p>
<p>> <span> For Docker actions, it rebuilds images from the Dockerfile and checks for semantic equivalence</span></p>
<p><br>
</p>
<p>"rebuild" here means we build it twice. This is because there's actually two types of Docker actions: Docker-image based vs Dockerfile based. So far we have some coverage for dockerfile-based action (thanks to Aman).</p>
<p>For Docker-image based actions we could test reproducibility in a manner closer resembling that of the JavaScript-based actions, namely we would build the image on our side and compare it to the hosted image.</p>
<p>There's an open PR to support this (<a href="https://github.com/ericcornelissen/reproducing-actions/pull/232" class="OWAAutoLink">https://github.com/ericcornelissen/reproducing-actions/pull/232</a>) but it's proving very difficult to actually reproduce any
(action) image in this way.</p>
<p><br>
</p>
<p><br>
</p>
<p>> <span>reproducibility for composite actions</span></p>
<p><br>
</p>
<p>The reproducibility of composite actions is a bit of an odd one. As opposed to JavaScript or Docker-based actions, which typically have a build. Composite actions typically don't have a build of their own. Technically speaking they could, in which case the
build could be literally anything (there's nothing stopping you from compiling, say, a Go binary, committing it, and invoking it in a composite action), but I haven't seen it.</p>
<p><br>
</p>
<p>Another aspect for composite actions is if all actions it is composed of (if any) are reproducible. This is a similar (practically identical, theoretically different) question to asking if a given workflow or repository is "composed" of reproducible actions,
for which there's an open question: <a href="https://github.com/ericcornelissen/reproducing-actions/issues/73" class="OWAAutoLink">https://github.com/ericcornelissen/reproducing-actions/issues/73</a>.</p>
<p><br>
</p>
<p><br>
</p>
<p>Regards,</p>
<p>Eric</p>
<p><br>
</p>
<p><br>
</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Arnout Engelen <arnout@bzzt.net><br>
<b>Sent:</b> Thursday, April 16, 2026 9:56:40 PM<br>
<b>To:</b> Aman Sharma; rb-general<br>
<b>Cc:</b> Eric Cornelissen<br>
<b>Subject:</b> Re: Monitoring reproducibility of GitHub Actions</font>
<div> </div>
</div>
<div>
<div>On Thu, Apr 16, 2026, at 19:22, Aman Sharma wrote:</div>
<blockquote type="cite" id="qt" style="">
<div id="qt-divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Garamond,Georgia,serif,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p>Thanks for sharing the infrastructure! It is quite cool.</p>
<p><br>
</p>
<p>> <span>At the ASF, we explicitly allowlist action versions,</span></p>
<p><br>
</p>
<p>I am interested to know more about it. I see in your repository that the final output is <span>approved_patterns.yml</span>. How do you enforce this regularly? Is there are CI job updating
<a href="https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#allowing-select-actions-and-reusable-workflows-to-run" class="qt-OWAAutoLink">
"<span>Allow or block specified actions and reusable workflow </span>" under "Org Settings > Action > General"</a>? We also do it, but we don't have a lot of actions so we are okay with doing this manually.</p>
</div>
</blockquote>
<div><br>
</div>
<div>Basically yes, <a href="https://github.com/apache/infrastructure-gha-allowlist-manager">https://github.com/apache/infrastructure-gha-allowlist-manager</a>. Though we'll likely have to figure out something else, as that configuration has a maximum of 'only'
1000 entries ;)</div>
<div><br>
</div>
<blockquote type="cite" id="qt" style="">
<div id="qt-divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Garamond,Georgia,serif,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p>I also see many <a href="https://github.com/apache/infrastructure-actions/blob/8a059befd17ed98f4942c5cf3a67b7378045b669/approved_patterns.yml#L26-L28" class="qt-OWAAutoLink">
unpinned actions</a> in that file. You may want to pin them :)</p>
</div>
</blockquote>
<div><br>
</div>
<div>Yes, those are a remnant of before we required pinning, we do want to remove those but gradually, without breaking too many workflows.</div>
<div><br>
</div>
<blockquote type="cite" id="qt" style="">
<div id="qt-divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Garamond,Georgia,serif,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p>By the way, does your infrastructure also check reproducibility for composite actions?</p>
</div>
</blockquote>
<div><br>
</div>
<div>I don't think so</div>
<div><br>
</div>
<div><br>
</div>
<div>Kind regards,</div>
<div><br>
</div>
<div>Arnout</div>
<div><br>
</div>
<blockquote type="cite" id="qt" style="">
<div id="qt-divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Garamond,Georgia,serif,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p>> This already led to <a href="https://github.com/SonarSource/sonarqube-scan-action/pull/228" target="_blank" rel="noopener noreferrer">https://github.com/SonarSource/sonarqube-scan-action/pull/228</a> .</p>
<p><br>
</p>
<p>Nice! We also have some "trophies" which we record <a href="https://github.com/ericcornelissen/reproducing-actions#trophies" class="qt-OWAAutoLink">
here</a>.</p>
<p><br>
</p>
<p>> <span>Will definitely keep your projects in mind when we plan to extend that!</span></p>
<p><br>
</p>
<p>We will also look at the actions you have and create monitors for them.</p>
<p><br>
</p>
<div id="qt-Signature">
<div id="qt-divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<div id="qt-m_4935352394101912768Signature">
<div name="divtagdefaultwrapper"><span class="size" style="font-size:13px"><span class="color" style="color:rgb(128,128,128)"><span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif"><span class="size" style="font-size:12pt">
<div style="margin-top:0px; margin-bottom:0px"><span style="color:rgb(0,0,0)"><span class="font" style="font-family:Garamond,Georgia,serif">Regards,</span></span></div>
<div><span class="font" style="font-family:Garamond,Georgia,serif"></span><span class="font" style="font-family:Garamond,Georgia,serif"></span><span style="color:rgb(0,0,0)"></span><span class="font" style="font-family:Garamond,Georgia,serif"></span><span class="font" style="font-family:Garamond,Georgia,serif"></span><br>
</div>
<div style="margin-top:0px; margin-bottom:0px"><span style="color:rgb(0,0,0)"><span class="font" style="font-family:Garamond,Georgia,serif">Aman Sharma</span></span></div>
</span></span></span></span></span></div>
<div name="divtagdefaultwrapper"><span class="size" style="font-size:13px"><span class="color" style="color:rgb(128,128,128)"><span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif"></span></span><span class="qt-im">PhD
Student<br style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">
<span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">KTH Royal Institute of Technology</span></span><br style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">
</span><span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">School of Electrical Engineering and Computer Science (EECS)</span></span><br style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">
<span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">Department of Theoretical Computer Science (TCS)</span></span><span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif"><a href="http://www.kth.se" target="_blank" id="qt-LPNoLP"></a><a href="https://www.kth.se/profile/amansha" class="qt-OWAAutoLink" id="qt-LPNoLP"></a><a href="https://www.kth.se/profile/amansha" class="qt-OWAAutoLink" id="qt-LPNoLP"></a></span></span></span></span></div>
</div>
<div><a href="https://www.kth.se/profile/amansha" class="qt-OWAAutoLink" id="qt-LPNoLP"><span class="size" style="font-size:10pt"></span></a><a href="https://algomaster99.github.io/" class="qt-OWAAutoLink" id="qt-LPNoLP">https://algomaster99.github.io/</a></div>
</div>
</div>
</div>
<div>
<hr style="display:inline-block; width:98%">
<br>
</div>
<div id="qt-divRplyFwdMsg" dir="ltr">
<div><span class="font" style="font-family:Calibri,sans-serif"><span class="color" style="color:rgb(0,0,0)"><b>From:</b> Arnout Engelen <arnout@bzzt.net><br>
<b>Sent:</b> Thursday, April 16, 2026 6:23:47 PM<br>
<b>To:</b> rb-general<br>
<b>Cc:</b> Aman Sharma; Eric Cornelissen<br>
<b>Subject:</b> Re: Monitoring reproducibility of GitHub Actions</span></span></div>
<div> </div>
</div>
<div>
<div>Hi,</div>
<div><br>
</div>
<div>Nice!</div>
<div><br>
</div>
<div>At the ASF, we explicitly allowlist action versions, and recently started checking actions with 'compiled' javascript for reproducibility before allowlisting. You can read more about this at
<a href="https://github.com/apache/infrastructure-actions?tab=readme-ov-file#management-of-organization-wide-github-actions-allow-list">
https://github.com/apache/infrastructure-actions?tab=readme-ov-file#management-of-organization-wide-github-actions-allow-list</a>, in particular
<a href="https://github.com/apache/infrastructure-actions/tree/main/utils/verify_action_build">
https://github.com/apache/infrastructure-actions/tree/main/utils/verify_action_build</a> and
<a href="https://github.com/apache/infrastructure-actions/blob/main/.github/workflows/verify_dependabot_action.yml">
https://github.com/apache/infrastructure-actions/blob/main/.github/workflows/verify_dependabot_action.yml</a>.</div>
<div><br>
</div>
<div>This already led to <a href="https://github.com/SonarSource/sonarqube-scan-action/pull/228">https://github.com/SonarSource/sonarqube-scan-action/pull/228</a> .</div>
<div><br>
</div>
<div>Will definitely keep your projects in mind when we plan to extend that!</div>
<div><br>
</div>
<div><br>
</div>
<div>Kind regards,</div>
<div><br>
</div>
<div>Arnout</div>
<div><br>
</div>
<div>On Thu, Apr 16, 2026, at 18:02, Aman Sharma via rb-general wrote:</div>
<blockquote type="cite" id="qt-qt" style="">
<div id="qt-qt-divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Garamond,Georgia,serif,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p>Hi all,</p>
<p><br>
</p>
<p><br>
</p>
<p><span>I wanted to briefly share a project from our group at KTH Royal Institute of Technology. <a href="https://www.ericcornelissen.dev/" class="qt-qt-OWAAutoLink">Eric Cornelissen</a>, a PhD student in our
<a href="https://chains.proj.kth.se/" class="qt-qt-OWAAutoLink">CHAINS</a> group, is maintaining an open-source project that monitors the reproducibility of GitHub Actions:</span><br>
<a href="https://github.com/ericcornelissen/reproducing-actions" id="qt-qt-LPlnk718057"><span>https://github.com/ericcornelissen/reproducing-actions</span></a></p>
<p><br>
</p>
<div><span>The goal of the project is to assess whether GitHub Actions can be reproduced. Currently, it focuses on two types of Actions: JavaScript-based actions and Docker-based actions (composite actions are not considered). For JavaScript actions, the project
rebuilds the distributed files and compares them bit-by-bit with the repository contents. For Docker actions, it rebuilds images from the Dockerfile and checks for semantic equivalence, using <a href="https://github.com/reproducible-containers/diffoci" class="qt-qt-OWAAutoLink">https://github.com/reproducible-containers/diffoci</a>,
across builds.</span></div>
<p><br>
</p>
<p><span></span><br>
</p>
<p><span>More details about current actions being monitored are available on README. I am one of its contributors, so would be happy to talk about it.</span></p>
<p><br>
</p>
<div id="qt-qt-Signature">
<div id="qt-qt-divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<div id="qt-qt-m_4935352394101912768Signature">
<div name="divtagdefaultwrapper"><span class="size" style="font-size:13px"><span class="qt-color" style="color:rgb(128,128,128)"><span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif"><span class="size" style="font-size:12pt">
<div style="margin-top:0px; margin-bottom:0px"><span style="color:rgb(0,0,0)"><span class="font" style="font-family:Garamond,Georgia,serif">Regards,</span></span></div>
<div><span class="font" style="font-family:Garamond,Georgia,serif"></span><span class="font" style="font-family:Garamond,Georgia,serif"></span><span style="color:rgb(0,0,0)"></span><span class="font" style="font-family:Garamond,Georgia,serif"></span><span class="font" style="font-family:Garamond,Georgia,serif"></span><br>
</div>
<div style="margin-top:0px; margin-bottom:0px"><span style="color:rgb(0,0,0)"><span class="font" style="font-family:Garamond,Georgia,serif">Aman Sharma</span></span></div>
</span></span></span></span></span></div>
<div name="divtagdefaultwrapper"><span class="size" style="font-size:13px"><span class="qt-color" style="color:rgb(128,128,128)"><span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif"></span></span><span class="qt-qt-im">PhD
Student<br style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">
<span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">KTH Royal Institute of Technology</span></span><br style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">
</span><span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">School of Electrical Engineering and Computer Science (EECS)</span></span><br style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">
<span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif">Department of Theoretical Computer Science (TCS)</span></span><span style="background-color:rgb(255,255,255)"><span class="font" style="font-family:Arial,"Helvetica Neue",helvetica,sans-serif"><a href="http://www.kth.se" target="_blank" id="qt-qt-LPNoLP"></a><a href="https://www.kth.se/profile/amansha" class="qt-qt-OWAAutoLink" id="qt-qt-LPNoLP"></a><a href="https://www.kth.se/profile/amansha" class="qt-qt-OWAAutoLink" id="qt-qt-LPNoLP"></a></span></span></span></span></div>
</div>
<div><a href="https://www.kth.se/profile/amansha" class="qt-qt-OWAAutoLink" id="qt-qt-LPNoLP"><span class="size" style="font-size:10pt"></span></a><a href="https://algomaster99.github.io/" class="qt-qt-OWAAutoLink" id="qt-qt-LPNoLP">https://algomaster99.github.io/</a></div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div id="qt-sig124436424">
<div class="qt-signature">-- </div>
<div class="qt-signature">Arnout Engelen</div>
<div class="qt-signature">Engelen Open Source</div>
<div class="qt-signature"><a href="https://engelen.eu">https://engelen.eu</a></div>
</div>
<div><br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div id="sig124436424">
<div class="signature">-- </div>
<div class="signature">Arnout Engelen</div>
<div class="signature">Engelen Open Source</div>
<div class="signature"><a href="https://engelen.eu">https://engelen.eu</a></div>
</div>
<div><br>
</div>
</div>
</div>
</body>
</html>