Monitoring reproducibility of GitHub Actions

[Aman Sharma]

Hi Arnout,


Thanks for sharing the infrastructure! It is quite cool.


> At the ASF, we explicitly allowlist action versions,


I am interested to know more about it. I see in your repository that the final output is approved_patterns.yml. How do you enforce this regularly? Is there are CI job updating "Allow or block specified actions and reusable workflow " under "Org Settings > Action > General"<https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#allowing-select-actions-and-reusable-workflows-to-run>? We also do it, but we don't have a lot of actions so we are okay with doing this manually.


I also see many unpinned actions<https://github.com/apache/infrastructure-actions/blob/8a059befd17ed98f4942c5cf3a67b7378045b669/approved_patterns.yml#L26-L28> in that file. You may want to pin them :)


By the way, does your infrastructure also check reproducibility for composite actions?


> This already led to https://github.com/SonarSource/sonarqube-scan-action/pull/228 .


Nice! We also have some "trophies" which we record here<https://github.com/ericcornelissen/reproducing-actions#trophies>.


> Will definitely keep your projects in mind when we plan to extend that!


We will also look at the actions you have and create monitors for them.


Regards,
Aman Sharma

PhD Student
KTH Royal Institute of Technology
School of Electrical Engineering and Computer Science (EECS)
Department of Theoretical Computer Science (TCS)
<http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
<https://www.kth.se/profile/amansha>https://algomaster99.github.io/
________________________________
From: Arnout Engelen <arnout at bzzt.net>
Sent: Thursday, April 16, 2026 6:23:47 PM
To: rb-general
Cc: Aman Sharma; Eric Cornelissen
Subject: Re: Monitoring reproducibility of GitHub Actions

Hi,

Nice!

At the ASF, we explicitly allowlist action versions, and recently started checking actions with 'compiled' javascript for reproducibility before allowlisting. You can read more about this at https://github.com/apache/infrastructure-actions?tab=readme-ov-file#management-of-organization-wide-github-actions-allow-list, in particular https://github.com/apache/infrastructure-actions/tree/main/utils/verify_action_build and https://github.com/apache/infrastructure-actions/blob/main/.github/workflows/verify_dependabot_action.yml.

This already led to https://github.com/SonarSource/sonarqube-scan-action/pull/228 .

Will definitely keep your projects in mind when we plan to extend that!


Kind regards,

Arnout

On Thu, Apr 16, 2026, at 18:02, Aman Sharma via rb-general wrote:

Hi all,



I wanted to briefly share a project from our group at KTH Royal Institute of Technology. Eric Cornelissen<https://www.ericcornelissen.dev/>, a PhD student in our CHAINS<https://chains.proj.kth.se/> group, is maintaining an open-source project that monitors the reproducibility of GitHub Actions:
https://github.com/ericcornelissen/reproducing-actions


The goal of the project is to assess whether GitHub Actions can be reproduced. Currently, it focuses on two types of Actions: JavaScript-based actions and Docker-based actions (composite actions are not considered). For JavaScript actions, the project rebuilds the distributed files and compares them bit-by-bit with the repository contents. For Docker actions, it rebuilds images from the Dockerfile and checks for semantic equivalence, using https://github.com/reproducible-containers/diffoci, across builds.



More details about current actions being monitored are available on README. I am one of its contributors, so would be happy to talk about it.


Regards,

Aman Sharma
PhD Student
KTH Royal Institute of Technology
School of Electrical Engineering and Computer Science (EECS)
Department of Theoretical Computer Science (TCS)<http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
<https://www.kth.se/profile/amansha>https://algomaster99.github.io/

--
Arnout Engelen
Engelen Open Source
https://engelen.eu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20260416/68251e05/attachment.htm>