[Git][reproducible-builds/reproducible-presentations][master] toulouse talk: wip, almost there..
Holger Levsen (@holger)
gitlab at salsa.debian.org
Fri Nov 15 21:53:43 UTC 2024
Holger Levsen pushed to branch master at Reproducible Builds / reproducible-presentations
Commits:
0722e024 by Holger Levsen at 2024-11-15T22:53:33+01:00
toulouse talk: wip, almost there..
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
2 changed files:
- 2024-11-16-R-B-rebuilding-what-is-distributed-from-ftp.debian.org/index.html
- 2024-11-16-R-B-rebuilding-what-is-distributed-from-ftp.debian.org/todo
Changes:
=====================================
2024-11-16-R-B-rebuilding-what-is-distributed-from-ftp.debian.org/index.html
=====================================
@@ -598,6 +598,20 @@ lunarⒶdebian.org / https://lunar.anargeek.net
<li class="fragment">all the rest</li>
</section>
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h2>Resources about unreproducibilities:</h2>
+ <ul>
+ <li class="fragment">430 known issue types in reproducible-notes.git<li>
+ <li class="fragment">Lunar's talk at CCCamp 2015</li>
+ <li class="fragment">https://reproducible-builds.org/docs/</li>
+ <span class="fragment">
+ <li>It's much easier to show common pitfalls making a package unreproducible than the opposite:<ul>
+ <li style="font-size: 85%">https://github.com/bmwiedemann/theunreproduciblepackage</li></ul></li</ul></li>
+ </span>
+ </ul>
+ </section>
+
+
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
<h2>SOURCE_DATE_EPOCH</h2>
<ul>
@@ -637,33 +651,6 @@ lunarⒶdebian.org / https://lunar.anargeek.net
</section>
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h3>3919 reprodubility related bugs fixed (mostly upstreamed), 298 patches pending...</h3>
- <img src="images/stats_bugs_sin_ftbfs_state.png">
-
- </section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h2>Resources about unreproducibilities:</h2>
- <ul>
- <li class="fragment">430 known issue types in reproducible-notes.git<li>
- <li class="fragment">Lunar's talk at CCCamp 2015</li>
- <li class="fragment">https://reproducible-builds.org/docs/</li>
- <span class="fragment">
- <li>It's much easier to show common pitfalls making a package unreproducible than the opposite:<ul>
- <li style="font-size: 85%">https://github.com/bmwiedemann/theunreproduciblepackage</li></ul></li</ul></li>
- </span>
- </ul>
- </section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h3>37400 bugs in 11 years ~= 9 per day</h3>
- <span class="fragment"><img src="images/stats_bugs_state.png">
- <p>we rebuild constantly and find lots of FTBFS bugs</p></span>
- </section>
-
-
-
<section data-background-color="white">
<img src="images/logo.png" width="584">
<h3>https://reproducible-builds.org</h3>
@@ -671,6 +658,7 @@ lunarⒶdebian.org / https://lunar.anargeek.net
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
<h2>Reproducible Builds Summits</h2>
+ <ul style="font-size: 90%">
<li>2015 Athens</li>
<li>2016 Berlin</li>
<li>2017 Berlin</li>
@@ -680,6 +668,7 @@ lunarⒶdebian.org / https://lunar.anargeek.net
<li>2023 Hamburg</li>
<li>2024 Hamburg</li>
<li class="fragment">2025 location needed!</li>
+ </ul>
</section>
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
@@ -759,7 +748,7 @@ Warpforge.
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
<h3>Reproducible-builds.org funding</h3>
<ul>
- <li class="fragment">r-b.o is a Software Freedom Conservancy (SFC) project since 2018, currently funding Chris Lambs, Mattia Rizzolo, Vagrant Cascadian and myself.</li>
+ <li class="fragment">r-b.o is a Software Freedom Conservancy (SFC) project since 2018, currently funding Chris Lambs, Mattia Rizzolo, Vagrant Cascadian, myself ß <i>kpcyrd</i>.</li>
<li class="fragment">Funding needed to support our continous work: community work, fixing upstreams, developing software, designing processes, the yearly summit...</li>
<li class="fragment">Thank you, CIP, OTF & STF & all past sponsors too ❤️ </li>
</ul>
@@ -769,6 +758,16 @@ Warpforge.
<h3>Short summary of Reproducible Debian</h3>
</section>
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h3>Reproducible Builds for some parts of Debian are available today:</h3>
+ <ul>
+ <li class="fragment">reproducible docker/podman images: docker.debian.net</li>
+ <li class="fragment">reproducible live images: cdimage.debian.org</li>
+ <li class="fragment">individual packages, useful for both developers and some users</li>
+ </ul>
+ </section>
+
+
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
<h3>CI results Debian unstable, 20150131</h3>
<img src="images/stats_pkg_state_20150131.png">
@@ -780,112 +779,96 @@ Warpforge.
</section>
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h3>CI reproducibility of Debian amd64</h3>
- <table>
- <tr><th>suite</th><th>reproducible</th><th style="font-size: 84%">unreproducible</th><th style="font-size: 88%">fails to build</th><th>other</th></tr>
-<tr>
-<td>stretch</td>
-<td>23040(93.2%)</td>
-<td>1514(6.1%)</td>
-<td>85(0.3%)</td>
-<td>80(0.4%)</td>
-</tr><tr>
-<td>buster</td>
-<td>26653(93.9%)</td>
-<td>1405(4.9%)</td>
-<td>232(0.8%)</td>
-<td>108(0.4%)</td>
-</tr><tr>
-<td>bullseye</td>
-<td>29698(96.2%)</td>
-<td>761(2.5%)</td>
-<td>274(0.9%)</td>
-<td>127(0.4%)</td>
-</tr><tr>
-<td>bookworm</td>
-<td>33240(96.9%)</td>
-<td>670(2.0%)</td>
-<td>260(0.8%)</td>
-<td>124(0.4%)</td>
-</tr><tr>
-<td>trixie</td>
-<td>34275(96.2%)</td>
-<td>764(2.1%)</td>
-<td>379(1.1%)</td>
-<td>207(0.5%)</td>
-</tr>
-</table>
- </section>
+ <h3>CI results for Debian trixie, 20241115</h3>
+ <img src="images/stats_pkg_state_trixie_20241115.png">
+ </section>
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h3>CI reproducibility of Debian amd64</h3>
- <table>
- <tr><th>suite</th><th>reproducible</th><th style="color: #ff0000; font-size: 84%">unreproducible</th><th style="font-size: 88%">fails to build</th><th>other</th></tr>
-<tr>
-<td>stretch</td>
-<td>23040(93.2%)</td>
-<td><span style="color: #ff0000">1514(6.1%)</span></td>
-<td>85(0.3%)</td>
-<td>80(0.4%)</td>
-</tr><tr>
-<td>buster</td>
-<td>26653(93.9%)</td>
-<td><span style="color: #ff0000">1405(4.9%)</span></td>
-<td>232(0.8%)</td>
-<td>108(0.4%)</td>
-</tr><tr>
-<td>bullseye</td>
-<td>29698(96.2%)</td>
-<td><span style="color: #ff0000">761(2.5%)</span></td>
-<td>274(0.9%)</td>
-<td>127(0.4%)</td>
-</tr><tr>
-<td>bookworm</td>
-<td>33240(96.9%)</td>
-<td><span style="color: #ff0000">670(2.0%)</span></td>
-<td>260(0.8%)</td>
-<td>124(0.4%)</td>
-</tr><tr>
-<td>trixie</td>
-<td>34275(96.2%)</td>
-<td><span style="color: #ff0000">764(2.1%)</span></td>
-<td>379(1.1%)</td>
-<td>207(0.5%)</td>
-</tr>
-</table>
- </section>
+
+
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h3>3919 reprodubility related bugs fixed (mostly upstreamed), 298 patches pending...</h3>
+ <img src="images/stats_bugs_sin_ftbfs_state.png">
+
+ </section>
+
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h3>37400 bugs in 11 years ~= 9 per day</h3>
+ <img src="images/stats_bugs_state.png">
+ <p>we rebuild constantly and find lots of FTBFS bugs</p>
+ </section>
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h3>CI results for Debian trixie, 20241115</h3>
- <img src="images/stats_pkg_state_trixie_20241115.png">
- </section>
+ <h2><code>snapshot.debian.org</code><br/> fixed in July 2024!</h2>
+ <h1>🥳</h1>
+ <ul class="fragment">
+ <li>Huge thanks to Linux Nordberg and DSA!</li>
+ <li>In the last 2(?) years many snapshots were not imported,</li>
+ <li>also access was severely throttled.</li>
+ <li>There are still some smaller issues but in general the service is finally reliable and usable again.</li>
+ </ul>
+ </section>
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h2>Debian <em>testing</em> migration</h2>
+ <h2>Debian <em>testing</em> migration, soon we'll be getting real!</h2>
<ul style="font-size: 94%">
- <li>Since the end of 2023, CI reproducible-builds results are included in the excuses output for Debian testing migration, but there is no penalty nor bonus yet.</li>
- <li>In 2025 for Debian 14 "<em>forky</em>" however there could penalties for violating:
- <ul>
- <li class="fragment">reproducible packages <em>must not</em> regress (to be allowed into <code>testing</code> and therefore into <code>stable</code>).</li>
- <li class="fragment">NEW packages <em>must</em> build reproducibly (to be allowed into <code>testing</code> and therefore into <code>stable</code>).</li>
+ <li>2023: CI reproducible-builds results included in excuses output for Debian testing migration, but there is no penalty nor bonus yet.</li>
+ <li class="fragment">July 2024: <code>snapshot.debian.org</code> got fixed and we can now do <b>rebuilds where the build is compared against what we distribute on ftp.debian.org</b> instead of CI builds.</li>
+ <li class="fragment">September 2024: <code>debootsnap</code> and <code>debrebuild</code> (both from <code>devscripts</code>) fixed for good.</li>
+ <li class="fragment">October 2024: work on https://reproduce.debian.net began.</li>
</ul>
- <li class="fragment">At first there could be whitelisting of some needed packages, but less over time until we can drop those exceptions.</li>
+ </section>
+
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h3>how to use debrebuild from src:devscripts in trixie</h3>
+ <ul><code>
+ <li class="fragment">wget https://buildinfos.debian.net/ftp-master.debian.org/buildinfo/2024/01/16/crun_1.13-1_amd64.buildinfo</li>
+ <li class="fragment">debrebuild --builder=sbuild+unshare libaacs_0.11.1-3_amd64-source.buildinfo</li></code>
+ </ul>
+ </section>
+
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h2>about <em>rebuilderd</em></h2>
+ <ul>
+ <li class="fragment">has support for rebuilding Arch, Debian and Tails</li>
+ <li class="fragment">several instances for Arch exist</li>
+ <li class="fragment">written in Rust by kpcyrd</li>
+ <li class="fragment">available at https://github.com/kpcyrd/rebuilderd</li>
+ <li class="fragment">installation with sudo make install, soon with sudo apt install</li>
+ <li class="fragment">rebuilderd, rebuilder-worker, rebuilderctl</li>
</ul>
</section>
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h2>Debian policy</h2>
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h2><code>https://reproduce.debian.net</code></h2>
<ul>
- <li>2017: packages <em>should</em> build reproducibly.</li>
- <li class="fragment">2025? reproducible packages <em>must not</em> regress.</li>
- <li class="fragment">2025? NEW packages <em>must</em> build reproducibly.</li>
- <li class="fragment">2027? packages <em>must</em> build reproducibly.</li>
+ <li class="fragment">a <code>rebuilderd</code> instance</li>
+ <li class="fragment">rebuilding and comparing against what we distribute on ftp.debian.org</li>
+ <li class="fragment">setup still at its infancy</li>
</ul>
</section>
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h2>100%!</h2>
+ <h3>https://gitlab.archlinux.org/archlinux/rebuilderd-website</h3>
+ <img src="images/Screenshot_2024-11-15_Arch_Linux_Reproducible_Status.png">
+ </section>
+
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h3>the difference between theory and practice?</h3>
+ <span class="fragment">
+ <h1>63% !</h1>
+ <h3>(96% vs 33%)</h3>
+ </span>
+ </section>
+
+
+
+
+
+
+
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h2>How to reach 100% in practice</h2>
<ul>
<li class="fragment">100% reproducible is a political decision and nothing technical.</h3>
<li class="fragment">We need to change <code>debian-policy</code>!</li>
@@ -895,6 +878,17 @@ Warpforge.
</ul>
</section>
+
+ <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
+ <h2>Debian policy</h2>
+ <ul>
+ <li>2017: packages <em>should</em> build reproducibly.</li>
+ <li class="fragment">2025? reproducible packages <em>must not</em> regress.</li>
+ <li class="fragment">2025? NEW packages <em>must</em> build reproducibly.</li>
+ <li class="fragment">2027? packages <em>must</em> build reproducibly.</li>
+ </ul>
+ </section>
+
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
<h3>future reproducibility of Debian amd64</h3>
<table>
@@ -938,73 +932,6 @@ Warpforge.
</table>
</section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h2>Debian <em>testing</em> migration, an even better outlook</h2>
- <ul style="font-size: 94%">
- <li>2023: CI reproducible-builds results included in excuses output for Debian testing migration, but there is no penalty nor bonus yet.</li>
- <li>2025: (for Debian 14 "<em>forky</em>"): testing migration penalities for reproducibility regressions or new unreproducible packages.
- <li class="fragment">July 2024: <code>snapshot.debian.org</code> got fixed and thus it seems we'll finally be able to base this on <b>rebuilder</b> instead of CI builds.</li>
- </ul>
- </section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h2><code>snapshot.debian.org</code> got fixed!</h2>
- <h1>🥳</h1>
- <p class="fragment">huge thanks to Linux Nordberg and DSA!</p>
- </section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h3>debootsnap and debrebuild from src:devscripts in unstable</h3>
- <ul><code>
- <li class="fragment">wget https://buildinfos.debian.net/ftp-master.debian.org/buildinfo/2024/01/16/crun_1.13-1_amd64.buildinfo</li>
- <li class="fragment">debrebuild --builder=sbuild libaacs_0.11.1-3_amd64-source.buildinfo</li></code>
- </ul>
- </section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h2>about <em>rebuilderd</em></h2>
- <ul>
- <li class="fragment">has support for rebuilding Arch, Debian and Tails</li>
- <li class="fragment">several instances for Arch exist</li>
- <li class="fragment">written in Rust by kpcyrd</li>
- <li class="fragment">available at https://github.com/kpcyrd/rebuilderd</li>
- <li class="fragment">installation with sudo make install, soon with sudo apt install</li>
- <li class="fragment">rebuilderd, rebuilder-worker, rebuilderctl</li>
- </ul>
- </section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h2><code>https://reproduce.debian.net</code></h2>
- <ul>
- <li class="fragment">a <code>rebuilderd</code> instance</li>
- <li class="fragment">rebuilding and comparing against what we distribute on ftp.debian.org</li>
- <li class="fragment">setup still at its infancy</li>
- </ul>
- </section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h3>https://gitlab.archlinux.org/archlinux/rebuilderd-website</h3>
- <img src="images/Screenshot_2024-11-15_Arch_Linux_Reproducible_Status.png">
- </section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h3>the difference between theory and praxis?</h3>
- <span class="fragment">
- <h1>63% !</h1>
- <h3>(96% vs 33%)</h3>
- </span>
- </section>
-
- <section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
- <h3>more Debian Reproducible Builds successes</h3>
- <ul>
- <li class="fragment">reproducible docker/podman images: docker.debian.net</li>
- <li class="fragment">reproducible live images: cdimage.debian.org</li>
- <li class="fragment">reproducible debian-installer (in theory, not tested atm)</li>
- </ul>
- </section>
-
<section data-background="images/Capitole_du_libre_logo.png" data-background-size="10%" data-background-position="93% 9%" data-transition="none">
<h3>Theory vs Praxis</h3>
<ul>
=====================================
2024-11-16-R-B-rebuilding-what-is-distributed-from-ftp.debian.org/todo
=====================================
@@ -1,8 +1,10 @@
thanks: tell people how awesome they are while they are alive.
-order: other bugs, ftbfs bugs
-early slide: the diff between theory and praxis? 70% (or whatever then number then will be)
- nach history
- vor rebuilder
+
+last three slides need some more love
+also explain why we are at 33% only now (missing snapshots, fakeroot, snapshot still a bit flaky)
+
+
+explain how we were at 20% 2 weeks or so
<Oejet> | h01ger: And 57% in recent days! jq 'map(select(.built_at >= "2024-11-14")) | group_by(.status) | map([.[0].status, length])' < $CACHE/list
<Oejet> That command returned: [["BAD",810],["GOOD",1065]]
<Oejet> 159 `^debsnap failed$` out of latest 1000 logs.
@@ -11,7 +13,6 @@ early slide: the diff between theory and praxis? 70% (or whatever then number th
mention kp is funded by rb now too
ask people for $arch hardware to setup rebuilderd instances
mention riscv64 nodes
-mention netbsd table?
different "Summary, looking forward"
'Short summary of Reproducible Debian' is also not the best title
View it on GitLab: https://salsa.debian.org/reproducible-builds/reproducible-presentations/-/commit/0722e024ba058d670eea7c794d02aaeca713a36c
--
View it on GitLab: https://salsa.debian.org/reproducible-builds/reproducible-presentations/-/commit/0722e024ba058d670eea7c794d02aaeca713a36c
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-commits/attachments/20241115/13427e04/attachment.htm>
More information about the rb-commits
mailing list