Yosys docs reproducibility (Was: Yosys in Debian)

[Daniel Gröber]

Hi Emil,
Hi all (CC'ing rb-general to fact check my recomendation and incentives
analysis below),

On Mon, Jul 08, 2024 at 05:34:19PM +0200, Emil Jiří Tywoniak wrote:
> Thanks for the reply. I saw that PR
> <https://github.com/YosysHQ/yosys/pull/3951> - so the blocker is related to
> how Debian uses faketime in chroot, and triggered by yosys using faketime
> on pdfs? Do you have some related links?

That turned out to be a false-positive. Faketime was never really the
problem, my assertion that graphviz supports SOURCE_DATE_EPOCH was also
wrong see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1054170.

If you look at
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/yosys.html

this still shows the old repro problems with CreationDate differing.

    <key>CreationDate</key>
-   <value><string·size="22">D:20240629125837-12'00</string></value>
+   <value><string·size="22">D:20260905043016+14'00</string></value>

IIRC I was able to suppress that problem in the latest experimental
upload. It turned out to be a makefile variable override fuckup on my part.

Unfortunately the experimental package currently fails to build so the
diffoscope output is not on the CI server anymore. Last I built it what I
was seeing was (suspected) sort() reordering of help output depending on
locale. This is incorporated into the docs build hence the repro problem.

Looked something like this according to my notes:

-synthesis for Intel (Altera) FPGAs.
-synth_intel
 synthesis for ALM-based Intel (Altera) FPGAs.
 synth_intel_alm
+synthesis for Intel (Altera) FPGAs.
+synth_intel
 synthesis for Lattice FPGAs
 synth_lattice
 <bram_type>

The problem comes from me (quite intentionally) only overriding LC_*/TZ for
the docs build, not the main yosys binary build. So my guess is the help
texts are hardcoded into the binary at build time somehow and this depends
on locale. This is where you it would help if you know or could ask your
collegues if that hypothesis makes any sense or what :)

I did an audit of all the sort()ish calls in yosys during mDebConf Berlin
to find the culprit but alas I forgot to write down my suspect :] Note:
keep in mind this could also be an unstable ordering rather than a local
dependent sort() I have not yet proved the sort() hypothesis.

An easy fix would be to override LC_* in Debian but we don't like to do
that because ideally your upstream version shouldn't have this
reproducibility hazard in the first place. This way other distros can
benefit too.

BTW: The reason overriding in our packaging is OK for latex/graphviz IMO is
that the bugs are problems with those packages not yours so we'll either
fix them upstream or in the corresponding Debian packages. If
reproducibility is a concern you care about too you could opt to do this
override in your build system but I would perhaps recommend against doing
it in the released yosys Makefile as it would lessen the incentives for the
root cause upstreams to fix their problems.

The help text reordering does represent a problem in yosys though so it
should be fixed there.

Thanks,
--Daniel