Call for real-world scenarios prevented by RB practices

[David A. Wheeler]

> On Mar 25, 2022, at 12:00 AM, John Gilmore <gnu at toad.com> wrote:
> 
> On 22/03/2022 13.46, Chris Lamb wrote:
>> Just wondering if anyone on this list is aware of any real-world
>> instances where RB practices have made a difference and flagged
>> something legitimately "bad"?
> 
> The GNU compilers are already tested for complete reproducibility.  We
> at Cygnus Support built that infrastructure back in the 1990s, when we
> made gcc into a cross-compiler (compiling on any architecture + OS,
> targeting any other).  ...


And for that, I'm grateful!!

I did have some challenges recompiling gcc version 3.0.4 when using
a *different* grandparent compiler, as described in my dissertation
"Fully Countering Trusting Trust through Diverse Double-Compiling"
<https://dwheeler.com/trusting-trust/> section 7.3.2.
These might have been fixed since then (that was a while back).

In particular (quoting myself):

1. Master result directory: One piece of critical information that had to be recorded is the full pathname of the “master result” directory that contains the source code and object directories. This value is passed to the build process through the DEST environment variable, and this value embedded in the final executable. In the experiment this value was “/home/dwheeler/thesis/work”, but this specific value is unimportant; the key is making certain that DDC uses the same value as was used when creating the compiler-under-test. This doesn't prevent reproduction, but it does create an unnecessary complication.
2. Obsolete format for tail: The build process for the chosen version of GCC (3.0.4), as part of its “make compare” step, uses an obsolete format for the “tail” command. For example, it uses “tail +16c” to skip the first 16 characters. This format is no longer accepted by default by modern GNU implementations of “tail”, which interpret “tail +16c” as an attempt to read from a file named “+16c”. This was resolved by setting the environment variable “_POSIX2_VERSION” to “199209” before the build is performed; GNU tail will notice that this environment variable is set and use the older (GCC-expected) semantics.
3. Libiberty library: It turned out that GCC 3.0.4 did not fully rebuild itself when later build stages were requested, even though the GCC recompilation documents stated that they did, due to the way the GCC build process handles its “libiberty” run-time library routines. The GCC compiler documentation explains that its normal full build process, called a “bootstrap”, can be broken into “stages”. The command “make bootstrap” is supposed to build GCC three times—once with the native compiler, once with the native-built compiler it just built, and once with the compiler it built the second time. Each step of this process is called a “stage” [GNU2002, section 14]. The last two stages should produce the same results; “make compare” checks if this is true (this is a “compiler bootstrap” test). This recompilation process includes recompilation of the “libiberty” library, a collection of lower-level subroutines used by various GNU programs. Unfortunately, actual GCC build behavior does not match the GCC documentation for “make bootstrap”. The stage1 compiler was not used to recompile the internal libiberty library when creating stage2; instead, the results of stage1 were directly copied into stage2. This appears to be a side-effect of how the makefiles were written; when stage2 was performed, the make program determined that the libiberty object file was dated after the source, and skipped rebuilding it. Because of this, the resulting executable was actually a hodgepodge that combined the results of two different compilers into a single executable. After a long effort to track down this problem, it was noted that there was a hint about this defect in the GCC documentation, though its significance was not obvious at the time: “Libiberty [is only] built twice... fixing this, so that libiberty is built three times, has long been on the to-do list.” [GNU2002, section 14]

The third one, failing to recompile the libiberty library, was especially annoying.

Hopefully all of these have been addressed now. I can't remember if I reported these to the GCC developers, I hope I did but it's been years now.

The moral of the story is that if you want something to work through different versions, you have to run automated tests for it :-).

--- David A. Wheeler