GCC, binutils, and Debian's build-essential set

[Vagrant Cascadian]

I have been poking at gcc and binutils this month; they take a good long
while to build...

Inspired by how close we are to making the Debian build-essential set
reproducible and how important that set of packages are in general... I
have some progress, some hope, and I daresay, some fears...

  https://tests.reproducible-builds.org/debian/bookworm/amd64/pkg_set_build-essential-depends.html

Fixing issues in these packages should in general help most all distros,
although some of the specific issues addressed are pretty
Debian-specific.


With a few patches, and a lot of caveats, I was able to get
binutils building reproducibly:

  https://bugs.debian.org/1033958
  files in source tarball in arbitrary order

  https://bugs.debian.org/1033958
  build paths embedded in debug symbols

This required disabling PGO (profile guided optimization) by passing
DEB_BUILD_OPTIONS=nopgo, so that configure is not passed
--enable-pgo-build=lto ... without that, some non-determinism ended up
in the binaries.

Also, disabling the test suites with DEB_BUILD_OPTIONS=nocheck, as this
embeds various timing and other information, as these results are
intentionally embedded in the package. This was discussed a while back,
with some ideas on how to move forward:

  https://bugs.debian.org/950585

Most of these issues are arguably debian-specific issues (e.g. we do not
need to test build paths, the tarball is specific to debian packaging,
upstream does not embed test suites in the packages, etc.), although the
speed improvements of PGO will likely tempt many distros to adopt
it...


Fixing the embedded test suite results in a more generalized way will
also be necessary to fix...

GCC!

The GCC packages in Debian also embed the test suite results... but
thankfully, puts all the test suite results into a single package on
Debian, so that is easy enough to exclude, at least!

One caveat is building gcc with:

  DEB_BUILD_PROFILES="nocheck nodoc nopgo nolto"

For many of the same reasons as with binutils...

At first, I worked on the gcc-12 packages a fair amount, discovered some
bugs in GNU Modula2 which were embedding timestamps, only to find that
gcc-13 includes GNU Modula2... in a way that does not appear to embed
the timestamps! Yay!

Curiously, even though I have been building gcc-* with
DEB_BUILD_OPTIONS=nodoc, but it is clearly still building some
documentation! So there may be some documentation that is successfully
not included that has other issues.

I did find some documentation that embeds the timestamp:

  https://gcc.gnu.org/git?p=gcc.git;a=blob;f=libstdc%2B%2B-v3/scripts/run_doxygen;h=42ed9eb4f5dfa8ed7697f4ac50353133ff6a8e6b;hb=c\
c035c5d8672f87dc8c2756d9f8367903aa72d93#l133

Have some builds going testing a fix for that right now... moderate
hopes they will be successful... will see.

Also, there are _formulas.log files that get shipped in the gcc-*
packages and embed some timestamps and timing data produced by texlive,
but I suspect/hope that these can simply be removed from the packages;
they appear to be build artifacts from some documentation generation
process.

There may be build path issues, but those are somewhat ignorable
(although much debian tooling defaults to randomizing the build path, so
would be nice to fix still)...


Broadening the horizons a bit, I poked at a few of the
build-essential-build-depends package set (the packages needed to build
build-essential):

  https://tests.reproducible-builds.org/debian/bookworm/amd64/pkg_set_build-essential-depends.html

And fixed issues in php8.2 (with a nice little patch from Jelle!), and
the maintainer uploaded the fixes already!

  https://bugs.debian.org/1034892
  Timestamps in phar files

  https://bugs.debian.org/1034423
  Paths to sed in php-config8.2 and and phpize8.2

And twisted:

  https://bugs.debian.org/1034499
  timestamp embedded in .html documentation

And boost:

  https://bugs.debian.org/1034740
  build date and time embedded in .html documentation

And qemu:

  https://bugs.debian.org/1034431
  FTBFS creating vof.bin when /bin/sh -> bash


For all of this, I have leveraged the reprotest --auto-build feature
extensively, which will do a normalized build, another normalized build,
a everything-you-asked-to-vary build, followed by a build for each type
of variation... mostly just to get it to the point where the two
normalized builds build reproducibly or nearly so, to be able to focus
on the smallest number of issues at a time, and it has been paying off,
if slowly...


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20230430/37f3bf29/attachment.sig>