[rb-general] progress in rpm and openSUSE in 2019

Fri Nov 15 15:31:29 UTC 2019

Hi

like last year (see
https://lists.reproducible-builds.org/pipermail/rb-general/2018-December/001301.html
)
as preparation to our summit in December, I wanted to once again collect
last year's changes in rpm and openSUSE that were relevant to
reproducible builds.

The collection below is in a slightly raw form, but I hope still useful
to many of us.

The period is roughly covered by
https://salsa.debian.org/reproducible-builds/reproducible-website/tree/master/_blog/posts/188.md
to _reports/2019-11.md

In my reproducibleopensuse repo, I added

https://github.com/bmwiedemann/reproducibleopensuse/blob/master/howtodebug

with details on how to find, debug and fix reproducibility issues.
I also added an rbplot.pl script to graph the package status over time
like Debian does.
http://rb.zq1.de/compare.factory/graph.png shows the current state.

I started a binary archive to be able to rebuild with old binaries. This
allowed to verify that published (reproducible) packages were not
tampered with during build.
https://lizards.opensuse.org/2019/04/03/experimental-opensuse-mirror-via-ipfs/

Slightly related is also this package source mirror:
https://github.com/bmwiedemann/openSUSE/
It is not yet used for r-b, but at some point could provide a way to
snapshot a whole distribution source tree with a simple "git tag"

I discovered one issue in OBS
https://github.com/openSUSE/open-build-service/issues/6690 new binaries
published under old names confuse our other tools

https://build.opensuse.org/package/rdiff/openSUSE:Leap:15.1/post-build-checks?linkrev=base&rev=26
added FORCE_SOURCE_DATE=1 for latex
and fixed suse-ignored-rpaths.conf (the old version caused i586 or
x86_64 builds to be unverifiable)

https://github.com/openSUSE/brp-check-suse/pull/10 was merged,
allowing for bit-reproducible .a files

https://github.com/openSUSE/pesign-obs-integration/pull/13 pass through
rpm %licence filetype tag

https://github.com/openSUSE/pesign-obs-integration/pull/14 to better
keep rpm bits was merged, but then reverted because it caused trouble
for VirtualBox.

A number of fixes have been done in rpm. Like dpkg in Debian, rpm is the
low-level package manager used in openSUSE, Mandriva, Fedora, Qubes OS
and various derivatives. rpm also includes rpmbuild.

https://github.com/rpm-software-management/rpm/pull/656 properly
initialize some rpm metadata

https://github.com/rpm-software-management/rpm/pull/785 (allow for
unreproducible Build Date and make it the default)

https://github.com/rpm-software-management/rpm/pull/931 toolchain, keep
at least one changelog entry

https://github.com/rpm-software-management/rpm/pull/935 regression-fix
to allow to override the Build Date header again

https://github.com/rpm-software-management/rpm/pull/936 fix header
generation order

In 2019-07, openSUSE enabled builds with Link Time Optimization (LTO) in
all packages. This introduced some unreproducibility that has now all
been fixed.

https://bugzilla.opensuse.org/show_bug.cgi?id=1140896 =
https://bugzilla.opensuse.org/show_bug.cgi?id=1141319 -flto introduces
number of CPUs that causes variations in rpm OPTFLAGS and  debuginfo in
.a files and similar
https://bugzilla.opensuse.org/show_bug.cgi?id=1141323 packages embed
CFLAGS with -flto : fldigi gmp haproxy ImageMagick lyx neovim tboot tcl znc

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91307
LTO-induced indeterminism from global constructors

https://bugzilla.opensuse.org/show_bug.cgi?id=1143905 fwupd computed a
hash over unreproducible LTO data

https://github.com/openSUSE/brp-check-suse/pull/29 I proposed to strip
LTO data from .o files

https://build.opensuse.org/request/show/732635 updated rpm to use
-flto=auto (since 2019-10-21) to not embed the number of CPUs anymore in
the resulting artifacts.

https://github.com/openSUSE/build-compare/pull/31 ignore javadoc
dc.created date

https://github.com/openSUSE/osc/issues/547 report multibuild dep bug

https://github.com/openSUSE/obs-build/pull/510 use gzip -n in Debian
package build

https://github.com/bmwiedemann/theunreproduciblepackage/ got 8 commits,
including one on how floating point introduces non-determinism.
It also adds notes on solutions to some issues.

https://bugzilla.opensuse.org/show_bug.cgi?id=1133809 tracks progress
towards bit-reproducible OBS Factory pkgs.

And finally we had some toolchain and high profile openSUSE packages
patched:

https://github.com/python/cpython/pull/12341 a toolchain patch was
merged to sort readdir when building C-extensions for python.
openSUSE python + python3 packages got backports

We added a pip install macro to handle python's wheel (.whl) files
without creating unreproducible .pyc files
https://bugzilla.opensuse.org/show_bug.cgi?id=1094323

https://build.opensuse.org/request/show/705693 gettext-runtime use SDE
for mtime to make acl package build reproducibile

fix build time race in MozillaFirefox + Thunderbird translations
https://bugzilla.opensuse.org/show_bug.cgi?id=1137970
https://bugzilla.mozilla.org/show_bug.cgi?id=1568145 and use a fixed date

Ciao
Bernhard M.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20191115/ae8a1049/attachment.sig>