[rb-general] new tool: autoprovenance

[Bernhard M. Wiedemann]

Hi

when debugging reproducibility issues, it is often helpful to understand
how and where unreproducible files are created.
To help with this task, I created a new tool:

https://github.com/bmwiedemann/reproducibleopensuse/blob/master/autoprovenance

It is not specific to openSUSE.
It is just 60 lines of code, but amazingly powerful.


In openSUSE, I mostly use it with 1-year-old
https://github.com/bmwiedemann/reproducible-faketools/blob/master/bin/rpmbuild-strace
https://github.com/bmwiedemann/reproducibleopensuse/blob/master/stracebuild

which provides/uses a build wrapper to run the build with strace.



E.g. I used it with
https://build.opensuse.org/package/show/openSUSE:Factory/python-service_identity
that has unreproducible index.html files
and 'autoclassify' already told me that the diff is from parallelism

I attached a file with example autoprovenance output (to keep the long
lines intact)

There you can see that index.html is produced in the 'docs' dir by
sphinx-build
in the line below, you see that it was called by
["make", "-j8", "man", "singlehtml"]
which was called by /var/tmp/rpm-tmp.*
So that call is coming directly from the .spec file.

Below that, you can also see 2 sections on how the install part happens
for python2 and python3, but it is not so relevant here.

At some point, I also had the idea to create .dot graphs with graphviz
visualizing the call traces or another graph to show the flow of input
and output.

Ideas and PRs are welcome.

Ciao
Bernhard M.
-------------- next part --------------
stracebuild
autoprovenance index.html
"/home/abuild/rpmbuild/BUILD/service_identity-17.0.0/docs/_build/singlehtml/index.html" written
    by pid=7262 dir=/home/abuild/rpmbuild/BUILD/service_identity-17.0.0/docs exec="/usr/bin/sphinx-build", ["sphinx-build", "-b", "singlehtml", "-d", "_build/doctrees", ".", "_build/singlehtml"] - started
    by pid=7258 dir=/home/abuild/rpmbuild/BUILD/service_identity-17.0.0/docs exec="/usr/bin/make", ["make", "-j8", "man", "singlehtml"] - started
    by pid=7235 dir=/home/abuild/rpmbuild/BUILD/service_identity-17.0.0/docs exec="/bin/sh", ["/bin/sh", "-e", "/var/tmp/rpm-tmp.ndfEbx"] - started

"/home/abuild/rpmbuild/BUILDROOT/python-service_identity-17.0.0-0.x86_64/usr/share/doc/packages/python2-service_identity/index.html" written
    by pid=7394 dir=/home/abuild/rpmbuild/BUILD/service_identity-17.0.0 exec="/usr/bin/cp", ["cp", "-pr", "docs/_build/singlehtml/index.html", "/home/abuild/rpmbuild/BUILDROOT/python-service_identity-17.0.0-0.x86_64/usr/share/doc/packages/python2-service_identity"] - started
    by pid=7390 dir=/home/abuild/rpmbuild/BUILD/service_identity-17.0.0 exec="/bin/sh", ["/bin/sh", "-e", "/var/tmp/rpm-tmp.bSskWW"] - started

"/home/abuild/rpmbuild/BUILDROOT/python-service_identity-17.0.0-0.x86_64/usr/share/doc/packages/python3-service_identity/index.html" written
    by pid=7546 dir=/home/abuild/rpmbuild/BUILD/service_identity-17.0.0 exec="/usr/bin/cp", ["cp", "-pr", "docs/_build/singlehtml/index.html", "/home/abuild/rpmbuild/BUILDROOT/python-service_identity-17.0.0-0.x86_64/usr/share/doc/packages/python3-service_identity"] - started
    by pid=7542 dir=/home/abuild/rpmbuild/BUILD/service_identity-17.0.0 exec="/bin/sh", ["/bin/sh", "-e", "/var/tmp/rpm-tmp.MbjMcu"] - started