[Git][reproducible-builds/reproducible-website][master] 2019-10: Initial draft
Chris Lamb
gitlab at salsa.debian.org
Sun Nov 3 21:33:49 UTC 2019
Chris Lamb pushed to branch master at Reproducible Builds / reproducible-website
Commits:
74907abb by Chris Lamb at 2019-11-03T21:33:18Z
2019-10: Initial draft
- - - - -
15 changed files:
- _reports/2019-10.md
- + images/reports/2019-10/archlinux.png
- + images/reports/2019-10/blug.jpg
- + images/reports/2019-10/coreboot.png
- + images/reports/2019-10/debian.png
- + images/reports/2019-10/diffoscope.svg
- + images/reports/2019-10/gnu.png
- + images/reports/2019-10/guix.png
- + images/reports/2019-10/opensuse.png
- + images/reports/2019-10/openwrt.png
- + images/reports/2019-10/reproducible-builds.png
- + images/reports/2019-10/summit.jpg
- + images/reports/2019-10/testframework.png
- + images/reports/2019-10/webmin.png
- + images/reports/2019-10/website.png
Changes:
=====================================
_reports/2019-10.md
=====================================
@@ -2,100 +2,234 @@
layout: report
year: "2019"
month: "10"
-month_name: "October"
-title: "Reproducible builds in October 2019"
+title: "Reproducible Builds in October 2019"
draft: true
---
-## misc
+**Welcome to the October 2019 report from the [Reproducible Builds]({{ "/" | prepend: site.baseurl }}) project.**
+{: .lead}
-* FIXME: Holger learned about [Reproducible Science is good. Replicated Science is better.](https://rescience.github.io/) from profpatch.
+[![]({{ "/images/reports/2019-10/reproducible-builds.png#right" | prepend: site.baseurl }})]({{ "/" | prepend: site.baseurl }})
-## Distro work
+In our monthly reports we outline the most important things that we have been up to recently. As a reminder what our project is all about, whilst anyone can inspect the source code of free software for malicious changes, most software is distributed to end users or servers as precompiled binaries. The motivation behind the reproducible builds effort is to ensure zero changes have been introduced during these compilation processes. This is achieved by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
-#### Debian
+In this month's report, we cover:
-* FIXME: Holger started a new section at https://wiki.debian.org/ReproducibleBuilds#Solved_issues to document progress on Debian.
+* **Media coverage & conferences** — *Reproducible builds in Belfast and in science*
+* **2019 Reproducible Builds Summit** — *Registration and attendees, etc.*
+* **Distribution work** — *The latest work in Debian, OpenWrt, OpenSuse, etc.*
+* **Software development** — *More diffoscope development, etc.*
+* **Getting in touch** — *How to contribute and get in touch*
-* FIXME: Holger sponsored an upload of in-toto_0.4.0-1_amd64.changes by Lukas Pühringer to Debian NEW queue. in-toto_0.4.0-1 ACCEPTED into unstable, 0.4.0-2 MIGRATED to testing.
+If you are interested in contributing to our project, please visit our [*Contribute*]({{ "/contribute/" | prepend: site.baseurl }}) page on our website.
-* FIXME: Holger [restarted the discussion](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774415#270) on "[#774415](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774415): devscripts: please add the srebuild wrapper for reproducible builds".
+---
-* FIXME: Johannes 'josch' Schauer explained that [mmdebstrap](https://tracker.debian.org/mmdebstrap) can [create bit by bid identical Debian chroots of unstable and buster](https://lists.debian.org/debian-devel/2019/10/msg00101.html), for both the --variant=essential and --variant=minbase variants.
+## Media coverage & conferences
-* [bit by bit identical chroot creation](https://alioth-lists.debian.net/pipermail/reproducible-builds/Week-of-Mon-20191007/011759.html)
+[Jonathan McDowell](https://www.earth.li/~noodles/) gave [an introduction on Reproducible Builds in Debian](https://www.meetup.com/belfast-lug/events/264951460/) at the [Belfast Linux User Group](https://www.meetup.com/belfast-lug/):
-* FIXME: bug in dpkg: #940571: LTO/PGO and reproducible builds
+[![]({{ "/images/reports/2019-10/blug.jpg" | prepend: site.baseurl }})](https://twitter.com/FarsetLabs/status/1187070654383841280)
-* Holger noted that the essential package set in bullseye became unreproducible again, due to perl 5.30, see #791362.
+Whilst not strictly related to reproducible *builds*, Sean Gallagher from [Ars Technica](https://arstechnica.com/) wrote an article entitled [*Researchers find bug in Python script may have affected hundreds of studies*](https://arstechnica.com/information-technology/2019/10/chemists-discover-cross-platform-python-scripts-not-so-cross-platform/):
-#### Archlinux
+> A programming error in a set of Python scripts commonly used for computational analysis of chemistry data returned varying results based on which operating system they were run on.
-* [FIXME](https://github.com/archlinux/archweb/issues/244)
+---
-#### openSUSE
+## 2019 Reproducible Builds Summit
-Bernhard M. Wiedemann posted [his monthly Reproducible Builds status update](https://lists.opensuse.org/opensuse-factory/2019-10/msg00367.html) for the [openSUSE](https://opensuse.org/) distribution.
+[![]({{ "/images/reports/2019-10/summit.jpg#right" | prepend: site.baseurl }})]({{ "/events/Marrakesh2019/" | prepend: site.baseurl }})
- * [`rpm`](https://build.opensuse.org/request/show/732635) was updated to run most builds with -flto=auto saving mirror disk space and bandwidth since 2019-10-21
- * [`maven-javadoc-plugin`](https://build.opensuse.org/request/show/735873) got a toolchain patch [from Debian](https://salsa.debian.org/java-team/maven-javadoc-plugin/blob/master/debian/patches/reproducible-footer.patch) to normalize a date
+Registration for [our fifth annual Reproducible Builds summit]({{ "/events/Marrakesh2019/" | prepend: site.baseurl }}) that will take place between 1st → 8th December in Marrakesh, Morocco has opened and [personal invitations](https://lists.reproducible-builds.org/pipermail/rb-general/2019-September/001651.html) have been sent out.
-#### OpenWrt
+Similar to previous incarnations of the event, the heart of the workshop will be three days of moderated sessions with surrounding "hacking" days and will include a huge diversity of participants from Arch Linux, coreboot, Debian, F-Droid, GNU Guix, Google, Huawei, in-toto, MirageOS, NYU, openSUSE, OpenWrt, Tails, Tor Project and many more. If you would like to learn more about the event and how to register, please visit our [our dedicated event page]({{ "/events/Marrakesh2019/" | prepend: site.baseurl }}).
-* FIXME: Paul Spooren posted a patch to the OpenWrt mailinglist that... [kernel.mk: add KCFLAGS to make kmods reproducible](https://lists.infradead.org/pipermail/openwrt-devel/2019-October/019248.html)
+---
-#### Upstream issues
+### Distribution work
- * [`sphinx-doc`](https://github.com/sphinx-doc/sphinx/issues/6714) (report nondeterminism from parallelism via sphinx)
- * [`vlc`](https://mailman.videolan.org/pipermail/vlc-devel/2019-October/128188.html) (sort tar)
- * [`keeperrl`](https://github.com/miki151/keeperrl/pull/1489) (merged, date)
- * various expiring SSL test-certs have been extended to 2049 to fix future builds:
- [`python-thriftpy2`](https://github.com/Thriftpy/thriftpy2/pull/91),
- [`python-aiosmtplib`](https://github.com/cole/aiosmtplib/pull/92),
- [`python-geventhttpclient`](https://github.com/gwik/geventhttpclient/pull/115),
- [`python-distlib`](https://bitbucket.org/pypa/distlib/pull-requests/44/extend-test-cert-validity-to-2049/diff),
- [`python-M2Crypto`](https://gitlab.com/m2crypto/m2crypto/merge_requests/235),
- [`python-oslo.service`](https://review.opendev.org/687822),
- [`python-moto`](https://github.com/spulec/moto/pull/2500) (has a remaining year 2038 bug)
+[![]({{ "/images/reports/2019-10/guix.png#right" | prepend: site.baseurl }})](http://guix.gnu.org/)
+
+[GNU Guix](http://guix.gnu.org/) announced that [they had significantly reduced the size of their "bootstrap seed"](https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/) by replacing [binutils](https://en.wikipedia.org/wiki/GNU_Binutils), [GCC](https://gcc.gnu.org/) and [glibc](https://www.gnu.org/software/libc/) and replacing them with smaller alternatives, resulting in the package manager possessing a formal description of how to build all underlying software in a reproducible way from a 120MB seed.
+
+[OpenWrt](https://openwrt.org/) is a Linux-based operating system targeting embedded devices such as wireless network routers. This month, Paul Spooren (*aparcar*) posted [a patch to their mailing list that add KCFLAGS to the build flags](https://lists.infradead.org/pipermail/openwrt-devel/2019-October/019248.html) to make it easier to rebuild the official kernel binaries.
+
+[![]({{ "/images/reports/2019-10/opensuse.png#right" | prepend: site.baseurl }})](https://www.opensuse.org/)
+
+Bernhard M. Wiedemann posted [his monthly Reproducible Builds status update](https://lists.opensuse.org/opensuse-factory/2019-10/msg00367.html) for the [openSUSE](https://opensuse.org/) distribution which describes how [`rpm` was updated](https://build.opensuse.org/request/show/732635) to run most builds with the `-flto=auto` argument saving mirror disk space/bandwidth and [`maven-javadoc-plugin` received a toolchain patch](https://build.opensuse.org/request/show/735873) that [originated from a patch in Debian](https://salsa.debian.org/java-team/maven-javadoc-plugin/blob/master/debian/patches/reproducible-footer.patch) to normalise a date.
+
+#### Debian
-* [FIXME](https://github.com/pypa/setuptools/pull/1305#issuecomment-538810632)
+[![]({{ "/images/reports/2019-10/debian.png#right" | prepend: site.baseurl }})](https://debian.org/)
-* [FIXME](https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/)
+In Debian this month...
-#### Unsorted
+Didier *OdyX* Raboud started a discussion on the [debian-devel](https://lists.debian.org/debian-devel/) mailing list regarding on building Debian source packages in a reproducible manner ([thread index](https://lists.debian.org/debian-devel/2019/10/threads.html#00301)).
+Holger Levsen started a new section on the [Debian wiki](https://wiki.debian.org/) to centralise to [document the progress made on various Debian-specific reproducibility issues](https://wiki.debian.org/ReproducibleBuilds#Solved_issues) and noticed that the ["essential" package set in the *bullseye* distribution](https://tests.reproducible-builds.org/debian/buster/amd64/pkg_set_essential.html) became unreproducible again [due to a bug in Perl](https://bugs.debian.org/791362) version 5.30.
-* [942009 https://github.com/ctmarinas/stgit/pull/43](forwarded)
+Lukas Pühringer prepared an upload of [in-toto](https://in-toto.io/) to the Debian archive that was prepared, a framework to protect supply chain integrity by the [Secure Systems Lab](https://ssl.engineering.nyu.edu/) at [New York University](https://engineering.nyu.edu/) which was uploaded by Holger Levsen.
-* [FIXME](https://github.com/ctmarinas/stgit/pull/43#issuecomment-541256140)
+Holger [restarted a discussion](https://bugs.debian.org/774415#270) on Debian bug [#774415](https://bugs.debian.org/774415) which asks that the `devscripts` collection of scripts that "make the life of a Debian package maintainer easier" to add a script/wrapper to enable easier end-user testing of reproducible builds.
-* [942342 https://github.com/ipython/traitlets/pull/535](forwarded)
+Johannes *josch* Schauer explained that their [`mmdebstrap`](https://tracker.debian.org/mmdebstrap) tool [can create bit-for-bit identical](https://lists.debian.org/debian-devel/2019/10/msg00101.html) Debian [chroots](https://en.wikipedia.org/wiki/Chroot) of the *unstable* and *buster* distributions for both the `essential` and `minbase` [bootstrap "variants"](https://sources.debian.org/src/debootstrap/1.0.116/debootstrap.8/#L78-L85) and Bernhard M. Wiedemann [contributed to a discussion](https://bugs.debian.org/940571#26) regarding adding a "global" build switch to enable/disable [Profile-Guided Optimisation](https://en.wikipedia.org/wiki/Profile-guided_optimization) (PGO) and [Link-time optimisation](https://en.wikipedia.org/wiki/Interprocedural_optimization) in the `dpkg-buildflags` tool, nothing that "overall it is still very hard to get reproducible builds with PGO enabled."
-* [FIXME](https://arstechnica.com/information-technology/2019/10/chemists-discover-cross-platform-python-scripts-not-so-cross-platform/)
+64 reviews of Debian packages were added, 10 were updated and 35 were removed this month adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html). Three new types were added by Chris Lamb: [`nondeterminstic_output_in_code_generated_by_ros_genpy`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/d219f18a), [`nondeterministic_ordering_in_include_graphs_generated_by_doxygen`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/25771b7c) & [`nondetermistic_defaults_in_documentation_generated_by_python_traitlets`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/3ecdc853).
-* [FIXME](https://github.com/KhronosGroup/SPIRV-Tools/pull/2982)
+There was a far-reaching discussion regarding the correctness and suitability of setting the `TZ` [environment variable](https://en.wikipedia.org/wiki/Environment_variable#Unix_2) to `UTC` when [it was noted that the value `UTC0`](https://lists.reproducible-builds.org/pipermail/rb-general/2019-October/001697.html) was "technically" more correct.
-* [FIXME](https://gitlab.gnome.org/GNOME/libchamplain/merge_requests/9)
+---
+
+## Software development
+
+#### Upstream patches
+
+The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:
+
+* Bernhard M. Wiedemann:
+ * [`keeperrl`](https://github.com/miki151/keeperrl/pull/1489) (merged, date)
+ * [`sphinx-doc`](https://github.com/sphinx-doc/sphinx/issues/6714) (nondeterminism from parallelism via [Sphinx](http://www.sphinx-doc.org/en/master/))
+ * [`vlc`](https://mailman.videolan.org/pipermail/vlc-devel/2019-October/128188.html) (sort tar)
+ * A number of expiring SSL testing certificates have been extended to 2049 to fix future builds:
+ * [`python-M2Crypto`](https://gitlab.com/m2crypto/m2crypto/merge_requests/235),
+ * [`python-aiosmtplib`](https://github.com/cole/aiosmtplib/pull/92),
+ * [`python-distlib`](https://bitbucket.org/pypa/distlib/pull-requests/44/extend-test-cert-validity-to-2049/diff),
+ * [`python-geventhttpclient`](https://github.com/gwik/geventhttpclient/pull/115),
+ * [`python-moto`](https://github.com/spulec/moto/pull/2500) (has a remaining year 2038 bug)
+ * [`python-oslo.service`](https://review.opendev.org/687822),
+ * [`python-thriftpy2`](https://github.com/Thriftpy/thriftpy2/pull/91),
+
+* Chris Lamb:
+ * [#934698](https://bugs.debian.org/934698) filed against [`libchamplain`](https://tracker.debian.org/pkg/libchamplain) ([merged upstream](https://gitlab.gnome.org/GNOME/libchamplain/merge_requests/9)).
+ * [#941714](https://bugs.debian.org/941714) filed against [`bst-external`](https://tracker.debian.org/pkg/bst-external).
+ * [#941715](https://bugs.debian.org/941715) filed against [`checkinstall`](https://tracker.debian.org/pkg/checkinstall).
+ * [#941716](https://bugs.debian.org/941716) filed against [`gobject-introspection`](https://tracker.debian.org/pkg/gobject-introspection).
+ * [#942005](https://bugs.debian.org/942005) filed against [`elph`](https://tracker.debian.org/pkg/elph).
+ * [#942006](https://bugs.debian.org/942006) filed against [`squeak-plugins-scratch`](https://tracker.debian.org/pkg/squeak-plugins-scratch).
+ * [#942009](https://bugs.debian.org/942009) filed against [`stgit`](https://tracker.debian.org/pkg/stgit) ([forwarded upstream](https://github.com/ctmarinas/stgit/pull/43)).
+ * [#942342](https://bugs.debian.org/942342) filed against [`traitlets`](https://tracker.debian.org/pkg/traitlets) ([forwarded upstream](https://github.com/ipython/traitlets/pull/535)).
+ * [#942479](https://bugs.debian.org/942479) filed against [`frobby`](https://tracker.debian.org/pkg/frobby).
+ * [#942767](https://bugs.debian.org/942767) filed against [`python-oslo.reports`](https://tracker.debian.org/pkg/python-oslo.reports).
+ * [#942847](https://bugs.debian.org/942847) filed against [`cloudkitty`](https://tracker.debian.org/pkg/cloudkitty).
+ * [#942848](https://bugs.debian.org/942848) filed against [`designate`](https://tracker.debian.org/pkg/designate).
+ * [#942867](https://bugs.debian.org/942867) & [#942870](https://bugs.debian.org/942870) filed against [`r-base`](https://tracker.debian.org/pkg/r-base) (not respecting `nocheck` and `nodoc` [Debian build profiles](https://wiki.debian.org/BuildProfileSpec)).
+ * [#943471](https://bugs.debian.org/943471) filed against [`khard`](https://tracker.debian.org/pkg/khard) ([forwarded upstream](https://github.com/scheibler/khard/pull/233)).
+ * [#943674](https://bugs.debian.org/943674) filed against [`flask`](https://tracker.debian.org/pkg/flask) ([forwarded upstream](https://github.com/pallets/flask/pull/3408)).
+ * [#943694](https://bugs.debian.org/943694) filed against [`ros-genpy`](https://tracker.debian.org/pkg/ros-genpy) ([forwarded upstream](https://github.com/ros/genpy/pull/110)).
+ * [#943829](https://bugs.debian.org/943829) filed against [`pmemkv`](https://tracker.debian.org/pkg/pmemkv).
+ * [#943954](https://bugs.debian.org/943954) filed against [`tm-align`](https://tracker.debian.org/pkg/tm-align).
+ * [#943956](https://bugs.debian.org/943956) filed against [`snakemake`](https://tracker.debian.org/pkg/snakemake) ([forwarded upstream](https://github.com/snakemake/snakemake/pull/80))
+ * [`spirv-tools`](https://github.com/KhronosGroup/SPIRV-Tools/pull/2982).
+
+* Mattias Ellert:
+ * [#942671](https://bugs.debian.org/942671) filed against [`doxygen`](https://tracker.debian.org/pkg/doxygen).
+
+Lastly, a request from [Steven Engler](https://github.com/stevenengler) to sort fields in the `PKG-INFO` files in the [setuptools](https://pypi.org/project/setuptools/) Python build and install Python packages to ensure they are reproducible [was resolved](https://github.com/pypa/setuptools/pull/1305#issuecomment-538810632) by [Jason R. Coombs](https://www.jaraco.com/) and Vagrant Cascadian added [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/docs/source-date-epoch/) support to [LTSP](https://ltsp.github.io/)'s manual page generation.
+
+#### strip-nondeterminism & reprotest
+
+[strip-nondeterminism](https://tracker.debian.org/pkg/strip-nondeterminism) is our tool to remove specific non-deterministic results from successful builds. This month, Chris Lamb made a number of changes including [uploading version `1.6.1-1` was to Debian unstable](https://tracker.debian.org/news/1071922/accepted-strip-nondeterminism-161-1-source-into-unstable/) that dropped the `bug_803503.zip` test fixture as it is no longer compatible with the latest version of Perl's [Archive::Zip](https://metacpan.org/pod/Archive::Zip). ([#940973](https://bugs.debian.org/940973))
+
+`reprotest` is our end-user tool to build same source code twice in different environments and then check the binaries produced by each build for differences. This month, Iñaki Malerba updated our [Salsa CI](https://salsa.debian.org) scripts [[...](https://salsa.debian.org/reproducible-builds/reprotest/commit/a547967)] as well as adding a `--control-build` parameter [[...](https://salsa.debian.org/reproducible-builds/reprotest/commit/52f6eeb)]. Holger Levsen uploaded the package as `0.7.10`, bumping the Debian ["standards version"](https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-standards-version) to `4.4.1` [[...](https://salsa.debian.org/reproducible-builds/reprotest/commit/fa0e286)].
+
+#### [diffoscope](https://diffoscope.org)
+
+[![]({{ "/images/reports/2019-10/diffoscope.svg#right" | prepend: site.baseurl }})](https://diffoscope.org)
+
+[`diffoscope`](https://diffoscope.org) is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. It is run countless times a day on [our testing infrastructure](https://tests.reproducible-builds.org/debian/reproducible.html) and is essential for identifying fixes and causes of non-deterministic behaviour.
+
+This month, Chris Lamb made the following changes, including uploading versions `126`, `127`, `128` and `129` to the Debian *unstable* distribution:
+
+* Disassembling and reporting on files related to the [R (programming language)](https://en.wikipedia.org/wiki/R_(programming_language)):
+
+ * Expose an `.rdb` file's absolute paths in the semantic/human-readable output, not hidden deep in a hexdump. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/f1e80ca)]
+ * Rework and refactor the handling of `.rdb` files with respect to locating the parallel `.rdx` prior to inspecting the file to ensure that we do not add files to the user's filesystem in the case of directly comparing two `.rdb` files or — worse — overwriting a file in is place. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/ea4c94a)]
+ * Query the container for the full path of the parallel `.rdx` file to the `.rdb` file as well as looking in the same directory. This ensures that comparing two Debian packages shows any varying path. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/322a568)]
+ * Correct the matching of `.rds` files by also detecting newer versions of this file format. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/2c9fbc1)]
+ * Don't read the site and user environment when comparing `.rdx`, `.rdb` or `.rds` files by using `Rscript`'s `--vanilla` option. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/b8236d4)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/f8e436d)]
+ * Ensure all object names are displayed, including ones beginning with a fullstop (`.`) [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/1f89609)] and sort package fields when dumping data from `.rdb` files [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/9f60724)].
+ * Mask/hide standard error when processing `.rdb` files [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/0092be0)] and don't include useless/misleading `NULL` when dumping data from them. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/cb83076)]
+ * Format package contents as `foo = bar` rather than using ugly and misleading brackets, etc. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/343d01d)] and include the object's type [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/895f398)].
+ * Don't pass our long script to parse `.rdb` files via the command line; use [standard input](https://en.wikipedia.org/wiki/Standard_streams#Standard_input_(stdin)) instead. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/07a013f)]
+ * Call the`deparse` function to ensure that we do not error out and revert to a binary diff when processing `.rdb` files with internal "vector" types; they do not automatically coerce to strings. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/91d7029)]
+ * Other misc/cosmetic changes. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/c23651e)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/face6fb)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/f23f2b4)]
+
+* Output/logging:
+ * When printing an error from a command, format the command for the user. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/138aac1)]
+ * Truncate very long command lines when displaying them as an external source of data. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/ecccd71)]
+ * When formatting command lines ensure newlines and other metacharacters appear escaped as `\n`, etc. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/691ce88)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/338dbdf)]
+ * When displaying the standard error from commands, ensure we use the escaped version. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/bbfdb57)]
+ * Use "exit code" over "return code" terminology when referring to UNIX error codes in displayed differences. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/6a8251d)]
+
+* Internal API:
+ * Add ability to pass [bytestring](https://docs.python.org/3/library/stdtypes.html#bytes) input to external commands. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/c525ba9)]
+ * Split out command-line formatting into a separate utility function. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/f784d2c)]
+ * Add support for easily masking the standard error of commands. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/9b5c5fd)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/2e33ad6)]
+ * To match the [libarchive](https://www.libarchive.org/) container, raise a `KeyError` exception if we request an invalid member from a directory. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/c98e40f)]
+ * Correct string representation output in the traceback when we cannot locate a specific item in a container. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/2478e9c)]
+
+* Misc:
+ * Move build-dependency on `python-argcomplete` to its Python 3 equivalent to facilitate Python 2.x removal. ([#942967](https://bugs.debian.org/942967))
+ * Track and report on missing Python modules. ([#72](https://salsa.debian.org//diffoscope/reproducible-builds/diffoscope))
+ * Move from deprecated `$ADTTMP` to `$AUTOPKGTEST_TMP` in the [autopkgtests](https://ci.debian.net/). [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/f06c44f)]
+ * Truncate the tcpdump expected diff to 8KB (from ~600KB). [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/c6517e6)]
+ * Try and ensure that new test data files are generated dynamically, ie. at least no new ones are added without "good" reasons. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/e83b360)]
+ * Drop unused `BASE_DIR` global in the tests. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/7b44c80)]
+
+In addition, Mattia Rizzolo updated our tests to run against all supported Python versions [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/23c6112)] and to exit with a [UNIX exit status](https://en.wikipedia.org/wiki/Exit_status) of `2` instead of `1` in case of running out of disk space [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/59267e8)]. Lastly Vagrant Cascadian updated diffoscope [126](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c3704ecaa537f96dfca2f820c3af5357a6208ce6) and [129](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=d332fd860f89ed426a2b05eaa8f7a76ff6aab58f) in [GNU Guix](https://guix.gnu.org/), and updated inputs for added additional [test suite coverage](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=36f5f23c1af640782aa47dbfed6352e3d4c957ff).
+
+[trydiffoscope](https://try.diffoscope.org) is the web-based version of [diffoscope](https://diffoscope.org) and this month Chris Lamb migrated the tool to depend on the `python3-docutils` package over `python-docutils` to allow for Python 2.x removal ([#943293](https://bugs.debian.org/943293)) as well as updating the packaging to the latest Debian standards and conventions [[...](https://salsa.debian.org/reproducible-builds/trydiffoscope/commit/75e8b14)][[...](https://salsa.debian.org/reproducible-builds/trydiffoscope/commit/95d7faf)][[...](https://salsa.debian.org/reproducible-builds/trydiffoscope/commit/01df0a4)].
+
+#### Project website
+
+[![]({{ "/images/reports/2019-10/website.png#right" | prepend: site.baseurl }})](https://reproducible-builds.org/)
+
+There was yet more effort put into our [our website](https://reproducible-builds.org/) this month, including Chris Lamb improving the formatting of reports [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/db5e808)][[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/4594e05)][[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/f338c38)][[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/46b66fc)][[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/ad390e8)] and tidying the new ["Testing framework"](https://tests.reproducible-builds.org/debian/reproducible.html) links [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/137e4bd)], etc.
+
+In addition, Holger Levsen add the Tor Project's [Reproducible Builds Manager](https://rbm.torproject.org/) to our "[Who is Involved?](https://reproducible-builds.org/who/)" page and Mattia Rizzolo droped a literal `<br>` HTML element [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/8b69f7d)].
+
+#### Test framework
+
+[![]({{ "/images/reports/2019-10/testframework.png#right" | prepend: site.baseurl }})](https://tests.reproducible-builds.org/)
+
+We operate a comprehensive [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](https://tests.reproducible-builds.org). This month, the following changes were made:
+
+* Holger Levsen:
+ * [Debian](https://debian.org/)-specific changes:
+ * Add a script to ease powercycling `x86` and `arm64` nodes. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/8a69efc8)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/64d87e9e)]
+ * Don't create suite-based directories for [buildinfos.debian.net](https://buildinfos.debian.net/). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/e4a15fc4)]
+ * Make all four suites being tested shown in a single row on the performance page. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/cd8f363f)]
+
+ * [OpenWrt](https://openwrt.org) changes:
+ * Only run jobs every third day. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d75af2d4)]
+ * Create jobs to run the `reproducible_openwrt_rebuild.py` script today and in the future. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/fa9febb0)]
-* [FIXME](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=942867)
+* Mattia Rizzolo:
+ * Add some packages that were lost while updating to *buster*. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/69c765d7)]
+ * Fix the auto-offline functionality by checking the content of the `permalinks` file instead of following the `lastSuccessfulBuild` that no longer being updated. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/a395b84f)]
-* [FIXME](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=942870)
+* Paul Spooren ([OpenWrt](https://openwrt.org)):
+ * Add a `reproducible_common` utilities file. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/94dcfb4c)]
+ * Update the `openwrt-rebuild` script to to use `schroot`. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f73cf72f)]
+ * Use [unbuffered](https://eklitzke.org/stdout-buffering) Python output [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/e2630fb7)] as well as fixing newlines [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/dcbacce5)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/0443a133)]
-* diffoscope 127 uploaded to Debian by lamby and to Archlinux by sangy
+The usual node maintenance was performed by Holger Levsen [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/cfbc58fb)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/5f9424da)], Mattia Rizzolo [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/9d3df188)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/88db9f0a)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/5cdad8fd)] and Vagrant Cascadian [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/974bca24)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/8d4b533c)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/3da81a76)]
-* Jonathan McDowell gave a 15min introduction (plus Q&A) to Reproducible Builds of Debian at the Belfast LUG:
- the Meetup event page: https://www.meetup.com/belfast-lug/events/264951460/
- a tweet from our Hackerspace that hosted: https://twitter.com/FarsetLabs/status/1187070654383841280
+---
+
+## Getting in touch
+If you are interested in contributing the Reproducible Builds project, please visit our [*Contribute*](https://reproducible-builds.org/contribute/) page on our website. However, you can get in touch with us via:
-* [943674 https://github.com/pallets/flask/pull/3408](forwarded)
+ * Mailing list: [`rb-general at lists.reproducible-builds.org`](https://lists.reproducible-builds.org/listinfo/rb-general)
-* [943694 https://github.com/ros/genpy/pull/110](forwarded)
+ * IRC: `#reproducible-builds` on `irc.oftc.net`.
-* [943471 https://github.com/scheibler/khard/pull/233](forwarded)
+ * Twitter: [@ReproBuilds](https://twitter.com/ReproBuilds)
-* [FIXME](https://lists.debian.org/debian-devel/2019/10/msg00301.html)
+<br>
-* Vagrant Cascadian updated diffoscope [126](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c3704ecaa537f96dfca2f820c3af5357a6208ce6) and [129](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=d332fd860f89ed426a2b05eaa8f7a76ff6aab58f) in [GNU Guix](https://guix.gnu.org/), and updated inputs for added additional [test suite coverage](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=36f5f23c1af640782aa47dbfed6352e3d4c957ff).
+---
-* Vagrant Cascadian added [SOURCE_DATE_EPOCH](https://reproducible-builds.org/docs/source-date-epoch/) support to [LTSP](https://ltsp.github.io/) man page generation.
+This month's report was written by Chris Lamb. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.
=====================================
images/reports/2019-10/archlinux.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/archlinux.png differ
=====================================
images/reports/2019-10/blug.jpg
=====================================
Binary files /dev/null and b/images/reports/2019-10/blug.jpg differ
=====================================
images/reports/2019-10/coreboot.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/coreboot.png differ
=====================================
images/reports/2019-10/debian.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/debian.png differ
=====================================
images/reports/2019-10/diffoscope.svg
=====================================
@@ -0,0 +1,110 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ version="1.1"
+ width="128"
+ height="128"
+ id="svg2">
+ <defs
+ id="defs4" />
+ <metadata
+ id="metadata7">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title></dc:title>
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ transform="matrix(1.0692573,0,0,1.0692573,-328.34726,-503.5515)"
+ id="layer1">
+ <g
+ id="g5409">
+ <g
+ transform="translate(5.418238,0)"
+ id="g5386">
+ <rect
+ width="90.304001"
+ height="50.999996"
+ x="316.36414"
+ y="472.80621"
+ id="rect4667-3"
+ style="fill:none;stroke:none" />
+ <g
+ id="text4673-8"
+ style="font-size:64px;font-style:normal;font-variant:normal;font-weight:500;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#008000;fill-opacity:1;stroke:none;font-family:Inconsolata;-inkscape-font-specification:Inconsolata Medium">
+ <path
+ d="m 316.36413,483.82622 0,3.968 26.304,0 0,-3.968"
+ id="path5371"
+ style="fill:#c00000;fill-opacity:1" />
+ <path
+ d="m 348.36413,483.82622 0,3.968 26.304,0 0,-3.968"
+ id="path5373"
+ style="fill:#c00000;fill-opacity:1" />
+ <path
+ d="m 380.36413,483.82622 0,3.968 26.304,0 0,-3.968"
+ id="path5375"
+ style="fill:#c00000;fill-opacity:1" />
+ </g>
+ <g
+ id="text5366"
+ style="font-size:64px;font-style:normal;font-variant:normal;font-weight:500;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#008000;fill-opacity:1;stroke:none;font-family:Inconsolata;-inkscape-font-specification:Inconsolata Medium">
+ <path
+ d="m 327.69213,496.99019 0,10.88 -11.328,0 0,3.968 11.328,0 0,11.968 4.032,0 0,-11.968 10.944,0 0,-3.968 -10.944,0 0,-10.88 -4.032,0"
+ id="path5378" />
+ <path
+ d="m 359.69213,496.99019 0,10.88 -11.328,0 0,3.968 11.328,0 0,11.968 4.032,0 0,-11.968 10.944,0 0,-3.968 -10.944,0 0,-10.88 -4.032,0"
+ id="path5380" />
+ <path
+ d="m 391.69213,496.99019 0,10.88 -11.328,0 0,3.968 11.328,0 0,11.968 4.032,0 0,-11.968 10.944,0 0,-3.968 -10.944,0 0,-10.88 -4.032,0"
+ id="path5382" />
+ </g>
+ </g>
+ <use
+ id="use5399"
+ x="0"
+ y="0"
+ width="744.09448"
+ height="1052.3622"
+ xlink:href="#g5386" />
+ <use
+ transform="matrix(0.8,0,0,0.8,82.417275,133.65028)"
+ id="use5401"
+ style="opacity:0.85"
+ x="0"
+ y="0"
+ width="744.09448"
+ height="1052.3622"
+ xlink:href="#g5386" />
+ <use
+ transform="matrix(0.6,0,0,0.6,164.83455,260.05454)"
+ id="use5403"
+ style="opacity:0.7"
+ x="0"
+ y="0"
+ width="744.09448"
+ height="1052.3622"
+ xlink:href="#g5386" />
+ <use
+ transform="matrix(0.4,0,0,0.4,247.25182,379.25208)"
+ id="use5405"
+ style="opacity:0.55"
+ x="0"
+ y="0"
+ width="744.09448"
+ height="1052.3622"
+ xlink:href="#g5386" />
+ </g>
+ </g>
+</svg>
=====================================
images/reports/2019-10/gnu.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/gnu.png differ
=====================================
images/reports/2019-10/guix.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/guix.png differ
=====================================
images/reports/2019-10/opensuse.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/opensuse.png differ
=====================================
images/reports/2019-10/openwrt.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/openwrt.png differ
=====================================
images/reports/2019-10/reproducible-builds.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/reproducible-builds.png differ
=====================================
images/reports/2019-10/summit.jpg
=====================================
Binary files /dev/null and b/images/reports/2019-10/summit.jpg differ
=====================================
images/reports/2019-10/testframework.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/testframework.png differ
=====================================
images/reports/2019-10/webmin.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/webmin.png differ
=====================================
images/reports/2019-10/website.png
=====================================
Binary files /dev/null and b/images/reports/2019-10/website.png differ
View it on GitLab: https://salsa.debian.org/reproducible-builds/reproducible-website/commit/74907abb3abeb422aa9f8c3b88b845d64b2d9701
--
View it on GitLab: https://salsa.debian.org/reproducible-builds/reproducible-website/commit/74907abb3abeb422aa9f8c3b88b845d64b2d9701
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-commits/attachments/20191103/366f20a4/attachment.htm>
More information about the rb-commits
mailing list