Need help with getting a package to build reproducibly on arm*

[Vagrant Cascadian]

On 2021-01-07, Vagrant Cascadian wrote:
> On 2021-01-07, Michael Biebl wrote:
>> Am 07.01.21 um 18:24 schrieb Michael Biebl:
>>> as can be seen at [1], systemd does not build reproducibly on armhf and
>>> arm64 (while there is no problem on amd64 and i386).
>>> 
>>> The problem is, I have no idea what the diffoscope diff [2] means and
>>> how I can make the package build reproducibly everywhere or how I can
>>> further investigate this.
>>> 
>>> Any help here is greatly appreciated as I think reproducible-builds are
>>> a great effort and I'd like to support that as much as I can.
>>> 
>>> Regards,
>>> Michael
>>> 
>>> 
>>> [1]
>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd.html
>>> [2]
>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/systemd.html
>
> My best wild guesses would be parallelism, filesystem ordering or locale
> differences causing various sort ordering differences.
>
> I'm running a local build on arm64 with "reprotest --auto-build" to see
> if it can help give us any better leads, will see if that shows anything
> more useful... it could take some time on not particularly fast
> hardware.

First attempt was reproducible for me (two normalized builds and one
varied build), though I couldn't vary the clock with reprotest
(libfaketime appears to trigger issues with building systemd)... or
fileordering, user, group or hostname due to some limitations my my
typical test environment. The command I ran was:

  reprotest  --verbose --min-cpus=1 --vary=-user_group,-domain_host,-fileordering,-time auto -- null

So maybe one of those disabled variations, but all those are also varied
on all the platforms that tests.reproducible-builds.org tests for
Debian, so... hrm.


Another possibility is the locale used... reprotest picks more or less
at random, while:

  https://tests.reproducible-builds.org/debian/index_variations.html

  amd64: LANG="et_EE.UTF-8"
  i386: LANG="de_CH.UTF-8"
  arm64: LANG="nl_BE.UTF-8"
  armhf: LANG="it_CH.UTF-8"

Similar for LC_ALL and LANGUAGE.

But I somewhat doubt both nl_BE and it_CH would break in the same
way...


The other thing that's maybe a bit different is parallelism:

  XXX on amd64: 16 or 15
  XXX on i386: 10 or 9
  XXX on armhf: 5 or 3

But the difference between 3-5 cores and 9-10 or 15-16 doesn't seem very
likely to trigger issues either...


Was hoping reprotest would at least point us in a clearer direction for
what to test for ... but not today.

I'll chew on it a bit more and possibly try to stir up some more
possibilities.


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20210108/fba6e33d/attachment.sig>