Need help with getting a package to build reproducibly on arm*

[Vagrant Cascadian]

On 2021-01-08, Vagrant Cascadian wrote:
> On 2021-01-08, Vagrant Cascadian wrote:
>> On 2021-01-07, Vagrant Cascadian wrote:
>>> On 2021-01-07, Michael Biebl wrote:
>>>> Am 07.01.21 um 18:24 schrieb Michael Biebl:
>>>>> as can be seen at [1], systemd does not build reproducibly on armhf and
>>>>> arm64 (while there is no problem on amd64 and i386).
>>>>> 
>>>>> The problem is, I have no idea what the diffoscope diff [2] means and
>>>>> how I can make the package build reproducibly everywhere or how I can
>>>>> further investigate this.
>>>>> 
>>>>> Any help here is greatly appreciated as I think reproducible-builds are
>>>>> a great effort and I'd like to support that as much as I can.
>>>>> 
>>>>> Regards,
>>>>> Michael
>>>>> 
>>>>> 
>>>>> [1]
>>>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd.html
>>>>> [2]
>>>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/systemd.html
>>>
>>> My best wild guesses would be parallelism, filesystem ordering or locale
>>> differences causing various sort ordering differences.
>>>
>>> I'm running a local build on arm64 with "reprotest --auto-build" to see
>>> if it can help give us any better leads, will see if that shows anything
>>> more useful... it could take some time on not particularly fast
>>> hardware.
>>
>> First attempt was reproducible for me (two normalized builds and one
>> varied build), though I couldn't vary the clock with reprotest
>> (libfaketime appears to trigger issues with building systemd)... or
>> fileordering, user, group or hostname due to some limitations my my
>> typical test environment. The command I ran was:
>>
>>   reprotest  --verbose --min-cpus=1 --vary=-user_group,-domain_host,-fileordering,-time auto -- null
> ...
>
> But the second attempt for some reason did produce some interesting
> results... why it didn't happen the first time suggests it is not
> deterministic.
>
> │ │ │ ├── ./usr/bin/bootctl
> ...
> │ │ │ │ ├── strings --all --bytes=8 {}
> │ │ │ │ │ @@ -250,15 +250,15 @@
> │ │ │ │ │  SystemdOptions
> │ │ │ │ │  Failed to set SystemdOptions EFI variable: %m
> │ │ │ │ │  supported
> │ │ │ │ │  not supported
> │ │ │ │ │  Failed to query reboot-to-firmware state: %m
> │ │ │ │ │  Failed to parse argument: %s
> │ │ │ │ │  Failed to set reboot-to-firmware option: %m
> │ │ │ │ │ -/EFI/systemd/systemd-bootaa64.efi
> │ │ │ │ │ +/EFI/systemd/systemd-bootarm.efi
> │ │ │ │ │  Failed to access EFI variables. Is the "efivarfs" filesystem mounted?
> │ │ │ │ │  Failed to determine current boot order: %m
>
> This suggests to me that the running kernel is somehow used to determine
> the userspace architecture, effectively similar to:
>
>   https://tests.reproducible-builds.org/debian/issues/unstable/captures_build_arch_issue.html
>
>
> The armhf builders on tests.reproducible-builds.org for Debian do not
> systematically test this. I'm not sure about the arm64 builders, but I
> think they run the same kernel, so it seems unlikely to be the only
> issue. Also surprised the i386 builder doesn't catch this. Then again,
> it only happened on my second try, which suggests this is
> non-deterministic in some way; maybe the slower armhf/aarch64
> architectures trigger it more often?
>
> I'll later post the results of the diffoscope output somewhere and give
> a closer look.

And those results I promised:

  https://people.debian.org/~vagrant/reproducible/systemd.20210108.dE8pOx/

Nothing terribly obvious to me, though as mentioned, the running kernel
may be a factor for the arm64 and armhf platforms.


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20210211/68c018b9/attachment.sig>