[diffoscope] Diffoscope inconsistency

[Aman Sharma]

Hi Fay,


Thanks for looking into it!


> which suggest the issue is in a dependency, not diffoscope itself.


We also think so, and `file` if developers are convinced, they should fix this. Anyway, diffoscope should also take care of parsing this non-deterministic output.


> Can you give us the output of directly running file(1) on the .class
files?


[cid:ac0258e5-3c04-4bdf-a61f-517c2d336331]

[cid:350eccdb-1f46-4b2e-94a3-cbdd0228c12d]

I have also attached the classfile with this email.


These outputs are from Dardel<https://www.pdc.kth.se/hpc-services/computing-systems/dardel-1.1043529>, supercomputer at KTH. We have already provided the environment details. But maybe this information is also relevant.


Regards,
Aman Sharma

PhD Student
KTH Royal Institute of Technology
School of Electrical Engineering and Computer Science (EECS)
Department of Theoretical Computer Science (TCS)
<http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
<https://www.kth.se/profile/amansha>https://algomaster99.github.io/
________________________________
From: Fay Stegerman <flx at obfusk.net>
Sent: Friday, August 23, 2024 6:26:14 PM
To: diffoscope users and developers
Cc: Javier Ron Arteaga; Aman Sharma
Subject: Re: [diffoscope] Diffoscope inconsistency

Hi,

* Javier Ron via diffoscope <diffoscope at lists.reproducible-builds.org> [2024-08-23 17:18]:
> Hello Diffoscope team,
>
> We discovered inconsistent behavior while running the tool for compiled java
> classes.
>
> When running in Ubuntu Jammy with libmagic 5.42, the java classes were being
> compared as java bytecode,
> as expected.
>
> When running in SUSE 15.5, with libmagic 5.32 the java classes were being
> compared as hexdumps.
>
> We tracked the issue and found that the retrieved magic contained "Cafe
> Babe" at the beginning, thus
> the corresponding java regex <https://salsa.debian.org/reproducible-builds/diffoscope/-/blob/master/diffoscope/comparators/java.py#L78>
> failed to match.

A quick look at the file(1) source code [1] shows that both versions should
output the string "compiled Java class data" for .class files (though there have
been some other changes, so it's possible the older libmagic gives an incorrect
result).  So the regex used by diffoscope should be correct even for the older
libmagic, which suggest the issue is in a dependency, not diffoscope itself.

I'm not sure what you mean by "the retrieved magic contained 'Cafe Babe'"
exactly?  Can you give us the output of directly running file(1) on the .class
files?  For example:

$ file Foo.class
Foo.class: compiled Java class data, version 65.0

I get the same result when using the old magic/Magdir/cafebabe from 5.32 (though
with a more recent version of file(1)).

- Fay

[1] https://github.com/file/file
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/diffoscope/attachments/20240826/cef88105/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pastedImage.png
Type: image/png
Size: 14761 bytes
Desc: pastedImage.png
URL: <http://lists.reproducible-builds.org/pipermail/diffoscope/attachments/20240826/cef88105/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pastedImage.png
Type: image/png
Size: 7141 bytes
Desc: pastedImage.png
URL: <http://lists.reproducible-builds.org/pipermail/diffoscope/attachments/20240826/cef88105/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pastedImage.png
Type: image/png
Size: 15611 bytes
Desc: pastedImage.png
URL: <http://lists.reproducible-builds.org/pipermail/diffoscope/attachments/20240826/cef88105/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AnnotationAccessFlagTestCase.class
Type: application/java-vm
Size: 1375 bytes
Desc: AnnotationAccessFlagTestCase.class
URL: <http://lists.reproducible-builds.org/pipermail/diffoscope/attachments/20240826/cef88105/attachment.class>