Analysing differences in images produced with mkimg and makefs

Fabian Keil fk at fabiankeil.de
Sat Dec 4 09:03:42 UTC 2021 

After rebasing ElectroBSD from FreeBSD stable/11 to stable/12
I recently noticed that the "memstick" images are unfortunately
still not 100% reproducible.

I already tried to fix this myself a couple of years ago but
got distracted and unfortunately nobody else seems to have
fixed the problem in the mean time, either.

IIRC, last time I checked the differences "only" occurred
in the UFS data which was created with makefs (so mkimg was
blameless).

I'd like to look into this again and as a first step it would
be helpful to understand which parts of the UFS file system
actually vary.

Is anyone aware of a tool that can parse images created with
mkimg that contain an UFS file system created with makefs,
or a tool that can't deal with the whole image but can work
with an UFS file system image?

As a work around ElectroBSD contains image-checksum.sh since 2015:
<https://www.fabiankeil.de/sourcecode/electrobsd/ElectroBSD-20211130-817fe7d130ce/image-checksum.sh>

While image-checksum.img can interpret the UFS filesystem
from the "outside" by mounting it first, the relevant
differences seem to be below the "POSIX layer" and
as far as image-checksum.sh is concerned the images are
"reproducible enough" to get the same "partial image checksums".

    [fk at elektrobier3 ~/scratch/ElectroBSD-20211130-817fe7d130ce-2021-12-01]$ sudo /usr/src/release/scripts/image-checksum.sh -v /dpool/images/elektrobier3/ElectroBSD-20211130-817fe7d130ce-j2/ElectroBSD-20211130-817fe7d130ce.img /dpool/images/test-vm/ElectroBSD-20211130-817fe7d130ce-2021-12-01-18\:47/ElectroBSD-20211130-817fe7d130ce.img 
    [...]
    gpart checksum: 63089c3730bb9a55342f1a71132bc2fad29ab6b0b5746e3e13ea0ee2208fd9e4
    Boot code checksum: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
    mtree checksum: 67ca25a06160e79acb8160f74c49b33c40b7a31f2abcfe4c801a7cf235594007
    ls checksum: f9b944005ac17ecea86f354a19e7f41e7037bb72d0cbae1eb46148c2f98c4350
    Partial image checksum for /dpool/images/elektrobier3/ElectroBSD-20211130-817fe7d130ce-j2/ElectroBSD-20211130-817fe7d130ce.img: a9e1d8158b47f0a5c5636cb3aadcad916c13e002f04d7c3d634a7884f501d169
    [...]
    gpart checksum: 63089c3730bb9a55342f1a71132bc2fad29ab6b0b5746e3e13ea0ee2208fd9e4
    Boot code checksum: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
    mtree checksum: 67ca25a06160e79acb8160f74c49b33c40b7a31f2abcfe4c801a7cf235594007
    ls checksum: f9b944005ac17ecea86f354a19e7f41e7037bb72d0cbae1eb46148c2f98c4350
    Partial image checksum for /dpool/images/test-vm/ElectroBSD-20211130-817fe7d130ce-2021-12-01-18:47/ElectroBSD-20211130-817fe7d130ce.img: a9e1d8158b47f0a5c5636cb3aadcad916c13e002f04d7c3d634a7884f501d169

While I can diff the hd/hexdump(1) output manually,
this requires lots of cpu time and RAM, is therefore
slow, and the output isn't particularly helpful.

Quoting from top while diff was running:

      PID   JID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
    [...]
    96115     0 fk            1 102    0  2740M   924M CPU1     1 102:43  98.23% diff -u ElectroBSD-20211130-817fe7d130ce.img.elektrobier3.txt ElectroBSD-20211130-817fe7d130ce.img.test-vm.txt

And the complete output as example:

    [fk at elektrobier3 ~/scratch/ElectroBSD-20211130-817fe7d130ce-2021-12-01]$ time diff -u ElectroBSD-20211130-817fe7d130ce.img.elektrobier3.txt ElectroBSD-20211130-817fe7d130ce.img.test-vm.txt > hd.diff

    real    124m35.611s
    user    117m35.689s
    sys     0m26.180s
    [fk at elektrobier3 ~/scratch/ElectroBSD-20211130-817fe7d130ce-2021-12-01]$ cat hd.diff 
    --- ElectroBSD-20211130-817fe7d130ce.img.elektrobier3.txt       2021-12-02 13:56:56.879948000 +0100
    +++ ElectroBSD-20211130-817fe7d130ce.img.test-vm.txt    2021-12-02 14:22:08.070199000 +0100
    @@ -5212,7 +5212,7 @@
     000da4c0  6d 64 36 34 00 00 00 00  00 00 00 00 00 00 00 00  |md64............|
     000da4d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
     *
    -000da540  00 c0 73 01 08 00 00 00  00 d0 73 01 08 00 00 00  |..s.......s.....|
    +000da540  00 d0 73 01 08 00 00 00  00 e0 73 01 08 00 00 00  |..s.......s.....|
     000da550  00 00 00 00 00 00 00 00  00 00 00 00 00 80 00 00  |................|
     000da560  00 00 00 00 00 00 00 00  48 9d 03 00 00 00 00 00  |........H.......|
     000da570  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    @@ -5257,7 +5257,7 @@
     000e24c0  6d 64 36 34 00 00 00 00  00 00 00 00 00 00 00 00  |md64............|
     000e24d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
     *
    -000e2540  00 c0 73 01 08 00 00 00  00 d0 73 01 08 00 00 00  |..s.......s.....|
    +000e2540  00 d0 73 01 08 00 00 00  00 e0 73 01 08 00 00 00  |..s.......s.....|
     000e2550  00 00 00 00 00 00 00 00  00 00 00 00 00 80 00 00  |................|
     000e2560  00 00 00 00 00 00 00 00  48 9d 03 00 00 00 00 00  |........H.......|
     000e2570  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    @@ -47913116,7 +47913116,7 @@
     364fa4c0  6d 64 36 34 00 00 00 00  00 00 00 00 00 00 00 00  |md64............|
     364fa4d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
     *
    -364fa540  00 c0 73 01 08 00 00 00  00 d0 73 01 08 00 00 00  |..s.......s.....|
    +364fa540  00 d0 73 01 08 00 00 00  00 e0 73 01 08 00 00 00  |..s.......s.....|
     364fa550  00 00 00 00 00 00 00 00  00 00 00 00 00 80 00 00  |................|
     364fa560  00 00 00 00 00 00 00 00  48 9d 03 00 00 00 00 00  |........H.......|
     364fa570  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

So it looks like only a couple of bytes vary but it's not
obvious (to me at least) what those varying bytes mean.

BTW, the diffoscope output isn't particularly helpful either:

    --- /dpool/images/elektrobier3/ElectroBSD-20211130-817fe7d130ce-j2/ElectroBSD-20211130-817fe7d130ce.img
    +++ /dpool/images/test-vm/ElectroBSD-20211130-817fe7d130ce-2021-12-01-18:47/ElectroBSD-20211130-817fe7d130ce.img
    │┄ xxd not available in path. Falling back to Python hexlify.
    @@ -27940,15 +27940,15 @@
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000456c656374726f4253442d31322e332d535441424c452d61
     6d64363400000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
    -00c073010800000000d073010800000000000000000000000000000000800000
    +00d073010800000000e073010800000000000000000000000000000000800000
     0000000000000000489d03000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     00000000000000000000010000000000c703000000000000bc07000000000000
     ff27000000000000050000000000000000000000000000000000000000000000
     000000000000000000000000000000007814a56100000000489d030000000000
    @@ -28964,15 +28964,15 @@
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000456c656374726f4253442d31322e332d535441424c452d61
     6d64363400000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
    -00c073010800000000d073010800000000000000000000000000000000800000
    +00d073010800000000e073010800000000000000000000000000000000800000
     0000000000000000489d03000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     00000000000000000000010000000000c703000000000000bc07000000000000
     ff27000000000000050000000000000000000000000000000000000000000000
     000000000000000000000000000000007814a56100000000489d030000000000
    @@ -28474660,15 +28474660,15 @@
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000456c656374726f4253442d31322e332d535441424c452d61
     6d64363400000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
    -00c073010800000000d073010800000000000000000000000000000000800000
    +00d073010800000000e073010800000000000000000000000000000000800000
     0000000000000000489d03000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     0000000000000000000000000000000000000000000000000000000000000000
     00000000000000000000010000000000c703000000000000bc07000000000000
     ff27000000000000050000000000000000000000000000000000000000000000
     000000000000000000000000000000007814a56100000000489d030000000000

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20211204/6daa3489/attachment.sig>

More information about the rb-general mailing list