Analysing differences in images produced with mkimg and makefs

[Fabian Keil]

Fabian Keil <fk at fabiankeil.de> wrote on 2021-12-04 at 10:03:42:

> After rebasing ElectroBSD from FreeBSD stable/11 to stable/12
> I recently noticed that the "memstick" images are unfortunately
> still not 100% reproducible.
> 
> I already tried to fix this myself a couple of years ago but
> got distracted and unfortunately nobody else seems to have
> fixed the problem in the mean time, either.

Good news everyone, I've finally found the time to investigate
and fix the remaining issue(s) and with ElectroBSD based on
FreeBSD stable/14 the "memstick" image is reproducible.

> IIRC, last time I checked the differences "only" occurred
> in the UFS data which was created with makefs (so mkimg was
> blameless).

When I looked into this again a couple of weeks ago there
were additionally differences in the FAT partition containing
the boot loader. In 2021 there was no FAT partition, yet.

> I'd like to look into this again and as a first step it would
> be helpful to understand which parts of the UFS file system
> actually vary.
> 
> Is anyone aware of a tool that can parse images created with
> mkimg that contain an UFS file system created with makefs,
> or a tool that can't deal with the whole image but can work
> with an UFS file system image?
> 
> As a work around ElectroBSD contains image-checksum.sh since 2015:
> <https://www.fabiankeil.de/sourcecode/electrobsd/ElectroBSD-20211130-817fe7d130ce/image-checksum.sh>

As it turned out this script had a fatal flaw and did not
actually calculate the bootcode checksum but calculated the
checksum for a dd error message (which was 100% reproducible):

|  generate_partial_image_checksum() {
[...]
|    md_unit=$(mdconfig -o readonly -n -f "${image_file}")
|    if [ $? != 0 ]; then
|        return 1
|    fi
[ ... Snipped code to calculate some other checksums ... ]
|    mdconfig -d -u "${md_unit}" || return 1
|
|    bootcode_checksum=$(dd if=/dev/md${md_unit}s1 2>/dev/null | sha256)
|    verbose_log "Boot code checksum: ${bootcode_checksum}"

In 2021, the bootcode of the memstick image was actually reproducible,
but the bug prevented me from noticing the FAT-related regression right
away ...

I'm currently looking into which patches are actually needed
to get reproducible memstick images to see what can be upstreamed.

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20260109/dbdec2e3/attachment.sig>