Reproducible XFS Filesystems Builds for VMs

[Simon Josefsson]

Luca DiMaio via rb-general <rb-general at lists.reproducible-builds.org>
writes:

> - For FAT32 partitions: `mkfs.vfat --invarian -i $EFI_UUID` with
> `$SOURCE_DATE_EPOCH` and populating via mtools
> - For EXT4 partitions: `mkfs.ext4 -E hash_seed=$EXT4_HASH_SEED -U
> $ROOTFS_UUID` with `$SOURCE_DATE_EPOCH` plus the `-d
> /path/to/rootfs.tar.gz` to populate it

Wow that great work!

Out of curiosity, do mkfs.ext4 and mkfs.vfat upstream natively react on
the SOURCE_DATE_EPOCH environment variable?

I think generally it is unfortunate if upstream tools start to respect
build-specific environment variable.  Tools should support mechanisms
for reproducibility.  But I find reacting to build environment variables
problematic.  Should upstream tools start to listen on Gentoo's
NO_NETWORK build environment variable too?  Should ping stop sending
packets when that environment variable is set?  There are many other
build environment variables that are problematic to use in native code.

FWIW, I ran into a similar problem with 'help2man' which behave
different depending on SOURCE_DATE_EPOCH setting and there is no way
except to use datefudge/faketime to make help2man use another time than
current time.  Or to perform post-processing of the output files.  Or
give up and hard-code SOURCE_DATE_EPOCH when calling help2man (which is
what I settled with), but that invalidates the callers ability to pass
in another SOURCE_DATE_EPOCH value and this nests badly.  Help2man could
have accepted the time to use as a parameter on the command-line
instead.

> I'd like to hear your inputs on the approach here

You probably already thought of this, but I'd like to mention it.

To summarize your approach to reaching a reproducible file system image,
it appears to be:

1) mkfs on a loop device, with some magical parameters
2) mount it, copy files to it, unmount it
3) Compare checksum

I think making that work for all file systems is a worthy goal.  However
if you or anyone encounter a file system where the above procedure
cannot be implemented (for technical or social reasons), there is
another approach:

1) Write a tool that create a valid file system image containing a
couple of files passed as input to the tool, for example:

tar2xfs mytarball > myxfsimage

This tar2xfs can be a separate tool you develop not related to the
particular file system developers at all.  The only goal of this is to
create a valid looking file system image that, say, the Linux kernel
would accept to read files from, and hopefully also one that fsck.xfs
would not barf on.

Depending on file system, creating this tool may be simpler than
creating the recipes (and herding upstream patches into releases) that
you are working on.  You can make many shortcuts that are not possible
to do inside 'mkfs' or during read/write/unmount in the Linux kernel.

/Simon
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1251 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20250412/ba2a7613/attachment.sig>