Status of bit-for-bit reproducible Docker/OCI images (and FOSDEM talk on Sat 12:55)

[Akihiro Suda]

Hi there, let me just share the latest status of reproducible Docker/OCI
container image builds with BuildKit: https://github.com/moby/buildkit
(OCI = "Open Container Initiative", not "Oracle Cloud Infrastructure")

BuildKit v0.11 was released in the last month with very preliminary support
for SOURCE_DATE_EPOCH and bit-for-bit reproducible builds:

```dockerfile
FROM debian:bullseye-20230109
ARG SOURCE_DATE_EPOCH
RUN echo "hello ${SOURCE_DATE_EPOCH}" >/hello

# === Workarounds below will not be needed when
https://github.com/moby/buildkit/pull/3560 is merged ===
# Limit the timestamp upper bound to SOURCE_DATE_EPOCH.
# Workaround for https://github.com/moby/buildkit/issues/3180
RUN find $( ls / | grep -E -v "^(dev|mnt|proc|sys)$" ) -newermt
"@${SOURCE_DATE_EPOCH}" -writable -xdev | xargs touch
--date="@${SOURCE_DATE_EPOCH}" --no-dereference
# Squash the entire stage for resetting the whiteout timestamps.
# Workaround for https://github.com/moby/buildkit/issues/3168
FROM scratch
COPY --from=0 / /
```

Currently, the SOURCE_DATE_EPOCH value is only applied to the timestamps in
the image metadata (e.g., the `docker history` timestamps), but not
automatically applied to the timestamps of the layered files.
So, BuildKit v0.11 still requires a very complex Dockerfile, but this will
be significantly simplified when https://github.com/moby/buildkit/pull/3560
gets merged.


The above Dockerfile can be deterministically built with the following
commands:

```bash
# Make sure to pin the BuildKit version
docker run -d --name buildkitd --privileged --restart=always
moby/buildkit:v0.11.0

docker cp buildkitd:/usr/bin/buildctl /usr/local/bin/buildctl

export BUILDKIT_HOST=docker-container://buildkitd

SOURCE_DATE_EPOCH=1675298208

# Change to "true" for pushing the image to the registry
PUSH=false

buildctl build \
  --frontend dockerfile.v0 \
  --local dockerfile=. \
  --local context=. \
  --metadata-file metadata.json \
  --output type=image,name=
example.com/foo:$SOURCE_DATE_EPOCH,buildinfo=false,push=$PUSH \
  --opt build-arg:SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH

# Verify the image digest
[ "$(jq -r '."containerimage.digest"' < metadata.json)" =
"sha256:b313cd9751ed3e0c3f7185c034fde857302d56642ac5518f1ebbf7fc2e8eed93" ]
```

Note that the buildctl CLI does not automatically propagate the
$SOURCE_DATE_EPOCH env var to containers. (While the buildx CLI does.)
So, the `--opt build-arg:SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH` flag is
explicitly required.

I'll talk further about this at FOSDEM:
https://fosdem.org/2023/schedule/event/container_reproducible_dockerfile/
(Sat 12:55-13:15, Containers devroom, UB2.252A (Lameere))

My talk will also mention how to pin the apt/dnf/apk/pacman packages with
my "repro-get" tool ( https://github.com/reproducible-containers/repro-get
).

Looking forward to meeting you all at FOSDEM.

Regards,
Akihiro Suda
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20230202/ce52eb43/attachment.htm>