[rb-general] offical Debian docker images reproducible? (Re: Reproducible system images)

[Tianon Gravi]

On Mon, 6 Jan 2020 at 06:13, Holger Levsen <holger at layer-acht.org> wrote:
> On Thu, Jan 02, 2020 at 09:36:58AM +0100, Sylvain Beucler wrote:
> > On 02/01/2020 00:57, Chris Lamb wrote:
> > >> Are you familiar with this project?
> > >> https://github.com/debuerreotype/debuerreotype
> > > Gosh, its somewhat odd to read your own name in other peoples' README
> > > files. Whilst this was based on some work I did on debootstrap in 2015
> > > [1] speaking in early 2020 (happy New Year all, by the way) I wonder
> > > how much of this can or should be moved into upstream debootstrap
> > > itself.
>
> (leaving this context for Paul and Tianon to smile ;)

Sorry Chris!  If the reference is offensive of off-putting, I'll
remove it right away -- my goal was to credit the source of
inspiration for the project, not to put you on the spot.

I'd personally be happy to see more of this as part of debootstrap
itself, but from what I've seen debootstrap isn't very actively
maintained these days (which is understandable, given that for the
purposes of things like d-i, it "just works", and any changes in it
tend to have somewhat of a ripple effect).

I'm also excited about / interested in what projects like mmdebstrap
can offer, so will be watching that space (and really d-i in general).
:)

> > Maybe there's an opportunity for cooperation?
>
> I'd certainly hope so!
>
> Right now I'm now quite sure, where we (r-b.o) should promote them, eg on
> https://reproducible-builds.org/who/#Debian or better create
> https://reproducible-builds.org/who/#Docker? Or only on
> https://wiki.debian.org/ReproducibleBuilds?

We'd be flattered with any reference to it, but it's built upon/only
possible thanks to the work of y'all anyhow! :)

> Then upon reading https://hub.docker.com/_/debian/ I miss:
>  - checksums of the images
>  - instructions how to recreate those images (eg which SOURCE_DATE_EPOCH
>    was used)

For me, the fact that Holger's asking these questions is a sign we
need to a better job explaining what we mean in the image description,
so to that end I've got
https://github.com/docker-library/docs/pull/1633 open and would really
love any feedback anyone from this list might have on how it could be
improved. :)

I hope that docs PR clarifies reasonably, but it might also help to
add this bit of color I included in a private reply to Holger (paultag
took the conversation off-list to clarify something else and I'd
replied there):

> As for hashes, I think that's a bit more complicated.  We don't
> maintain either directly on that image description because we'd have
> to have an explicit "update the hashes" PR to
> https://github.com/docker-library/docs for every image update, which
> is hopefully understandably not something we're interested in doing
> (and would be disruptive for more than just us).  Some of that can be
> gleaned from "docker image inspect xxx", which then has a content
> digest, but by the time that digest is generated it's round-tripped
> through a Docker graph driver, and I'm not 100% sure they all can
> handle full reproducibility (AUFS, btrfs, devmapper, etc).
>
> What I mean when I say the images are reproducible, is that the
> "rootfs.tar.xz" has been reproducible, which you can find a hash for
> in that same directory referenced above (per published tag) under
> "rootfs.tar.xz.sha256" ([2] for example).  It will also be relevant to
> take "rootfs.debuerreotype-version" into account, because the
> reproducibility of a given tarball has traditionally been dependent on
> the version of debuerreotype (sometimes we have no choice but to make
> a hash-breaking change, sometimes it's for the "greater good" like
> adding more clues in the images themselves for traceability, such as
> those provenance comments in "sources.list").

Maybe this is already explained reasonably in my PR, or maybe I need
to expand on it there?

(Please keep me on CC in any replies -- I'm not subscribed to this
particular list.  /o\)

Thank you all for your work on reproducible builds! :D

♥,
- Tianon
  4096R / B42F 6819 007F 00F8 8E36  4FD4 036A 9C25 BF35 7DD4