[rb-general] Clarification re SOURCE_BUILD_EPOCH and clamping

[Ximin Luo]

Holger Levsen:
> [..]
> 
> tagging that 1.1 would make it official, without quotes. but please don't do
> this next week, just a week before the summit, where we have a much better
> oportunity to release this with a nice consensus. that said, I'm all for preparing
> this consensus in the days leading to the summit!
> 

Here's an updated spec, based on the comments we got from the summit:

https://anonscm.debian.org/cgit/reproducible/source-date-epoch-spec.git/diff/?id=pu/1.1&id2=1.0

Summary of the major changes:

+			[..] The intended result is a build where the
+			output looks as if the build had happened instantly
+			at that timestamp.

This is added to cover potential future issues that might arise, to prevent interpretations that go against "the spirit" of the spec. For example, timestamp clamping was not explicitly part of the spec, and on some ungenerous interpretations is forbidden by it, but is compatible with our intentions.

+			The value MUST be reproducible (deterministic) across
+			different executions of the build, depending only on the
+			source code [..]
     [..]
-			Build systems MUST NOT overwrite this variable for
-			child processes to consume if it is already present.
     [..]
+			Build processes MUST NOT unset this variable for child
+			processes if it is already present.

This allows upstream buildsystems to set the variable themselves from a changelog or other file or VCS commit, etc. Originally I thought it best to prohibit rewinding the value, but on second thoughts it's valid to set it to 0 or 1 (as Bazel and some others intend), or perhaps to an older value for older child modules that didn't change.

Some programs like to start with a clean environment and hence would violate the "MUST NOT unset" part, but I don't think it's necessary for us to endorse that. If they want to comply with the spec, they can simply whitelist this variable by default, or set it to 0 or 1.

(In an earlier draft, I had "discriminately unset" instead of "unset" and this would make those programs already-compliant. Then I changed my mind and thought it's unnecessary, but we could change it back if people disagree with my reasoning here.)

That concludes the changes to the spec. Then there are some additional examples:

+			Distributions could set <envar>SOURCE_DATE_EPOCH</envar> by using
+			the value of a changelog file.  For instance, Debian packages can
+			set it to the value of the latest entry of
+			<filename>debian/changelog</filename>.

We moved stuff about d/changelog to Examples, to make the main spec less Debian-oriented.

+			Developers could set <envar>SOURCE_DATE_EPOCH</envar> to the date
+			of the latest commit in their version control system. In this case,
+			it is recommended to also update source file timestamps to this
+			value, or else this may interact badly with any timestamp clamping.

This was a complexity brought up at the meeting. We had some heated discussion about this but in the end I think it's an implementation-specific issue and the spec should warn about it, rather than forbidding timestamp clamping altogether (and we already do it in practise anyway).

+			Though it is not forbidden to set <envar>SOURCE_DATE_EPOCH</envar>
+			several times during a build, such as for different child modules,
+			build processes doing this should ensure that the differing values
+			do not interfere with each other in a nondeterministic way.

Another complexity brought up at the meeting, that we had some heated discussion about. Similar to the previous, I think it's an implementation-specific issue and we should warn about it, but not forbid the whole technique of setting S_D_E several times.

+			One can reasonably assume that all source timestamps are
+			before <envar>SOURCE_DATE_EPOCH</envar> and all builds take
+			place after it. This means we can efficiently both preserve
+			source-based timestamps and omit build-specific timestamps,
+			by rewriting timestamps more recent than <envar>SOURCE_DATE_EPOCH</envar>
+			back to the latter. See for example the <option>--clamp-mtime</option>
+			option to GNU tar.

Unchanged since the meeting (no-one said anything about it), this is my attempt to explain timestamp clamping in a concise way, shorter than dkg's original example in commit 189c1d5.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git