[rb-general] [arch-projects] [devtools] [PATCH] Support reproducible builds

[Eli Schwartz]

On 10/30/2017 11:17 AM, Eli Schwartz wrote:
> Recent development versions of makepkg support reproducible builds
> through the environment variable SOURCE_DATE_EPOCH. Pass this variable
> through makechrootpkg to makepkg when available.
> 
> Also initialize SOURCE_DATE_EPOCH whenever running archbuild to enforce
> reproducible builds for repository packages.
> 
> Signed-off-by: Eli Schwartz <eschwartz at archlinux.org>
> ---
>  archbuild.in     | 7 ++++++-
>  lib/archroot.sh  | 6 ++++--
>  makechrootpkg.in | 5 +++--
>  3 files changed, 13 insertions(+), 5 deletions(-)
> 
> diff --git a/archbuild.in b/archbuild.in
> index 8339aef..1e5b582 100644
> --- a/archbuild.in
> +++ b/archbuild.in
> @@ -39,7 +39,7 @@ while getopts 'hcr:' arg; do
>  	esac
>  done
>  
> -check_root
> +check_root SOURCE_DATE_EPOCH
>  
>  # Pass all arguments after -- right to makepkg
>  makechrootpkg_args+=("${@:$OPTIND}")
> @@ -74,5 +74,10 @@ else
>                  pacman -Syu --noconfirm || abort
>  fi
>  
> +# Always build official packages reproducibly
> +if [[ ! -v SOURCE_DATE_EPOCH ]]; then
> +	export SOURCE_DATE_EPOCH=$(date +%s)
> +fi
> +
>  msg "Building in chroot for [%s] (%s)..." "${repo}" "${arch}"
>  exec makechrootpkg -r "${chroots}/${repo}-${arch}" "${makechrootpkg_args[@]}"
> diff --git a/lib/archroot.sh b/lib/archroot.sh
> index 98fd2cf..f279603 100644
> --- a/lib/archroot.sh
> +++ b/lib/archroot.sh
> @@ -6,13 +6,15 @@
>  CHROOT_VERSION='v4'
>  
>  ##
> -#  usage : check_root
> +#  usage : check_root $keepenv
>  ##
>  orig_argv=("$0" "$@")
>  check_root() {
> +	local keepenv=$1
> +
>  	(( EUID == 0 )) && return
>  	if type -P sudo >/dev/null; then
> -		exec sudo -- "${orig_argv[@]}"
> +		exec sudo --preserve-env=$keepenv -- "${orig_argv[@]}"
>  	else
>  		exec su root -c "$(printf ' %q' "${orig_argv[@]}")"
>  	fi
> diff --git a/makechrootpkg.in b/makechrootpkg.in
> index be9b33f..d43a130 100644
> --- a/makechrootpkg.in
> +++ b/makechrootpkg.in
> @@ -200,6 +200,7 @@ EOF
>  	{
>  		printf '#!/bin/bash\n'
>  		declare -f _chrootbuild
> +		declare -p SOURCE_DATE_EPOCH 2>/dev/null
>  		printf '_chrootbuild "$@" || exit\n'
>  
>  		if [[ $run_namcap = true ]]; then
> @@ -226,7 +227,7 @@ _chrootbuild() {
>  	# use "$" in arguments to commands with "sudo -i".  ${foo} or
>  	# ${1} is OK, but $foo or $1 isn't.
>  	# https://bugzilla.sudo.ws/show_bug.cgi?id=765
> -	sudo -iu builduser bash -c 'cd /startdir; makepkg "$@"' -bash "$@"
> +	sudo --preserve-env=SOURCE_DATE_EPOCH -iu builduser bash -c 'cd /startdir; makepkg "$@"' -bash "$@"
>  }
>  
>  _chrootnamcap() {
> @@ -338,7 +339,7 @@ main() {
>  	[[ -n $makepkg_user && -z $(id -u "$makepkg_user") ]] && die 'Invalid makepkg user.'
>  	makepkg_user=${makepkg_user:-${SUDO_USER:-$USER}}
>  
> -	check_root
> +	check_root SOURCE_DATE_EPOCH
>  
>  	# Canonicalize chrootdir, getting rid of trailing /
>  	chrootdir=$(readlink -e "$passeddir")

As requested by h0lger, I'm sending this on to the Reproducible Builds
mailing list as you guys are probably interested in this too. ;)

The basic idea of this patch is that:

1) makepkg will use SOURCE_DATE_EPOCH, if available, as the canonical
release date of a pacman package, and additionally make sure that
(assuming the underlying system environment is the same), software is
built in a reproducible manner e.g. by unifying source file mtime,
ensuring SOURCE_DATE_EPOCH is exported, ensuring that makepkg-created
metadata files are reproducible e.g. the .MTREE of packaged files, the
builddate, etc.

2) makechrootpkg passes an existing SOURCE_DATE_EPOCH through sudo and
into our systemd-nspawn build container.

3) archbuild is used to build official repo packages, and will make
SOURCE_DATE_EPOCH default to the date archbuild was initially run at,
which will become the canonical package release date.

See for example makepkg patches surrounding:
https://lists.archlinux.org/pipermail/pacman-dev/2017-May/022034.html
https://lists.archlinux.org/pipermail/pacman-dev/2017-August/022113.html

-- 
Eli Schwartz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20171030/eaa4195e/attachment.sig>