[rb-general] Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

[Philip Hands]

"Chris Lamb" <lamby at debian.org> writes:

> [adding rb-general at lists.reproducible-builds.org to CC]
>
> Hi Colin,
>
>> This is all from dubious memory, but I suspect my setup at the time was
>> roughly an amd64 system with:
>> 
>>   deb [arch=amd64] <local partial mirror>
>>   deb <official mirror>
>> 
>> ... on the grounds that my local partial mirror didn't have space for
>> both amd64 and i386.
>
> Apologies for the delay in getting back to you all here.
>
> I've got this working locally here although we require the following
> change to the gen-sources.list.udeb script. Basically, we need print
> three columns if we have "[options]", otherwise we just print two:
>
> diff --git a/build/util/gen-sources.list.udeb b/build/util/gen-sources.list.udeb
> index 539345a45..ac416266a 100755
> --- a/build/util/gen-sources.list.udeb
> +++ b/build/util/gen-sources.list.udeb
> @@ -36,10 +36,9 @@ get_mirrors() {
>  		[ -s $file ] || continue
>  		grep '^deb[[:space:]]' $file | \
>  		   grep -v '^deb[[:space:]]\+cdrom:' | \
> -		   sed 's,^deb \[[^]]*\] ,deb ,' | \
>  		   grep -v '\(security.debian.org\|volatile.debian.\(net\|org\)\)' | \
>  		   grep '[[:space:]]main' | \
> -		   awk '{print $1 " " $2}' | \
> +		   awk '{ print (substr($2, 0, 1) == "[") ? $1 " " $2 " " $3 : $1 " " $2 }' | \
>  		   sed 's,^deb file,deb copy,' | \
>  		   sed 's,/* *$,,'
>  	done
>
> How does this look to you? Shell "golf" suggestions welcome,
> naturally. (I tried a few sed variants but it got a bit messy.)

I failed to resist that, so I _think_ this sed command implements the
same effect as those greps/seds & awk:

  sed -ne '/^deb[[:space:]]\+cdrom/d;
           /\(security.debian.org\|volatile.debian.\(net\|org\)\)/d;
           /^deb[[:space:]]\+.*[[:space:]]\+main/{
             s,\(deb[[:space:]]\+\(\[[^]]*\][[:space:]]\+\|\)[^[:space:]]*\).*$,\1,;
             s,^deb file,deb copy,;
             s,/* *$,,;
             p
           }'

I only tested that with a few fairly simple test cases, so hopefully
you've got some nice test data.

BTW I note that in the original (and therefore in this too) that the
exclusion of cdrom: and the 'deb file' to 'deb copy' bits only work if
there's no [option] bit in the line -- was that an oversight?

Also, I'm asking myself: Why all the [[:space:]]\+ stuff if one then
seds for '^deb file' and '/* *$' at the end? -- I think one should
choose to do one or the other of those throughout.

As you say though, the sed for handling the optional bit is rather nasty
to read.  I'm now wondering if the whole thing might not be better done
in a single awk invocation...

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/    http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,    GERMANY
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20190606/433f6b50/attachment.sig>