[rb-general] Core Debian reproducibility: 57% and rising!

[Holger Levsen]

On Sun, Oct 28, 2018 at 05:42:58PM -0700, Vagrant Cascadian wrote:
> On 2018-10-27, Vagrant Cascadian wrote:
> > Ok, I've found at least one package in the required set, with three
> > distinct .buildinfo files that converged on the same .deb:
> ...
> > We're now officially beyond mere theory!
> 
> And now I've found 88 binary packages!
> 
> I used the crude script below [...}

I've put this script now on jenkins and exteneded it a bit, so it now
analyses all the packages in unstable/amd64.

The result can currently be seen at
https://jenkins.debian.net/job/reproducible_compare_Debian_sha1sums/47/console
and while this run hasnt finished yet, these are it's current results:

reproducible packages: 685: (53.9700%)
unreproducible packages: 584: (46.0200%)

Besides some conceptual problems (eg no verification of the buildinfo files
signatures or the signers) there is another performance problem:

wget https://buildinfo.debian.net/api/v1/buildinfos/checksums/sha1/$sha1

takes 15 seconds on average, so downloadling 60000 .buildinfo files
takes 250 hours. (this is still bad from a user POV, eg, I have 1800
binary packages installed on my machine here, downloading those 1800
.buildinfo files will still take >7h.)

I currently don't have an idea how to address this nicely.

still, it's very great that we're finally starting to gather real world
data for Debian.


-- 
tschau,
	Holger

-------------------------------------------------------------------------------
               holger@(debian|reproducible-builds|layer-acht).org
       PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20190220/fe449f0f/attachment.sig>