[rb-general] Reproducible builds and distributed CI

[Morten Linderud]

On Sun, May 19, 2019 at 01:09:40PM +0300, Lars Wirzenius wrote:
> Greetings,

Yo!
 
> The distributed idea would be that anyone can run their own
> controller, and workers, and can offer workers for any controller.
> A lot of people run a desktop machine, or other similar machine, and
> could provide some of the resources of that, at least some of the
> time, as a worker. Workers can run in isolation, in a highly
> restricted VM, and so this is reasonably safe to do.

Yes. There has been work towards this for a little while. What we
currently have for Debian is a rebuilder setup where we have three
components.

- scheduler which pulls BUILDINFO files and adds to a Redis queue
- worker which pops from the Redis queue and build the package
- visualizer that accepts the rebuild submissions

These rebuild submissions consists of a BUILDINFO file, and a in-toto
link metadata file [0]. Currently there are two rebuilders that has been
rebuilding packages as part of a research project [1].

> This immediately brings up the question of how a controller can trust
> the output of a worker. Otherwise there's a tempatation to run workers
> that produce malicious output.
> 
> I'm thinking that if there's enough workers available, the controller
> could give the same build to more than one worker, and compare the
> result. This is easy if builds are bitwise reproducible. It is not
> very easy otherwise.

We discussed during the 2018 summit the possibility of saying that you
have 10 rebuilders, we need 8 ACKs (same hash) and only accept 2 NACKS
at most. What rebuilders to trust is up to the user, or a baseline can
be distributed by the distribution for ease of use [2].

There is currently an APT transport written for the setup mentioned
above [3].

> What do you think, you who have a lot of experience with reproducible
> builds? Is there any merit in what I'm thinking? Any problems you
> foresee in relying in reproducible builds like this? Is it
> unreasonable to require random software developers to achieve
> reproducible builds?

Yes. We do want multiple independent parties to rebuild packages and
publish the results, this has been discussed for a while I believe [4].


[0] https://in-toto.github.io/
[1] https://ssl.engineering.nyu.edu/blog/2019-01-18-in-toto-paris
[2] https://reproducible-builds.org/events/paris2018/report/#Toc11384_331763073
[3] https://github.com/in-toto/apt-transport-in-toto
[4] https://reproducible-builds.org/docs/sharing-certifications/ 

-- 
Morten Linderud
PGP: 9C02FF419FECBE16
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20190519/12facf4f/attachment.sig>