[rb-general] [FOSDEM16] Reproducible FreeBSD and variants

Holger Levsen holger at layer-acht.org
Fri Feb 12 15:04:40 CET 2016

Hi Steven,

(I believe) somehow your mail did not make it to the rb-general list, are you 
subscribed? (leaving full quoted context for that reason…)

On Donnerstag, 11. Februar 2016, Steven Chamberlain wrote:
> Holger Levsen wrote:
> > > I've been trying to get started testing Debian GNU/kFreeBSD package
> > > reproducibility (and I'd like it being mass-rebuilt anyway to find
> > > FTBFS or other bugs sooner).  Just a few weeks ago I got Jenkins
> > > working on kfreebsd (after porting some dependency) so it can run as a
> > > master or slave.
> > 
> > cool. what are the specs, roughly?
> Currently this is a Core i7-980X, 24 GiB RAM, 2x 1.4TB disks.


> It is a server I rent out for business uses, but outside office hours
> it was quite idle so I schedule jobs to run overnight and on weekends:
> some rebootstrap jobs, and rebuilding debian-installer per Git commit.
> I'd like to add reproducible jobs once I figure out how they work.

I'm not sure I'll be comfortable building "anything+everything" on this box 
then. ("rented for business uses")

The jenkins set up is not designed with security in mind, in the sense that I 
don't trust the box on *anything*, it's ok to produce QA results, but… I'm not 
comfortable giving it access to trusted machines. (even in jails, etc & YMMV, 

> I've decided to open up the Jenkins web interace now (except HTTP POST
> because I really don't trust its security!)
> http://jenkins.kfreebsd.eu/jenkins/


> I've put Jenkins master in its own separate chroot jail.  The jobs run
> one at a time, in another separate sid chroot jail.  Jails prevent
> access to files or devices of the host, and I can firewall their network
> access if I want to.
> > I dont see us moving to DSA
> > maintained host. patches for that (="less sudo usage…") welcome ;)
> This is a major concern for me at the moment, as I allow Jenkins to do
> many things on the host (outside of any jail) via sudo to set up the sid
> chroot.  I'm experimenting with better ways to do this.

well, things should probably work nicely in a jail where jenkins thinks it can 
do what it wants :-)

> I'd started out with sbuild, but it is incompatible with jails.  It was
> also really slow, and that's a major concern for me with the limited
> resources I have.  Optimizing is fun also.

I dont think I want to build reproducible Debian packages on one arch in a 
different way than on the others. So pbuilder for now (until we maybe switch 
to sbuild with the patch for reproducible rebuilds once we're rebuilding 
against sid…)

> With ZFS I can set sync=disabled on a whole chroot which is similar to
> using Linux 'eatmydata', and makes APT/dpkg stages run really fast.
> I'm also trying ccache, and preserving the cache directory between job
> runs.  Hasn't helped much though.
> What might be nice is if the host's sshd could set up so that, upon
> login as user 'jenkins', it would give a root shell in a freshly-created
> jail.  ZFS snapshots and clones could make that really fast.  In that
> kind of setup, Jenkins need not run on build machines (saving hundreds
> of MiB RAM).
> Mostly I'd like to reduce the setup/teardown time so that a small
> package like 'hello' takes only the smallest amount of time to build.
> I think this is where most time would be wasted given how many small
> packages there are in the archive.

I don't think that time is *that* relevant as long as you use tmpfs (or 
something similar) and have enough RAM. maybe then it even becomes irrelevant 

Also we have packages in the archive which build twice in <60s (on our amd64 
"hw"), while the average time is 8min. 

So currently we're building 60*24/8*32=5760 packages a day (/8 because average 
build time is 8min and *32 because we have 32 amd64 builders).

If you bring down the setup time to 0s you would "only" build 6560 packages a 
day (60*24/7*32), or 13.8% faster.

I suspect the numbers will look similar for armhf, but I'll leave that as an 
excercize for the reader :)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20160212/61a06ad4/attachment.sig>

More information about the rb-general mailing list