Call for real-world scenarios prevented by RB practices

David A. Wheeler dwheeler at
Sat Mar 26 17:23:12 UTC 2022

On Sat, 26 Mar 2022 10:06:21 +0100, Dan Shearer <dan at> wrote:
> Reproducibility ... has driven code reviews and many
> bugfixes, and there is a boolean pass/fail way of measuring reproducibility.
> But reproducibility doesn't improve security any more or less than better
> coding practices, chain of custody efforts or commit signing ...

I disagree; there are fundamental *differences* in these approaches.
Attackers will typically use the kind of attack that has the best impact/cost ratio
for them, and different attacks require *different* countermeasures.
In general, attackers keep moving backwards in the software production process
as the later parts of the process are hardened.

Today, most attackers focus on attacking operational systems through the vulnerabilities
already present in them. Training on how to develop secure software,
code reviews, static analysis tools, fuzzers, etc.,
are useful countermeasures for this kind of attack. Most software projects need to
up their game here. The good news is that some projects *have* significantly
upped their game against this attack, but this means that attackers start
applying *other* kinds of attack by moving further up the supply chain.

Some attackers are now attacking the *distribution* channels through
typosquatting, subverted package repos or developer accounts, etc.
Chain of custody & signing can help here.

A few attackers are now targeting the *build* process (which is earlier than the
distribution process). SolarWinds' Orion subversion is an example.
You can (and should) build environments as a countermeasure, but that's
a weak approach; the attacker only has to succeed once for that to fail.
Verified reproducible builds are the main strong countermeasure to this attack.
Such attacks have *already* occurred, so this isn't theoretical.

Of course, if your project hasn't worked on finding vulnerabilities in the source code,
or haven't hardened your distribution channel, using
reproducible builds should probably *NOT* be
your focus. For those projects, I agree that reproducibility should not
be "high on their priority lists"... because they have other problems.
But some projects *are* starting to get sufficiently strong in their countermeasures
against those attacks that subverted build environments are becoming a real concern.
In addition, it will take a long time for reproducible builds to be widely deployed.
So we (the IT community) have to work on reproducible builds *now*, even though
other kinds of attacks are currently more common.

A few attackers are also attacking the earlier source repositories and developer
environments. There are other (different) countermeasures for those as well.

>  - and, for
> example, none of these are a defence again Ken Thompson's famous 1984 paper
> despite some of the best minds in computer science working on it. 

Correct but irrelevant. If you want to counter the trusting trust attack - which is
a WAY earlier attack - then you need another *different* countermeasure.
In this case the main countermeasures are Diverse Double-Compiling (DDC) (yay) and
bootstappable builds. Those *DO* counter the trusting trust attack.
I guess you could consider these two countermeasures as competitors,
but they really aren't; they work well together.
DDC builds on reproducible builds, so once you get
reproducible builds of compilers it's not a huge step.

In short: because attackers can attack different parts of the software development /
distribution / operations process, using different kinds of attacks, you have to have
different countermeasures to counter them all along the way.

Once you counter the trusting trust attack, combined with those other types,
I believe you have reached the end of
software attacks (there is no "lower turtle").
Attacks "even earlier" are possible, but now you're switching to hardware attacks
or physical threats to people ("write this code or I'll kill your family").
Forcing attackers to switch to those kinds of attacks, which are often
more challenging & often don't scale as well, is a win in my opinion.

> Nevertheless my point stands I think, that none of the approaches I
> listed address the fundamental trust problem.

No *single* countermeasure addresses all problems, quite true.
But a combination of countermeasures *does* dramatically reduce risk
(likelihood & impact). We don't need a silver bullet, just a set of
countermeasures to reduce risk to acceptable levels.

> For that reason, I don't think reproducibility is a good single-purpose driver
> for general-purpose projects.

I agree with you there, 110%.
Verified reproducible builds give you confidence in the build process,
but you may have built garbage.

You need different countermeasures for different kinds of attacks.

> And we certainly don't see good
> solutions being adopted to the issues of unbounded complexity, or why it takes
> a stack multiple millions of lines of lines of code high to present me with a
> dialogue box that says "Hello". These are human problems, not really technical ones.

Those are definitely problems!

Those are human problems - especially economic ones -
but I think there are ways technologies can help. In particular,
most users don't have any insight into this absurd complexity, and
there's no pushback against it. I have hopes that the recent efforts for
more transparency, especially software bill of materials (SBOMs)
and eliminating dependencies with known vulnerabilities,
may help here. Once people realize they depend on 5,000 components
with known vulnerabilities, they'll demand they be fixed, and the
true cost of reusing unnecessary components will be revealed. That
will finally give some organizations a reason to reduce their dependencies
(because there's an economic cost to them).
It won't be perfect (nothing ever is), but it will at least start to
provide a counterweight to endlessly adding unnecessary complexity.
If you have other suggestions on how to counter these I'd
love to hear them :-).

--- David A. Wheeler

More information about the rb-general mailing list