whatsrc new feature: Cargo.lock file tracking

kpcyrd kpcyrd at archlinux.org
Tue May 7 01:21:54 UTC 2024


hello,

I developed and deployed a new feature:

- when discovering a Cargo.lock file (or similar), import it into the 
database

This can be seen when looking at the page for the ripgrep 14.1.0 source 
code release:

https://whatsrc.org/artifact/sha256:b21e1926af548e7dcc47bcfa7b121344cf035f8ed5225740d7f055ca4f7f540a

There's now a section titled "Dependency Lockfiles", in case of ripgrep 
it lists one file (ripgrep-14.1.0/Cargo.lock) containing a resolved 
dependency graph for the Rust library ecosystem (crates). This file is 
essential if one would want to reproduce the pre-compiled binaries 
provided by upstream (however it's not the only file needed), or reason 
about vulnerable dependencies in those binaries:

https://github.com/BurntSushi/ripgrep/releases/tag/14.1.0

This exact same dependency tree of additional source code is also used 
by Arch Linux and Alpine, since they integrate with crates.io natively.

(The kind of data that's needed for reproducible builds but not tracked 
by Cargo.lock is downstream-specific, e.g. Arch Linux records this in 
.BUILDINFO files (gcc version, binutils version, rustc version, ...) - 
the reason those are operating system specific is because as soon as you 
start having a concept of those tools you are either using an existing 
operating system or making your own)

The concept of Cargo.lock files is also mentioned in these two emails:

- 
https://lists.reproducible-builds.org/pipermail/rb-general/2024-March/003307.html
- 
https://lists.reproducible-builds.org/pipermail/rb-general/2024-March/003317.html

The indexing is still ongoing, and I currently do not yet import the 
source code referenced in Cargo.lock files, so many links are 404 at the 
moment because they refer to a sha256 I haven't observed yet.

However, a surprising amount of links already works:

- aho-corasick 1.1.2: 
https://whatsrc.org/artifact/sha256:b2969dcb958b36655471fc61f7e416fa76033bdd4bfed0678d8fee1e2d07a1f0
- autocfg 1.1.0: 
https://whatsrc.org/artifact/sha256:d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa
- cc 1.0.83 
https://whatsrc.org/artifact/sha256:f1174fb0b6ec23863f8b971027804a42614e347eafb0a95bf0b12cdae21fc4d0
- cfg-if 1.0.0 
https://whatsrc.org/artifact/sha256:baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd

This is because many .tar.gz files referenced by sha256 in the 
Cargo.lock are also 100% identical with the _orig.tar files uploaded to 
the rust-* namespace in Debian by the Debian-Rust team:

https://wiki.debian.org/Teams/RustPackaging

cheers,
kpcyrd

PS: this also works in some cases now:

https://whatsrc.org/artifact/git:64a5f90b44bc845a4c59f37cb49d9b7693cde6b5


More information about the rb-general mailing list