whatsrc new feature: Cargo.lock file tracking
kpcyrd
kpcyrd at archlinux.org
Tue May 7 01:21:54 UTC 2024
hello,
I developed and deployed a new feature:
- when discovering a Cargo.lock file (or similar), import it into the
database
This can be seen when looking at the page for the ripgrep 14.1.0 source
code release:
https://whatsrc.org/artifact/sha256:b21e1926af548e7dcc47bcfa7b121344cf035f8ed5225740d7f055ca4f7f540a
There's now a section titled "Dependency Lockfiles", in case of ripgrep
it lists one file (ripgrep-14.1.0/Cargo.lock) containing a resolved
dependency graph for the Rust library ecosystem (crates). This file is
essential if one would want to reproduce the pre-compiled binaries
provided by upstream (however it's not the only file needed), or reason
about vulnerable dependencies in those binaries:
https://github.com/BurntSushi/ripgrep/releases/tag/14.1.0
This exact same dependency tree of additional source code is also used
by Arch Linux and Alpine, since they integrate with crates.io natively.
(The kind of data that's needed for reproducible builds but not tracked
by Cargo.lock is downstream-specific, e.g. Arch Linux records this in
.BUILDINFO files (gcc version, binutils version, rustc version, ...) -
the reason those are operating system specific is because as soon as you
start having a concept of those tools you are either using an existing
operating system or making your own)
The concept of Cargo.lock files is also mentioned in these two emails:
-
https://lists.reproducible-builds.org/pipermail/rb-general/2024-March/003307.html
-
https://lists.reproducible-builds.org/pipermail/rb-general/2024-March/003317.html
The indexing is still ongoing, and I currently do not yet import the
source code referenced in Cargo.lock files, so many links are 404 at the
moment because they refer to a sha256 I haven't observed yet.
However, a surprising amount of links already works:
- aho-corasick 1.1.2:
https://whatsrc.org/artifact/sha256:b2969dcb958b36655471fc61f7e416fa76033bdd4bfed0678d8fee1e2d07a1f0
- autocfg 1.1.0:
https://whatsrc.org/artifact/sha256:d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa
- cc 1.0.83
https://whatsrc.org/artifact/sha256:f1174fb0b6ec23863f8b971027804a42614e347eafb0a95bf0b12cdae21fc4d0
- cfg-if 1.0.0
https://whatsrc.org/artifact/sha256:baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd
This is because many .tar.gz files referenced by sha256 in the
Cargo.lock are also 100% identical with the _orig.tar files uploaded to
the rust-* namespace in Debian by the Debian-Rust team:
https://wiki.debian.org/Teams/RustPackaging
cheers,
kpcyrd
PS: this also works in some cases now:
https://whatsrc.org/artifact/git:64a5f90b44bc845a4c59f37cb49d9b7693cde6b5
More information about the rb-general
mailing list