[Git][reproducible-builds/diffoscope][master] 4 commits: Add support for nar files via Guix.
Chris Lamb (@lamby)
gitlab at salsa.debian.org
Fri Sep 25 17:45:41 UTC 2026
Chris Lamb pushed to branch master at Reproducible Builds / diffoscope
Commits:
df30572e by Chris Lamb at 2026-09-25T10:42:43-07:00
Add support for nar files via Guix.
Nix archive files, or normalised archive files in the case of Guix is
a simple serialisation format.
- - - - -
b4f193dc by Chris Lamb at 2026-09-25T10:43:30-07:00
Update copyright years.
- - - - -
8237da72 by Chris Lamb at 2026-09-25T10:43:30-07:00
Update debian/tests/control.
- - - - -
cbdec07c by Chris Lamb at 2026-09-25T10:44:22-07:00
releasing package diffoscope version 331
- - - - -
15 changed files:
- debian/changelog
- debian/control
- debian/tests/control
- diffoscope/__init__.py
- diffoscope/comparators/__init__.py
- diffoscope/comparators/decompile.py
- + diffoscope/comparators/nar.py
- diffoscope/external_tools.py
- + diffoscope/scripts/restore-nar.scm
- + tests/comparators/test_nar.py
- + tests/data/nar_listing_expected_diff
- + tests/data/test1.nar
- + tests/data/test2.nar
- + tests/data/test3.nar
- + tests/data/test4.nar
Changes:
=====================================
debian/changelog
=====================================
@@ -1,8 +1,14 @@
-diffoscope (331) UNRELEASED; urgency=medium
+diffoscope (331) unstable; urgency=medium
- * WIP (generated upon release).
+ [ Chris Lamb ]
+ * Support radare2 >= 5.9.0. (Closes: reproducible-builds/diffoscope#432)
+ * Update debian/tests/control.
+ * Update copyright years.
+
+ [ Christopher Baines ]
+ * Add support for .nar files via Guix.
- -- Chris Lamb <lamby at debian.org> Mon, 21 Sep 2026 13:37:57 -0700
+ -- Chris Lamb <lamby at debian.org> Fri, 25 Sep 2026 10:44:20 -0700
diffoscope (330) unstable; urgency=medium
=====================================
debian/control
=====================================
@@ -39,6 +39,7 @@ Build-Depends:
gnumeric <!nocheck>,
gnupg-utils <!nocheck>,
gpg <!nocheck>,
+ guix <!nocheck>,
hdf5-tools <!nocheck>,
help2man,
html2text <!nocheck>,
=====================================
debian/tests/control
=====================================
@@ -7,7 +7,7 @@
# $ mv debian/tests/control.tmp debian/tests/control
Tests: pytest-with-recommends
-Depends: python3-all, diffoscope, black, python3-pytest, python3-h5py, file, linux-image-amd64 [amd64] | linux-image-generic [amd64], 7zip, abootimg, acl, apksigner, apktool, binutils-multiarch, bzip2, caca-utils, colord, coreboot-utils [!risv64], db-util, default-jdk-headless | default-jdk | java-sdk, device-tree-compiler, docx2txt, e2fsprogs, enjarify, ffmpeg, fontforge-extras, fonttools, fp-utils [!riscv64 !s390x], genisoimage, gettext, ghc, ghostscript, giflib-tools, gnumeric, gnupg-utils, gpg, hdf5-tools, html2text, imagemagick, jsbeautifier, libarchive-tools, libxmlb-utils, llvm, lz4, lzip, mono-devel (>= 6.14.1+ds-3) [!riscv64] | mono-utils (<< 6.14.1+ds-3) [!riscv64], ocaml, odt2txt, oggvideotools [!s390x], openssh-client, openssl, perl, pgpdump, poppler-utils, procyon-decompiler, python3-pdfminer, r-base-core, rpm2cpio, sng, sqlite3, squashfs-tools, systemd-ukify, tcpdump, u-boot-tools, unzip, wabt, xmlbeans, xxd, xz-utils, zip, zstd, python3-argcomplete, python3-binwalk, python3-defusedxml, python3-distro, python3-guestfs [amd64 arm64 loong64 ppc64el riscv64 s390x], python3-jsondiff, python3-progressbar, python3-pypdf, python3-debian, python3-pyxattr, python3-rpm, python3-tlsh
+Depends: python3-all, diffoscope, black, python3-pytest, python3-h5py, file, linux-image-amd64 [amd64] | linux-image-generic [amd64], 7zip, abootimg, acl, apksigner, apktool, binutils-multiarch, bzip2, caca-utils, colord, coreboot-utils [!risv64], db-util, default-jdk-headless | default-jdk | java-sdk, device-tree-compiler, docx2txt, e2fsprogs, enjarify, ffmpeg, fontforge-extras, fonttools, fp-utils [!riscv64 !s390x], genisoimage, gettext, ghc, ghostscript, giflib-tools, gnumeric, gnupg-utils, gpg, guix, hdf5-tools, html2text, imagemagick, jsbeautifier, libarchive-tools, libxmlb-utils, llvm, lz4, lzip, mono-devel (>= 6.14.1+ds-3) [!riscv64] | mono-utils (<< 6.14.1+ds-3) [!riscv64], ocaml, odt2txt, oggvideotools [!s390x], openssh-client, openssl, perl, pgpdump, poppler-utils, procyon-decompiler, python3-pdfminer, r-base-core, rpm2cpio, sng, sqlite3, squashfs-tools, systemd-ukify, tcpdump, u-boot-tools, unzip, wabt, xmlbeans, xxd, xz-utils, zip, zstd, python3-argcomplete, python3-binwalk, python3-defusedxml, python3-distro, python3-guestfs [amd64 arm64 loong64 ppc64el riscv64 s390x], python3-jsondiff, python3-progressbar, python3-pypdf, python3-debian, python3-pyxattr, python3-rpm, python3-tlsh
Tests: pytest
Depends: python3-all, diffoscope, python3-pytest, python3-h5py, file, python3-tlsh
=====================================
diffoscope/__init__.py
=====================================
@@ -17,4 +17,4 @@
# You should have received a copy of the GNU General Public License
# along with diffoscope. If not, see <https://www.gnu.org/licenses/>.
-VERSION = "330"
+VERSION = "331"
=====================================
diffoscope/comparators/__init__.py
=====================================
@@ -58,6 +58,7 @@ class ComparatorManager:
("bzip2.Bzip2File",),
("cpio.CpioFile",),
("deb.DebFile",),
+ ("nar.NarFile",),
("hdf.Hdf5File",),
("dex.DexFile",),
("elf.ElfFile",),
=====================================
diffoscope/comparators/decompile.py
=====================================
@@ -2,7 +2,7 @@
# diffoscope: in-depth comparison of files, archives, and directories
#
# Copyright © 2020 Jean-Romain Garnier <salsa at jean-romain.com>
-# Copyright © 2020-2022 Chris Lamb <lamby at debian.org>
+# Copyright © 2020-2022, 2026 Chris Lamb <lamby at debian.org>
#
# diffoscope is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
=====================================
diffoscope/comparators/nar.py
=====================================
@@ -0,0 +1,245 @@
+#
+# diffoscope: in-depth comparison of files, archives, and directories
+#
+# Copyright © 2024 Chris Lamb <lamby at debian.org>
+# Copyright © 2022 Christopher Baines <mail at cbaines.net>
+#
+# diffoscope is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# diffoscope is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with diffoscope. If not, see <https://www.gnu.org/licenses/>.
+
+import os
+import stat
+import pathlib
+import logging
+import subprocess
+
+from diffoscope.difference import Difference
+from diffoscope.tools import tool_required
+from diffoscope.tempfiles import get_temporary_directory
+
+from .directory import Directory
+from .symlink import Symlink
+
+from .utils.file import File
+from .utils.archive import Archive, ArchiveMember
+
+logger = logging.getLogger(__name__)
+
+# The top-level node of a NAR is unnamed, so use this when referring to it in
+# file listings.
+NAR_ROOT = "."
+
+# "guix archive --extract" would be the obvious way to unpack a NAR, but it
+# insists on connecting to guix-daemon, so use "guix repl" to call the
+# underlying (guix serialization) code directly instead.
+RESTORE_NAR_SCRIPT = str(
+ pathlib.Path(__file__).parent.parent.joinpath("scripts", "restore-nar.scm")
+)
+
+
+class NarMember(ArchiveMember):
+ @property
+ def path(self):
+ # We unpack the archive in one go, so point at the already-extracted
+ # file rather than letting ArchiveMember create a temporary directory
+ # per-member.
+ return self.container.get_member_path(self._name)
+
+
+class NarDirectory(Directory, NarMember):
+ def __init__(self, container, member_name):
+ NarMember.__init__(self, container, member_name)
+
+ def compare(self, other, source=None):
+ return None
+
+ def has_same_content_as(self, other):
+ return False
+
+ def is_directory(self):
+ return True
+
+ def get_member_names(self):
+ raise ValueError("archives are compared as a whole.") # noqa
+
+ def get_member(self, member_name):
+ raise ValueError("archives are compared as a whole.") # noqa
+
+
+class NarSymlink(Symlink, NarMember):
+ def __init__(self, container, member_name, destination):
+ NarMember.__init__(self, container, member_name)
+ self._destination = destination
+
+ @property
+ def symlink_destination(self):
+ return self._destination
+
+ def is_symlink(self):
+ return True
+
+
+class NarContainer(Archive):
+ def open_archive(self):
+ return True
+
+ def close_archive(self):
+ pass
+
+ def get_member_names(self):
+ self.ensure_unpacked()
+
+ if not self._root_is_directory:
+ return [self._root_member_name]
+
+ return list(self.walk())
+
+ def get_member(self, member_name):
+ path = self.get_member_path(member_name)
+
+ if os.path.islink(path):
+ return NarSymlink(self, member_name, os.readlink(path))
+
+ if os.path.isdir(path):
+ return NarDirectory(self, member_name)
+
+ return NarMember(self, member_name)
+
+ def get_member_path(self, member_name):
+ self.ensure_unpacked()
+
+ if not self._root_is_directory:
+ if member_name != self._root_member_name:
+ raise KeyError(member_name)
+ return self._root
+
+ path = os.path.join(self._root, member_name)
+ if not os.path.lexists(path):
+ raise KeyError(member_name)
+
+ return path
+
+ def extract(self, member_name, dest_dir):
+ # Ignore dest_dir; the archive is already unpacked in one go.
+ return self.get_member_path(member_name)
+
+ def walk(self, relative_to=""):
+ """
+ Yield the name of every node underneath the top-level directory,
+ relative to it and in the order NAR stores them (ie. sorted), with
+ directories appearing before their contents.
+ """
+
+ directory = os.path.join(self._root, relative_to)
+
+ for name in sorted(os.listdir(directory)):
+ member_name = os.path.join(relative_to, name)
+ yield member_name
+
+ path = os.path.join(self._root, member_name)
+ if not os.path.islink(path) and os.path.isdir(path):
+ yield from self.walk(member_name)
+
+ def listing(self):
+ """
+ Describe the archive in terms of what NAR actually records: the type of
+ each node, whether regular files are executable and where symlinks
+ point.
+ """
+
+ self.ensure_unpacked()
+
+ def describe(name, path):
+ mode = os.lstat(path).st_mode
+
+ if stat.S_ISLNK(mode):
+ return f"symlink {name} -> {os.readlink(path)}\n"
+ if stat.S_ISDIR(mode):
+ return f"directory {name}\n"
+ if mode & stat.S_IXUSR:
+ return f"executable {name}\n"
+
+ return f"regular {name}\n"
+
+ yield describe(NAR_ROOT, self._root)
+
+ if self._root_is_directory:
+ for member_name in self.walk():
+ yield describe(
+ member_name, os.path.join(self._root, member_name)
+ )
+
+ def compare(self, other, **kwargs):
+ differences = []
+
+ # Always include the file list; without it, added, removed or
+ # newly-executable files would only show up as a binary diff.
+ if other.source:
+ differences.append(
+ Difference.from_text_readers(
+ self.listing(),
+ other.listing(),
+ self.source.path,
+ other.source.path,
+ source="file list",
+ )
+ )
+
+ differences.extend(super().compare(other, **kwargs))
+
+ return differences
+
+ @tool_required("guix")
+ def ensure_unpacked(self):
+ if hasattr(self, "_root"):
+ return
+
+ self._temp_dir_object = get_temporary_directory(suffix="nar")
+ root = os.path.join(self._temp_dir_object.name, "contents")
+
+ logger.debug("Extracting %s to %s", self.source.path, root)
+
+ with open(self.source.path, "rb") as f:
+ subprocess.run(
+ ("guix", "repl", "--", RESTORE_NAR_SCRIPT, root),
+ stdin=f,
+ # The script is tiny, so byte-compiling it would cost more than
+ # it saves (and would need a writable cache directory).
+ env=dict(os.environ, GUILE_AUTO_COMPILE="0"),
+ capture_output=True,
+ check=True,
+ )
+
+ self._root = root
+ self._root_is_directory = not os.path.islink(root) and os.path.isdir(
+ root
+ )
+
+ if self._root_is_directory:
+ # NAR restores directories read-only, which would prevent us from
+ # removing the unpacked files again later.
+ for dirpath, _, _ in os.walk(root):
+ os.chmod(dirpath, os.lstat(dirpath).st_mode | stat.S_IWUSR)
+ else:
+ # A NAR containing a single file or symlink does not record a name
+ # for it, so fall back to naming it after the archive itself.
+ self._root_member_name = self.get_compressed_content_name(".nar")
+
+
+class NarFile(File):
+ DESCRIPTION = "nar files"
+ CONTAINER_CLASSES = [NarContainer]
+
+ FALLBACK_FILE_TYPE_HEADER_PREFIX = (
+ len("nix-archive-1").to_bytes(8, byteorder="little") + b"nix-arch"
+ )
=====================================
diffoscope/external_tools.py
=====================================
@@ -86,6 +86,7 @@ EXTERNAL_TOOLS = {
"guix": "gnupg",
},
"guestfs": {"debian": "python3-guestfs"},
+ "guix": {"debian": "guix"},
"gzip": {"debian": "gzip", "arch": "gzip", "guix": "gzip"},
"h5dump": {"debian": "hdf5-tools", "arch": "hdf5", "guix": "hdf5"},
"html2text": {"debian": "html2text", "guix": "html2text"},
=====================================
diffoscope/scripts/restore-nar.scm
=====================================
@@ -0,0 +1,11 @@
+;;; Restore the NAR read from standard input into the directory named
+;;; by the first command-line argument.
+;;;
+;;; We cannot use "guix archive --extract" for this as it handles
+;;; --extract from within a (with-store ...) form, and therefore
+;;; insists on connecting to guix-daemon even though restoring a NAR
+;;; needs nothing from the store. This should be fixed in the future.
+
+(use-modules (guix serialization))
+
+(restore-file (current-input-port) (cadr (command-line)))
=====================================
tests/comparators/test_nar.py
=====================================
@@ -0,0 +1,87 @@
+#
+# diffoscope: in-depth comparison of files, archives, and directories
+#
+# Copyright © 2024 Chris Lamb <lamby at debian.org>
+# Copyright © 2022 Christopher Baines <mail at cbaines.net>
+#
+# diffoscope is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# diffoscope is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with diffoscope. If not, see <https://www.gnu.org/licenses/>.
+
+import pytest
+
+from diffoscope.comparators.nar import NarFile
+
+from ..utils.data import load_fixture, assert_diff
+from ..utils.nonexisting import assert_non_existing
+from ..utils.tools import skip_unless_tools_exist
+
+# test1.nar and test2.nar contain a single regular file; test3.nar and
+# test4.nar contain a "dir" directory holding a "text" file and a "link"
+# symlink.
+nar1 = load_fixture("test1.nar")
+nar2 = load_fixture("test2.nar")
+nar3 = load_fixture("test3.nar")
+nar4 = load_fixture("test4.nar")
+
+
+def test_identification(nar1):
+ assert isinstance(nar1, NarFile)
+
+
+def test_no_differences(nar1):
+ difference = nar1.compare(nar1)
+ assert difference is None
+
+
+ at pytest.fixture
+def single_file_differences(nar1, nar2):
+ return nar1.compare(nar2).details
+
+
+ at pytest.fixture
+def differences(nar3, nar4):
+ return nar3.compare(nar4).details
+
+
+ at skip_unless_tools_exist("guix")
+def test_single_file(single_file_differences):
+ # A NAR holding a single file does not record a name for it, so it is
+ # named after the archive itself.
+ assert single_file_differences[0].source1 == "test1"
+ assert single_file_differences[0].source2 == "test2"
+ assert_diff(single_file_differences[0], "text_ascii_expected_diff")
+
+
+ at skip_unless_tools_exist("guix")
+def test_listing(differences):
+ assert_diff(differences[0], "nar_listing_expected_diff")
+
+
+ at skip_unless_tools_exist("guix")
+def test_symlinks(differences):
+ assert differences[1].source1 == "dir/link"
+ assert differences[1].source2 == "dir/link"
+ assert differences[1].comment == "symlink"
+ assert_diff(differences[1], "symlink_expected_diff")
+
+
+ at skip_unless_tools_exist("guix")
+def test_text_file(differences):
+ assert differences[2].source1 == "dir/text"
+ assert differences[2].source2 == "dir/text"
+ assert_diff(differences[2], "text_ascii_expected_diff")
+
+
+ at skip_unless_tools_exist("guix")
+def test_compare_non_existing(monkeypatch, nar3):
+ assert_non_existing(monkeypatch, nar3)
=====================================
tests/data/nar_listing_expected_diff
=====================================
@@ -0,0 +1,6 @@
+@@ -1,4 +1,4 @@
+ directory .
+ directory dir
+-symlink dir/link -> broken
++symlink dir/link -> really-broken
+ regular dir/text
=====================================
tests/data/test1.nar
=====================================
Binary files /dev/null and b/tests/data/test1.nar differ
=====================================
tests/data/test2.nar
=====================================
Binary files /dev/null and b/tests/data/test2.nar differ
=====================================
tests/data/test3.nar
=====================================
Binary files /dev/null and b/tests/data/test3.nar differ
=====================================
tests/data/test4.nar
=====================================
Binary files /dev/null and b/tests/data/test4.nar differ
View it on GitLab: https://salsa.debian.org/reproducible-builds/diffoscope/-/compare/ba64aec70232a76c7e668a5e679cc7daef2281c2...cbdec07c6adfbca6f286b2c6c18488d9e82552e7
--
View it on GitLab: https://salsa.debian.org/reproducible-builds/diffoscope/-/compare/ba64aec70232a76c7e668a5e679cc7daef2281c2...cbdec07c6adfbca6f286b2c6c18488d9e82552e7
You're receiving this email because of your account on salsa.debian.org. Manage all notifications: https://salsa.debian.org/-/profile/notifications | Help: https://salsa.debian.org/help
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-commits/attachments/20260925/d27ab4b9/attachment.htm>
More information about the rb-commits
mailing list