[Git][reproducible-builds/diffoscope][split-pkgs] 5 commits: Add a missing comma.

Mattia Rizzolo gitlab at salsa.debian.org
Wed Nov 25 15:32:08 UTC 2020



Mattia Rizzolo pushed to branch split-pkgs at Reproducible Builds / diffoscope


Commits:
f4952da2 by Chris Lamb at 2020-11-23T10:55:24+00:00
Add a missing comma.

- - - - -
a615cc31 by Chris Lamb at 2020-11-23T11:03:04+00:00
Don't depends on radare2 in the Debian autopkgtests; it will not be in bullseye due to security considerations (see #950372). (Closes: #975313)

- - - - -
793376de by Chris Lamb at 2020-11-25T10:47:27+00:00
Don't rely on radare2 being available. Thanks, Matthias Klose. (Re: #975313)

- - - - -
edf77bc3 by Mattia Rizzolo at 2020-11-25T16:30:19+01:00
Add a list of big or niche tools.

Signed-off-by: Mattia Rizzolo <mattia at debian.org>

- - - - -
2ed38723 by Mattia Rizzolo at 2020-11-25T16:30:50+01:00
debian: Split the diffoscope package into 2, so users who wants to do so can more easily avoid the slew of Recommends

diffoscope-minimal: contains all the files, but avoid some recommended
packages that are considered too "huge" for the common use.
diffoscope: keeps behaving like the current one, installing as much as possible

Closes: #975261
Signed-off-by: Mattia Rizzolo <mattia at debian.org>

- - - - -


11 changed files:

- debian/clean
- debian/control
- + debian/diffoscope-minimal.docs
- debian/diffoscope.install → debian/diffoscope-minimal.install
- debian/diffoscope.manpages → debian/diffoscope-minimal.manpages
- debian/rules
- debian/tests/control
- debian/tests/control.sh
- debian/tests/pytest
- diffoscope/external_tools.py
- diffoscope/main.py


Changes:

=====================================
debian/clean
=====================================
@@ -1,5 +1,5 @@
 debian/diffoscope.1
-debian/diffoscope.bash-completion
+debian/diffoscope-minimal.bash-completion
 diffoscope.egg-info/
 .cache/
 .pytest_cache/


=====================================
debian/control
=====================================
@@ -94,18 +94,43 @@ Homepage: https://diffoscope.org
 Vcs-Git: https://salsa.debian.org/reproducible-builds/diffoscope.git
 Vcs-Browser: https://salsa.debian.org/reproducible-builds/diffoscope
 
-Package: diffoscope
+Package: diffoscope-minimal
 Architecture: all
-Suggests:
- libjs-jquery,
 Depends:
  python3-distutils,
  python3-pkg-resources,
  ${misc:Depends},
  ${python3:Depends},
+Recommends:
+ ${diffoscope-minimal:Recommends},
+ ${python3:Recommends},
+Breaks: diffoscope (<< 162~)
+Replaces: diffoscope (<< 162~)
+Description: in-depth comparison of files, archives, and directories
+ diffoscope will try to get to the bottom of what makes files or directories
+ different. It will recursively unpack archives of many kinds and transform
+ various binary formats into more human-readable form to compare them. It can
+ compare two tarballs, ISO images, or PDF just as easily. The differences can
+ be shown in a text or HTML report.
+ .
+ This package only recommends a partial set of the supported 3rd party tools
+ needed to produce file-format-specific comparisons, excluding those that are
+ considered too large or too niche for the general use.
+ .
+ diffoscope is developed as part of the “reproducible builds” project.
+
+Package: diffoscope
+Architecture: all
+Suggests:
+ libjs-jquery,
+Depends:
+ diffoscope-minimal,
+ ${misc:Depends},
 Recommends:
  ${diffoscope:Recommends},
  ${python3:Recommends},
+Enhances:
+ diffoscope-core,
 Description: in-depth comparison of files, archives, and directories
  diffoscope will try to get to the bottom of what makes files or directories
  different. It will recursively unpack archives of many kinds and transform
@@ -113,6 +138,9 @@ Description: in-depth comparison of files, archives, and directories
  compare two tarballs, ISO images, or PDF just as easily. The differences can
  be shown in a text or HTML report.
  .
+ This is a dependency package that recommends the full set of external tools,
+ to support as many type of files as possible.
+ .
  ${diffoscope:Description}
  .
  diffoscope is developed as part of the “reproducible builds” project.


=====================================
debian/diffoscope-minimal.docs
=====================================
@@ -0,0 +1 @@
+diffoscope.docs
\ No newline at end of file


=====================================
debian/diffoscope.install → debian/diffoscope-minimal.install
=====================================


=====================================
debian/diffoscope.manpages → debian/diffoscope-minimal.manpages
=====================================


=====================================
debian/rules
=====================================
@@ -18,7 +18,7 @@ ifneq ($(DEB_VERSION),$(PY_VERSION))
 endif
 
 export PATH := $(CURDIR)/bin:$(PATH)
-export PYBUILD_DESTDIR = debian/diffoscope
+export PYBUILD_DESTDIR = debian/diffoscope-minimal
 export PYBUILD_TEST_ARGS = -vv -r sxX -l --cov=diffoscope --cov-report=term-missing --cov-report=html
 export PYBUILD_BEFORE_TEST = flake8 --config=/dev/null --select=F821
 export PYBUILD_AFTER_TEST = debian/tests/basic-command-line
@@ -37,20 +37,29 @@ endif
 doc/%.1:
 	$(MAKE) -C doc $*.1
 
-debian/diffoscope.bash-completion:
+debian/diffoscope-minimal.bash-completion:
 	register-python-argcomplete3 diffoscope > $@
 
-execute_after_dh_auto_build: debian/diffoscope.bash-completion
+execute_after_dh_auto_build: debian/diffoscope-minimal.bash-completion
 	dh_auto_build -O--buildsystem=makefile -Ddoc
 
 override_dh_python3:
-	dh_python3 -p diffoscope \
+	# dh_python3 will only work on binaries shipping files
+	dh_python3 -p diffoscope-minimal \
 		--depends-section=distro_detection \
 		--recommends-section=cmdline \
-		--recommends-section=comparators \
+		--recommends-section=comparators
+	grep Recommends debian/diffoscope-minimal.substvars >> debian/diffoscope.substvars
+	# Remove big Recommends from the -minimal package, leaving them only in the full one
+	sed -i 's/python3-guestfs,//' debian/diffoscope-minimal.substvars
+	sed -i 's/python3-binwalk,//' debian/diffoscope-minimal.substvars
 
 execute_before_dh_gencontrol:
-	bin/diffoscope --list-debian-substvars >> debian/diffoscope.substvars
+	bin/diffoscope --list-debian-substvars | tee -a \
+		debian/diffoscope.substvars \
+		debian/diffoscope-minimal.substvars \
+		> /dev/null
+	head debian/*.substvars
 
 diffoscope/presenters/icon.py: favicon.png
 	(echo '# Generated from $@'; \


=====================================
debian/tests/control
=====================================
@@ -7,7 +7,7 @@
 #   $ mv debian/tests/control.tmp debian/tests/control
 
 Tests: pytest-with-recommends
-Depends: python3-all, diffoscope, black, python3-pytest, python3-h5py, file, linux-image-amd64 [amd64] | linux-image-generic [amd64], abootimg, acl, apksigner, apktool [!ppc64el !s390x], binutils-multiarch, bzip2, caca-utils, colord, db-util, default-jdk-headless | default-jdk | java-sdk, device-tree-compiler, docx2txt, e2fsprogs, enjarify, ffmpeg, fontforge-extras, fp-utils [!ppc64el !s390x], genisoimage, gettext, ghc, ghostscript, giflib-tools, gnumeric, gnupg, gnupg-utils, hdf5-tools, imagemagick, jsbeautifier, libarchive-tools, llvm, lz4 | liblz4-tool, mono-utils, ocaml-nox, odt2txt, oggvideotools [!s390x], openssh-client, openssl, pgpdump, poppler-utils, procyon-decompiler, python3-pdfminer, r-base-core, radare2, rpm2cpio, sng, sqlite3, squashfs-tools, tcpdump, unzip, wabt, xmlbeans, xxd | vim-common, xz-utils, zip, zstd, python3-argcomplete, python3-binwalk, python3-defusedxml, python3-distro, python3-guestfs, python3-jsondiff, python3-progressbar, python3-pypdf2, python3-debian, python3-pyxattr, python3-rpm, python3-tlsh
+Depends: python3-all, diffoscope, black, python3-pytest, python3-h5py, file, linux-image-amd64 [amd64] | linux-image-generic [amd64], abootimg, acl, apksigner, apktool [!ppc64el !s390x], binutils-multiarch, bzip2, caca-utils, colord, db-util, default-jdk-headless | default-jdk | java-sdk, device-tree-compiler, docx2txt, e2fsprogs, enjarify, ffmpeg, fontforge-extras, fp-utils [!ppc64el !s390x], genisoimage, gettext, ghc, ghostscript, giflib-tools, gnumeric, gnupg, gnupg-utils, hdf5-tools, imagemagick, jsbeautifier, libarchive-tools, llvm, lz4 | liblz4-tool, mono-utils, ocaml-nox, odt2txt, oggvideotools [!s390x], openssh-client, openssl, pgpdump, poppler-utils, procyon-decompiler, python3-pdfminer, r-base-core, rpm2cpio, sng, sqlite3, squashfs-tools, tcpdump, unzip, wabt, xmlbeans, xxd | vim-common, xz-utils, zip, zstd, python3-argcomplete, python3-binwalk, python3-defusedxml, python3-distro, python3-guestfs, python3-jsondiff, python3-progressbar, python3-pypdf2, python3-debian, python3-pyxattr, python3-rpm, python3-tlsh
 
 Tests: pytest
 Depends: python3-all, diffoscope, python3-pytest, python3-h5py, file, python3-tlsh


=====================================
debian/tests/control.sh
=====================================
@@ -15,6 +15,8 @@ sed -i "s#%PYRECOMMENDS%#$(python3 -c "import distutils.core; \
 	print(', '.join(sorted(['python3-{}'.format(x) for y in setup.extras_require.values() for x in y])))" \
 )#" debian/tests/control.tmp
 
+# Don't test-depend on radare2; not in bullseye for security reasons. (#950372)
+sed -i "s#radare2, ##" debian/tests/control.tmp
 
 sed -i "s,python3-python-debian,python3-debian," debian/tests/control.tmp
 sed -i "s,python3-rpm-python,python3-rpm," debian/tests/control.tmp


=====================================
debian/tests/pytest
=====================================
@@ -9,7 +9,7 @@ fi
 
 export LIBGUESTFS_MEMSIZE=128
 if [ "$(basename "$0")" = "pytest-with-recommends" ]; then
-    export DIFFOSCOPE_FAIL_TESTS_ON_MISSING_TOOLS="apktool zipinfo pedump oggDump ppudump cbfstool otool lipo"
+    export DIFFOSCOPE_FAIL_TESTS_ON_MISSING_TOOLS="apktool zipinfo pedump oggDump ppudump cbfstool otool lipo radare2"
 fi
 
 for py in $(py3versions -s); do


=====================================
diffoscope/external_tools.py
=====================================
@@ -239,3 +239,23 @@ REMAPPED_TOOL_NAMES = {}
 # If we only use POSIX CLI options and the output is identical to the system
 # version (so that our tests don't break) then it's unnecessary to add it here.
 GNU_TOOL_NAMES = {"diff", "readelf", "objcopy", "objdump"}
+
+# Set of tools considered "large" their installation size, or too niche in
+# their target users.  To be easily excluded from installation if not
+# specifically required.
+# These are the names of the tools, not package names.
+HUGE_TOOLS = {
+    "ghc",
+    "ocamlobjinfo",
+    "llvm-bcanalyzer",
+    "llvm-config",
+    "llvm-dis",
+    "ppudump",
+    "javap",
+    "ssconvert",
+    "apktool",
+    "apksigner",
+    "pedump",
+    "radare2",
+    "dumpxsb",
+}


=====================================
diffoscope/main.py
=====================================
@@ -49,7 +49,7 @@ from .profiling import ProfileManager, profile
 from .tempfiles import clean_all_temp_files, get_tempdir_free_space
 from .difference import Difference
 from .comparators import ComparatorManager
-from .external_tools import EXTERNAL_TOOLS
+from .external_tools import EXTERNAL_TOOLS, HUGE_TOOLS
 from .presenters.html import JQUERY_SYSTEM_LOCATIONS
 from .presenters.formats import PresenterManager
 from .comparators.utils.compare import compare_root_paths
@@ -564,7 +564,7 @@ class ListDebianSubstvarsAction(argparse._StoreTrueAction):
         ComparatorManager().reload()
 
         # ... however for the generated substvar to be effective/deterministic
-        # regardless of the currently installed packages we special-case some
+        # regardless of the currently installed packages, we special-case some
         # tools (NB. not package names) as their modules may not have been
         # imported by the `ComparatorManager().reload()` call above. (#908072)
         tools = set(
@@ -573,15 +573,20 @@ class ListDebianSubstvarsAction(argparse._StoreTrueAction):
         tools.update(tool_required.all)
 
         packages = set()
+        packages_basic = set()
         for x in tools:
             try:
-                packages.add(EXTERNAL_TOOLS[x]["debian"])
+                pkg = EXTERNAL_TOOLS[x]["debian"]
             except KeyError:
                 pass
+            packages.add(pkg)
+            if x not in HUGE_TOOLS:
+                packages_basic.add(pkg)
 
         # Exclude "Required" packages
         for x in ("gzip", "tar", "coreutils", "diffutils", "findutils"):
             packages.discard(x)
+            packages_basic.discard(x)
 
         description = "File formats supported include: {}".format(
             ComparatorManager().format_descriptions(),
@@ -591,6 +596,12 @@ class ListDebianSubstvarsAction(argparse._StoreTrueAction):
         print("diffoscope:Description={}".format(wrapped))
 
         print("diffoscope:Recommends={}".format(", ".join(sorted(packages))))
+        print(
+            "diffoscope-minimal:Recommends={}".format(
+                ", ".join(sorted(packages_basic))
+            )
+        )
+
         sys.exit(0)
 
 



View it on GitLab: https://salsa.debian.org/reproducible-builds/diffoscope/-/compare/1f1e7bd12a46df2d505aca97b367bb432df5c16b...2ed38723fa2147ceca3f5bf7be0ec2046b523fce

-- 
View it on GitLab: https://salsa.debian.org/reproducible-builds/diffoscope/-/compare/1f1e7bd12a46df2d505aca97b367bb432df5c16b...2ed38723fa2147ceca3f5bf7be0ec2046b523fce
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-commits/attachments/20201125/4bc1e706/attachment.htm>


More information about the rb-commits mailing list