[diffoscope] Tests on FreeBSD
Ed Maste
emaste at freebsd.org
Mon Jun 12 20:33:47 UTC 2023
Hello Diffoscope folks, it's been a while!
I'd like to get FreeBSD into diffoscope CI, either by using
Cirrus-CI's hosted service, or via GitLab CI. But to start off I had a
look at the state of tests on FreeBSD and I see a few failures to sort
out. There are also a number of skipped tests; I'm leaving those aside
for now.
Summary (run on my laptop):
======================= short test summary info ========================
FAILED tests/comparators/test_device.py::test_diff - AssertionError:
assert '@@ -1,3 +1,2...t laborum..\n' == '@@ -1,3 +...
FAILED tests/comparators/test_device.py::test_diff_reverse -
AssertionError: assert '@@ -1,28 +1,... 12.\n' == '@@ -1,28 ...
FAILED tests/comparators/test_git.py::test_diff - AssertionError
FAILED tests/comparators/test_macho.py::test_llvm_diff -
FileNotFoundError: [Errno 2] No such file or directory: 'llvm-config'
FAILED tests/comparators/test_pdf.py::test_text_diff - IndexError:
list index out of range
FAILED tests/comparators/test_pdf.py::test_compare_non_existing - AssertionError
FAILED tests/comparators/test_rlib.py::test_num_items - assert 0 == 4
FAILED tests/comparators/test_rlib.py::test_item2_rust_metadata_bin -
IndexError: list index out of range
FAILED tests/comparators/test_rlib.py::test_compare_non_existing -
AssertionError
FAILED tests/comparators/test_squashfs.py::test_symlink_root -
AssertionError: assert ' crw-r--r-- root/root 1, 3 201...
FAILED tests/comparators/test_zip.py::test_commented - IndexError:
list index out of range
============= 11 failed, 557 passed, 147 skipped in 47.72s =============
A couple of the failures come from llvm_version() invoking
llvm-config; in FreeBSD we have Clang and other primary LLVM tools in
the base system but do not include llvm-config. Perhaps we can check
the version of the specific tool we're using (i.e., llvm-dis /
llvm-readobj / llvm-objdump)?
tests/comparators/test_device.py::test_diff and ::test_diff_reverse,
and tests/comparators/test_git.py::test_diff all fail because device
ID (major/minor) do not match. I'm not sure how best to handle this,
but I think we ought to have diffoscope elide them rather than having
the test be permissive.
It wasn't immediately clear to me what the issue is with the other
tests (tests/comparators/test_pdf.py, tests/comparators/test_rlib.py,
tests/comparators/test_squashfs.py, tests/comparators/test_zip.py).
More information about the diffoscope
mailing list