[rb-general] Finding Packages With Tests

[Omar Navarro Leija]

> What, in more detail, are you trying to achieve by identifying packages
with tests?
Our main goal is to show that if you build a package under our tool (using
./ourTool dpkg-buildpackage -uc -us -b):
1) the package will always build the same way (assuming same environment,
compiler version, libraries, etc). We verify this using reprotest +
continuous integration system.
2) The reviewers of our last publication submission were concerned about
"correctness". That is, does our generated deb file have equivalent
semantics to that of a deb file built without our tool. Basically, they're
concerned out dynamic manipulation leads to an incorrect build. So we want
to run the unit tests for a few packages (a dozen or so) to show that the
package still works as intended after going through our tool.

That's why I'm trying to identify some packages with tests to run.

On Thu, Feb 21, 2019 at 11:35 AM Chris Lamb <lamby at debian.org> wrote:

> Hi Omar,
>
> > Is there some way to identify which packages which have built in tests?
>
> Unfortunately there is no canonical or general way to identify
> Debian packages that "run" a testuite.
>
> Not only will many packages will always tests as part of their
> build, there are a myriad of ways in which the various Debian
> build systems will attempt locate and run tests.
>
> Just as a concrete example, the "debian/rules" package for
> countless packages is merely:
>
>     #!/usr/bin/make -f
>
>     %:
>             dh $@
>
> .. which will automatically look for a Makefile (or similar) and
> then magically work out whether there is a "tests" target, etc. etc.
> There are parallel ones for Python, Go, CMake, etc. etc. Even if we
> could list them all, I am sure we would be missing 50% of them.
>
> Furthermore, some test failures are deliberately ignored for good
> reasons so knowing that a package "had" a testsuite would not make
> running those tests particularly meanintful.
>
> What, in more detail, are you trying to achieve by identifying
> packages with tests? I suspect there may be a better solution.
>
>
> Regards,
>
> --
>       ,''`.
>      : :'  :     Chris Lamb
>      `. `'`      lamby at debian.org 🍥 chris-lamb.co.uk
>        `-
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20190221/e6a74e61/attachment.html>