[rb-general] Bug#917386: apt: Difference in installation depending on real tty or not

[Johannes 'josch' Schauer]

Package: apt
Version: 1.8.0~alpha3
Severity: normal

Hi,

this bug shows how apt leaves the system in a different state depending
on whether its stdout/stderr are connected to a real tty or not. I'm not
appending my package versions or apt config because to reproduce this
problem, we are using apt from an up-to-date Debian chroot:

$ sudo debootstrap --variant=minbase unstable debian-unstable
$ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 chroot debian-unstable apt-get --yes install python3
$ md5sum debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc
2c6fdc51b035428a3881f3eef70e3a5b  debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc

You can remove the "debian-unstable" directory and repeat this step as
often as you want. The md5sum will remain the same. Now try the same
thing but connect the stdout and stderr to "cat":

$ sudo debootstrap --variant=minbase unstable debian-unstable
$ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 chroot debian-unstable apt-get --yes install python3 2>&1 | cat
$ md5sum debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc
df6fe61fe176e4858ce2062233d2280e  debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc

Again, you can repeat this as often as you want, the md5sum will remain
stable.

Some observations:

 - SOURCE_DATE_EPOCH has to be set for reproducible output
 - LC_ALL has to be set. Without it, the md5sum of the second test will
   be equal to the first even if stderr and stdout get redirected
   (why??)
 - the pyc get compiled in the postinst of the Python packages but a
   different pyc gets compiled by different postinst scripts. To verify
   this, try to remove a pyc and then invoke different postints. I thus
   speculate, that the problem might be that in both scenarios, the
   postinst scripts get invoked in different order.
 - there are more pyc files that differ. The example I picked is just
   one with a lot of differences.

So this might just as well be a bug in the Python packages where the
postinst scripts should output the same pyc independent of which
postinst script is being invoked. But right now I don't even know
whether postinst invokation order is the culprit and all I see is a
difference when invoking apt differently. Thus I'm filing the bug with
apt first.

Thanks!

cheers, josch