tests.reproducible-builds.org: build logs sent as gzip but no content-encoding, but only sometimes?

[наб]

On Wed, Nov 15, 2023 at 09:43:38AM +0000, Holger Levsen wrote:
> > On opening
> >   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd-cron.html
> > I see garbage in the log view
> > (screenshot at https://101010.pl/@nabijaczleweli/111410835559109343).
> > 
> > In the network log I see a request for
> >   https://tests.reproducible-builds.org/debian/rbuild/unstable/amd64/systemd-cron_2.3.0-1.rbuild.log.gz
> > which returns an expectedly-gzipped answer, but with no
> > Content-Encoding: gzip header.
> >
> > I tested this on four machines total:
> >   my own           (win32   firefox, "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0"):   bad
> >   my brother's     (x11     firefox):                                                                                     good
> >   my father's      (win32   firefox, "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/119.0"): bad
> >   another tester's (wayland firefox, "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/119.0"):           good
> so good on linux and bad on windows?
This is what I kept coming back to, but refused to accept firefox would
do something this weird just based on the target.

Forgot, of course, that both bad machines have ESET running,
and disabling "HTTP(S) scanning" in ESET fixes this.
Sorry for the noise, I s'pose.

> > Requesting with curl/7.88.1 I see Content-Encoding: gzip.
> > But adding --compressed I see Content-Encoding: gzip, gzip.
> > Either way it's valid, but not an ideal approach.
> what would you propose?
I mostly meant "not an ideal approach to read the logs".

That said, I think it's a miracle of standards conformance that
curl and firefox understand the replies: Content-Encoding: gzip, gzip
means that the reply is gzip(gzip("plain"))
which apparently neither lynx nor ESET cope with
(I'm assuming they do a string match instead of tokenising).

But, indeed, t.r-b.o returns valid responses:
  $ curl -v https://tests.reproducible-builds.org/debian/rbuild/unstable/amd64/systemd-cron_2.3.0-1.rbuild.log.gz | gzip -d
  < Content-Encoding: gzip
  Sat Oct 21 17:20:54 UTC 2023  I: starting to build systemd-cron/unstable/amd64 on jenkins on '2023-10-21 17:20'
  Sat Oct 21 17:20:54 UTC 2023  I: The jenkins build log is/was available at https://jenkins.debian.net/userContent/reproducible/debian/build_service/amd64_12/2133/console.log
  Sat Oct 21 17:20:54 UTC 2023  I: Downloading source for unstable/systemd-cron=2.3.0-1
  ...
vs
  $ curl -H "accept-encoding: gzip" -sv https://tests.reproducible-builds.org/debian/rbuild/unstable/amd64/systemd-cron_2.3.0-1.rbuild.log.gz | gzip -d | gzip -d
  > accept-encoding: gzip
  < Content-Encoding: gzip, gzip
  Sat Oct 21 17:20:54 UTC 2023  I: starting to build systemd-cron/unstable/amd64 on jenkins on '2023-10-21 17:20'
  Sat Oct 21 17:20:54 UTC 2023  I: The jenkins build log is/was available at https://jenkins.debian.net/userContent/reproducible/debian/build_service/amd64_12/2133/console.log
  Sat Oct 21 17:20:54 UTC 2023  I: Downloading source for unstable/systemd-cron=2.3.0-1
  ...

But this naturally means it's recompressing gzipped data:
  $ curl -H "accept-encoding: gzip" -s https://tests.reproducible-builds.org/debian/rbuild/unstable/amd64/systemd-cron_2.3.0-1.rbuild.log.gz | wc -c
  1976
  $ curl -H "accept-encoding: gzip" -s https://tests.reproducible-builds.org/debian/rbuild/unstable/amd64/systemd-cron_2.3.0-1.rbuild.log.gz | gzip -d | wc -c
  1953
  $ curl -H "accept-encoding: gzip" -s https://tests.reproducible-builds.org/debian/rbuild/unstable/amd64/systemd-cron_2.3.0-1.rbuild.log.gz | gzip -d | gzip -d | wc -c
  3511

So I'm assuming if you made it not do that then you'd both
save on load and have a wider compatibility range?

Best,
наб
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.reproducible-builds.org/pipermail/rb-general/attachments/20231115/0ea6581a/attachment.sig>