[rb-general] SOURCE_PREFIX_MAP and Occam's Razor

Tue Jan 24 09:38:00 CET 2017

John Gilmore:
>> talk is cheap, show me the code
> 
> Arrogance is often a useful attitude for a young person, but it's
> useful to first determine whether one is in the wrong.  When working
> on code that's decades old, reading ChangeLog files is often
> educational.
> 

Sure, reading old stuff is useful. Which is why I'm annoyed because none of what you said references any of the work we did recently.

Instead you're referencing vague general principles in a way that sounds like I'm against those principles, which is a bit of a straw man as far as the current situation is concerned. Calling young people arrogant is also a bit cliché and again of questionable relevance to the current situation.

Sure, so let's take a look at the stuff you posted. (By "documentation" I actually meant something slightly different, but never mind.) Bear in mind this is taking away time from me actually working on the topic of the thread, so forgive me if I drop out early or take longer to reply.

> [..]
> 
> Nine months later, we shipped our fourth major multi-platform
> supported GNU compiler release (93q2) in June 1993.  Compiling the
> same source files on 8 of the 9 different host systems, for the
> Motorola 68000 target (among 10 target environments using 5 different
> CPU chips), produced exactly bit-for-bit identical object files.  Part
> of making this happen was by adding a "-gnodir" switch to gcc that
> suppressed the directory from the debug information.  (RMS, who was
> maintaining GCC at the time, suggested merely dropping the directory
> name in all cases.  We left that issue unresolved pending a later
> merge with his version of the code.)
> 
> [..]
> 
> To: ian at cygnus.com (Ian Lance Taylor)
> Cc: gumby at cygnus.com, p3 at cygnus.com, gnu
> Subject: Re: -gnodir 
> Date: Wed, 28 Oct 92 13:09:17 -0800
> From: gnu at cygnus.com
> 
>> Do you think I should improve it to use the basename of the argument,
>> or do you think that since this is just an undocumented hack anyhow
>> there's no point?
> 
> I do not think that this should be changed.  If the user passes in an
> absolute pathname, they should get what they get.  Cutting it down to
> the basename would be the wrong thing; it would not enable the
> debugger to tell the difference between ../bfd/core.c and
> ../gdb/core.c, for example.
> 
> I think that -gnodir should become a supported feature, but we should get
> some further experience with it, first.  Did it accomplish its intended
> purpose of making it possible to compare all the P3 .o's easily?
> 
> [..]
> 
> From: ian at cygnus.com (Ian Lance Taylor)
> Date: Wed, 28 Oct 92 16:27:47 EST
> To: gnu at cygnus.com
> Cc: gumby at cygnus.com, p3 at cygnus.com, ian at cygnus.com
> Subject: -gnodir 
> 
>    Date: Wed, 28 Oct 92 13:09:17 -0800
>    From: gnu at cygnus.com
> 
>    I think that -gnodir should become a supported feature, but we should get
>    some further experience with it, first.  Did it accomplish its intended
>    purpose of making it possible to compare all the P3 .o's easily?
> 
> Yes, it did.  I did not an exhaustive comparison of everything (too
> much other stuff to do, but this should definitely be done for p4).
> For the ones that I did compare the only differences were the floating
> point differences and the COFF timestamps.
> 
> In the collection of mail you sent me, rms suggested never generating
> the directory name at all instead of adding the -gnodir option.  I'm
> not sure I know how to find out whether that would ever cause any
> problems.  dbx on the Sun4 seems to be able to cope.
> 
> [..]

Firstly, it's not a surprise that adding -gnodir helped - today, --debug-prefix-map already helps with quite a lot of things. However, it doesn't help with other things, such as __FILE__ and other programs.

Secondly, there are the "[potential] problems" mentioned in the thread you're quoting. From what you quoted, we don't know what they investigated afterwards, so we can't predict what sort of wider problems it might have caused. I would guess that --debug-prefix-map is an evolution of -gnodir, probably in response to *some* problems that were later uncovered. Of course it would be good to have a GNU person here to tell us.

But I do suspect that "$(basename "$path")" really is not acceptable for a large variety of reasons, which is why personally I don't want to spend time pursuing that approach. I am not saying it's impossible - you are welcome to try it yourself - but with *your* time and resources not mine. ("talk is cheap, show me the code").

For example, from a debugging session I did a couple of days ago:

(gdb) bt
#0  0xce401bc6 in get_index (addr=addr at entry=2232868864) at headers.c:207
#1  0xce401e19 in GC_install_counts (h=0xff96e000, sz=4456448) at headers.c:279
#2  0xce3faf45 in GC_allochblk_nth (sz=4456448, kind=1, flags=1, n=60, may_split=1) at allchblk.c:791
#3  0xce3fb361 in GC_allochblk (sz=4456448, kind=1, flags=1) at allchblk.c:624
#4  0xce4022f1 in GC_alloc_large (lb=4456448, k=1, flags=1) at malloc.c:64
#5  0xce40304b in GC_generic_malloc_ignore_off_page (lb=4456448, k=1) at mallocx.c:192
#6  0xce4031ba in GC_malloc_ignore_off_page (lb=4456448) at mallocx.c:223
#7  0xce5a8678 in ecl_alloc_unprotected (n=4456448) at /build/ecl-IrXgwb/ecl-15.3.7+dfsg1/src/c/alloc_2.d:717
#8  0xce5a8769 in ecl_alloc (n=4456448) at /build/ecl-IrXgwb/ecl-15.3.7+dfsg1/src/c/alloc_2.d:732
#9  0xce58d251 in alloc_pointerfull_memory (l=1114112) at /build/ecl-IrXgwb/ecl-15.3.7+dfsg1/src/c/array.d:530
#10 0xce58d2ed in ecl_array_allocself (x=0xff920de0) at /build/ecl-IrXgwb/ecl-15.3.7+dfsg1/src/c/array.d:545
#11 0xce58dc95 in si_make_vector (etype=0xce6ef61c <cl_symbols+28>, dim=0x440003, adj=0x1, fillp=0x1, displ=0x1, disploff=0x3) at /build/ecl-IrXgwb/ecl-15.3.7+dfsg1/src/c/array.d:521
#12 0xce4e32d3 in _eclWWewOka7_1m3gSs21 (flag=0x0) at lsp/format.c:17846
#13 0xce566eec in ecl_init_module (block=<optimized out>, entry_point=0xce4e3140 <_eclWWewOka7_1m3gSs21>) at /build/ecl-IrXgwb/ecl-15.3.7+dfsg1/src/c/read.d:2444
#14 0xce48b62f in init_lib_LSP (cblock=0x0) at eclinitFTPx4A.c:193
#15 0xce566eec in ecl_init_module (block=<optimized out>, entry_point=0xce48b430 <init_lib_LSP>) at /build/ecl-IrXgwb/ecl-15.3.7+dfsg1/src/c/read.d:2444
#16 0xce48a57e in cl_boot (argc=<optimized out>, argv=<optimized out>) at /build/ecl-IrXgwb/ecl-15.3.7+dfsg1/src/c/main.d:779
#17 0xce7204f4 in __pyx_pf_4sage_4libs_3ecl_4init_ecl (__pyx_self=<optimized out>) at ./sage/src/build/cythonized/sage/libs/ecl.c:5438
#18 0xce71fe51 in __Pyx_PyObject_Call (kw=<optimized out>, arg=<optimized out>, func=<optimized out>) at ./sage/src/build/cythonized/sage/libs/ecl.c:13253
#19 __Pyx_PyObject_CallNoArg (func=<optimized out>) at ./sage/src/build/cythonized/sage/libs/ecl.c:13532
#20 0xce72effb in initecl () at ./sage/src/build/cythonized/sage/libs/ecl.c:13034
#21 0xf74f3c22 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Debugging large programs is really annoying especially if you're already under time pressure and I don't really want to make this worse by wasting people's time having to figure out which "malloc.c" they should be looking at.

I'd rather spend more time now, doing BUILD_PATH_PREFIX_MAP, when we don't have any immediate critical time-pressure on our hands.

Thirdly, there are the other issues from the "background" thread from gcc-patches that I linked in my first post, around dealing with this same problem across many build tools (in 2017, GNU is no longer "the FOSS world") and across 1000s of packages, some of which may be hostile to reproducible builds. I really don't want to repeat myself so I'll just ask you to read that instead.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git