<div dir="ltr"><div dir="ltr">Hi,<br><br>Thanks for your detailed answer! I should be subscribed to the mailing list, so no need to keep me in CC.<br><br>To try to give a more concrete example of the types of changes I made, without entering into implementation details, here are a few modifications I made to how ELF files are handled:<br>• Ignore some sections (essentially debugging sections, as well as those prone to changing if a file is recompiled). This could be done using the --exclude-command option, so it's not entirely necessary to add a new feature. It does make for a pretty long command line, though.<br>• Filter out some strings in specific sections of readelf, for example to ignore offsets in relocs,<br>• Same goes for objdump, for example to ignore offsets referencing functions that resolve to a name (among other things).<br><br>These last two changes could be handled using the new --diff-mask option, but it might mask details in other files or sections that I would rather keep (I can't, for example, just mask every 0xXXX value or I'll lose a lot of valuable information).<br>To illustrate how implementing those could impact the codebase, I opened some merge requests to serve as examples:<br>• Filtering the output of readelf: <a href="https://salsa.debian.org/reproducible-builds/diffoscope/-/merge_requests/59">https://salsa.debian.org/reproducible-builds/diffoscope/-/merge_requests/59</a><br>• Filtering the output of objdump: <a href="https://salsa.debian.org/reproducible-builds/diffoscope/-/merge_requests/60">https://salsa.debian.org/reproducible-builds/diffoscope/-/merge_requests/60</a><br>• Filtering the output of strings(1) and removing binary diff fallback: <a href="https://salsa.debian.org/reproducible-builds/diffoscope/-/merge_requests/61">https://salsa.debian.org/reproducible-builds/diffoscope/-/merge_requests/61</a><br><br>As you can see, the overall idea is to remove as much "noise" (with regards to my use case) as possible. Since I only want to focus on changes related to features or fixes, I ignore essentially everything that makes a build non-reproducible.<br>I realize these changes could be beneficial for a subset of diffoscope users, and I think that shows in some issues that have been opened. Since it goes pretty much against the flow of diffoscope's original purpose, I wasn't very optimistic about finding a clean way of merging these upstream.<br>It would require either adding an option for each feature (which we don't really want), or having an overall "detail-level" option to manage several of those features. I believe this is similar to what is described in issue #129.<br>You definitely know better than I do however, so you can check out the mentioned merge requests if you wish.<br><br>Going back to the feature proposal, I must say your insight and experience are very valuable.<br>Right now, my implementation consists in basically cloning diffoscope, and replacing the comparator files with my own tweaked versions, which is what led me to this proposal.<br>I understand that it would add another layer of complexity, which might not be beneficial for the project. This is the main reason why I decided to send an email about it, rather than trying to implement something that doesn't fit the project's objectives.<br><br>Thanks again for your feedback, and let me know if you have any further ideas or comments.<br><br>Best,<br><div>Jean-Romain Garnier</div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 9, 2020 at 9:30 AM Chris Lamb <<a href="mailto:chris@reproducible-builds.org">chris@reproducible-builds.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">[keeping Jean in CC; let me know if you are on this list]<br>
<br>
Hi Jean-Romain,<br>
<br>
I think we share the same concerns about the scope of diffoscope.<br>
<br>
There are already a lot of moving parts, comparators and code that<br>
could do with rewriting, yet alone optimising.<br>
<br>
However, I will add the general comment that there is a tendency in<br>
free software development to prematurely add layers of abstraction. I<br>
have nothing against plugin systems and similar abstractions — as<br>
ideas, they are also very satisfying to my geek mind as well. But, in<br>
my experience, they sometimes do not improve the codebase as a whole<br>
as the wider and longer-term negatives are not fully appreciated or<br>
even considered. Also, if they are added prematurely they often don't<br>
really fix the original issue to begin with as they are not understood<br>
properly.<br>
<br>
> This use case is very different from the need to spot all<br>
> differences in the context of reproducible builds, so simply<br>
> merging it upstream doesn't seem straightforward.<br>
<br>
I think specifics might be better addressed in the MR rather than this<br>
email but what, in general terms, is preventing you from merging this<br>
functionality? It may not be Platonically ideal, but merging it in<br>
some form may be the best step for diffoscope as a whole.<br>
<br>
Even if it hacks around existing functionality (eg. --exclude-command<br>
or whatever… again, I won't engage in the specifics in this thread I'm<br>
afraid), but if we *directly* experience what problems we see with<br>
this approach in real life we can make a real judgement about where we<br>
go from there and possibly even see patterns across different<br>
features/problems/whatever.<br>
<br>
Compare this to imagining these problems and jumping to the conclusion<br>
that "just adding" abstraction X, plugin system Y or autodetection<br>
system Z would magically fix it … all requiring delicate, nuanced and<br>
exhausting responses by the maintainers that would have to maintain<br>
them.<br>
<br>
Hope that makes sense.<br>
<br>
<br>
Best wishes,<br>
<br>
--<br>
o<br>
⬋ ⬊ Chris Lamb<br>
o o <a href="http://reproducible-builds.org" rel="noreferrer" target="_blank">reproducible-builds.org</a> 💠<br>
⬊ ⬋<br>
o<br>
</blockquote></div></div>