How to Add a New Project within Reproducible Builds

[Jared Parsons]

> It would really have been nice to have you at Reproducible Builds Summit last
> week in our "language packaging" track: we had many ecosystems, and we saw
> that we were missing a few ones like .NET. I hope we'll have you next year :)

Sad we missed this. Definitely would be interested in participating next year. I can talk endlessly about all the fun we've had tracking down sources of non-determinism in our tooling and all the tools we've invested in to make this easy in the compiler. :) 

> IIUC, this is about .NET compiler producing binaries in a reproducible way =
> executing the compiler twice from the same source code gives the same binary
> output?

It's about the entire set of processes involved in a `dotnet build` command. That includes most prominently msbuild, nuget and the c# compiler. While I'm involved in most aspects of this my area of focus is the C# compiler. That is fully deterministic and reproducible. This is verified on every PR to the compiler source (compiler code base is built three times, on different drives, and verified that everything remains byte for byte equivalent). 

The conversation about reproducible is more nuanced. File paths definitely make their way into .NET DLL / EXE / PDB. So by default building the same code on different paths will produce different output. However the compiler supports an option /pathmap, which allows you to normalize paths between builds (and that's used with great success in our own repos).

> Is there a minimum .NET SDK required?

All supported .NET SDK have deterministic builds. If you go back to some of the earlier releases, say to 2.1, there will be a few aspects like windows PDB files, the occasional tool that put time stamps in generated files, etc ... which resulted in non-deterministic builds. That has been addressed in recent years though to the point all supported SDKS should be deterministic on the main build paths. 

> Is there some publicly visible documentation on this?

There are a couple of items we've written about this:

- This doc describes all of the inputs that contribute to a deterministic build. Essentially if these item are the same, then the output should be the same. https://github.com/dotnet/roslyn/blob/main/docs/compilers/Deterministic%20Inputs.md
- This describes the history of why the compiler has a /deterministic switch vs. always having this behavior (it's on by default in .NET SDK) https://blog.paranoidcoding.com/2016/04/05/deterministic-builds-in-roslyn.html

If there was a different style of doc you're looking for please let me know. 

> Does this reproducible builds scenario cover NuGet packages output?

I believe there are still date issues that cause NuPkg to be non-deterministic. The support was added but then removed because it broke tools that looked at dates (sigh). This is the issue tracking. 

https://github.com/NuGet/Home/issues/8601

-- 
jaredpar
https://twitter.com/jaredpar


From: rb-general <rb-general-bounces at lists.reproducible-builds.org> on behalf of Hervé Boutemy <hboutemy at apache.org>
Sent: Friday, November 11, 2022 12:48 AM
To: General discussions about reproducible builds <rb-general at lists.reproducible-builds.org>
Subject: Re: How to Add a New Project within Reproducible Builds 
 
[You don't often get email from hboutemy at apache.org. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Hi Adrian,

> We’re looking to submit .NET SDK.
It would really have been nice to have you at Reproducible Builds Summit last
week in our "language packaging" track: we had many ecosystems, and we saw
that we were missing a few ones like .NET. I hope we'll have you next year :)

> For all of our core scenarios this is a
> fully reproducible toolset. This is true out of the box with no real
> configuration changes needed. There is one caveat though: some less
> mainline scenarios are not reproducible at the moment (e.g. AOT). I’ve
> added Jared Parsons to this thread who is the lead engineer (and has also
> joined the rb-general list) and can answer any follow-up questions.

I have a few questions on the currently achieved status and eventually
remaining tasks (planned or not) in the .NET ecosystem

IIUC, this is about .NET compiler producing binaries in a reproducible way =
executing the compiler twice from the same source code gives the same binary
output?
Is there a minimum .NET SDK required?
Is there some publicly visible documentation on this?

Does this reproducible builds scenario cover NuGet packages output?

Regards,

Hervé