FrameworkReference

[Terry Aney]

I have a library dll I wrote that includes the FrameworkReference where that csproj has 'FrameworkReference Include="Microsoft.AspNetCore.App.  I'll work on maybe reoving.

In my excel Add-In, when I try to call into that api, I get the following exception:

Exception thrown: 'System.IO.FileNotFoundException' in KAT.Extensibility.Excel.AddIn: 'Could not load file or assembly 'Microsoft.Extensions.Configuration.Abstractions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.'

Since I have that framework reference, it doesn't copy any dlls to the folder as it expects the runtime to be insalled and make them available.  Any idea which setting to modify to make this work?

[Terry Aney]

FYI, this is because my project depends on those., and have following packages specified:

        <PackageReference Include="Microsoft.Extensions.Configuration" Version="7.0.0" />

        <PackageReference Include="Microsoft.Extensions.Configuration.Binder" Version="7.0.0" />

        <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="7.0.0" />

[Terry Aney]

I guess I should have specified.

1. I have the PackageReferences mentioned above.

2. If I don't reference my library dll that references FrameworkReference, then the Microsoft.Extensions.Configuration.* files are copied to output folder.

3. If I do reference my library dll, none of the Microsoft.Extensions.Configuration.* files are copied to output folder after build.

Terry

[Terry Aney]

So, looking in the *.deps file, I see the following in lines 17-19.  Is this a possible bug with the packing of the addin in that it needs to also check the 'framework folder' as well for the files?

So 

[Govert van Drimmelen]

Hi Terry,

I assume the Microsoft.Extensions.Configuration assemblies are part of AspNet SDK, but not part of the regular Sdk.

Perhaps (I've not checked this) the output of your library targets the AspNet SDK whenever you reference the library that targets this SDK.

Once your output targets the AspNet SDK, it will no longer require (and hence no longer include) the extra M.E.C. assemblies.
In addition to the .deps.json files, you should also look at the .runtimeconfig.json files in the two cases.

Maybe you can post all four of these?

We have under Excel-DNA version 1.8.0-alpha3 some experimental support for using the add-in library output runtimeconfig.json when Excel-DNA initially loads the .NET runtime. This would allow you to load the AspNet Sdk for your add-in, presumably making those M.E.C. assemblies available. The downside is that this only works if your add-in is the first to load into the Excel process (sometimes it's hard to ensure this).

I suggest you rather figure out whether you can make your dependency somehow not target the AspNet Sdk.

There are a few other settings you can experiment with, if you really just want those libraries to be copied to the output.

I'm not much of an expert about these.

There are options on PackageReference when you set PrivateAssets:

<PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="87.0.4280.8800">

    <PrivateAssets>contentfiles;analyzers</PrivateAssets>

</PackageReference>

Finally, there a setting like this, which affects what gets copied from NuGet packages to the output. See https://learn.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#locking-dependencies

<PropertyGroup> 

    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies> 

</PropertyGroup>

-Govert

[Terry Aney]

I'll look to see if I can remove the 'FrameworkReferene' inside the helper assembly, but it is used by many other projects in our company, so I'm a bit scared to do so, but will test it out.  In the meantime, here are all my files.

1. KAT.Extensibility.Excel.AddIn is my addin I'm writing.

2. KAT.Camelot.Domain is the helper assembly that references FrameworkReference.

Thanks,

Terry

[Terry Aney]

OK, I removed all the `FrameworkReference` items from my projects and am still getting the same problem. I've attached `Excel.AddIn.csproj` and the two assembly projects of ours as well: `Camelot.Api.Excel.Contracts.csproj` and `Camelot.Domain.csproj`.  The following patterns are observable:

1) Currently `Camelot.Api.Excel.Contracts.csproj` and `Camelot.Domain.csproj` are commented out and when I build `Excel.AddIn.csproj`, the `Microsoft.Extensions.Configuration*` files are in the output folder (along with others) and all my code works that attempts to use `Microsoft.Extensions.Configuration` code.

2) If I uncomment the `Camelot.Api.Excel.Contracts.csproj` project reference (none of its code is actually used) and build the project, the same output as #1 occurs.

3) If I uncomment the `Camelot.Domain.csproj` project reference (again, no code used) and build the project, none of the `Microsoft.Extensions.Configuration*` files are in the output folder.  I've tried looking at the difference between my two helper assembly project files and there isn't much difference except Domain has many Serilog package references.  Looking at the Frameworks Tab on nuget.org, it looks like Serilog.AspNetCore might do the <FrameworkReference/> itself which is causing the problem? :(

[Terry Aney]

As in, it is going to be too hard to pull out `<FrameworkReference/>` elements.  I was going to try something like this:

    <Target Name="PostBuild" AfterTargets="PostBuildEvent">

        <Exec Command="powershell -Command &quot;Get-ChildItem -Path '$(TargetFrameworkDirectory)' -Filter 'Microsoft.Extensions.Configuration*.dll' | ForEach-Object { Copy-Item -Path $_.FullName -Destination '$(OutDir)' }&quot;" />

    </Target>  

But it seems to run too late.  Is there an MSBuild target I can leverage that will run before ExcelDnaBuild and/or ExcelDnaPack?  Unless you have any other ideas.  At the end of the day I am going to try and get some better separation of concerns for our 'helper dlls', but that is going to take time and approvals, so I'm hoping for a miracle workaround until then so I can continue to migrate our addin to .net core :)

[Govert van Drimmelen]

Hi Terry,

Can you confirm the following?

1. In the case where the M.E.C. files are _not_ copied to the output, then you have this entry in the runtimeconfig.json

            {
                "name": "Microsoft.AspNetCore.App",
                "version": "7.0.0"
            }

2. In the case where the M.E.C. files are copied to the output, then you do _not_ have such an entry in the runtimeconfig.json.

-Govert

[Terry Aney]

Yes, that is the exact behavior that is being displayed.

FYI - I started a major refactor to try and get rid of the FrameworkReferences in the assemblies that I'm going to be using but am stuck on the following

1. I use IFormFile, IFormFileCollection and ProblemDetails that are now part of the FrameworkReference.  They are also part of a couple obsolete packages from MS but the consensus on the web is to avoid them, so I'm scared to use them.

2. I use Serilog in an assembly to enrich the LogEvent with some properties for diagnostic information, but since Serilog has FrameworkReference that is a problem I can't change.  I've (hopefully temporarily) tried to remove my use of Serilog, but not thrilled about losing the functionality - and haven't tested how Logger<> stuff will behave in my new AddIn or in my existing clients (Sites/Apis).

As for the FrameworkReference, you tried to explain above, but I didn't follow exactly with regard to AddIn has to load first.  My naive understanding is that the packed xll contains all the dlls/packages I need?  Couldn't the packing just scan the developer box/framework directory for any of the dlls excluded due to FrameworkReference?  Sure is is more complicated than that, but thought I'd ask in case there is a 2 sentence layman response :)

Thanks again for looking at it.

[Govert van Drimmelen]

Hi Terry,

.NET core (for our purposes .NET 6+) has three different 'runtime' editions, which include sets of libraries that are loaded and work together:

* .NET Runtime

* .NET Desktop Runtime (includes the .NET Runtime)

* ASP.NET Core Runtime (includes the .NET Runtime)

When .NET core is first loaded into a process, the load call specifies which of these runtimes to load (together with the version). After the .NET core runtime is loaded with whatever version and Runtime options are given, that process cannot load other .NET core versions or runtime options.

Excel-DNA targets the .NET Desktop Runtime, and this is what it loads into the process. So as long as your add-in only targets the .NET Runtime or .NET Desktop Runtime, things work fine.

But if your add-in targets the ASP.NET Core Runtime, as indicated by the presence of the "Microsoft.AspNetCore.App" in your output runtimeconfig.json file, then it probably won't work with since some of the assemblies it might depend on won't be available given the runtime that is loaded.

The target runtime of your library is set by either having a <FarmeworkReference> entry in the project file, or by setting the project file Sdk at the top like this:

<Project Sdk="Microsoft.NET.Sdk.Web">

From what I understand, your library itself is not targeting the ASP.NET Core Runtime (which means you don't have either a FrameworkReference or the 'Web' Sdk. But you are referencing other projects that do target the ASP.NET Core Runtime, and that results in the build process adding this additional runtime entry in your output. So, by referencing a library that targets the ASP.NET Core Runtime, your own library now implicitly also targets that runtime.

A consequence of your library targeting the ASP.NET Core Runtime is that some assemblies in packages will _not_ be copied to the output, since the version that is part of the Runtime is assumed to be available. So that's why Microsoft.Extensions.Configuration etc. are not in the output anymore. However, there may be many other assemblies that your library dependencies need, that are also not put in the output directory because they are part of the ASP.NET Core Runtime. So just copying the M.E.C. assemblies to the output folder are not likely to be a good fix.

Excel-DNA does not check what runtime your library targets, so the error you run into would likely arise only when some type is loaded that now has a missing reference.

One of the snags in this is what I said earlier - "After the .NET core runtime is loaded with whatever version and Runtime options are given, that process cannot loaded other .NET core versions or runtime options." So, if different add-ins will be loaded into the same Excel process, it's hard to decide what to do in terms of both the .NET version to load, and the exact Runtime to load. The first add-in loaded decides, and if any other add-in needs a different version or runtime option, it's too late to change and it will fail to load. Up to Excel-DNA version 1.7.0 we've been conservative, and only support one .NET core version with one runtime configuration - we support only .NET 6 with the .NET Desktop Runtime. (This is in addition to the .NET Framework 4.x which we continue to support, which can happily run in parallel, which only has a single version installed on the computer, which we load, and which does not have these Runtime flavours.)

For the next Excel-DNA version, we're making things more flexible, but then also exposing more of the complexity of these decisions to the add-in author. We now support other .NET core versions (so you can target .NET 7 or .NET 8 instead of .NET 6). But the danger if you are targeting .NET 8 is that another add-in loads .NET 6 first, and then your add-in will fail to load. You need to deal with that complexity. You can also use the <RollForward> project option to set some more subtle options. E.g. you can target .NET 6 but say that the newest installed version of the runtime should be loaded. That might break other add-ins that need .NET 6. But you need to deal with that complexity.

Now in the latest pre-release version of Excel-DNA - version 1.8.0-alpha3 - we also support loading the runtime using the runtime configuration options from your project. To enable this you add the <ExcelAddInCustomRuntimeConfiguration>true</ExcelAddInCustomRuntimeConfiguration> option to your project file. With this option enabled, and if your add-in is the first to load .NET core into the Excel process, it will load the version of the runtime, _and the Runtime option_ that is specified in your project's output runtimeconfig.json. So, your add-in can load the ASP.NET Core Runtime in this way. But if another add-in was loaded first, then it might have loaded a different .NET core version, or might have loaded with the default runtime option, and your add-in will fail to load. So, you get this additional option, but you need to deal with the complexity in add-in interactions which follows.

That gives you different ways of dealing with the problem of dependencies pulling in the ASP.ENT Core Runtime requirement. You can try take the above approach with the experimental Excel-DNA 1.8.0-alpha3, or you can try to get rid of the ASP.NET Core Runtime dependency in the other projects.

For example, with SeriLog, you can probably keep the core SeriLog library, but should not have the additional SeriLog.AspNetCore involved, which (from its name) probably uses and needs the extra runtime.

I hope that gives you some ideas on what path to take.

-Govert

[Terry Aney]

Thanks for the detailed answer.  It gives me ideas.  My only question left unanswered from above is those 'api objects'.

I have a couple 'Contracts' assemblies for some web apis I'm going to work with in my add-in.  The Contracts simply exposes Reponse types, Request types, and endpoint url generation in a typed/safe fashion.  But there are three types I don't know how to work around.  I'm not sure if .NET Runtime or .NET Desktop Runtime have them, but they are IFormFile, IFormFileCollection and ProblemDetails.

1. Need to figure out how to change the 'target types'

2. I could probably just replicate a simple ProblemDetails clone in my own code base, but will have to see if replicating IFormFile is any tricker :)

[Terry Aney]

Well, I did it! lol.  This is about the 4th round of commits of this size for these repositories, but now when I reference 'what I need' from Excel-DNA, nothing is a 'web app' and all the assemblies are in the output directory.  Phew, that was exhausting, but worth it in the long run.  Thanks for the help.