Optimizing .NET Core self-contained deployment - deployment

I am developing a simple, cross-platform .NET application. I need to distribute it as a self-contained deployment as I can't assume if .NET is already installed.
.NET publish for Windows 10 64 bit generates a 64 MB directory. I am pretty sure most of the DLL files are not needed. Is there a way to optimize the distribution so that only the necessary DLL file are kept?

I'm facing the same problem. So far, here's what I found:
Microsoft released a Trim tool that finds unused assemblies and removes them.
With .NET CLI:
dotnet add package Microsoft.Packaging.Tools.Trimming -v 1.1.0-preview1-25818-01
dotnet publish -r win-x64 -c release /p:TrimUnusedDependencies=true
Your application's footprint is now much smaller (normally)
For a better optimization you can use it with ILLinker:
dotnet new nuget
Open nuget.config and add <add key="dotnet-core" value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json" />between <packageSource>
Here is an example of what your nuget.config should look like.
Then execute the following commands:
dotnet add package ILLink.Tasks -v 0.1.4-preview-981901
dotnet add package Microsoft.Packaging.Tools.Trimming -v 1.1.0-preview1-25818-01
dotnet publish -r win-x64 -c release /p:TrimUnusedDependencies=true
Using these two tools I managed to divide the size of my application by three.
Source: Reducing the size of self-contained .NET Core applications
I'm still trying to get better optimization. I want to run my ASP.NET Core application (currently 72 MB) in an embedded system with only 10 MB.
If anyone has found a better optimization, I'm a buyer.

Since .NET Core 3.0, it's as simple as adding a flag to your project's .csproj file:
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RuntimeIdentifier>win-x64</RuntimeIdentifier>
<PublishSingleFile>true</PublishSingleFile>
<PublishTrimmed>true</PublishTrimmed> <!-- Add this flag to enable trimming -->
</PropertyGroup>
Then you can just use dotnet publish to generate a trimmed executable.
And here's a warning from the documentation:
It's important to consider that applications or frameworks (including
ASP.NET Core and WPF) that use reflection or related dynamic features,
will often break when trimmed. This breakage occurs because the linker
doesn't know about this dynamic behavior and can't determine which
framework types are required for reflection. The IL Linker tool can be
configured to be aware of this scenario.
Above all else, be sure to test your app after trimming.
A typical reduction result for a simple console applications seems to be from about 70 MB to about 28 MB.

Update 2020: This is now part of the toolchain, see Pawel's answer.
This task can be done by using the IL linker. By adding a NuGet package to your project it can reduce the size of the published self-contained application by removing code that is not needed. However, it is still in preview (as of 2017). See the announcement post and sample instructions for self-contained applications.

Microsoft.DotNet.ILCompiler seems to be an even better option than ILLinker. See this reply.
My testing of .NET Core 2.2 self-contained outputs on the same source code gave:
Default self-contained publish: 68 MB
ILLinker: 36 MB
ILCompiler: 5 MB

Related

Entity Framework 6 .net Framework Migrations / Package Management Console - How Do You Run These In An Azure Pipeline?

I am setting up an Azure Release Pipeline and I need to execute any pending DB Migrations as part of the release.
I have been scouring the internet for over an hour and everything I can find is about dotnet Core, while the database is EF6 on .Net Framework, not dotnet Core (I've done this several times before for Core).
The problem, as I see it, is that EF6 works using Visual Studio's built in Package Manager Console - This just doesn't exist in an Azure Pipeline; It's a Visual Studio weirdness.
There seems to be several ways I can skin this cat, in my head, but I can't figure out how to start with either of them within the context of the pipeline...
OPTION 1: Run the Migrations on the Pipeline - but... how?
OPTION 2: SQL Scripts - Requires running the Package Manager to generate them so they can be run (if I could do that on the pipeline then I'd just run it anyway so these would have to be created locally and committed with the code which is somewhat backward as a solution IMO)
OPTION 3: Write a console app - Do I really have to do this??
You can try Entity Framework Migration Extensions.
This task allows a Build / Release to provide database connection parameters and execute an Entity Framework 6 migration against the database.
Build your project to an output folder and include the migrate.exe executable that comes with Entity Framework 6.
Create an automated build that packages up your files and makes them accessible during a Release.
Create a Release definition for the relevant Build
Add an EF6 Migration task. Once that task is added to an environment within the release, you'll need to enter the appropriate parameters to configure it. All file path parameters should be within the file system for the build, none of them are for TFS source control paths.
You can also check this article.
The answer here is to use the ef6.exe command line tool and make sure it gets shipped with your build.
This could be useful to anyone here until Microsoft update the non-existent docs: http://github.com/dotnet/EntityFramework.Docs/issues/1740 - This contains a table with a kind of translation matrix between the two.

Debug NuGet package with Azure Devops and Source Link

I am trying to get SourceLink to work with a private NuGet package.
I am running a netcore2.1 web application which references a netstandard2.0 NuGet package hosted on our Azure Devops NuGet feed.
Question 1: Does Source Link support .NET Standard packages?
I have followed the instructions in the guide here https://learn.microsoft.com/en-us/azure/devops/artifacts/symbols/setting-up-github-sourcelinking?view=vsts, which is basically:
Add the Index Sources and Publish symbols package to my Azure Devops build.
In Visual Studio, add our VSTS server as a symbols server
In Visual Studio, enable Source Link support. I also tried enabling Source server support.
The Build pipeline Publish symbols path appears to be working - in the logs I see:
Succeeded processing D:\a\1\s\src\MyCompany.Core.Services.SnapshotClient\bin\release\netstandard2.0\MyCompany.Core.Services.SnapshotClient.pdb:
When I start debugging my application I see a bunch of output in the VS Output window:
'dotnet.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\2.1.4\Microsoft.AspNetCore.Hosting.dll'. Cannot find or open the PDB file.
For my NuGet package I see "Symbols loaded" which seems promising.
FWIW I do not see the prompt from Visual Studio that "Source Link will download from the internet".
When I debug and attempt to Step-In to my NuGet package, it just steps over it.
I then tried:
Headed over to https://github.com/dotnet/sourcelink and followed their instructions and installed the Microsoft.SourceLink.Vsts.Git package (Question 2 is that necessary?)
When that didn't work, I upgraded every darn package in my application, which forced me to install .NET Core SDK 2.1.403
Tried adding some stuff to the .csproj of my NuGet package, after trawling GitHub issues
<PublishRepositoryUrl>true</PublishRepositoryUrl>
<AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
and
<DebugType>portable</DebugType>
<ci>true</ci>
Now my .nupkg includes .pdb files too, which weren't there before. Still doesn't help me step in debug though.
installed the sourcelink cli tools from https://www.nuget.org/packages/sourcelink/ and ran sourcelink print-urls on the .pdb from my .nupkg. Looks correct, I think? URLs are present.
Disabled indexing after seeing a comment https://github.com/MicrosoftDocs/vsts-docs/issues/1336#issuecomment-414415049 from #mitchdenny . Still doesn't work.
And now I'm stumped as to why it's not working.
I wrote a complete blog on how to do this using .NET Core & AzureDevops, but the steps should work for .NET Standard projects as well.
That said, some key takeaways that are missing from Microsofts documentation that you should know are:
The project's debugging information needs to change from "Portable" to "Full"
The AzureDevOps Nuget(restore, build, pack & push) need to use the .NET Core task.
The .NET Core build task should have an argument "--configuration" that passes in the value "debug". Doing so generates the .PDB file
The .NET Core pack task should use the "custom" command, with the custom command being "pack" and have the following arguments: "--include-symbols -v d" and a "-c" that passes in the value "debug". Doing so tells the pack command to include the .PDB file in the package.
Question 1: Does Source Link support .NET Standard packages?
Yes. I successfully built a .NET Standard 2.0 library on Azure DevOps Pipeline, after which it was pushed to our private Azure DevOps Artifacts NuGet feed. Then, in a local project, I was able to step into the library (Visual Studio prompted me with a pop-up about downloading remote source code).
Here are the changes I had to make in the library's .csproj file:
<PropertyGroup>
<PublishRepositoryUrl>true</PublishRepositoryUrl>
<EmbedUntrackedSources>true</EmbedUntrackedSources>
<AllowedOutputExtensionsInPackageBuildOutputFolder>
$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb
</AllowedOutputExtensionsInPackageBuildOutputFolder>
</PropertyGroup>
...
<ItemGroup>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta-63127-02" PrivateAssets="All"/>
</ItemGroup>
Question 2: is that [PackageReference to Microsoft.SourceLink.GitHub] necessary?
I'm not sure. The docs suggest it is. But I removed the reference, re-built on Azure DevOps, and was still able to step through the library. Perhaps it's necessary for different environments (I'm keeping it just in case).
FWIW:
I'm debugging using Visual Studio 15.8.9
My latest installed .NET Core SDK is 2.1.403
My library's consumer is a .NET Core 2.1 executable
I compiled my library using Cake, which I have call into dotnet msbuild

Can MSBuild binaries be used on their own?

I've been using NAnt for sometime and although it works pretty well most of the time, some functionality is so limited you find yourself doing a page worth of work instead of couple of lines.
My biggest problem is returning results from exec task.. you HAVE to output to file then read the file then in my case substring what you read.
Multiply that by 8 different queries and you have lots of unnecessary work.
Anyway I would like to switch MSBuild which has much better support in this case.
But the problem is that I am using CI + Nant to deploy on staging and sometimes production and Nant is just a directory with files that you copy.
But As far as I know MSBuild must be installed and registered.
Is there such a thing as a standalone MSBuild download?
Thanks
I don't have time today to write up a rough history for MSBuild (I already covered ASP.NET on IIS and C# compilers), but I would tell you that MSBuild is not only standalone today, but open source at GitHub,
https://github.com/microsoft/msbuild
You can download the installers from Microsoft Downloads, or you can compile your own.
Reference: Rough History of MSBuild

Is there a way to make nant build using .NET 4.5.1?

We have just updated a project from 4.0 to 4.5.1 and we are having problems building it on NAnt. I just checked and it looks like it is dead.
Is there a way to build it using framework 4.5.1? The last version just supports up to framework 4.
An alternative approach if you want to keep your existing build scripts / msbuild task: We're actually still running NAnt 0.85, but building directly with .Net 4. In order to support newer framework versions, you need to do two things:
Firstly, edit nant.exe.config. There is a section dealing with the frameworks - you'll see entries for net-2.0, net-4.0 etc. Find the one for net-4.0 and duplicate it. Then go through and update the paths/.Net version numbers as needed.
Secondly, at the start of your build script, specify which framework you want to use:
<property name="nant.settings.currentframework" value="net-4.5" />
You can switch between frameworks as you wish during the build - parts of our legacy system switch between .Net 2 and .Net 4 as needed.
Instead of using the NANT MSBUILD task, we use the NANT EXEC task and invoke the MSBUILD.exe directly by passing the attributes
"program" - location of the (v3.5, 4.0, 4.5) msbuild.exe
"command line" - location of csproj or solution file, additional properties
This way we control which version of MSBUILD we use and does not depend on NANT supporting the latest framework. On a side note, if you want to handle incremental builds in this scenario then you should pass the MSBUILD build/rebuild argument as a property to the exec task instead of hardcoding.

Incremental Build with MSBuild.exe

I'm building a Visual Studio 2010 solution through Python with a call to subprocess. When called directly from the command line it takes devenv.com ~15 seconds to start. But when called from Python this jumps up to ~1.5 minutes.
Naturally I'm hoping to remove that dead time from our build. So I decided to test out MSBuild.exe (from .NET 4). It looks like MSBuild.exe runs instantly. But... it seems to do a full build every time and not an incremental.
The command I'm using is
"C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe" "C:\path\to\my\project.sln" /target:build /maxcpucount:8 /property:Configuration=Release
It seems like this should support an incremental build. But I've seen posts online indicating that msbuild may not be able to support a incremental build like this.
Is this possible? If so what am I doing wrong?
Update:
I've read into this a bit more. Based on
http://msdn.microsoft.com/en-us/library/ms171483.aspx
and
http://www.digitallycreated.net/Blog/67/incremental-builds-in-msbuild-and-how-to-avoid-breaking-them
It seems like I need the Input and Output properties set in my .vcxproj files. Checking out my files these are indeed missing.
When would they be generated? Most my .vcxproj files were converted over from Visual Studio 2008. But I also generated a new project which is missing the Input and Output properties as well.
Does VS2010 not create projects with these properties?
Update: We've since upgrade to VS 2013. Now msbuild supports incremental builds. Never got to the bottom of the VS 2010 issue.
I think that fact that Incremental builds are not supported is a false Statement from according to official sources,Managed Incremental Build this feature and was included in VS2010 SP1
We first introduced the managed incremental build feature in VS2008.
In VS2010, we were not able to re-implement the managed incremental
build feature with the build system moving to MSBuild. We received
strong customer requests for this feature. As a result, we
re-implemented this feature and it is included in VS2010 SP1.
Other Solutions I found on Web
Projects should build incrementally already (just make sure that you
do Build instead of Rebuild). The best way to check if incremental
building works is to run the build from the command line. The second
time you build it should take almost no time.
If things are still getting rebuilt, then perhaps you've modified
your projects in some way that's messing up with the build order.
Looking at the build logs (via the /v option) can help you poinpoint
what's going on.
Other reason which can cause problems with the incremental build is GenerateResource.TrackFileAccess PropertyThis API supports the .NET Framework infrastructure and is not intended to be used directly from your code.
Gets or sets a switch that specifies whether we should be tracking file access patterns.