dotnet build vs publish on Azure DevOps - azure-devops

I have a .NET Core 2.0 console app. I can successfully build or publish this app and run it locally. I can also successfully build and publish this app in Azure DevOps. However, if I build the app in Azure DevOps, I cannot run the result.
In Azure DevOps, I tried building using:
dotnet build -c Release -r win-x64 -o app
This generates a small number of files with just the project related files. It does not include all of the System.*.dll files, etc that seem excessive for most of my cases. This command works fine when I run it on my local machine and I can successfully click the MyApp.exe file and run my console app. However, if I run the same command on Azure DevOps, the MyApp.exe file that gets generated does not run as expected. Instead, it starts then immediately quits. Nothing is printed in the console app. I see no errors. The app is very basic, includes a "try-catch" around everything and has a Console.ReadLine at the end. So, I thought it would stay open.
When I run:
dotnet publish -c Release -r win-x64 -o app
I get the same files, but with all of the System.*.dll files, etc. included. This time, I've noticed that I can successfully run MyApp.exe and it behaves as expected.
Why does dotnet build ... work locally, but I don't seem to get the same behavior when I run dotnet build ... in Azure DevOps. It seems I'm forced to use dotnet publish. My issue is, the resulting .zip file goes from ~500kb to 30MB. This is big difference.

An answer from the horse's mouth:
The dotnet build command builds the project and its dependencies into
a set of binaries. The binaries include the project's code in
Intermediate Language (IL) files with a .dll extension and symbol
files used for debugging with a .pdb extension. A dependencies JSON
file (*.deps.json) is produced that lists the dependencies of the
application. A *.runtimeconfig.json file is produced, which specifies
the shared runtime and its version for the application.
If the project has third-party dependencies, such as libraries from
NuGet, they're resolved from the NuGet cache and aren't available with
the project's built output. With that in mind, the product of dotnet
build isn't ready to be transferred to another machine to run.
dotnet build - Builds a project and all of its dependencies.
dotnet publish - Packs the application and its dependencies into a folder for deployment to a hosting system. (PS - this also builds the application before packing)
The description is actually very good considering it's coming directly from Microsoft so I will not duplicate the words here.
As an exercise create a solution with multiple projects. For one of the projects add a reference to another project. Add some static files which your code references and a few NuGet packages. And run these commands at the solution root level and at the project level and observe the output in the bin folder.
Commands to run:
dotnet build
dotnet publish
dotnet clean to clean the bin folder
Also, run this at the root level and observe the output with the self-contained flag enabled:
dotnet publish -o ./output --runtime win10-x64 --self-contained
More info on self-contained builds

The different between them is that:
For publish, the necessary assembly files (packages) will be included in build folder and the app uses these assemblies.
But for build, the app references packages that are in the user’s folder. That’s why the zip file just 500 kb.
Since it references packages that are in the user’s folder, so the app needs to be built under the same user’s account, then you can run the app without publishing. So, you need to change build agent’s service account to your account (log on as), then restart the service and queue a new build.
Otherwise, you need to publish the app.

Related

How to test nuget packages without officially publishing?

If I have one repo that holds libraries (that are published to Nuget) and a separate repo that holds the application code (that consumes the Nuget packages), is there an easy way to test changes to the library code within the application repo without publishing to the official Nuget feed?
Your build script could be something like this
step 1. build your package and copy your .nupkg's to %buildroot%\newPackages\
step 2. create a nuget.config file in your application code's root that adds %buildroot%\newPackages\ as a packageSource. If your application code is a functional test, then you can probably check in the nuget.config, so it doesn't need to be recreated by the build machine every build.
step 3. Have a shell script or small program that updates your application code's references to the newly built package, to match the version that was just built
step 4 build/test your applicationCode

Copying files and deploying to Azure without building using Visual Studio Team Services

I'm attempting to deploy a web site to Azure using VSTS. Basically, I commit code to the GIT repo and have it setup to run CI, so it begins building as soon as I commit. However, once it hits the release section, it never copies the code to the Azure web app, rather, it gives me this line:
Info: Updating file ({projectname}\error.txt).
It doesn't copy the files I changed, but rather always just copies this file. I checked and there is indeed an error.txt file in my website directory in Azure, but it is always blank.
This build/deploy process isn't "standard" because the build step only downloads from source code, it doesn't build, because the website isn't a "web application", but rather just a "web site", meaning it doesn't need to be built.
So my build step is as follows:
Get Sources
Run on Agent - this step is empty
so the idea is that it just downloads everything from source control, that's it.
Then, my release step is as follows:
Artefacts are from build step above
deploy to environment 1 (dev)
Azure app service deploy, using "package or folder" as $(System.DefaultWorkingDirectory)/
Any idea what I might be doing wrong here?
So I actually figured this out and will leave this here in case anyone else needs it.
I admit I'm pretty new to the Azure/VSTS world, so maybe someone else is making my mistake as well.
If you don't need to "build" your project, then don't. I resolved it by simply skipping the build step altogether. What I was really after was to just download the files from source control and deploy them as-is.
In your release editor, you can specify which "artifact" you want to use to release, and one of the options is source control, which is what I did.
This would be useful for websites like mine where you don't need to build them (mine is DNN/DotNetNuke, so you don't build it before deploying).

LPRUN on AppVeyor with a reference to a custom NUGET repo reference

I am using a LinqPad script to automate an internal health check via AppVeyor.
The script references a custom nuget package hosted on our appveyor account.
The build does the following:
Pulls down a GitHub repo
via LPRUN executes health-check.linq
Locally this works.
On AppVeyor it does not.
I have the following build process
SETUP
Via chocolatey --> install Linqpad5
choco install linqpad5
BUILD
nuget install Example.Package
[this is our own NUGET package hosted on AppVeyor [SUCCESS]]
xcopy "c:\projects\example_project\utilities" %AppData%\LINQPad\ /i
[copy our custom NuGetSources.xml file containing our nuget repo location to linqpads folder]
cd "C:\Program Files (x86)\LINQPad5\"
lprun "C:\projects\example_project\utilities\health-check.linq"
ERROR
'Error downloading 'Example.Package' - An error occurred while retrieving package metadata for 'Example.Package' from source 'Example Company Repo'.
Does anyone have any hints on how to reference a custom NUGET repo from a LINQPAD script on APPVEYOR?
More Info
We use AppVeyor for our CI. It allows us to write our own custom NUGET packages for internal use within our own projects.
We have a repository ('FinPad') that contains numerous .linqpad files that automate our processes & house keeping.
Each FinPad script contains a reference to a package called 'FairGo.FinPower' on our own AppVeyor nuget repo. This custom nuget package contains numerous 3rd party .Net DLLs & our own custom code to connect to a 3rd party financial loan management system we use as a backend - http://www.finpower.com.au/ (hosted by us on Azure)
One such script is a 'Health Check' - this confirms a specific environment is operating OK.
For our TEST environment I wanted to schedule our 'health check linqpad' script to run every 15 minutes (and on failure alert stackify & slack)
The process works as follows ( using a custom Azure build machine from AppVeyor)
every 15 minutes
run a custom AppVeyor build called 'Health Check TEST' -->
pull down the GitHub project 'FinPad' to c:\projects\finpad
before build --> choco install linqpad5
Run the below command to copy our own nuget.config (with our own FairGo nuget repo references) to the location linqpad wants for reference
copy "c:\projects\finpad\utilities\nuget.config" "C:\Users\appveyor\AppData\Roaming\NuGet"
*** the above nuget.config contains the url / username /password to our appveyor nuget repo in plain text
build --> cd "C:\Program Files (x86)\LINQPad5\"
after build --> lprun "C:\projects\FinPad\utilities\health-check-test.linq"
For reference the Health check linqpad file is
http://share.linqpad.net/tuthme.linq
Locally this works fine (I manually configured our NUGET repo by the LinqPad GUI locally & assumed it only updated the nuget.config in 'C:\Users\ME\AppData\Roaming\NuGet\nuget.config'. (so I added this file to the repo & copy it every build)
On AppVeyor the build gives the below error
Downloading NuGet package FairGo.FinPower and dependencies from https://api.nuget.org/v3/index.json
Error downloading 'FairGo.FinPower' - Unable to find package 'FairGo.FinPower'.
Command exited with code 1
I have this running through a Console App now - but would really like to get this working through a LinqPad script if possible.

Where is the nuget packages folder located on a hosted build server using TFS?

I need to execute a command line utility from a package that is downloaded as part of nuget package restore in the TFS build process.
On my local computer that is stored in c:\users\me.nuget*
I've tried every permutation of that on TFS without success. I've also tried \mydir\packages with no success as well.
The biggest problem is that I have to run the package restore step before being able to see any sort of feedback from the log. That's some slow debugging.
Any ideas? Thanks ahead.
With the latest nuget/msbuild the packages folder is held under the active user's profile directory, so an appropriate Powershell command is
Get-ChildItem $(UserProfile)\.nuget\packages
This currently evaluates on the VSTS 2017 Hosted build agent to C:\Users\VssAdministrator\.nuget\packages but by using the variable you are insulated from any changes made.
Just an addition to #Paul Hatcher's answer:
I also faced the same problem in Azure DevOps build pipeline where a specific package and nuget packages directory could not be found.
It is a Xamarin.Forms app based on a .net standard library where no packages folder exists. I later noticed in build logs that the packages are restored to nuget folder under user's profile. However this particular case is not documented on https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=vsts#agent-variables.
That means #Paul Hatcher's answer is also valid if you try to reference nuget package folder directly from your build pipeline. This ($(UserProfile).nuget\packages) should actually be a (standard) predefined build variable.
The Nuget package cache folder is in C:\Users\buildguest.nuget\packages, but it will be cleaned after build if you are using Hosted build server.
The simple way to verify:
Add NuGet restore or .Net Core Restore build step to restore packages
Add PowerShell build step to list files in C:\Users\buildguest.nuget\packages
Code:
Get-ChildItem -Path C:\Users\buildguest\.nuget\packages
Queue build and check the PowerShell step log (the packages’ will be listed in the log)
Remove/disable NuGet restore or .Net Core Restore build step > Save build definition
Queue build
The build will be failed, because the path does not exist.
So, the packages need to be restored before build solution/project if aren’t existing. You can add packages to source control and map to build agent to deal with the issue of too long time takes to restore packages.

Azure website GIT deployment: server compiled .DLL different from local built .DLL

I have an MVC4 + EF4.0 .NET 4.5 project (say, MyProject) I'm able to run the project locally just fine. When I FTP deploy it to Azure Websites (not cloud service) it runs fine too. However, if I do a GIT deploy, the site 'runs' for the most part until it does some EF5.0 database operations. I get an exception Unable to load the specified metadata resource.
Upon debugging I noticed that if I:
GIT deploy the entire MVC4 project (as before)
FTP in and then replace bin\MyProject.dll with the bin\MyProject.dll file that I just built locally (Windows 8 x64, VS2012, Oct'12 Azure tools) after the GIT push (i.e. same source)
then the Azure hosted website runs just fine (even the EF5.0 database functionality portion).
The locally built .dll is about 5KB larger than the Azure GIT publish built one and both are 'Release' mode. It's obvious that the project as built after the GIT push (inside Azure) is being built differently than as on my own PC. I checked the portal and it's set to .NET 4.5. I'm also GIT pushing the entire solution folder (with just one project) and not just small bits and pieces.
When I load the locally built as well as the remotely built MyProject.dll files, I noticed the following difference(FrameworkDisplayName)
local: System.Runtime.Versioning.TargetFrameworkAttribute(".NETFramework,Version=v4.5", FrameworkDisplayName = ".NET Framework 4.5"),
remote: System.Runtime.Versioning.TargetFrameworkAttribute(".NETFramework,Version=v4.5", FrameworkDisplayName = ""),
Anyone knows why this is happening and what the fix might be?
Yes, this is a bug that will be fixed in the next release. The good news is that it's possible to work around it today:
First, you need to use a custom deployment script, per this post.
Then you need to change the MSBuild command line in the custom script per this issue.
Credit goes to David above for the pointers and hints. I voted him up but I'll also post the exact solution to the issue here. I've edited my original post because I found there was a major bug that I didn't notice until I started from scratch (moved GIT servers). So here is the entire process, worked for me.
Download Node.JS (it's needed even for .NET projects because the GIT deploy tools use it)
Install the azure-cli tool (open regular command prompt => npm install azure-cli -g)
In the command prompt, cd to the root of your repository (cd \projects\MyRepoRoot)
In there, type azure site deploymentscript --aspWAP PathToMyProject\MyProject.csproj -s PathToMySolution.sln (obviously adjust the paths as needed)
This will create the .deployment and deploy.cmd files
Now edit the deploy.cmd file, find the line starting with %MSBUILD_PATH% (will be just one)
Insert the /t:Build parameter. For example:
[Before] %MSBUILD_PATH% <blah blah> /verbosity:m /t:pipelinePreDeployCopyAllFilesToOneFolder
[After] %MSBUILD_PATH% <blah blah> /verbosity:m /t:Build /t:pipelinePreDeployCopyAllFilesToOneFolder)
Push to GIT (check the GIT output if everything went ok)
Browse to your website and confirm it works!
I'll be glad when it's fixed in the next revision so we won't maintain the build script