Azure DevOps Pipeline: can't get tests to compile - azure-devops

Question

I have a solution containing 4 projects (names simplified for clarity)
MyApplication (.Net Standard 2.0)
MyApplication.Test.1 (.Net Core 3.1)
MyApplication.Test.2 (.Net Core 3.1)
MyApplication.Test.3 (.Net Core 3.1)
The solution folder contains the *.sln file, and one folder per project (plus stuff for Git source control).
The three test projects each consume a private NuGet package, "MyNugetPackage"; this package exists on our Azure environment as an artifact. They also use NUnit3 - the test project's NuGet references are as follows:
<PackageReference Include="MyNugetPackage" Version="2020.1.1" />
<PackageReference Include="nunit" Version="3.12.0" />
<PackageReference Include="NUnit3TestAdapter" Version="3.16.1">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.4.0" />
I created the following YAML file for our pipeline:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
buildConfiguration: release
platform: x64
steps:
- task: NuGetToolInstaller#1
inputs:
versionSpec:
checkLatest: true
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '**/*.sln'
feedsToUse: 'select'
vstsFeed: '{GuidA}/{GuidB}'
- task: MSBuild#1
inputs:
solution: '**/*.sln'
msbuildArchitecture: 'x64'
platform: 'Any CPU'
configuration: 'Release'
msbuildArguments: '-m'
clean: true
- task: DotNetCoreCLI#2
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration Release'
Initially, this failed at the MSBuild step, until I worked out how to set up the permissions for it to pull from our Azure artifact (the vstsFeed property). Having got the MSBuild step to execute successfully, it then fails when the DonNetCoreCLI (Test) step executes: I get multiple messages similar to:
MyTestFile.cs(13,22): error CS0234: The type or namespace name 'ABC'
does not exist in the namespace 'MyNugetPackage' (are you missing an
assembly reference?)
[d:\a\1\s\MyApplication.Test.1\MyApplication.Test.1.csproj]
I'm assuming that this means the Test projects can't find our private 'MyNugetPackage'. But how could this be when MSBuild succeeded?
I tried swapping the MSBuild step out for a Visual Studio build, but to no avail (again, build step worked, test step failed):
- task: VSBuild#1
inputs:
solution: '**\*.sln'
platform: 'any cpu'
configuration: 'release'
clean: true
maximumCpuCount: true
msbuildArchitecture: 'x64'
I then tried swapping the MSBuild step for the .NET Core build so that it matched the job for the tests
- task: DotNetCoreCLI#2
displayName: Build
inputs:
command: build
projects: '**/*.csproj'
arguments: '--configuration Release'
And now the BUILD step failed.
So it seems to me that I have two routes forward:
Either stick with either MSBuild or Visual Studio Build, and work out how to then execute my tests, or
Work out how to compile successfully with DotNetcoreCLI#2 rather than use MSBuild/Visual Studio Build, and hope that the tests will then work as expected
The documentation from Microsoft seemed so simple (Run your tests)....hoping someone can help guide me on the right path.

Answer 1

You can run dotnet test command on your local machine to check if it works fine locally. Make sure you can successfully build and test your projects locally first.
In you pipeline, You can try use vstest task after msbuild task to run the tests.
- task: VSTest#2
inputs:
testAssemblyVer2: |
**\*[Tt]est*.dll
!**\*[Tt]estAdapter*.dll
!**\obj\**
searchFolder: $(system.defaultworkingdirectory)
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
You can also run Dotnet restore task to restore the packages before dotnet test task
- task: DotNetCoreCLI#2
displayName: 'dotnet restore'
inputs:
command: restore
projects: '**/*.csproj'
feedsToUse: select
vstsFeed: 'nugettest'

Answer 2

So the solution that (eventually) worked for me was to use the .Net Core (DotNetCoreCLI#2) build, and to break every step apart, e.g.
# ASP.NET Core
# Build and test ASP.NET Core projects targeting .NET Core.
# Add steps that run tests, create a NuGet package, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/dotnet-core
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
buildConfiguration: 'Release'
platform: x64
steps:
- task: DotNetCoreCLI#2
displayName: 'Restore for MyApplication'
inputs:
command: 'restore'
projects: 'MyApplication/*.csproj'
feedsToUse: 'select'
vstsFeed: '{GuidA}/{GuidB}'
- task: DotNetCoreCLI#2
displayName: 'Restore for MyApplication.Test.1'
inputs:
command: 'restore'
projects: 'MyApplication.Test.1/*.csproj'
feedsToUse: 'select'
vstsFeed: '{GuidA}/{GuidB}'
- task: DotNetCoreCLI#2
displayName: 'Restore for MyApplication.Test.2'
inputs:
command: 'restore'
projects: 'MyApplication.Test.2/*.csproj'
feedsToUse: 'select'
vstsFeed: '{GuidA}/{GuidB}'
- task: DotNetCoreCLI#2
displayName: 'Restore for MyApplication.Test.3'
inputs:
command: 'restore'
projects: 'MyApplication.Test.3/*.csproj'
feedsToUse: 'select'
vstsFeed: '{GuidA}/{GuidB}'
- task: DotNetCoreCLI#2
displayName: 'Build MyApplication'
inputs:
command: 'build'
projects: 'MyApplication/*.csproj'
- task: DotNetCoreCLI#2
displayName: 'Build Test.1'
inputs:
command: 'build'
projects: 'MyApplication.Test.1/*.csproj'
- task: DotNetCoreCLI#2
displayName: 'Build Test.2'
inputs:
command: 'build'
projects: 'MyApplication.Test.2/*.csproj'
- task: DotNetCoreCLI#2
displayName: 'Build Test.3'
inputs:
command: 'build'
projects: 'MyApplication.Test.3/*.csproj'
- task: DotNetCoreCLI#2
displayName: 'Execute Test.1s'
inputs:
command: 'test'
projects: 'MyApplication.Test.1/*.csproj'
- task: DotNetCoreCLI#2
displayName: 'Execute Test.2s'
inputs:
command: 'test'
projects: 'MyApplication.Test.2/*.csproj'
- task: DotNetCoreCLI#2
displayName: 'Execute Test.3s'
inputs:
command: 'test'
projects: 'MyApplication.Test.3/*.csproj'