Is it possible? (It seems not. Does it not support NUnit? What should I use instead?)
Here is my test project.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<IsTestProject>true</IsTestProject>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" />
<PackageReference Include="NUnit" Version="3.12.0" />
<PackageReference Include="NUnit3TestAdapter" Version="3.17.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\TheProjectToBeTested\TheProjectToBeTested.csproj" />
</ItemGroup>
</Project>
And here is my azure-pipelines.yml
pool: 'TFSBuild'
variables:
# The web app will not build because it is netcore3.1 and the server only supports netcore3.0.
solution: '**/*Build.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#0
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip" /p:DeployIisAppPath="Default Web Site"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
# I added this and I don't undertand it.
- task: VSTest#2
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: |
**\$(BuildConfiguration)\*test*.dll
**\$(BuildConfiguration)\**\*test*.dll
!**\*Microsoft.VisualStudio.TestPlatform*
!**\obj\**
searchFolder: '$(System.DefaultWorkingDirectory)'
However the tests don't get run.
##[section]Starting: VSTest
==============================================================================
Task : Visual Studio Test
Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
Version : 2.153.9
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/vstest
==============================================================================
SystemVssConnection exists true
SystemVssConnection exists true
SystemVssConnection exists true
Running tests using vstest.console.exe runner.
======================================================
Test selector : Test assemblies
Test filter criteria : null
Search folder : E:\TFS\8\s
VisualStudio version selected for test execution : latest
Attempting to find vstest.console from a visual studio installation.
Run in parallel : false
Run in isolation : false
Path to custom adapters : null
Other console options : null
Code coverage enabled : false
Diagnostics enabled : false
SystemVssConnection exists true
Run the tests locally using vstest.console.exe
========================================================
Test selector : Test assemblies
Test assemblies : **\Release\*test*.dll,**\Release\**\*test*.dll,!**\*Microsoft.VisualStudio.TestPlatform*,!**\obj\**
Test filter criteria : null
Search folder : E:\TFS\8\s
Run settings file : E:\TFS\8\s
Run in parallel : false
Run in isolation : false
Path to custom adapters : null
Other console options : null
Code coverage enabled : false
Diagnostics enabled : false
Rerun failed tests: false
VisualStudio version selected for test execution : latest
Attempting to find vstest.console from a visual studio installation.
========================================================
======================================================
[command]"E:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" #E:\TFS\_temp\126341f1-33c9-11eb-a545-b3a68600e11b.txt
Microsoft (R) Test Execution Command Line Tool Version 16.3.0-preview-20190715-02
Copyright (c) Microsoft Corporation. All rights reserved.
vstest.console.exe
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\TheProjectToBeTested.Test.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CommunicationUtilities.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CoreUtilities.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CrossPlatEngine.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.PlatformAbstractions.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.Utilities.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\NUnit3.TestAdapter.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\testhost.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
/logger:"trx"
/TestAdapterPath:"E:\TFS\8\s"
Starting test execution, please wait...
Test run will use DLL(s) built for framework .NETFramework,Version=v4.0 and platform X86. Following DLL(s) do not match framework/platform settings.
TheProjectToBeTested.Test.dll is built for Framework .NETCoreApp,Version=v3.0 and Platform AnyCPU.
Microsoft.TestPlatform.CommunicationUtilities.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
Microsoft.TestPlatform.CoreUtilities.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
Microsoft.TestPlatform.CrossPlatEngine.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
Microsoft.TestPlatform.PlatformAbstractions.dll is built for Framework .NETCoreApp,Version=v2.1 and Platform AnyCPU.
Microsoft.TestPlatform.Utilities.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
NUnit3.TestAdapter.dll is built for Framework .NETCoreApp,Version=v2.1 and Platform AnyCPU.
testhost.dll is built for Framework .NETCoreApp,Version=v2.1 and Platform AnyCPU.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.
No test is available in E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\TheProjectToBeTested.Test.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CommunicationUtilities.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CoreUtilities.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CrossPlatEngine.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.PlatformAbstractions.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.Utilities.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\NUnit3.TestAdapter.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\testhost.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CrossPlatEngine.resources.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Results File: E:\TFS\8\s\TestResults\TFSBUILD$_TFSBUILD_2020-12-01_11_33_48.trx
No Result Found to Publish 'E:\TFS\8\s\TestResults\TFSBUILD$_TFSBUILD_2020-12-01_11_33_48.trx'.
##[section]Async Command Start: Publish test results
##[section]Async Command End: Publish test results
##[section]Finishing: VSTest
Please move rather to dotnet command and instead of using VSTest run it using this:
If you have test projects in your repository, then use the .NET Core task to run unit tests by using testing frameworks like MSTest, xUnit, and NUnit.
steps:
# ...
# do this after other tasks such as building
- task: DotNetCoreCLI#2
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration)'
Run NUnit tests in Azure DevOps pipeline
To test .Net Core dlls with Visual Studio Test task, please try to add following additional settings of Visual Studio Test task:
Path to vstest.console.exe: E:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform.
Other console options: /framework:".NETCoreApp,Version=v3.0":
You maybe missing NUnit3TestAdapter package in one of project
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="NUnit" version="3.10.1" targetFramework="net45" />
<package id="NUnit3TestAdapter" version="4.1.0" targetFramework="net45" />
</packages>
After adding package this task was running all nunit tests
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
If you have a .net core application as you do, the other answer using DotNetCoreCLI#2 with test command is the correct answer.
However, if you are using framework and don't want to use VSTest or really have to use nunit there is another way to get this to work.
You can use the pipeline with a docker container that has nunit in. You can use the code from here which is a mono image that has nunit added. A requirement of Azure Devops is there is no entry point so you will need to use your own copy without the last line.
Once you have your container you can then run your pipeline in a container
and use a bash command block
- task: Bash#3
inputs:
targetType: 'inline'
script: '/nunit/nunit3-console.exe **/*Tests/*.csproj --configuration $(buildConfiguration)'
Related
I am trying to configure the VSTest#2 task in an Azure DevOps build pipeline to provide a code coverage metric.
When the pipeline runs, the logs for the task show this error (but the tests themselves run and pass):
Data collection : Unable to find a datacollector with friendly name 'Code Coverage'.
Data collection : Could not find data collector 'Code Coverage'
The target framework is net472 and the unit tests are implemented with NUnit and referencing NUnit 3.13.2 and NUnit3TestAdapter 4.1.0.
YAML task:
- task: VSTest#2
displayName: 'Run Tests'
inputs:
diagnosticsEnabled: true
codeCoverageEnabled: true
What is missing to enable code coverage for the VSTest#2 task? Does a data collector need to be configured and how can that be achieved?
TL;DR
I'm building a nuget package with Azure Devops pipelines, and the DotNetCoreCLI#2 / pack command is looking for the built assemblies and other files of dependent projects in the wrong folder because the name of the solution configuration the yaml uses is different from the name of the build configuration of the dependent projects the solution configuration uses.
In detail:
The project dependencies in the solution: Main --> DepA --> DepB.
The nuget package is built from Main.
The yaml uses a solution configuration called ReleaseMain, which ...
builds only the above projects
uses the Release build configuration of each project. I can't see any point in creating a separate build configuration called ReleaseMain in each project if Release can be referred to by ReleaseMain.
The relevant parts of the yaml:
variables:
buildConfiguration: 'ReleaseMain'
steps:
- task: NuGetToolInstaller#1
- task: DotNetCoreCLI#2
inputs:
command: 'restore'
projects: '**/Main.csproj'
feedsToUse: 'select'
vstsFeed: '<nuget feed ID>'
# I don't use a DotnetCoreCLI#2 / Build task because the pack command also builds everything it needs. This works with other pipelines.
- task: DotNetCoreCLI#2
inputs:
command: 'pack'
packagesToPack: '**/Main.csproj'
versioningScheme: 'off'
The pack task fails while building the DepB project because it is looking for obj\ReleaseMain\netstandard2.0\.NETStandard,Version=v2.0.AssemblyAttributes.cs but doesn't find it, since it is in the obj\Release\netstandard2.0\ folder. Then the log contains the same error for the DepA project, and the task fails.
So the problem is that the pack command is looking for release files in a subfolder named after the solution configuration instead of the build configuration of the projects.
In cases when the project from which the nuget package is built has no project dependencies, I can make the pipeline work by specifying the project build configuration name in the files/file XML elements of the nuspec file, see below.
<files>
<!-- instead of this -->
<file src="bin\$configuration$\netcoreapp3.1\Main.dll" target="lib\.NETCoreApp3.1" />
<!-- I hardcode this -->
<file src="bin\release\netcoreapp3.1\Main.dll" target="lib\.NETCoreApp3.1" />
</files>
But the build task is stuck before this step in the above scenario, so I only mention this as a side note.
There must be a solution for this because it seems a pretty frequent situation to me but I can't find it. I know I could create a project build configuration called ReleaseMain for each project but it seems unnecessary. Or is the way the nuget pack command works incompatible with this setup?
While running below command for the solution it works fine on command line on on-premise.
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\msbuild.exe" test.sln /t:Metrics -p:Configuration=Debug -p:Platform="Any CPU"
The task used in the pipeline is :
- task: CmdLine#2
inputs:
script: '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\msbuild.exe" Test.sln /p:AzureDevOps=true /verbosity:d /t:Metrics -p:Configuration=Debug -p:Platform="Any CPU"'
workingDirectory: 'C:\a\1\s\Ett\Test'
failOnStderr: true
Also tried the task :
- task: VSBuild#1
displayName: CodeMetrics
inputs:
solution: '**\CN.Test.sln'
msbuildArgs: '/t:Metrics'
platform: 'Any CPU'
configuration: 'debug'
createLogFile: true
logFileVerbosity: 'detailed'
However fails to run from the pipeline with below error :
The target "Metrics" does not exist in the project
The target 'Metrics' the nuget 'Microsoft.CodeAnalysis.Metrics' is installed in each project.
This is the key. That target won't be available until after you've restored the nuget packages. The packages will have been restored on your local machine though.
For your build pipeline to work make sure you perform a restore first, either as part of the existing invocation of MSBuild, or as a seperate step.
MSBuild /t:restore;metrics ...
Or:
Msbuild /t:restore ...
Msbuild /t:metrics ...
Based on your description and concern, in your second shared task, for msbuildArgs: '/t:Metrics' in the task, the specified target File structure '/t:Metrics' is not completed, that's why you met this issue.
To solve this issue, you could define the target to folder level where the spevified csproj file located.
For example: File structure: xx->xxx->metrics ->xx.csproj
msbuildArgs: ”/t:xx\xxx\metrics“
For more information, you could also refer to the doc: How to: Build specific targets in solutions by using MSBuild.exe .
I'm transitioning a traditional C++ code base build to the Azure DevOps Pipeline. We have a bunch of gtest unit tests in a module that is just an executable named "sdktests.exe". It links to gtest 1.8.1. The user has always just manually run this exe and observed the results.
But I can't seem to make it Azure Devops detect and run these tests. I'm guessing that it needs the tests in the form of a DLL that it loads instead of an EXE
Is that the case? Do I need to convert my google tests module from an EXE to a DLL to make this work?
This is my yaml pipeline step
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
testSelector: 'testAssemblies'
testAssemblyVer2: '**\*sdktests*.exe'
This is my Azure build output for the step:
Source filter: **\*sdktests*.exe
SystemVssConnection exists true
d:\a\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.170.1\Modules\DTAExecutionHost.exe --inputFile d:\a\_temp\input_484b64d0-c544-11ea-bc95-251e50750ee7.json
======================================================
##########################################################################
DtaExecutionHost version 18.170.30112.1.
Starting TestExecution Model...
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Result Attachments will be stored in LogStore
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Updated Run Settings:
<RunSettings>
<RunConfiguration>
<BatchSize>1000</BatchSize>
<ResultsDirectory>d:\a\_temp\TestResults</ResultsDirectory>
</RunConfiguration>
</RunSettings>
**************** Starting test execution *********************
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "#d:\a\_temp\jcgckxlo13t.tmp"
Microsoft (R) Test Execution Command Line Tool Version 16.6.0
Copyright (c) Microsoft Corporation. All rights reserved.
vstest.console.exe "d:\a\1\s\x64\Release\sdktests.exe"
/Settings:"d:\a\_temp\ktasri4fewz.tmp.runsettings"
/Logger:"trx"
/TestAdapterPath:"d:\a\1\s"
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
No test is available in d:\a\1\s\x64\Release\sdktests.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
#Joe,
I have run into a similar situation in the past. Here is what I found. If you are running GTest you should be running as you would normally, no need to us VSTest task. The test result file that is generated by GTest is in a JUnit formatted XML file. You can use the publish test results v2 task to publish to the build. The setup would look something like:
- task: CmdLine#1
displayName: Run Unit Tests (GTest)
inputs:
script: 'sdktests.exe'
- task: PublishTestResults#2
displayName: Publish Unit Test Results (GTest)
inputs:
testResultsFiles: '**/SDKTestResults.xml'
testRunTitle: 'GTest Results'
Here are references that you should use to create a solution that meets your needs:
Publish Test Results - Doc
Command Line - Doc
The build runs well but can not see the unit test result When added a build pipeline which used a classic edit or for my solution.
And the code coverage also blank. I have two projects and tests project in the solution. The unit test task returns the result as below:
A total of 14 test files matched the specified pattern.
No test is available in d:\a\1\s\src.net...
Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Results File: d:\a\_temp\TestResults\VssAdministrator_fv-az686_2019-12-26_06_10_28.trx
Attachments:
d:\a\_temp\TestResults\d301a26b-d99f-4b4e-bbe8-c95e588ee1c5\VssAdministrator_fv-az686 2019-12-26 06_10_20.coverage
Vstest.console.exe exited with code 0.
**************** Completed test execution *********************
Test results files: d:\a\_temp\TestResults\VssAdministrator_fv-az686_2019-12-26_06_10_28.trx
No Result Found to Publish 'd:\a\_temp\TestResults\VssAdministrator_fv-az686_2019-12-26_06_10_28.trx'.
Created test run: 1006840
Publishing test results: 0
Publishing test results to test run '1006840'.
TestResults To Publish 0, Test run id:1006840
Published test results: 0
Publishing Attachments: 2
Completed TestExecution Model...
And also I can not see anymore from the Tests windows.
The test YAML script as below:
steps:
- task: VSTest#2
displayName: 'Test Assemblies'
inputs:
testAssemblyVer2: |
**\$(BuildConfiguration)\*test*.dll
!**\obj\**
codeCoverageEnabled: true
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
There no any problem on your YAML, the same definition is all work for me.
A total of 14 test files matched the specified pattern.
No test is available in d:\a\1\s\src.net...
Based on these error message, the test files has been identified out from the folder which means the dlls has exists now. But it still say there's no test available, this is more relative with vstest.console.exe did not identify the test.
For example, if your test project type is NUnit, you could use one extension names NUnit test adapter to support this test type running in visual studio.
But, when the project is published into VSTS, this extension could not be published also, and this extension does not exists in VSTS hosted agent. Without this extension support, the vstest.console.exe could not identify the Nunit test, then it will prompt the message like No test is available in xxxxxxx. See this as refer to.
But, you can installed nuget packages to replace the role of that VS extension.
If it's type is Nunit test, you must ensure add the reference on packages NUnit and NUnit3TestAdapter in your csproj file:
<PackageReference Include="NUnit">
<Version>3.12.0</Version>
</PackageReference>
<PackageReference Include="NUnit3TestAdapter">
<Version>3.15.1</Version>
</PackageReference>
Similar method tp xUnit test.