I'm transitioning a traditional C++ code base build to the Azure DevOps Pipeline. We have a bunch of gtest unit tests in a module that is just an executable named "sdktests.exe". It links to gtest 1.8.1. The user has always just manually run this exe and observed the results.
But I can't seem to make it Azure Devops detect and run these tests. I'm guessing that it needs the tests in the form of a DLL that it loads instead of an EXE
Is that the case? Do I need to convert my google tests module from an EXE to a DLL to make this work?
This is my yaml pipeline step
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
testSelector: 'testAssemblies'
testAssemblyVer2: '**\*sdktests*.exe'
This is my Azure build output for the step:
Source filter: **\*sdktests*.exe
SystemVssConnection exists true
d:\a\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.170.1\Modules\DTAExecutionHost.exe --inputFile d:\a\_temp\input_484b64d0-c544-11ea-bc95-251e50750ee7.json
======================================================
##########################################################################
DtaExecutionHost version 18.170.30112.1.
Starting TestExecution Model...
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Result Attachments will be stored in LogStore
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Updated Run Settings:
<RunSettings>
<RunConfiguration>
<BatchSize>1000</BatchSize>
<ResultsDirectory>d:\a\_temp\TestResults</ResultsDirectory>
</RunConfiguration>
</RunSettings>
**************** Starting test execution *********************
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "#d:\a\_temp\jcgckxlo13t.tmp"
Microsoft (R) Test Execution Command Line Tool Version 16.6.0
Copyright (c) Microsoft Corporation. All rights reserved.
vstest.console.exe "d:\a\1\s\x64\Release\sdktests.exe"
/Settings:"d:\a\_temp\ktasri4fewz.tmp.runsettings"
/Logger:"trx"
/TestAdapterPath:"d:\a\1\s"
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
No test is available in d:\a\1\s\x64\Release\sdktests.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
#Joe,
I have run into a similar situation in the past. Here is what I found. If you are running GTest you should be running as you would normally, no need to us VSTest task. The test result file that is generated by GTest is in a JUnit formatted XML file. You can use the publish test results v2 task to publish to the build. The setup would look something like:
- task: CmdLine#1
displayName: Run Unit Tests (GTest)
inputs:
script: 'sdktests.exe'
- task: PublishTestResults#2
displayName: Publish Unit Test Results (GTest)
inputs:
testResultsFiles: '**/SDKTestResults.xml'
testRunTitle: 'GTest Results'
Here are references that you should use to create a solution that meets your needs:
Publish Test Results - Doc
Command Line - Doc
Related
I am trying to configure the VSTest#2 task in an Azure DevOps build pipeline to provide a code coverage metric.
When the pipeline runs, the logs for the task show this error (but the tests themselves run and pass):
Data collection : Unable to find a datacollector with friendly name 'Code Coverage'.
Data collection : Could not find data collector 'Code Coverage'
The target framework is net472 and the unit tests are implemented with NUnit and referencing NUnit 3.13.2 and NUnit3TestAdapter 4.1.0.
YAML task:
- task: VSTest#2
displayName: 'Run Tests'
inputs:
diagnosticsEnabled: true
codeCoverageEnabled: true
What is missing to enable code coverage for the VSTest#2 task? Does a data collector need to be configured and how can that be achieved?
While running below command for the solution it works fine on command line on on-premise.
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\msbuild.exe" test.sln /t:Metrics -p:Configuration=Debug -p:Platform="Any CPU"
The task used in the pipeline is :
- task: CmdLine#2
inputs:
script: '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\msbuild.exe" Test.sln /p:AzureDevOps=true /verbosity:d /t:Metrics -p:Configuration=Debug -p:Platform="Any CPU"'
workingDirectory: 'C:\a\1\s\Ett\Test'
failOnStderr: true
Also tried the task :
- task: VSBuild#1
displayName: CodeMetrics
inputs:
solution: '**\CN.Test.sln'
msbuildArgs: '/t:Metrics'
platform: 'Any CPU'
configuration: 'debug'
createLogFile: true
logFileVerbosity: 'detailed'
However fails to run from the pipeline with below error :
The target "Metrics" does not exist in the project
The target 'Metrics' the nuget 'Microsoft.CodeAnalysis.Metrics' is installed in each project.
This is the key. That target won't be available until after you've restored the nuget packages. The packages will have been restored on your local machine though.
For your build pipeline to work make sure you perform a restore first, either as part of the existing invocation of MSBuild, or as a seperate step.
MSBuild /t:restore;metrics ...
Or:
Msbuild /t:restore ...
Msbuild /t:metrics ...
Based on your description and concern, in your second shared task, for msbuildArgs: '/t:Metrics' in the task, the specified target File structure '/t:Metrics' is not completed, that's why you met this issue.
To solve this issue, you could define the target to folder level where the spevified csproj file located.
For example: File structure: xx->xxx->metrics ->xx.csproj
msbuildArgs: ”/t:xx\xxx\metrics“
For more information, you could also refer to the doc: How to: Build specific targets in solutions by using MSBuild.exe .
Is it possible? (It seems not. Does it not support NUnit? What should I use instead?)
Here is my test project.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<IsTestProject>true</IsTestProject>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" />
<PackageReference Include="NUnit" Version="3.12.0" />
<PackageReference Include="NUnit3TestAdapter" Version="3.17.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\TheProjectToBeTested\TheProjectToBeTested.csproj" />
</ItemGroup>
</Project>
And here is my azure-pipelines.yml
pool: 'TFSBuild'
variables:
# The web app will not build because it is netcore3.1 and the server only supports netcore3.0.
solution: '**/*Build.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#0
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip" /p:DeployIisAppPath="Default Web Site"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
# I added this and I don't undertand it.
- task: VSTest#2
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: |
**\$(BuildConfiguration)\*test*.dll
**\$(BuildConfiguration)\**\*test*.dll
!**\*Microsoft.VisualStudio.TestPlatform*
!**\obj\**
searchFolder: '$(System.DefaultWorkingDirectory)'
However the tests don't get run.
##[section]Starting: VSTest
==============================================================================
Task : Visual Studio Test
Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
Version : 2.153.9
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/vstest
==============================================================================
SystemVssConnection exists true
SystemVssConnection exists true
SystemVssConnection exists true
Running tests using vstest.console.exe runner.
======================================================
Test selector : Test assemblies
Test filter criteria : null
Search folder : E:\TFS\8\s
VisualStudio version selected for test execution : latest
Attempting to find vstest.console from a visual studio installation.
Run in parallel : false
Run in isolation : false
Path to custom adapters : null
Other console options : null
Code coverage enabled : false
Diagnostics enabled : false
SystemVssConnection exists true
Run the tests locally using vstest.console.exe
========================================================
Test selector : Test assemblies
Test assemblies : **\Release\*test*.dll,**\Release\**\*test*.dll,!**\*Microsoft.VisualStudio.TestPlatform*,!**\obj\**
Test filter criteria : null
Search folder : E:\TFS\8\s
Run settings file : E:\TFS\8\s
Run in parallel : false
Run in isolation : false
Path to custom adapters : null
Other console options : null
Code coverage enabled : false
Diagnostics enabled : false
Rerun failed tests: false
VisualStudio version selected for test execution : latest
Attempting to find vstest.console from a visual studio installation.
========================================================
======================================================
[command]"E:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" #E:\TFS\_temp\126341f1-33c9-11eb-a545-b3a68600e11b.txt
Microsoft (R) Test Execution Command Line Tool Version 16.3.0-preview-20190715-02
Copyright (c) Microsoft Corporation. All rights reserved.
vstest.console.exe
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\TheProjectToBeTested.Test.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CommunicationUtilities.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CoreUtilities.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CrossPlatEngine.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.PlatformAbstractions.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.Utilities.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\NUnit3.TestAdapter.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\testhost.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
/logger:"trx"
/TestAdapterPath:"E:\TFS\8\s"
Starting test execution, please wait...
Test run will use DLL(s) built for framework .NETFramework,Version=v4.0 and platform X86. Following DLL(s) do not match framework/platform settings.
TheProjectToBeTested.Test.dll is built for Framework .NETCoreApp,Version=v3.0 and Platform AnyCPU.
Microsoft.TestPlatform.CommunicationUtilities.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
Microsoft.TestPlatform.CoreUtilities.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
Microsoft.TestPlatform.CrossPlatEngine.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
Microsoft.TestPlatform.PlatformAbstractions.dll is built for Framework .NETCoreApp,Version=v2.1 and Platform AnyCPU.
Microsoft.TestPlatform.Utilities.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
NUnit3.TestAdapter.dll is built for Framework .NETCoreApp,Version=v2.1 and Platform AnyCPU.
testhost.dll is built for Framework .NETCoreApp,Version=v2.1 and Platform AnyCPU.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.
No test is available in E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\TheProjectToBeTested.Test.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CommunicationUtilities.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CoreUtilities.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CrossPlatEngine.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.PlatformAbstractions.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.Utilities.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\NUnit3.TestAdapter.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\testhost.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CrossPlatEngine.resources.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Results File: E:\TFS\8\s\TestResults\TFSBUILD$_TFSBUILD_2020-12-01_11_33_48.trx
No Result Found to Publish 'E:\TFS\8\s\TestResults\TFSBUILD$_TFSBUILD_2020-12-01_11_33_48.trx'.
##[section]Async Command Start: Publish test results
##[section]Async Command End: Publish test results
##[section]Finishing: VSTest
Please move rather to dotnet command and instead of using VSTest run it using this:
If you have test projects in your repository, then use the .NET Core task to run unit tests by using testing frameworks like MSTest, xUnit, and NUnit.
steps:
# ...
# do this after other tasks such as building
- task: DotNetCoreCLI#2
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration)'
Run NUnit tests in Azure DevOps pipeline
To test .Net Core dlls with Visual Studio Test task, please try to add following additional settings of Visual Studio Test task:
Path to vstest.console.exe: E:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform.
Other console options: /framework:".NETCoreApp,Version=v3.0":
You maybe missing NUnit3TestAdapter package in one of project
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="NUnit" version="3.10.1" targetFramework="net45" />
<package id="NUnit3TestAdapter" version="4.1.0" targetFramework="net45" />
</packages>
After adding package this task was running all nunit tests
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
If you have a .net core application as you do, the other answer using DotNetCoreCLI#2 with test command is the correct answer.
However, if you are using framework and don't want to use VSTest or really have to use nunit there is another way to get this to work.
You can use the pipeline with a docker container that has nunit in. You can use the code from here which is a mono image that has nunit added. A requirement of Azure Devops is there is no entry point so you will need to use your own copy without the last line.
Once you have your container you can then run your pipeline in a container
and use a bash command block
- task: Bash#3
inputs:
targetType: 'inline'
script: '/nunit/nunit3-console.exe **/*Tests/*.csproj --configuration $(buildConfiguration)'
I am using Visual Studio 2019 Community Edition and I have a multi-tier solution. Below is a screenshot of that.
Please note, it's a sample project to replicate a production solution, so, namings probably are not really important here.
Project specifications: (all netcoreapp3.1)
Sample.AzureFunction.Api - Target Framework: netcoreapp3.1and Azure Function version: 3.0.7
{ Application, Model, ClassLibrary1, and ClassLibrary2 } - Target Framework: netcoreapp3.1 and all are `.netcore class library' projects.
ApiTests - Target Framework: netcoreapp3.1
The Problem
The Azure Build Pipeline fails when there is a reference from the ApiTest project to the Sample.AzureFunction.Api. If I remove the project reference, the build continues to be green. Here is a screenshot from the build errors when the build step is running in the pipeline.
Basically, all the errors are complaining about not finding some dlls. For example, CSC : error CS0006: Metadata file 'D:\a\1\s\publish_output\Application.dll' could not be found [D:\a\1\s\Sample.AzureFunction.Api\Tests\ApiTests\ApiTests.csproj]
Few Notes:
The build step in the pipeline is .net core added automatically by Azure DevOps.
I don't have a dedicated Agent and I use the Azure Pipelines that comes by default when creating a new CI.
Agent specification is windows-2019
I used the classic view to create the pipeline (no YAML) but I could grab the following YAML from the generated steps by clicking on each of them and copying the YAML:
I've spent a day on resolving this issue and I'm running out of ideas now. Any thoughts would be appreciated.
If you remove --output argument your build will succeeded.
It looks like your folder publish_output was cleared before compiling test project. Thus it can't find these dlls there.
Furthermore, you don't need rather to publish as an artifact all dlls. Please use publish command to create artifact for code which is going to be deployed:
- task: DotNetCoreCLI#2
inputs:
command: publish
publishWebProjects: True
arguments: '--configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
zipAfterPublish: True
# this code takes all the files in $(Build.ArtifactStagingDirectory) and uploads them as an artifact of your build.
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: 'myWebsiteName'
I listed publish_output folder after running your original pipeline and you can find there these files:
Application.deps.json
ClassLibrary1.deps.json
Function1
Sample.AzureFunction.Api.deps.json
bin
host.json
The build runs well but can not see the unit test result When added a build pipeline which used a classic edit or for my solution.
And the code coverage also blank. I have two projects and tests project in the solution. The unit test task returns the result as below:
A total of 14 test files matched the specified pattern.
No test is available in d:\a\1\s\src.net...
Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Results File: d:\a\_temp\TestResults\VssAdministrator_fv-az686_2019-12-26_06_10_28.trx
Attachments:
d:\a\_temp\TestResults\d301a26b-d99f-4b4e-bbe8-c95e588ee1c5\VssAdministrator_fv-az686 2019-12-26 06_10_20.coverage
Vstest.console.exe exited with code 0.
**************** Completed test execution *********************
Test results files: d:\a\_temp\TestResults\VssAdministrator_fv-az686_2019-12-26_06_10_28.trx
No Result Found to Publish 'd:\a\_temp\TestResults\VssAdministrator_fv-az686_2019-12-26_06_10_28.trx'.
Created test run: 1006840
Publishing test results: 0
Publishing test results to test run '1006840'.
TestResults To Publish 0, Test run id:1006840
Published test results: 0
Publishing Attachments: 2
Completed TestExecution Model...
And also I can not see anymore from the Tests windows.
The test YAML script as below:
steps:
- task: VSTest#2
displayName: 'Test Assemblies'
inputs:
testAssemblyVer2: |
**\$(BuildConfiguration)\*test*.dll
!**\obj\**
codeCoverageEnabled: true
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
There no any problem on your YAML, the same definition is all work for me.
A total of 14 test files matched the specified pattern.
No test is available in d:\a\1\s\src.net...
Based on these error message, the test files has been identified out from the folder which means the dlls has exists now. But it still say there's no test available, this is more relative with vstest.console.exe did not identify the test.
For example, if your test project type is NUnit, you could use one extension names NUnit test adapter to support this test type running in visual studio.
But, when the project is published into VSTS, this extension could not be published also, and this extension does not exists in VSTS hosted agent. Without this extension support, the vstest.console.exe could not identify the Nunit test, then it will prompt the message like No test is available in xxxxxxx. See this as refer to.
But, you can installed nuget packages to replace the role of that VS extension.
If it's type is Nunit test, you must ensure add the reference on packages NUnit and NUnit3TestAdapter in your csproj file:
<PackageReference Include="NUnit">
<Version>3.12.0</Version>
</PackageReference>
<PackageReference Include="NUnit3TestAdapter">
<Version>3.15.1</Version>
</PackageReference>
Similar method tp xUnit test.