I'm trying to generate a code coverage report using the build pipeline for the C# unit tests (MSTest2). Report can be generated using the Reportgenerator.exe but expects .xml file as the input. I have added the Visual Studio test task which has generated a .coverage file in the build artifact. We could use CodeCoverage.exe to convert .coverage to .xml file.
To test this locally, I have copied the .coverage file and on running:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools\amd64>CodeCoverage collect /IIS /session:WebSession /output:'C:\CoverageFiles\test.coverage'
and
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools\amd64>CodeCoverage analyze /output:'c:\CoverageFiles\results.xml' 'c:\CoverageFiles\test.coverage'
the script is not throwing any error and xml file is also not generated.
Is there any other way to generate the .xml file from .coverage file? Any help on this is appreciated.
You can use coverlet to create a coverage report in different format. Coverlet will be invoked by msbuild. You can check my azure-pipelines.yml, where I use coverlet in combination with reporgenerator to create the code coverage in the build pipeline.
Parts of the yml:
- script: |
echo $(Build.SourcesDirectory)
cd src\Mwd.Exceptions.Solution
mkdir $(Build.SourcesDirectory)\results
dotnet test --logger trx /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura
copy Mwd.Exceptions.Test\coverage.cobertura.xml $(Build.SourcesDirectory)\results
dotnet tool install dotnet-reportgenerator-globaltool --tool-path . --version 4.0.0-rc4
.\reportgenerator "-reports:$(Build.SourcesDirectory)\results\coverage.cobertura.xml" "-targetdir:$(Build.SourcesDirectory)\results" "-reporttypes:HTMLInline;HTMLChart"
dotnet test --logger trx /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura creates the coverage in cobertura format, at the time the only format build pipelines are supporting.
.\reportgenerator "-reports:$(Build.SourcesDirectory)\results\coverage.cobertura.xml" "-targetdir:$(Build.SourcesDirectory)\results" "-reporttypes:HTMLInline;HTMLChart" generate the report.
Since coverlet adds build targets to msbuild, compiling your solution(s) via devenv.exe should also be supported.
Related
I am trying to publish the code coverage result for .NET framework 4.6.1 in Azure DevOps. I am using visual studio test task to run unit test cases and get the code coverage. After successful execution of Visual studio test task, I am getting a code coverage result in a '.coverage file'.I have opened the coverage file in visual studio in my laptop and verified it shows the number of line covered and not covered in unit test.
Using vstest.console.exe I am converting the coverage file to xml and using report generator to publish the code coverage result.
After successfully completed the above steps, when i click code coverage tab i am presented with html link to download the coverage file. I want to publish the coverage file in the code coverage tab itself.
I have also tried this as well other similar problem, when i try, i am getting a coverage.cobertura.xml with 1kb doesn't have any details about code covered in unit test.
&dotnet tool install dotnet-reportgenerator-globaltool --tool-path . --version 4.0.12
&dotnet tool install coverlet.console --tool-path . --version 1.4.1
"`nmake reports dir:"
mkdir .\reports
"`nrun tests:"
$unitTestFile = gci -Recurse | ?{ $_.FullName -like "*bin\*UnitTestProject2.dll" }
Write-Host "`$unitTestFile value: $unitTestFile"
$coverlet = "$pwd\coverlet.exe"
"calling $coverlet for $($unitTestFile.FullName)"
&$coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura"
"`ngenerate report(s)"
gci -Recurse |
?{ $_.Name -eq "coverage.cobertura.xml" } |
%{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reporttypes:HTMLInline;HTMLChart" }
Do you add Publish code coverage task to the pipeline? Code coverage Tab only supports code coverage data in Jacoco or Cobertura formats, result of the *.coverage file can not be shown by tables and graphs.
Add "Publish code coverage" task with these parameters:
Here is some tickets(case1,case2) with similar issue you can refer to.
I'm transitioning a traditional C++ code base build to the Azure DevOps Pipeline. We have a bunch of gtest unit tests in a module that is just an executable named "sdktests.exe". It links to gtest 1.8.1. The user has always just manually run this exe and observed the results.
But I can't seem to make it Azure Devops detect and run these tests. I'm guessing that it needs the tests in the form of a DLL that it loads instead of an EXE
Is that the case? Do I need to convert my google tests module from an EXE to a DLL to make this work?
This is my yaml pipeline step
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
testSelector: 'testAssemblies'
testAssemblyVer2: '**\*sdktests*.exe'
This is my Azure build output for the step:
Source filter: **\*sdktests*.exe
SystemVssConnection exists true
d:\a\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.170.1\Modules\DTAExecutionHost.exe --inputFile d:\a\_temp\input_484b64d0-c544-11ea-bc95-251e50750ee7.json
======================================================
##########################################################################
DtaExecutionHost version 18.170.30112.1.
Starting TestExecution Model...
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Result Attachments will be stored in LogStore
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Updated Run Settings:
<RunSettings>
<RunConfiguration>
<BatchSize>1000</BatchSize>
<ResultsDirectory>d:\a\_temp\TestResults</ResultsDirectory>
</RunConfiguration>
</RunSettings>
**************** Starting test execution *********************
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "#d:\a\_temp\jcgckxlo13t.tmp"
Microsoft (R) Test Execution Command Line Tool Version 16.6.0
Copyright (c) Microsoft Corporation. All rights reserved.
vstest.console.exe "d:\a\1\s\x64\Release\sdktests.exe"
/Settings:"d:\a\_temp\ktasri4fewz.tmp.runsettings"
/Logger:"trx"
/TestAdapterPath:"d:\a\1\s"
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
No test is available in d:\a\1\s\x64\Release\sdktests.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
#Joe,
I have run into a similar situation in the past. Here is what I found. If you are running GTest you should be running as you would normally, no need to us VSTest task. The test result file that is generated by GTest is in a JUnit formatted XML file. You can use the publish test results v2 task to publish to the build. The setup would look something like:
- task: CmdLine#1
displayName: Run Unit Tests (GTest)
inputs:
script: 'sdktests.exe'
- task: PublishTestResults#2
displayName: Publish Unit Test Results (GTest)
inputs:
testResultsFiles: '**/SDKTestResults.xml'
testRunTitle: 'GTest Results'
Here are references that you should use to create a solution that meets your needs:
Publish Test Results - Doc
Command Line - Doc
I need help. I am trying to configure roslyn code analyser in azure devops. I need the MsCommandLine to add this roslyn analyser.
and the build task generated is
- task: RoslynAnalyzers#3 inputs:
userProvideBuildInfo: 'msBuildInfo'
msBuildVersion: '16.0'
msBuildArchitecture: 'amd64'
msBuildCommandline: '$(Parameters.solution)'
But i am getting error
MSBUILD : error MSB1003: Specify a project or solution file. The
current working directory does not contain a project or solution
file.
What should be added in MSBuildCommandLine to specify source directory of the project?
I have referred to https://secdevtools.azurewebsites.net/helpRoslynAnalyzers.html
and https://www.1eswiki.com/wiki/Secure_Development_Tools_Extension_For_Azure_DevOps .
I am not getting any clue about this MSCommandLine. How to specify it?
The commandline should begin with a full path to MSBuild.exe.
The command will run with $(Build.SourcesDirectory) as the working directory. You may specify the full path to MSBuild.exe, and try again.
The commandline should begin with a full path to MSBuild.exe or dotnet.exe. So your msBuildCommandline statement should look like below
msBuildCommandline: "C:\Program Files\dotnet\dotnet.exe" build '$(Parameters.solution)' ...
When the "run code analysis" is started in the Azure DevOps the following error is shown:
==============================================================================
Task : Run Code Analysis
Description : Run scanner and upload the results to the SonarCloud server.
Version : 1.6.3
Author : sonarsource
Help : This task is not needed for Maven and Gradle projects since the scanner should be run as part of the build.
[More Information](http://redirect.sonarsource.com/doc/install-configure-scanner-tfs-ts.html)
==============================================================================
[command]D:\a\_tasks\SonarCloudPrepare_14d9cde6-c1da-4d55-aa01-2965cd301255\1.6.3\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.exe end
SonarScanner for MSBuild 4.6.2
Using the .NET Framework version of the Scanner for MSBuild
Post-processing started.
17:36:00.872 Fetching code coverage report information from TFS...
17:36:00.872 Attempting to locate a test results (.trx) file...
17:36:00.92 Looking for TRX files in: D:\a\1\TestResults
17:36:00.92 No test results files found
17:36:00.951 Did not find any binary coverage files in the expected location.
17:36:00.951 Falling back on locating coverage files in the agent temp directory.
17:36:00.951 Searching for coverage files in D:\a\_temp
17:36:00.951 No coverage files found in the agent temp directory.
WARNING: The following projects do not have a valid ProjectGuid and were not built using a valid solution (.sln) thus will be skipped from analysis...
D:\a\1\s\Domain.Core\Domain.Core.csproj, D:\a\1\s\Shared\Shared.csproj, D:\a\1\s\API\API.csproj, D:\a\1\s\Domain.Core\Domain.Core.csproj, D:\a\1\s\Shared\Shared.csproj, D:\a\1\s\Domain.Wallet\Domain.Wallet.csproj, D:\a\1\s\Domain.Store\Domain.Stores.csproj, D:\a\1\s\Domain.Order\Domain.Orders.csproj, D:\a\1\s\Domain.Users\Domain.Users.csproj, D:\a\1\s\App\App.csproj, D:\a\1\s\Domain.Core\Domain.Core.csproj, D:\a\1\s\Domain.Core\Domain.Core.csproj, D:\a\1\s\Domain.Store\Domain.Stores.csproj, D:\a\1\s\Shared\Shared.csproj, D:\a\1\s\Domain.Store\Domain.Stores.csproj, D:\a\1\s\Domain.Order\Domain.Orders.csproj, D:\a\1\s\Domain.Core\Domain.Core.csproj, D:\a\1\s\Shared\Shared.csproj, D:\a\1\s\Domain.Store\Domain.Stores.csproj, D:\a\1\s\Domain.Core\Domain.Core.csproj, D:\a\1\s\Shared\Shared.csproj,
WARNING: Duplicate ProjectGuid: "00000000-0000-0000-0000-000000000000". The project will not be analyzed by SonarQube. Project file: "D:\a\1\s\Domain.Core\Domain.Core.csproj"
WARNING: Duplicate ProjectGuid: "00000000-0000-0000-0000-000000000000". The project will not be analyzed by SonarQube. Project file:
##[error]No analysable projects were found. SonarQube analysis will not be performed. Check the build summary report for details.
No analysable projects were found. SonarQube analysis will not be performed. Check the build summary report for details.
Generation of the sonar-properties file failed. Unable to complete SonarQube analysis.
##[error]17:36:01.201 Post-processing failed. Exit code: 1
17:36:01.201 Post-processing failed. Exit code: 1
##[error]D:\a\_tasks\SonarCloudPrepare_14d9cde6-c1da-4d55-aa01-2965cd301255\1.6.3\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.exe failed with return code: 1
##[section]Finishing: Run Code Analysis
So I can not get the final report as expected.
In .Net core you must give to each project s special GUID.
So go to your each .csproj and inside the <PropertyGroup> add this line:
# Replace the guid here with a new one
<ProjectGuid>{13212313-gd5543-1321-3fdf1-313fdfs13}</ProjectGuid>
You can generate GUIDs to your projects here.
I was having the same issue in a pipeline, just remember to put the Sonar Prepare task before the build process and Sonar Analize at the end.
Cheers.
I am trying to use NuGet to package and publish the package with TFS Build 2015 to local NuGet Server. I am getting error , I am not sure what am i missing. Thanks for Help.
Here is Error
Starting task: NuGet Packager
Set workingFolder to default: C:\Lucky\agent\tasks\NuGetPackager\0.1.58
Executing the powershell script: C:\Lucky\agent\tasks\NuGetPackager\0.1.58\NuGetPackager.ps1
Checking pattern is specified
No Pattern found in solution parameter.
Found files: 1
--File: "C:\Lucky\agent_work\1\s\Dev\FabrikamFiber.CallCenter"
The property DirectoryName does not exist or was not found.
Creating Nuget Arguments:
--ARGS: pack "C:\Lucky\agent_work\1\s\Dev\FabrikamFiber.CallCenter" -OutputDirectory "C:\Lucky\agent_work\1\s" -Properties Configuration=Release
Invoking nuget with pack "C:\Lucky\agent_work\1\s\Dev\FabrikamFiber.CallCenter" -OutputDirectory "C:\Lucky\agent_work\1\s" -Properties Configuration=Release on
C:\Lucky\agent\agent\worker\tools\NuGet.exe pack "C:\Lucky\agent_work\1\s\Dev\FabrikamFiber.CallCenter" -OutputDirectory "C:\Lucky\agent_work\1\s" -Properties Configuration=Release
MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
Please specify a nuspec or project file to use.
Unexpected exit code 1 returned from tool NuGet.exe
Finishing task: NuGetPackager
Task NuGetPackager failed. This caused the job to fail. Look at the logs for the task for more details.
According to the error info:
Please specify a nuspec or project file to use. Unexpected exit code 1
returned from tool NuGet.exe
You may specified a wrong argument in nuget package task ,please double check you have followed below requirements:
Specify .csproj files (for example, **\*.csproj) for simple projects. In this case:
The packager compiles the .csproj files for packaging.
You must specify Configuration to Package (see below).
You do not have to check in a .nuspec file. If you do check one in, the packager honors its settings and replaces tokens such as $id$ and
$description$.
Specify .nuspec files (for example, **\*.nuspec) for more complex projects, such as multi-platform scenarios in which you need to
compile and package in separate steps. In this case:
The packager does not compile the .csproj files for packaging.
Each project is packaged only if it has a .nuspec file checked in.
The packager does not replace tokens in the .nuspec file (except the element, see Use build number to version package,
below). You must supply values for elements such as and
. The most common way to do this is to hardcode the
values in the .nuspec file.
Please double check your arguments , more details please refer this tutorial-- Pack NuGet packages.
Besides you could also enable verbose debug mode by adding system.debug=true to get a more detail build log info for troubleshooting.