Im currently setting up an azure pipeline for my repository. Currently it builds correctly and runs the unit tests. However the code coverage tab just spins infinitely. Any idea on what would cause this?
Details:
The artifact directory looks like this:
The console shows this error:
Error: Could not find route for route id
ms.vss-tfs-web.project-overview-route. Ensure that the requested route
is added to routes shared data.
This is how the test results are ran and generated:
dotnet tool install dotnet-reportgenerator-globaltool --tool-path .
dotnet test $(Build.SourcesDirectory)\RulesMadeEasy.Tests -c debug --logger trx --no-restore /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura --results-directory $(Build.SourcesDirectory)\TestResults\ /p:CoverletOutput=$(Build.SourcesDirectory)\TestResults\
.\reportgenerator -reports:$(Build.SourcesDirectory)\TestResults\coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)\TestResults\ -reporttypes:"HTMLInline_AzurePipelines;Badges" --version 4.0.0-rc4
The code coverage results are published using the PublishCodeCoverageResults#1 task with the following inputs
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(Build.SourcesDirectory)\TestResults\coverage.cobertura.xml'
reportDirectory: '$(Build.SourcesDirectory)\TestResults'
You have to enable the Boards service in you Azure DevOps project to let the error disappear.
Related
I tried to build my Angular 13 app on a self-hosted agent and created the following YAML snippet for this:
- task: NodeTool#0
displayName: 'Install Node.js'
inputs:
versionSpec: '14.x'
- script: |
npm install -g #angular/cli
npm install
ng build --configuration production --aot
displayName: 'npm install and build'
workingDirectory: '$(Build.SourcesDirectory)/src'
I can observe the /s directory of the agent _work-directory and after my task was running, there is no node_modules folder or dist folder inside.
But also no console output.
If I remove the line "npm install -g #angular/cli" from the line, a node_modules folder gets created, but no dist-folder.
I am pretty sure that the installation of angular cli fails, but I do not get any error output in my window.
It just looks like this:
How can I get more logs to find out why the angular cli is not installing correctly? I saw that the "script" file that is executed on the agent puts an #echo off by default in front of the script.
Why is that?
How can I get some output to find my problem?
To get more detailed log from the pipeline you can add the variable system.debug and set the value to true in your pipeline.
For YAML pipelines, you can select Variables in the upper right corner of the YAML edit page.
Add a new variable with the name System.Debug and value true.
For more info about logs, please refer to Review logs to diagnose pipeline issues.
I am trying to get code coverage results to show up on the pipeline run summary tab. To debug this, I used a sample project with the following pipeline.yaml file:
trigger:
- master
pool:
vmImage: ubuntu-latest
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '3.7'
displayName: 'Use Python 3.7'
- script: |
pip install -r requirements.txt
displayName: 'Install requirements'
- script: |
pip install pytest pytest-azurepipelines
pip install pytest-cov
pytest --doctest-modules --junitxml=junit/test-results.xml --cov=. --cov-report=xml
displayName: 'pytest'
- task: PublishTestResults#2
condition: succeededOrFailed()
inputs:
testResultsFiles: '**/test-*.xml'
testRunTitle: 'Publish test results'
The output of the PublishTestResults task is (Edit: As pointed out by the answers, this has nothing to do with the problem since it merely reports the test upload but not the code coverage report upload):
Starting: PublishTestResults
==============================================================================
Task : Publish Test Results
Description : Publish test results to Azure Pipelines
Version : 2.180.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/publish-test-results
==============================================================================
/usr/bin/dotnet --version
5.0.301
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Async Command Start: Publish test results
Publishing test results to test run '324'.
TestResults To Publish 1, Test run id:324
Test results publishing 1, remaining: 0. Test run id: 324
Published Test Run : https://dev.azure.com/.../Runs?runId=324&_a=runCharts
Async Command End: Publish test results
Finishing: PublishTestResults
So, it seems that everything works as expected. However, I do not get the results to display on the summary tab (and no code coverage tab appears):
Screenshot of pipeline summary
Since I am running out of ideas what to try: Is this feature broken or did I do something wrong?
Edit:
It has been suggested to add a PublishCodeCoverageResults-task. However, with the following at the end of the above yaml-file, it still does not work.
- task: PublishCodeCoverageResults#1
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(System.DefaultWorkingDirectory)/**/coverage.xml'
The output of the task is:
Starting: PublishCodeCoverageResults
==============================================================================
Task : Publish code coverage results
Description : Publish Cobertura or JaCoCo code coverage results from a build
2021-07-21T04:26:34: -reports:/home/vsts/work/1/s/**/coverage.xml
2021-07-21T04:26:34: -targetdir:/home/vsts/work/_temp/cchtml
2021-07-21T04:26:34: -reporttypes:HtmlInline_AzurePipelines
2021-07-21T04:26:35: Writing report file '/home/vsts/work/_temp/cchtml/index.html'
2021-07-21T04:26:35: Report generation took 0.5 seconds
Generated code coverage html report: /home/vsts/work/_temp/cchtml
Reading code coverage summary from '/home/vsts/work/1/s/coverage.xml'
Async Command Start: Publish code coverage
Publishing coverage summary data to TFS server.
Lines- 17 of 22 covered.
Branches- 0 of 0 covered.
Modifying Cobertura Index file
Publishing code coverage files to TFS server.
Uploading 8 files
Building file tree
Uploaded 0 out of 1,426,385 bytes.
Uploaded 1,426,385 out of 1,426,385 bytes.
Associating files
Total files: 8 ---- Associated files: 0 (0%)
File upload succeed.
Published '/home/vsts/work/_temp/cchtml' as artifact 'Code Coverage Report_1263'
Async Command End: Publish code coverage
Finishing: PublishCodeCoverageResults
To see code coverage I think you need to add task PublishCodeCoverageResults#1
https://learn.microsoft.com/en-us/azure/devops/pipelines/ecosystems/python?view=azure-devops#run-tests
Add the Publish Code Coverage Results task.
- task: PublishCodeCoverageResults#1
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(System.DefaultWorkingDirectory)/**/coverage.xml'
Estension https://marketplace.visualstudio.com/items?itemName=dazfuller.pyunittest-task
I'm trying to run a "dotnet tool install" to install cli tool on the build agent in my pipeline and the behavior is inconsistent. Sometimes it executes the command successfully but it also fails sometimes with below error and I cannot figure out why however the same step executes successfully when i use azure default pipeline instead of YAML based.
error NU1212: Invalid project-package combination for project 1.0.0. DotnetToolReference project style can only contain references of the DotnetTool type The tool package could not be restored. Tool 'project' failed to install. This failure may have been caused by:
* You are attempting to install a preview release and did not use the --version option to specify the version.
* A package by this name was found, but it was not a .NET Core tool.
* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
* You mistyped the name of the tool.
Here is the task:
- task: DotNetCoreCLI#2
displayName: 'Install project'
inputs:
command: custom
custom: tool
arguments: 'install -g --version 1.0.0 --add-source https://sample.jfrog.io/artifactory/project project'
Command:
dotnet tool install -g --version 1.0.0 --add-source https://sample.jfrog.io/artifactory/project project
These tasks are running on Windows-latest image on Microsoft Hosted agents
I developed WebAPI project using .NET Core 3.1.0 and integration tests using XUnit.
I added the below task in Azure DevOps CI Pipeline (azure-pipelines.yaml) to run the integration tests project.
- task: DotNetCoreCLI#2
displayName: 'Run API integration tests - $(buildConfiguration)'
inputs:
command: 'test'
arguments: '--configuration $(buildConfiguration)'
publishTestResults: true
projects: '**/IntegrationTests/IntegrationTests.csproj'
I got the below error during pipeline execution. How to resolve this error?
##[error]Error: The process '/usr/bin/dotnet' failed with exit code 1
##[warning].NET 5 has some compatibility issues with older Nuget versions(<=5.7), so if you are using an older Nuget version(and not dotnet cli) to restore, then the dotnet cli commands (e.g. dotnet build) which rely on such restored packages might fail. To mitigate such error, you can either: (1) - Use dotnet cli to restore, (2) - Use Nuget version 5.8 to restore, (3) - Use global.json using an older sdk version(<=3) to build
Info: Azure Pipelines hosted agents have been updated and now contain .Net 5.x SDK/Runtime along with the older .Net Core version which are currently lts. Unless you have locked down a SDK version for your project(s), 5.x SDK might be picked up which might have breaking behavior as compared to previous versions. You can learn more about the breaking changes here: https://learn.microsoft.com/en-us/dotnet/core/tools/ and https://learn.microsoft.com/en-us/dotnet/core/compatibility/ . To learn about more such changes and troubleshoot, refer here: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops#troubleshooting
##[error]Dotnet command failed with non-zero exit code on the following projects : /home/vsts/work/1/s/src/IntegrationTests/IntegrationTests.csproj
I've had a smiller issue but with .netcore 2.2. The problem was that the test tries to build before the test starts restoring the packages for the test, thus fails before the test runs or builds. One thinge that help me overcome this problem was this FAQ:
Most dotnet commands, including build, publish, and test include an implicit restore step. This will fail against authenticated feeds, even if you ran a successful dotnet restore in an earlier step, because the earlier step will have cleaned up the credentials it used.
To fix this issue, add the --no-restore flag to the Arguments textbox.
I've also read that the DotNetCLI had some issues when it came to tests like this one here
So I ended up using a script to solve this and other issues related to package restore.
- script: dotnet test '**/IntegrationTests/IntegrationTests.csproj' --configuration $(buildConfiguration) --logger trx;LogFileName=C:\temp\results
displayName: 'Run API integration tests - $(buildConfiguration)'
I hope that will help you or anyone who has similar issues.
I've had exactly the same problem, with the difference that my solution consisted of .net5 apps as well as .netcore3.1 apps.
I was able to solve this problem by specifying the newer dotnet runtime in the azure pipeline:
- task: UseDotNet#2
inputs:
version: '5.0.x'
packageType: runtime
I have an Azure DevOps pipeline that validates pull requests. I have configured dotnet test to collect code coverage metrics using the --collect "Code coverage" argument:
- task: DotNetCoreCLI#2
displayName: dotnet test
inputs:
command: 'test'
arguments: '--configuration $(BuildConfiguration) --collect "Code coverage" /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura'
workingDirectory: $(baseWorkingDirectory)
projects: 'tests/**/*.csproj'
nobuild: true
As you can see, I'm also passing /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura. This I have to do in order to generete a coverage report:
- task: CmdLine#2
inputs:
script: dotnet tool install -g dotnet-reportgenerator-globaltool
- task: CmdLine#2
inputs:
script: reportgenerator -reports:$(Build.SourcesDirectory)/tests/**/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/CodeCoverage -reporttypes:HtmlInline_AzurePipelines;Cobertura
- task: PublishCodeCoverageResults#1
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(Build.SourcesDirectory)/CodeCoverage/Cobertura.xml'
reportDirectory: '$(Build.SourcesDirectory)/CodeCoverage'
Documentation states that code coverage for pull requests is available only using the Visual Studio code coverage results format (file extension .coverage)
So:
I have to use Visual Studio code coverage --collect:"Code Coverage" to get code coverage for pull requests, because Cobertura format won't work.
I need to use Cobertura format in order to get a readable report on the Code Coverage tab in the pipeline, but the Cobertura report won't show up if I use --collect:"Code Coverage" at the same time.
It seems I can't get both code coverage for PRs and a full report in Cobertura format at the same time.
Other people seem to have the same problem, but the issue wasn't resolved in that thread.
Am I missing something?
Can you try use this?
You have to install coverlet.collector package in all your test projects. Please notice that I used argument --collect:"XPlat Code Coverage".
# You just added coverlet.collector to use 'XPlat Code Coverage'
- task: DotNetCoreCLI#2
displayName: Test
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage" -- RunConfiguration.DisableAppDomain=true'
workingDirectory: $(Build.SourcesDirectory)
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- script: ./reportgenerator -reports:$(Agent.TempDirectory)/**/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/coverlet/reports -reporttypes:"Cobertura"
displayName: Create reports
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(Build.SourcesDirectory)/coverlet/reports/Cobertura.xml
Please be aware that you may have different folder strcture.
If you want to use Code coverage for pull requests feature you should not use Cobertura:
Code coverage for pull requests capability is currently only available for Visual Studio code coverage (.coverage) formats. This can be used if you publish code coverage using the Visual Studio Test task, the test verb of dotnet core task and the TRX option of the publish test results task. Support for other coverage tools and result formats will be added in future milestones.
In the pipeline definition:
- task: DotNetCoreCLI#2
displayName: Test
inputs:
command: test
projects: 'path/to/test.csproj' # if multiple test projects are to be executed, create a dirs.proj and use it here. That will ensure the tests are executed in parallel
arguments: '--collect "Code Coverage" '
publishTestResults: true
In test project (or Packages.props if using https://github.com/microsoft/MSBuildSdks/tree/main/src/CentralPackageVersions)
...
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.0.0" />
...
According to https://github.com/microsoft/vstest/issues/981 and with Microsoft.NET.Test.Sdk 17.0.0+ coverlet is no longer needed as all coverage processing can be done using .coverage files on other-than Windows platforms using --collect:"Code Coverage". coverlet is no longer needed.
Unfortunately Azure DevOps is not supporting both code coverage PR support and readable reports at the same time.
My recommendation is to use dynamic code coverage (--collect "Code Coverage") and get *.coverage files. On PR build you can publish test results with publishRunAttachments: true. By this you should get code coverage PR support.
Additionally you can use dotnet-coverage to convert your *.coverage reports to cobertura. Then you can use report-generator to generate readable coverage report. This coverage report can be uploaded as zip into artifacts of build. In this case using PublishCodeCoverageResults#1 will not work as Azure DevOps is not supporting it.
On non-PR builds you can again use dynamic code coverage but when publishing test results specify publishRunAttachments: false. Then again use dotnet-coverage to merge and convert your reports to cobertura. Finally you can add
PublishCodeCoverageResults#1 task to get readable coverage report.
Both dotnet-coverage and report-generator are cross-plat dotnet global tools. You can easily install it in your pipeline.
Example script to generate html report and zip it:
dotnet-coverage merge -r -f cobertura -o merged.cobertura.xml *.coverage
reportgenerator -reports:merged.cobertura.xml -targetDir coverageReportHtml -reporttypes:HtmlInline
zip -r report.zip coverageReportHtml/*
There is one more workaround but I didn't try this. You could theoretically trigger 1 more build from your PR build. The second build could download *.coverage artifacts from first one and then use tooling like above to finally perform PublishCodeCoverageResults#1 task.