My team uses VSTS with hosted agents and a Visual Studio Test build task to run all tests and produce code coverage. However the tab Code coverage remains empty afterwards, only showing a link to download the *.coverage file.
I actually expect the code coverage results to be shown, with tables and graphs which projects are tested and it's respective coverage.
We are using the Visual Studio Test build task to test net461 assemblies with the Code Coverage enabled checked.
Somewhat related are both this and this issue, but not entirely, because we are just using MSTest Framework with built-in Visual Studio Test build task, which (I understand) should also automatically publish the code coverage results.
Am I missing something to get this Code coverage view working? Thanks!!
In Code coverage Tab, you need to use Publish Code Coverage Results task to show tables and graphs there.
While for Publish Code Coverage Results task, it only supports Code Coverage data in Jacoco or Cobertura formats. So the result of the *.coverage file can not be shown by tables and graphs in Code coverage Tab.
More details, you can refer the blog Browse Code Coverage reports and the issue How to publish the code coverage result with a .coverage file.
Related
Hi guys i have a problem with my code coverage of my azure pipeline. When the CI gets trigged for the master branch the agent has a job to test the .net core. When i use this test i have to configure the route and also enable the code coverage. After the CI has been done it ended succesfully and i can download a file with the code coverage. The problem i have is i cant open this file, not with visual studio or anything else. Aynone that knows how i can fix this problem and maby i didnt configure it right?
Picture of my settings:
picture of the file
picture of the error https://i.stack.imgur.com/fxqdI.png
Seems you are using the wrong configuration. I'm afraid --collect"Code coverage" will not work.
You could add a .NET core test task and add /p:CollectCoverage=true argument then use report generate task for adding code coverage reports.
More detail steps, please refer below tutorials:
ASP.NET Core code coverage reports on Azure DevOps
Azure DevOps, unit testing and code coverage with .Net Core
To generate code coverage please follow this steps
Restore nuget packages
Build the app
Run unit tests (You need to add nuegte package to test projects -> coverlet.collector to use 'XPlat Code Coverage')
Install report generator
Run report generator
Publish code coverage
In a multi-stage, yaml configured CI pipeline that builds in Windows and Linux via matrix strategy I'm generating:
A .cobertura.xml file in Linux os, used to produce an aggregated report via ReportsGenerator that is accesible from coverage tab in AzureDevops. I'm explicitly using PublishCodeCoverageResults in this os.
A .coverage file in Windows os, to feed SonarCloud with coverage data. I'm not using PublishCodeCoverageResults in this os.
My issue is that .coverage file overrides the expected report in coverage tab, allowing me to download it rather than seeing the report. How can I avoid that? Is there any way to get rid of that attachment to properly show the report?
If I don't run the Windows part, the report is shown as expected.
If I do it, I get this rather than the report.
I've tried removing .coverage files at the end of the pipeline, but the 'attachment' in coverage tab stays there.
As far I understood you want to have download avialble instead of displaying code coverage like this:
But your code coverage is also available as an artifact:
So you can still download it:
Does it help you somehow?
Answered here:
This is not a bug and the behavior is as designed as the .coverage
report is given precedence over other reports.
My guidance would be to stop publishing the test run as part of the
dotnet task in windows. There is an option to do so.
Copy the trx and coverage files out to separate locations. Upload the
trx file using the publish test results task but make sure the
coverage file is not available for this publish.
I need to publish my test coverage results into SonarCloud. My current pipeline is this
Update: - In run code analysis step this log can be found. But in SonarCloud coverage result is not displayed.
INFO: Parsing the Visual Studio coverage XML report D:\a\1\.\TestResults\VssAdministrator_fv-az185_2019-11-13_11_15_46\In\fv-az185\VssAdministrator_fv-az185 2019-11-13 11_15_34.coveragexml
INFO: Adding this code coverage report to the cache for later reuse: D:\a\1\.\TestResults\VssAdministrator_fv-az185_2019-11-13_11_15_46\In\fv-az185\VssAdministrator_fv-az185 2019-11-13 11_15_34.coveragexml
INFO: Coverage Report Statistics: 21 files, 20 main files, 20 main files with coverage, 1 test files, 0 project excluded files, 0 other language files.
How to publish azure pipeline test result to SonarCloud
According to the document:
From Team Foundation Server 2015 or Visual Studio Team Services
Optional: To import code coverage into SonarQube, add the Visual Studio Test build task after the build one and tick Code Coverage
Enabled
So, make sure you have checked Code coverage enabled in the VsTest task.
Besides, if you already checked that option, you can check if the version of your SonarQube is 6.5, there is an issue on that version:
SonarQube 6.5 – Code Coverage Result is not displayed
And, if you can get the Code Coverage in Azure devops but not in SonarCloud, check if this thread give any helps.
Hope this helps.
https://medium.com/#chameeradulanga87/running-asp-net-nunit-tests-in-azure-devops-build-pipeline-and-publishing-results-to-sonar-cloud-20d14dccf275
Complete Guide to publishing test results
I am using the ReportGenerator task in my Azure DevOps build to generate a code coverage report. The problem is that the generated report is empty.
In my VsTest task that runs the unit tests I have checked the option Code coverage enabled. The .coverage file is correctly generated and the unit tests are all successfully run. However, the generated code coverage report shows that no code was covered by any of the unit tests.
Here is a screenshot of my code coverage Report Generator task.
And here is the output from the code coverage report.
Here is a screenshot of the Azure DevOps log file for the task showing that it executed correctly without any errors.
Interestingly the log does show the following
Analyzing 0 classes
Is this because ReportGenerator cannnot find the assemblies / classes? I've tried specififying a value for Source directories but this has made no difference.
Why is the code coverage report showing no code coverage?
ReportGenerator can not handle .coverage files.
You can find the supported formats here: https://github.com/danielpalme/ReportGenerator#supported-input-and-output-file-formats
You can use a tool like OpenCover or coverlet to generate a coverage report in a format that ReportGenerator is able to parse.
Or you can convert the .coverage file format. Here you find instructions how this can be done: https://github.com/danielpalme/ReportGenerator/wiki/Visual-Studio-Coverage-Tools
I'm trying to fail builds in Visual Studio Online when Code Coverage is below a threshold.
Is there anyway to do it under Visual Studio Online or I have to do it using the XAML Build Definitions.
It is possible to control build result with code coverage percentage result in both vNext build system and XAML build.
In vNext build, you need to add the PowerShell step in the build definition, and run one PowerShell script to determine whether the build failed or succeed. Check this blog for the details of the PowerShell script: http://blogs.msdn.com/b/tfssetup/archive/2015/11/06/controlling-build-result-with-code-coverage-percentage-using-build-vnext.aspx
If you work with one XAML build, you need to create a custom build activity in which contains the logic to fail or pass a build based on code coverage result. Then include to use the build activity in the build definition. See: http://blogs.msdn.com/b/tfssetup/archive/2015/11/06/controlling-build-result-based-on-code-coverage-percentage-for-xaml-builds.aspx
My preferred method is to use the standard XAML build process in Azure DevOps pipeline. Use the code analysis task to fail the build if coverage is below a threshold.
See: https://learn.microsoft.com/en-us/archive/blogs/tfssetup/controlling-build-result-based-on-code-coverage-percentage-for-xaml-builds