I have an .Net Core application along with unit test cases. For that I have configured the Build pipeline in Azure DevOps. In that pipeline I have integrated SonarQube tasks (prepare analysis, run code analysis, and publish quality gate results).
I can see the report in SonarQube server after successful run. But in that report, I didn’t see the Code Coverage Results and Unit test results. Even though I used Cobertura for unit tests.
Your pipeline seems to contain the right steps, so there can be two issues:
1. Code coverage file is not generated (correctly)
The easiest way to validate if the code coverage file is generated correctly, is by publishing it as an artifact. Now check what format the output file is. If there is no output file, please check if you did include /p:CollectCoverage=true --logger trx to the test command. If you are running the build pipeline on Linux, you should also add /p:CoverletOutputFormat=opencover and install the coverlet.collector NuGet package in the .NET Test Project.
2. Code coverage file is not sent to Sonarqube
If you configured step 1 correctly, it is still possible that the generated files are not sent to Sonarqube. The best way to see what is going wrong, is by checking the build logs of the Run Code Analysis and Publish Quality Gate Results steps.
The most common issue, is that Sonarscanner is checking the wrong directory. In the prepare step, please specify where the files are located, like:
sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/**/coverage.opencover.xml
sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)/*.trx
Build pipeline generates code coverage files would it be coverlet or opencover, the problem is that Azure Devops pipeline creates the reports outside the working directory on the build agent, it uses the _temp folder inside _work whereas soanrquube searches inside the working directory. I am facing this issue with coverlet as well as Visual studio coverage tool. You can read the following 2 threads
https://github.com/coverlet-coverage/coverlet/issues/1399
https://github.com/microsoft/azure-pipelines-tasks/issues/11536.
I posted an issue on snoarqube community site but did not get any response so far. The solution I think is either make the test step in the build pipeline change the directory or configure somewher sonarqube to search the agent build directory _temp which is above the working directory
Related
I am using ReportGenerator task in my Azure DevOps to merge Cobertura based code coverage reports into one, end up with empty reports in pipeline CodeCoverage tab.
Below is my pipeline with three jobs.
Job1 – uses Windows agent pool1, builds java- (clean compile, test, cobertura:cobertura ), if build success, saves test reports, code coverage reports (only xml) to Azure pipeline artifacts.
Job2 – uses Windows agent pool2, builds .NET core- (restore, test, coverlet reports in Cobertura format), if build success, saves test reports, code coverage reports (only xml) to Azure pipeline artifacts.
Job3 - uses Windows agent pool3, downloads test and multiple coverage reports uploaded from previous jobs, merges all Cobertura reports into one using ReportGenerator. Publish code coverage reports.
But, if I go and see the Code Coverage report tab pipeline, assemblies, classes, files, package names data is there but no coverage data, when I click on a particular package class name it’s empty and showing “ ’/some relative path/ abc.java’ does not exist (any more)”.
Please suggest.
ReportGenerator needs the source code in order to create a complete report.
There is no way to avoid that.
You need to copy the source code or check it out again in the same directory.
Agree with Daniel's answer. And if you use Microsoft-hosted agents, there are different machines for each job so they don't share same build source directory.
You could try to deploy self-hosted agent and use it in pipeline so all jobs share the same build source directory, and then you don't need to check it out again.
This seems a pretty straightforward thing to do, but I cannot find relevant information. In Visual Studio is very easy to calculate code metrics for all projects and I would like to do the same during a build pipeline in Azure DevOps.
Has anyone done something like this?
In azure devops, you can review code coverage results. The results can be viewed and downloaded on the Code coverage tab.
Publish Code Coverage Results publishes code coverage results to
Azure Pipelines or TFS, which were produced by a build in Cobertura
or JaCoCo format.
Built-in tasks such as Visual Studio Test, .NET Core, Ant, Maven,
Gulp, Grunt, and Gradle provide the option to publish code coverage
data to the pipeline.
Here are some documents you can refer to:
Review code coverage results
Azure DevOps and the Code Coverage
In addition, you can get code analysis through SonarCloud integrated with Azure devops. SonarCloud is a cloud-based code quality and security service.
Here is the lab you can follow.
You can generate code metrics by adding the Microsoft.CodeAnalysis.Metrics nuget and building with the parameter that target the "Metrics" build, this will store the results in an xml file.
msbuild /t:Metrics
If you're building a solution with multiple projects, you have to add the nuget to every project otherwise the build will fail because the Metrics target won't be present for the other projects.
The build will produce one xml file per project.
You can also use the metrics command line tool from the roslyn analyzer repository.
See https://learn.microsoft.com/en-us/visualstudio/code-quality/how-to-generate-code-metrics-data?view=vs-2022
I have vs solution which contains multiple projects,
And im now configuring CI/CD pipeline for the solution in azure.
There is one project i dont want to be include in release.
Im trying to remove that project in restore and build tasks.
But still it included in restore and build.
if anyone has an idea about this ?
I've tested on my side using build task, the exclude pattern is working as expected, please check the following sample and compare with yours:
Task:
Log:
I have a dot net core version: '3.0.100', building on 'ubuntu-latest' and am trying to get code coverage reports.
I have been using Coverlet to generate Cobertura files which can then use the PublishCodeCoverageResults#1 to publish to the Devops pipelines code coverage viewer.
I have used one stage name build for building the project and generating artifacts and published them.
In another stage named unit tests i will be downloading the artifacts and using the artifacts of previous build.
But when I run unit tests by specifying the working directory where these artifacts were downloaded
Everything went fine up to this point,but when it is generating the code coverage reports in the unit test job it is generating NaN% value.
enter image description here
I have tried all possible ways playing with working path directory and restoring the nuget packages.But No Luck None worked for me. I have even searched for this issue,I found one but that didn't worked me
When i tried to run the build and unit tests in one stage itself it successfully generated the code coverage reports,but placing that in another stage and downloading the previous build artifacts made me run into the issue.
How would I achieve this ?
I use rake to build my project and one of the steps is running the unit, integration and fitnesse tests. If too many of these fail, I fail the rake script.
That part is working fine.
Unfortunately, after the build is failed, jenkins doesn't publish the html reports I generated from the unit, integration and fitnesse tests I generated, making it tad difficult to track down the failure reason.
Am I missing a configuration step to get the reports published?
Is Jenkins supposed to skip the post-build steps when the build fails?
It seems like it some for most of the plugins I am using.
You have to tell Jenkins which artifacts to archive in a post-build step (there is a check box under general 'Post-build actions' heading which is called 'Archive the Artifacts'). Important: the artifact path is determined relative to the workspace directory. Make sure that the option Discard all but the last successful/stable artifact to save disk space is not checked.
Finally figured it out, one of those I could have had a V8 moments...
I'm using a rake file to build and one of it's tasks is failing just before some reporting tasks that need to run in order to have the HTML pushed into the correct area to be published.