I am trying to publish the code coverage result for .NET framework 4.6.1 in Azure DevOps. I am using visual studio test task to run unit test cases and get the code coverage. After successful execution of Visual studio test task, I am getting a code coverage result in a '.coverage file'.I have opened the coverage file in visual studio in my laptop and verified it shows the number of line covered and not covered in unit test.
Using vstest.console.exe I am converting the coverage file to xml and using report generator to publish the code coverage result.
After successfully completed the above steps, when i click code coverage tab i am presented with html link to download the coverage file. I want to publish the coverage file in the code coverage tab itself.
I have also tried this as well other similar problem, when i try, i am getting a coverage.cobertura.xml with 1kb doesn't have any details about code covered in unit test.
&dotnet tool install dotnet-reportgenerator-globaltool --tool-path . --version 4.0.12
&dotnet tool install coverlet.console --tool-path . --version 1.4.1
"`nmake reports dir:"
mkdir .\reports
"`nrun tests:"
$unitTestFile = gci -Recurse | ?{ $_.FullName -like "*bin\*UnitTestProject2.dll" }
Write-Host "`$unitTestFile value: $unitTestFile"
$coverlet = "$pwd\coverlet.exe"
"calling $coverlet for $($unitTestFile.FullName)"
&$coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura"
"`ngenerate report(s)"
gci -Recurse |
?{ $_.Name -eq "coverage.cobertura.xml" } |
%{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reporttypes:HTMLInline;HTMLChart" }
Do you add Publish code coverage task to the pipeline? Code coverage Tab only supports code coverage data in Jacoco or Cobertura formats, result of the *.coverage file can not be shown by tables and graphs.
Add "Publish code coverage" task with these parameters:
Here is some tickets(case1,case2) with similar issue you can refer to.
Related
I'm trying to generate a code coverage report using the build pipeline for the C# unit tests (MSTest2). Report can be generated using the Reportgenerator.exe but expects .xml file as the input. I have added the Visual Studio test task which has generated a .coverage file in the build artifact. We could use CodeCoverage.exe to convert .coverage to .xml file.
To test this locally, I have copied the .coverage file and on running:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools\amd64>CodeCoverage collect /IIS /session:WebSession /output:'C:\CoverageFiles\test.coverage'
and
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools\amd64>CodeCoverage analyze /output:'c:\CoverageFiles\results.xml' 'c:\CoverageFiles\test.coverage'
the script is not throwing any error and xml file is also not generated.
Is there any other way to generate the .xml file from .coverage file? Any help on this is appreciated.
You can use coverlet to create a coverage report in different format. Coverlet will be invoked by msbuild. You can check my azure-pipelines.yml, where I use coverlet in combination with reporgenerator to create the code coverage in the build pipeline.
Parts of the yml:
- script: |
echo $(Build.SourcesDirectory)
cd src\Mwd.Exceptions.Solution
mkdir $(Build.SourcesDirectory)\results
dotnet test --logger trx /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura
copy Mwd.Exceptions.Test\coverage.cobertura.xml $(Build.SourcesDirectory)\results
dotnet tool install dotnet-reportgenerator-globaltool --tool-path . --version 4.0.0-rc4
.\reportgenerator "-reports:$(Build.SourcesDirectory)\results\coverage.cobertura.xml" "-targetdir:$(Build.SourcesDirectory)\results" "-reporttypes:HTMLInline;HTMLChart"
dotnet test --logger trx /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura creates the coverage in cobertura format, at the time the only format build pipelines are supporting.
.\reportgenerator "-reports:$(Build.SourcesDirectory)\results\coverage.cobertura.xml" "-targetdir:$(Build.SourcesDirectory)\results" "-reporttypes:HTMLInline;HTMLChart" generate the report.
Since coverlet adds build targets to msbuild, compiling your solution(s) via devenv.exe should also be supported.
I'm trying to set up a CI deployment for my NuGet package on VSTS so that when a new commit is made, a package is packed and sent to my feed. Unfortunately I'm not sure where to start; most of my experience with versioning has been manually updating a file that sits within the solution, hence this question, so if there is a better way to do this let me know.
I would like the name to be the version number in the AssemblyInfo.cs file ("0.0.1") with the build number of the automated build appended. So the final result would look something like "0.0.1.35".I would like to also avoid using date/time in my naming; a lot of the suggestions are to use this but I really wish to keep the version number clean so that I can release the packages.
I'm using the 'NuGet pack' task so I only have the options 'Use date-time', 'Use environment variable' or 'Use the build number'.
Date/time means I have to manually input a major, minor and patch which I would prefer to be automatic.
Environment variable, sounds like this could be it but I think I'm missing what I should put in this field.
I set my build name to be "$(BuildDefinitionName)_$(Year:yyyy).$(Month).$(DayOfMonth)$(Rev:.r)"but not getting the result I hoped.
Any help would be greatly appreciated!
Do you have the desire to have the file version match the NuGet package version? If so, you can use the following solution.
By default, the NuGet pack command will use the file version as the Package Version. To use this functionality to get the expected output that you want, you will need to update the file version during the build. This can be done easily with the Update Assembly Info task from the VSTS marketplace. There are a number of other similar tasks, but this one allows you to only modify the revision part of the file version independently of the Major, Minor, and Build versions.
Add the Update Assembly Info task to your VSTS account
Modify your build definition and add the 'Update Assembly Info' task to the build. Ensure that it is before your Visual Studio Build or MSBuild Task as you need to change the assembly info before the build occurs
Set the values in the Update Assembly Task to match what you need for your assemblies. By default it sets Revision to $(Build.BuildId) which is what you want based on your requirements
Turn 'Automatic Package Versioning' off in the Nuget Pack task
Add the 'Update Assembly Info' task to you build process and ensure that it is before your Visual Studio or MSBuild task.
Your build should now create a Nugetpackage of 0.0.1.{Build.BuildId}
Note: this was tested with version 2.* of the NuGet Task and Version 2.* of Update Assembly info task.
The simple workflow is using environment variable:
Add new variable to build definition (e.g. packageVersion)
Add PowerShell task to get version in AssemblyInfo.cs (you can refer to the code in Use a PowerShell script to customize your build process)
Update variable value in PowerShell script (step 2) by calling Write-Host "##vso[task.setvariable variable= packageVersion;]xxx" (Logging Commands)
Add NuGet pack task (Automatic package versioning: Use an environment variable; Environment variable:packageVersion)
I wrote this PowerShell script to do that:
param
(
[parameter()][string] $FolderPath,
[parameter()][string] $FileExtension
)
$RegularExpression = [regex] 'AssemblyInformationalVersionAttribute\(\"(.*)\"\)'
$path = Get-Location
# Get the files from folder that ends in $FileExtension content
$assemblyInfoFile = (Get-ChildItem -Path $FolderPath -Force -Recurse -File -Include *$FileExtension).Name
# Get the Content of the file and store it in the variable
$fileContent = Get-Content $FolderPath/$assemblyInfoFile
foreach($content in $fileContent)
{
$match = [System.Text.RegularExpressions.Regex]::Match($content, $RegularExpression)
if($match.Success) {
$version = $match.groups[1].value
}
}
# Check if variable has content
if ($version)
{
Write-Host $version
Write-Host ##vso[task.setvariable variable=packageversion]$version
}
To run the script locally:
.\powershellScriptName.ps1 -FolderPath "c:\Git\" -FileExtension "AssemblyInfo.cs"
FolderPath: the path to output build solution
FileExtension: part of the name where we gonna search the version
VSTS steps:
Build a task to build the solution and save de output directory in a variable;
Add PowerShell task to call your script with FolderPath and FileExtension as parameters;
In the end, packageversion should have the correct version
** Project technology: .netcore
I want to run NUnit on TeamCity and generate report with dotCover. But for some reasons I cannot get the coverage report for all my project assemblies.
TeamCity config as below
Teamcity: 6.5.3
NUnit: 2.5.10
.NET Runtime: Platform: x86
.NET Runtime: Version: v4.0
dotCover: bundled with TC not customized
I have checked that all *.dll *.xml *.pdb files are there in the directory as expected as below (reference DLLs are not listed)
MY.PROJECT.A.dll
MY.PROJECT.A.pdb
MY.PROJECT.A.xml
MY.PROJECT.B.dll
MY.PROJECT.B.pdb
MY.PROJECT.B.xml
MY.PROJECT.C.dll
MY.PROJECT.C.pdb
MY.PROJECT.C.xml
MY.PROJECT.Test.dll
MY.PROJECT.Test.pdb
MY.PROJECT.Test.xml
MY.PROJECT.Test.dll is executed with NUnit and this assembly is excluded in the coverage report by using *Test* filter. But only MY.PROJECT.A is in the dotCover coverage report whereas MY.PROJECT.B and MY.PROJECT.C are not included.
I checked the log but not error is found.
Any thoughts is much appreciated.
Finally, I figured out what is going on, and hope this answer is useful for those who are still struggling with this or similar problem.
Basically, dotCover only includes those assemblies are actually used (more precisely, those assemblies loaded by CLR) by the tests in the code coverage report.
In my case, only MY.PROJECT.A is used by tests, MY.PROJECT.B and MY.PROJECT.C are not used by tests due to external dependencies. and even with using MY.PROJECT.B directives in the tests, it does not count as CLR's lazy loading.
One dummy workaround to show 0% coverage report for these two assemblies is that either use anything in the assembly in the tests, or force load these assemblies by calling System.Reflection.Assembly.Load("MY.PROJECT.B")
Related Question:
Visual Studio Code Coverage Not Showing All Assemblies
What constitutes full code coverage ...
I want to use PowerShell with Psake and TeamCity to configure my CI. I used to standard Msbuild runner but now I wrote my own script for building solution but I have problem when msbuild failed.
When I was using Msbuild runner and build failed then on Overview page new section “Build errors” appears and I have there detail from msbuild. But when I wrote my custom scripts I got only error “Process exited with code 1” and I don’t know how to “create” this build errors section. Do you know how to do this? I know that I can use service messages but I can't handle failed log from msbuild.
task compile {
try {
exec { msbuild $code_dir\SampleSolution.sln /t:Build /p:Configuration=Release }
} catch {
Write-Host $_ #only information that error occured, but without any msbui details
} }
You can specify log for MSBuild
Parse error from log or attach whole log as build artifact.
##teamcity[publishArtifacts '.\msbuild.log']
To fail build you can use the following message
Write-Output "##teamcity[message text='MS Build failed' status='ERROR']"
You can get all messages that can be used here
I am using sonar-runner to run tests and code coverage over my C# code with the help of gallio plugin. The tests are running fine, but I am not able to see any code coverage on the sonar web UI.
My Sonar settings are as follows:
sonar-project.properties
mentioning only relevant bits
sonar.gallio.coverage.tool = NCover
sonar.NCover.installDirectory = C:/Program Files/NCover
sonar.donet.visualstudio.testProjectPattern = .Test
sonar.dotnet.buildConfigurations = "Release|x86"
Any idea what coule be missing??
sonar.projectKey=XXX:XXX
sonar.projectVersion=trunk
sonar.projectName=XXX
sources=.
sonar.language=cs
sonar.dotnet.visualstudio.solution.file=Project.sln
sonar.dotnet.excludeGeneratedCode=false
sonar.dotnet.4.0.sdk.directory=C:/WIndows/Microsoft.NET/Framework/v4.0.30319
sonar.dotnet.version=4.0
# Gallio
sonar.gallio.mode=
sonar.gallio.coverage.tool=NCover
sonar.gallio.runner=IsolatedAppDomain
sonar.NCover.installDirectory=c:/Program Files/NCover
sonar.gallio.installDirectory=C:/Program Files/Gallio
sonar.dotnet.test.assemblies=$(SolutionDir)/../**/bin/**/*.Tests.Unit.dll
# FXCop
sonar.fxcop.mode=
#StyleCop
sonar.stylecop.mode=
#NDeps
sonar.ndeps.mode=skip
sonar-runner.properties
You said
sonar.dotnet.buildConfigurations = "Release|x86"
If that's true, your build likely isn't generating .pdb files, which are needed to figure out the mapping between the binaries and your source files.
Does it work if you try it with a Debug build?
I was seeing this same behavior with NCover in Sonar. I found that Sonar was generating invalid arguments for Gallio's NCover runner.
Try piping the output from Sonar's runner into a text file so that you can examine the arguments more easily (on the command line, you can just type sonar-runner > output.txt to do this).
You will likely see a line like this in your output:
INFO .u.c.CommandExecutor - Executing command: C:\Program Files\Gallio\bin\Gallio.Echo.exe /r:Local /report-directory:E:\Reports\.sonar /report-name-format:gallio-report /report-type:Xml E:\Projects\UnitTests\bin\Release\UnitTests.dll /runner-property:NCoverCoverageFile=E:\Reports\.sonar\coverage-report.xml /runner-property:NCoverArguments=//ias MyFirstAssembly;MySecondtAssembly;MyThirdAssembly
If you attempt to execute this manually via Gallio on the command line, you will get an error:
Cannot find file 'MyFirstAssembly;MySecondtAssembly;MyThirdAssembly'
If you edit this list manually down to a single entry such as MyFirstAssembly*, everything will work as expected.
This seems to indicate that Sonar is generating invalid command line arguments for Gallio. As much as I love NCover, the easiest solution was to use OpenCover instead.