I'm trying to fail builds in Visual Studio Online when Code Coverage is below a threshold.
Is there anyway to do it under Visual Studio Online or I have to do it using the XAML Build Definitions.
It is possible to control build result with code coverage percentage result in both vNext build system and XAML build.
In vNext build, you need to add the PowerShell step in the build definition, and run one PowerShell script to determine whether the build failed or succeed. Check this blog for the details of the PowerShell script: http://blogs.msdn.com/b/tfssetup/archive/2015/11/06/controlling-build-result-with-code-coverage-percentage-using-build-vnext.aspx
If you work with one XAML build, you need to create a custom build activity in which contains the logic to fail or pass a build based on code coverage result. Then include to use the build activity in the build definition. See: http://blogs.msdn.com/b/tfssetup/archive/2015/11/06/controlling-build-result-based-on-code-coverage-percentage-for-xaml-builds.aspx
My preferred method is to use the standard XAML build process in Azure DevOps pipeline. Use the code analysis task to fail the build if coverage is below a threshold.
See: https://learn.microsoft.com/en-us/archive/blogs/tfssetup/controlling-build-result-based-on-code-coverage-percentage-for-xaml-builds
Related
Hi guys i have a problem with my code coverage of my azure pipeline. When the CI gets trigged for the master branch the agent has a job to test the .net core. When i use this test i have to configure the route and also enable the code coverage. After the CI has been done it ended succesfully and i can download a file with the code coverage. The problem i have is i cant open this file, not with visual studio or anything else. Aynone that knows how i can fix this problem and maby i didnt configure it right?
Picture of my settings:
picture of the file
picture of the error https://i.stack.imgur.com/fxqdI.png
Seems you are using the wrong configuration. I'm afraid --collect"Code coverage" will not work.
You could add a .NET core test task and add /p:CollectCoverage=true argument then use report generate task for adding code coverage reports.
More detail steps, please refer below tutorials:
ASP.NET Core code coverage reports on Azure DevOps
Azure DevOps, unit testing and code coverage with .Net Core
To generate code coverage please follow this steps
Restore nuget packages
Build the app
Run unit tests (You need to add nuegte package to test projects -> coverlet.collector to use 'XPlat Code Coverage')
Install report generator
Run report generator
Publish code coverage
I'm working on DevSecOps. Related to that, I want to secure my application with the help of Roslyn Analyzer. For that I created an MVC Web Application along with custom ruleset in Visual Studio and it's working fine with outputs clearly.
But, when I tried to do same thing with the help of VSTS Build task in my Build Definition in VSTS account. By referring the link, I known that with the help of MSBuild task we can get the Roslyn Analyser results by default. But I didn’t get the results of Roslyn Analyzer results after running my Build Definition. Can you suggest me to “How to overcome this situation”?
If that works locally when run msbuild in command line, then you can try below items to narrow down the issue:
Add a command line task to run the msbuild command line script
directly.
Change another hosted agent.
Deploy an private agent on your dev machine, then check if that works
with the private agent.
You can also reference below articles to troubleshoot the issue:
Using, configuring and distributing Roslyn analysers in teams
Running Code Analyzers on Build Server
My team uses VSTS with hosted agents and a Visual Studio Test build task to run all tests and produce code coverage. However the tab Code coverage remains empty afterwards, only showing a link to download the *.coverage file.
I actually expect the code coverage results to be shown, with tables and graphs which projects are tested and it's respective coverage.
We are using the Visual Studio Test build task to test net461 assemblies with the Code Coverage enabled checked.
Somewhat related are both this and this issue, but not entirely, because we are just using MSTest Framework with built-in Visual Studio Test build task, which (I understand) should also automatically publish the code coverage results.
Am I missing something to get this Code coverage view working? Thanks!!
In Code coverage Tab, you need to use Publish Code Coverage Results task to show tables and graphs there.
While for Publish Code Coverage Results task, it only supports Code Coverage data in Jacoco or Cobertura formats. So the result of the *.coverage file can not be shown by tables and graphs in Code coverage Tab.
More details, you can refer the blog Browse Code Coverage reports and the issue How to publish the code coverage result with a .coverage file.
In TFS post build script of a .proj file I want to find whether the project build is happening through TFS triggered build or manually triggered build.
Can someone suggest me how to do this using macros in Post Build event.
Short answer: you can make use of the IsDesktopBuild MSBUILD property within your csproj file to differentiate between TFS and local build.
Long Answer:
Developer or Team Build?
To differentiate the build environments we have to implement a mechanism that detects in which environment the build is being executed. In other words, we need to know if we running a local build that is executed by the developer or a team build running on the build server.
In fact, there are 3 different build environments we need to consider:
· Visual Studio Build – a build executed by a developer, on their own development machine inside the Visual Studio IDE
· Team Build – a build executed by TFS (manually or scheduled), on the build.
· Desktop Build – a build explicitly executed manually, on the development workstation using the command 'msbuild.exe tfsbuild.proj'.
A ‘DesktopBuild’ and a ‘TeamBuild’ are very similar in nature except that ‘DesktopBuild’ does not perform a ‘GetLatest’ function from source repository, will not ‘Label’ the source tree and will not determine the change set.
When using MSBUILD tasks (as we will use primarily in following sections), one common way to achieve this is to use the ‘IsDesktopBuild’ and ‘BuildingSolutionFile’ properties as conditions to test in the tasks.The ‘IsDesktopBuild’ property is declared in the ‘Microsoft.TeamFoundationBuild.targets’. The ‘BuildingSolutionFile’ property is declared and assigned automatically by MSBUILD.
The following table lists the values of each of these properties in each of the build environments.
Environment IsDesktopBuild BuildingSolutionFile
Visual Studio Build (empty) (empty)
Desktop Build true true
Team Build false true
One caveat with using the ‘IsDesktopBuild’ property is that it is not defined in many target files by default. This property will have an ‘empty’ value in a Visual Studio build, so we initialize it to a value of ‘true’ as the default value. Therefore we need to be explicitly define it in all MSBUILD target files where it will be tested.
We simply add the following element to all target files where we need to differentiate between a build on the development machine and a build on the build server (within the first section).
<IsDesktopBuild Condition="'$(IsDesktopBuild)' == ''">true</IsDesktopBuild>
Update: thank you #dbardakov. Starting VS 2012 we can use the property to find if the build is happening within Visual Studio:
BuildingInsideVisualStudio
MSDN SOURCE - for BuildingInsideVisualStudio
MSDN SOURCE
Following a MSDN web page, I am trying to manually run mstest within my tfsbuild.proj and put the results into the pass/fail logic so the build will fail if this particular test fails. It's kind of like running a FxCop or something else from CMD and capturing a "0" or "1" and force-fail the build.
MSTest /testcontainer:test.dll /publish:http://ourtfsmachine:8080 /teamproject:ProjectName /publishbuild:BuildNumber01 /platform:AnyCpu /flavor:Release
I could understand running this inside an Exec task, butI don't know what the BuildNumber is, for example.
Help?
Instructions for getting the Build Number from http://msdn.microsoft.com/en-us/library/ms243151%28VS.100%29.aspx:
Open Visual Studio and connect to a Team Foundation Server.
Open Team Explorer.
Open your team project and expand the team project node.
Under the build, double-click All Build Types or a specific build type to see its builds. Build names that you can use are in the Name column.