VSTest#2 Task is Updating / Overriding coverlet.runsettings and not respecting exclusions - azure-devops

Question

I am trying to add code coverage to a .NET Framework 4.8 solution in Azure DevOps. I have based my approach on the questions here and here and the coverlet docs, and am successfully getting code coverage results.
However this solution also contains a number of shared projects which have their unit tests in a different solution. I wish to exclude these projects, and the test projects themselves from the coverage report.
In my coverlet.runsettings file I have included the following lines based on the docs:
<Exclude>[coverlet.*.tests?]*,[*]Coverlet.Core*,[Company.*UnitTest]*</Exclude>
<Include>[Company.Application.*]*</Include>
However when the tests are run using the VSTest#2 I see the following in the logs
Provided settings file:
<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="XPlat code coverage">
<Configuration>
<Format>cobertura</Format>
<Exclude>[coverlet.*.tests?]*,[*]Coverlet.Core*,[Company.*UnitTests]*</Exclude>
<Include>[Company.Application.*]*</Include>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
Updated Run Settings:
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="XPlat code coverage">
<!-- As above -->
</DataCollector>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<!-- Lots of configuration ommitted for brevity -->
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
<RunConfiguration>
<MaxCpuCount>0</MaxCpuCount>
<BatchSize>1000</BatchSize>
<ResultsDirectory>D:\a\_temp\TestResults</ResultsDirectory>
</RunConfiguration>
</RunSettings>
**************** Starting test execution *********************
So it appears the VS test task is altering my runsettings on the fly and then not respecting the inclusions and exclusions values
My yaml is:
- task: VSTest#2
displayName: 'Run Tests'
inputs:
testAssemblyVer2: '**\*UnitTests*.dll'
searchFolder: '.\Output'
codeCoverageEnabled: true
runSettingsFile: .\Builds\coverlet.runsettings
Can anyone suggest a setting or some other way to respect the inclusions and exclusions?

Answer 1

It is counterintuitive, but in order to VS test task does not alter runsettings on the fly, need to remove "codeCoverageEnabled: true" and add "runSettingsFile: .\Builds\coverlet.runsettings":
- task: VSTest#2
displayName: 'Run Tests'
inputs:
testAssemblyVer2: '**\*UnitTests*.dll'
searchFolder: '.\Output'
runSettingsFile: .\Builds\coverlet.runsettings
Code coverage still will be calculated because of runsettings file.
In order to exclude something, you can check https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#sample-runsettings-file
I copy pasted here in case the link is broken
<!--
About include/exclude lists:
Empty "Include" clauses imply all; empty "Exclude" clauses imply none.
Each element in the list is a regular expression (ECMAScript syntax). See /visualstudio/ide/using-regular-expressions-in-visual-studio.
An item must first match at least one entry in the include list to be included.
Included items must then not match any entries in the exclude list to remain included.
-->
<!-- Match assembly file paths: -->
<ModulePaths>
<Include>
<ModulePath>.*\.dll$</ModulePath>
<ModulePath>.*\.exe$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*CPPUnitTestFramework.*</ModulePath>
</Exclude>
</ModulePaths>
<!-- Match fully qualified names of functions: -->
<!-- (Use "\." to delimit namespaces in C# or Visual Basic, "::" in C++.) -->
<Functions>
<Exclude>
<Function>^Fabrikam\.UnitTest\..*</Function>
<Function>^std::.*</Function>
<Function>^ATL::.*</Function>
<Function>.*::__GetTestMethodInfo.*</Function>
<Function>^Microsoft::VisualStudio::CppCodeCoverageFramework::.*</Function>
<Function>^Microsoft::VisualStudio::CppUnitTestFramework::.*</Function>
</Exclude>
</Functions>
<!-- Match attributes on any code element: -->
<Attributes>
<Exclude>
<!-- Don't forget "Attribute" at the end of the name -->
<Attribute>^System\.Diagnostics\.DebuggerHiddenAttribute$</Attribute>
<Attribute>^System\.Diagnostics\.DebuggerNonUserCodeAttribute$</Attribute>
<Attribute>^System\.CodeDom\.Compiler\.GeneratedCodeAttribute$</Attribute>
<Attribute>^System\.Diagnostics\.CodeAnalysis\.ExcludeFromCodeCoverageAttribute$</Attribute>
</Exclude>
</Attributes>
<!-- Match the path of the source files in which each method is defined: -->
<Sources>
<Exclude>
<Source>.*\\atlmfc\\.*</Source>
<Source>.*\\vctools\\.*</Source>
<Source>.*\\public\\sdk\\.*</Source>
<Source>.*\\microsoft sdks\\.*</Source>
<Source>.*\\vc\\include\\.*</Source>
</Exclude>
</Sources>
<!-- Match the company name property in the assembly: -->
<CompanyNames>
<Exclude>
<CompanyName>.*microsoft.*</CompanyName>
</Exclude>
</CompanyNames>
<!-- Match the public key token of a signed assembly: -->
<PublicKeyTokens>
<!-- Exclude Visual Studio extensions: -->
<Exclude>
<PublicKeyToken>^B77A5C561934E089$</PublicKeyToken>
<PublicKeyToken>^B03F5F7F11D50A3A$</PublicKeyToken>
<PublicKeyToken>^31BF3856AD364E35$</PublicKeyToken>
<PublicKeyToken>^89845DCD8080CC91$</PublicKeyToken>
<PublicKeyToken>^71E9BCE111E9429C$</PublicKeyToken>
<PublicKeyToken>^8F50407C4E9E73B6$</PublicKeyToken>
<PublicKeyToken>^E361AF139669C375$</PublicKeyToken>
</Exclude>
</PublicKeyTokens>