I see unreliable behaviour in the dotnet command when executed from a powershell script.
Executing in a powershell script:
dotnet build "$slnPath"
ignores any compilation errors and continues executing the script. I have to check $lastexitcode to see, if there are any errors.
On the other hand, the command:
dotnet test "$slnPath"
immediately terminates execution of the powershell script, if there are any failed unit tests.
Is that normal behavior? Do I need to write different error handling depending on the arguments of the dotnet command?
The dotnet test command launches the test runner console application specified for a project. The test runner executes the tests defined for a unit test framework (for example, MSTest, NUnit, or xUnit) and reports the success or failure of each test. If all tests are successful, the test runner returns 0 as an exit code; otherwise if any test fails, it returns 1.
But dotnet build building requires the project.assets.json file, which lists the dependencies of your application. The file is created when dotnet restore is executed. Without the assets file in place, the tooling can't resolve reference assemblies, which results in errors.
You can read what each command do(and may be found some -key to control they) at this adress tool_description
Related
We are trying to reduce the console verbosity of our test runs in our Azure DevOps pipelines.
Due to the number of tests and complexity, we wrapped the VSTest.Console.exe run in MSBuild targets.
We configured our vstest.console.exe runs with the /logger:console;verbosity=minimal. When running the vstest command manually through command prompt this works fine, it only outputs the skipped tests and the end result summary, as intended.
When executed through MSBuild targets, even with the simplest target we confirmed, the output to the console is the full VSTest console output as if its in normal verbosity.
We are also setting the MSBuild verbosity to minimal, but this has no effect on the VSTest output.
Here is the target which we can reproduce it with
<Target Name="RunTestWorker" Outputs="$(VSTestConsoleExitCode)">
<Exec Command=""C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\..\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" "ProjectA.SpecflowTest.dll" /Platform:x64 /logger:console;verbosity=q /logger:trx;LogFileName=TestTrxFile.trx"
ContinueOnError="True"
WorkingDirectory="$(TestResultsDirectory)..">
</Exec>
</Target>
Executed with this MSBuild call:
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\amd64\MSBuild.exe" "TestBuild.targets" /verbosity:m /t:RunTestWorker
We tried additional MSBuild Exec options like ConsoleToMSBuild and EchoOff, but this didn't really change much.
Is this a known issue?
This example from Microsoft's docs wraps the arguments after /logger: in quotes:
vstest.console.exe Tests.dll /logger:"console;verbosity=minimal"
I tried this from the command line without the trx logger and it did reduce logging compared to normal.
So in your case:
/logger:"console;verbosity=quiet" /logger:"trx;LogFileName=TestTrxFile.trx"
I want to use bullseye code coverage in my dos script.
And I have written below code. The test.cov file is created but result is not generated on test.cov.
SET MY_LOCAL_COV_FILE=c:\test.cov
SET COVFILE=%MY_LOCAL_COV_FILE%
SET COVBUILDZONE=%BUILD_NUMBER%
covselect --file "%MY_LOCAL_COV_FILE%" --add c:
cov01 --on
MSBuild ".\my.sln" /t:clean /p:Configuration="Debug"
cov01 --off
I think you have two problems.
You are not building the code, you have only run the 'clean' target from MSBuild, try running 'rebuild' which will clean and then compile the code so the code coverage instrumentation is inserted.
You are not running the built code so Bullseye can't get any meaningful coverage information. Before the 'conv01 --off' try running your executable, or unit test, or whatever it is you have built.
I've got a build that completed successfully, but the number of completed tests is about 50% lower than I was expecting. On closer inspection, it seems that the Mono instance running NUnit died with SIGABRT. (!!) Yet TeamCity still considers this a successful test run, since none of the tests returned a failure condition.
How do I force TeamCity to consider this a failed build? (TeamCity 8.x)
The same problem occurs with TeamCity 9 if an NUnit [OneTimeSetUp] function fails. No tests run, NUnit returns with exit code 1, but TeamCity considers it a success.
If there's anything in the build log relating to your SIGABRT, you can add a Failure Condition, e.g.:
Fail build if its build log ...
contains text matching regexp ...
"SIGABRT" or in my case "Process exited with code [^0]"
I want to use PowerShell with Psake and TeamCity to configure my CI. I used to standard Msbuild runner but now I wrote my own script for building solution but I have problem when msbuild failed.
When I was using Msbuild runner and build failed then on Overview page new section “Build errors” appears and I have there detail from msbuild. But when I wrote my custom scripts I got only error “Process exited with code 1” and I don’t know how to “create” this build errors section. Do you know how to do this? I know that I can use service messages but I can't handle failed log from msbuild.
task compile {
try {
exec { msbuild $code_dir\SampleSolution.sln /t:Build /p:Configuration=Release }
} catch {
Write-Host $_ #only information that error occured, but without any msbui details
} }
You can specify log for MSBuild
Parse error from log or attach whole log as build artifact.
##teamcity[publishArtifacts '.\msbuild.log']
To fail build you can use the following message
Write-Output "##teamcity[message text='MS Build failed' status='ERROR']"
You can get all messages that can be used here
I am using sonar-runner to run tests and code coverage over my C# code with the help of gallio plugin. The tests are running fine, but I am not able to see any code coverage on the sonar web UI.
My Sonar settings are as follows:
sonar-project.properties
mentioning only relevant bits
sonar.gallio.coverage.tool = NCover
sonar.NCover.installDirectory = C:/Program Files/NCover
sonar.donet.visualstudio.testProjectPattern = .Test
sonar.dotnet.buildConfigurations = "Release|x86"
Any idea what coule be missing??
sonar.projectKey=XXX:XXX
sonar.projectVersion=trunk
sonar.projectName=XXX
sources=.
sonar.language=cs
sonar.dotnet.visualstudio.solution.file=Project.sln
sonar.dotnet.excludeGeneratedCode=false
sonar.dotnet.4.0.sdk.directory=C:/WIndows/Microsoft.NET/Framework/v4.0.30319
sonar.dotnet.version=4.0
# Gallio
sonar.gallio.mode=
sonar.gallio.coverage.tool=NCover
sonar.gallio.runner=IsolatedAppDomain
sonar.NCover.installDirectory=c:/Program Files/NCover
sonar.gallio.installDirectory=C:/Program Files/Gallio
sonar.dotnet.test.assemblies=$(SolutionDir)/../**/bin/**/*.Tests.Unit.dll
# FXCop
sonar.fxcop.mode=
#StyleCop
sonar.stylecop.mode=
#NDeps
sonar.ndeps.mode=skip
sonar-runner.properties
You said
sonar.dotnet.buildConfigurations = "Release|x86"
If that's true, your build likely isn't generating .pdb files, which are needed to figure out the mapping between the binaries and your source files.
Does it work if you try it with a Debug build?
I was seeing this same behavior with NCover in Sonar. I found that Sonar was generating invalid arguments for Gallio's NCover runner.
Try piping the output from Sonar's runner into a text file so that you can examine the arguments more easily (on the command line, you can just type sonar-runner > output.txt to do this).
You will likely see a line like this in your output:
INFO .u.c.CommandExecutor - Executing command: C:\Program Files\Gallio\bin\Gallio.Echo.exe /r:Local /report-directory:E:\Reports\.sonar /report-name-format:gallio-report /report-type:Xml E:\Projects\UnitTests\bin\Release\UnitTests.dll /runner-property:NCoverCoverageFile=E:\Reports\.sonar\coverage-report.xml /runner-property:NCoverArguments=//ias MyFirstAssembly;MySecondtAssembly;MyThirdAssembly
If you attempt to execute this manually via Gallio on the command line, you will get an error:
Cannot find file 'MyFirstAssembly;MySecondtAssembly;MyThirdAssembly'
If you edit this list manually down to a single entry such as MyFirstAssembly*, everything will work as expected.
This seems to indicate that Sonar is generating invalid command line arguments for Gallio. As much as I love NCover, the easiest solution was to use OpenCover instead.