I am using sonar-runner to run tests and code coverage over my C# code with the help of gallio plugin. The tests are running fine, but I am not able to see any code coverage on the sonar web UI.
My Sonar settings are as follows:
sonar-project.properties
mentioning only relevant bits
sonar.gallio.coverage.tool = NCover
sonar.NCover.installDirectory = C:/Program Files/NCover
sonar.donet.visualstudio.testProjectPattern = .Test
sonar.dotnet.buildConfigurations = "Release|x86"
Any idea what coule be missing??
sonar.projectKey=XXX:XXX
sonar.projectVersion=trunk
sonar.projectName=XXX
sources=.
sonar.language=cs
sonar.dotnet.visualstudio.solution.file=Project.sln
sonar.dotnet.excludeGeneratedCode=false
sonar.dotnet.4.0.sdk.directory=C:/WIndows/Microsoft.NET/Framework/v4.0.30319
sonar.dotnet.version=4.0
# Gallio
sonar.gallio.mode=
sonar.gallio.coverage.tool=NCover
sonar.gallio.runner=IsolatedAppDomain
sonar.NCover.installDirectory=c:/Program Files/NCover
sonar.gallio.installDirectory=C:/Program Files/Gallio
sonar.dotnet.test.assemblies=$(SolutionDir)/../**/bin/**/*.Tests.Unit.dll
# FXCop
sonar.fxcop.mode=
#StyleCop
sonar.stylecop.mode=
#NDeps
sonar.ndeps.mode=skip
sonar-runner.properties
You said
sonar.dotnet.buildConfigurations = "Release|x86"
If that's true, your build likely isn't generating .pdb files, which are needed to figure out the mapping between the binaries and your source files.
Does it work if you try it with a Debug build?
I was seeing this same behavior with NCover in Sonar. I found that Sonar was generating invalid arguments for Gallio's NCover runner.
Try piping the output from Sonar's runner into a text file so that you can examine the arguments more easily (on the command line, you can just type sonar-runner > output.txt to do this).
You will likely see a line like this in your output:
INFO .u.c.CommandExecutor - Executing command: C:\Program Files\Gallio\bin\Gallio.Echo.exe /r:Local /report-directory:E:\Reports\.sonar /report-name-format:gallio-report /report-type:Xml E:\Projects\UnitTests\bin\Release\UnitTests.dll /runner-property:NCoverCoverageFile=E:\Reports\.sonar\coverage-report.xml /runner-property:NCoverArguments=//ias MyFirstAssembly;MySecondtAssembly;MyThirdAssembly
If you attempt to execute this manually via Gallio on the command line, you will get an error:
Cannot find file 'MyFirstAssembly;MySecondtAssembly;MyThirdAssembly'
If you edit this list manually down to a single entry such as MyFirstAssembly*, everything will work as expected.
This seems to indicate that Sonar is generating invalid command line arguments for Gallio. As much as I love NCover, the easiest solution was to use OpenCover instead.
Related
I've downloaded the binaries: or-tools_VisualStudio2022-64bit_v9.3.10497
I'm using vs2022 on win10. My shell has cygwin in the path if it's related.
I ran
%comspec% /k "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"
cl.exe is in the path, and which.exe finds it.
I ran make test_cc, but it complained
the cl command was not found in your PATH
exit 127
make: *** [Makefile:271: test_cc] Error 127
The var CXX_BIN was empty even though which cl returned the correct path. I set it manually to cl.
Then, there was a complaint about echo and a newline, which I commented out. Then, it couldn't find md, so I created manually md objs.
A few of the examples were built, but then it stopped with another error. For now, I just got what I want:
make run SOURCE=examples/cpp/solve.cc
but probably there was an easier way to get it?
I tried to build it from the source using cmake. Doesn't work off-the-shelf as well:
Build abseil-cpp: OFF
...
CMake Error at C:/prj-external-libs/vcpkg/scripts/buildsystems/vcpkg.cmake:824 (_find_package):
By not providing "Findabsl.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "absl", but
CMake did not find one.
Could not find a package configuration file provided by "absl" with any of
the following names:
abslConfig.cmake
absl-config.cmake
Add the installation prefix of "absl" to CMAKE_PREFIX_PATH or set
"absl_DIR" to a directory containing one of the above files. If "absl"
provides a separate development package or SDK, be sure it has been
installed.
Call Stack (most recent call first):
cmake/deps.cmake:33 (find_package)
CMakeLists.txt:304 (include)
If finds gurobi95.dll, but it can't find the function GRBtunemodeladv.
On failure, solve.exe crashes with (unknown) names in the stack trace. Need to add debug symbols and graceful error handling.
cmake looks more promising, and I was missing dependencies. Should give it a flag -DBUILD_DEPS:BOOL=ON.
OR-Tools depends on few external dependencies so CMake build will try to find them using the idiomatic find_package() => your distro/env(vcpkg ?) must provide them, just regular CMake stuff here.
ref: https://cmake.org/cmake/help/latest/command/find_package.html
note: we provide few findFoo.cmake here https://github.com/google/or-tools/tree/main/cmake
We also provide a meta option to build statically all our dependencies, simply pass -DBUILD_DEPS=ON cmake option at configure time.
You can also build only some of them, please take a look at
https://github.com/google/or-tools/tree/main/cmake#dependencies
Concerning Gurobi and GRBtunemodeladv symbol, this one has been removed by last version of Gurobi so we fix it in v9.4/main/stable branch...
see: https://github.com/google/or-tools/commit/d6e0feb8ae96368523deb99fe4318d32e80e8145
I have installed Robocorp Code as well as Robot Framework Language Server and have configured them. However, I am still having errors when trying to run the tests via the code lens options.
Repo - A webapi repo with a specific folder containing all tests. Lets call it regression.
RF - 4.1.3
Python - 3.8
This is what happens when I click on Run on the code lens for any of the tests -
`PS C:\git\xxxx\regression> C:; cd 'C:\git\xxxx\regression'; &
'C:\Users\xxxx\AppData\Local\Temp\rf-ls-run\run_env_00_smh5defr.bat'
'-u'
'c:\Users\xxxx.vscode\extensions\robocorp.robotframework-lsp-0.47.2\src\robotframework_debug_adapter\run_robot__main__.py'
'--port' '54331' '--no-debug' '--argumentfile'
'C:\git\xxxx\regression\args-local.txt' '--pythonpath'
'c:\git\xxxx\regression\common\lib' '--variable'
'EXECDIR:C:/git/xxxx/regression'
'--prerunmodifier=robotframework_debug_adapter.prerun_modifiers.FilteringTestsSuiteVisitor'
'c:\git\xxxx\regression\api\api_Test.robot'
[ ERROR ] Parsing'--pythonpath' failed: File or directory to execute does not exist.
However, the test starts if I remove the argumentfile parameter but it, of course, fails because its missing arguments from the file.
Do note that the folder specified in pythopath exists and has some python libraries needed for the tests.
I want to use bullseye code coverage in my dos script.
And I have written below code. The test.cov file is created but result is not generated on test.cov.
SET MY_LOCAL_COV_FILE=c:\test.cov
SET COVFILE=%MY_LOCAL_COV_FILE%
SET COVBUILDZONE=%BUILD_NUMBER%
covselect --file "%MY_LOCAL_COV_FILE%" --add c:
cov01 --on
MSBuild ".\my.sln" /t:clean /p:Configuration="Debug"
cov01 --off
I think you have two problems.
You are not building the code, you have only run the 'clean' target from MSBuild, try running 'rebuild' which will clean and then compile the code so the code coverage instrumentation is inserted.
You are not running the built code so Bullseye can't get any meaningful coverage information. Before the 'conv01 --off' try running your executable, or unit test, or whatever it is you have built.
I want to run NUnit on TeamCity and generate report with dotCover. But for some reasons I cannot get the coverage report for all my project assemblies.
TeamCity config as below
Teamcity: 6.5.3
NUnit: 2.5.10
.NET Runtime: Platform: x86
.NET Runtime: Version: v4.0
dotCover: bundled with TC not customized
I have checked that all *.dll *.xml *.pdb files are there in the directory as expected as below (reference DLLs are not listed)
MY.PROJECT.A.dll
MY.PROJECT.A.pdb
MY.PROJECT.A.xml
MY.PROJECT.B.dll
MY.PROJECT.B.pdb
MY.PROJECT.B.xml
MY.PROJECT.C.dll
MY.PROJECT.C.pdb
MY.PROJECT.C.xml
MY.PROJECT.Test.dll
MY.PROJECT.Test.pdb
MY.PROJECT.Test.xml
MY.PROJECT.Test.dll is executed with NUnit and this assembly is excluded in the coverage report by using *Test* filter. But only MY.PROJECT.A is in the dotCover coverage report whereas MY.PROJECT.B and MY.PROJECT.C are not included.
I checked the log but not error is found.
Any thoughts is much appreciated.
Finally, I figured out what is going on, and hope this answer is useful for those who are still struggling with this or similar problem.
Basically, dotCover only includes those assemblies are actually used (more precisely, those assemblies loaded by CLR) by the tests in the code coverage report.
In my case, only MY.PROJECT.A is used by tests, MY.PROJECT.B and MY.PROJECT.C are not used by tests due to external dependencies. and even with using MY.PROJECT.B directives in the tests, it does not count as CLR's lazy loading.
One dummy workaround to show 0% coverage report for these two assemblies is that either use anything in the assembly in the tests, or force load these assemblies by calling System.Reflection.Assembly.Load("MY.PROJECT.B")
Related Question:
Visual Studio Code Coverage Not Showing All Assemblies
What constitutes full code coverage ...
I am having problems running tests with the command line NUnit test runner.
I am using version 2.5.4 with .NET 4 on an x64 machine.
Using the following line results in a failure "Could not load file or assembly 'bar' or one of
its dependencies. The system cannot find the file specified."
nunit-console-x86 foo.dll bar.dll /framework=4.0.30319
If I reverse the dll file names it complains about not finding 'foo' instead...
It works if I run them separately like:
nunit-console-x86 foo.dll /framework=4.0.30319
Also the tests of the second file works if I run:
nunit-console-x86 bar.dll /framework=4.0.30319
Before upgrading our projects to 4.0 we used NUnit 2.5.2 and the same command line tool options and at that point the runner worked well with multiple assemblies. It seems like the ability to run tests on multiple files at the same time is broken...
Anyone that can see the same behavior or does it work indicating that my environment is somehow broken?
/Per
Assembly loading behaviour has changed between 2.5.4 and 2.5.3. It was causing problems for us, so we reverted to 2.5.3, since that still supports the 4.0 framework.
We use nunit-console.exe 2.5.3 with multiple assemblies in our msbuild script, which looks like this:
<Exec Command="%22$(NUnit_Install_Directory)bin\net-2.0\nunit-console.exe%22
/noshadow #(TestableAssemblies, ' ') /xml $(BuildFilesPath)\NUnit-Results.xml" />
On execution, it fills out like this (edited for readability):
"c:\Program Files\NUnit 2.5.3\bin\net-2.0\nunit-console.exe" /noshadow D:\BuildAgent\GojiSoft.Application.Test\bin\Release\GojiSoft.Application.Test.dll D:\BuildAgent\GojiSoft.Common.Test\bin\Release\GojiSoft.Common.Test.dll
/xml D:\BuildAgent\work\2f016459feee51ce\Build\NUnit-Results.xml
We don't use a 64bit machine for our unit tests, so YMMV.
I have a blog post about the addtional modification your should make to the nunit-console-x86.exe.config here: Getting .Net 4.0, Team City, MSBuild and Nunit to play nice.
I has the same problem. I got around it by creating an nunit project in the GUI that includes multiple dlls. Then I can run it with nunit-console.