This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
Solved: Look at the bottom of this question for the working configuration
I am attempting to set up NCover for usage in my FinalBuilder project, for a .NET 4.0 C# project, but my final coverage output file contains no coverage data.
I am using:
NCover 3.3.2
NUnit 2.5.4
FinalBuilder 6.3.0.2004
All tools are the latest official as of today.
I've finally managed to coax FB into running my unit tests under NCover for the .NET 4.0 project, so I get Tests run: 184, ..., which is correct.
However, the final Coverage.xml file output from NCover is almost empty, and looks like this:
<?xml version="1.0" encoding="utf-8"?>
<!-- saved from NCover 3.0 Export url='http://www.ncover.com/' -->
<coverage profilerVersion="3.3.2.6211" driverVersion="3.3.2" exportversion="3" viewdisplayname="" startTime="2010-04-22T08:55:33.7471316Z" measureTime="2010-04-22T08:55:35.3462915Z" projectName="" buildid="27c78ffa-c636-4002-a901-3211a0850b99" coveragenodeid="0" failed="false" satisfactorybranchthreshold="95" satisfactorycoveragethreshold="95" satisfactorycyclomaticcomplexitythreshold="20" satisfactoryfunctionthreshold="80" satisfactoryunvisitedsequencepoints="10" uiviewtype="TreeView" viewguid="C:\Dev\VS.NET\LVK.IoC\LVK.IoC.Tests\bin\Debug\Coverage.xml" viewfilterstyle="None" viewreportstyle="SequencePointCoveragePercentage" viewsortstyle="Name">
<rebasedpaths />
<filters />
<documents>
<doc id="0" excluded="false" url="None" cs="" csa="00000000-0000-0000-0000-000000000000" om="0" nid="0" />
</documents>
</coverage>
The output in FB log is:
...
***************** End Program Output *****************
Execution Time: 1,5992 s
Coverage Xml: C:\Dev\VS.NET\LVK.IoC\LVK.IoC.Tests\bin\Debug\Coverage.xml
NCover Success
My configuration of the FB step for NCover:
NCover what?: NUnit test coverage
Command: C:\Program Files (x86)\NUnit 2.5.4\bin\net-2.0\nunit-console.exe
Command arguments: LVK.IoC.Tests.dll /noshadow /framework:4.0.30319 /process=single /nothread
Note: I've tried with and without the /process and /nothread options
Working directory: %FBPROJECTDIR%\LVK.IoC.Tests\bin\Debug
List of assemblies to profile: %FBPROJECTDIR%\LVK.IoC.Tests\bin\Debug\LVK.IoC.dll
Note: I've tried just listing the name of the assembly, both with and without the extension.
The documentation for the FB step doesn't help, as it only lists minor sentences for each property, and fails to give examples or troubleshooting hints.
Since I want to pull the coverage results into NDepend to run build-time analysis, I want that file to contain the information I need.
I am also using TestDriven, and if I right-click the solution file and select "Test with NCover", NCover-explorer opens up with coverage data, and if I ask it to show me the folder with coverage files, in there is an .xml file with the same structure as the one above, just with all the data that should be there, so the tools I have is certainly capable of producing it.
Has anyone an idea of what I've configured wrong here?
Solved: The following configuration finally worked for me. It produces a code coverage basic statistics on the start results page for a build, with a link to a detailed report with everything.
Since I name my unit test projects the same as the project it tests, only with a .Tests suffix, I've excluded those from the code coverage. Also, I had to specifically include only my own assemblies (which all start with LVK.) otherwise I would get some assemblies loaded by the TeamCity runner, which slightly impacted my coverage statistics.
First, I installed latest of the tools (as of this edit):
NCover 3.4.8
TeamCity 5.1.2
NUnit 2.5.5
The configuration on the build-screen in TeamCity is then as follows:
Targets: Rebuild
Configuration: Debug
Platform: Any CPU
.NET Coverage Tool: NCover (3.x)
.NET Runtime: .NET Framework 4.0, Version: Native
Path to NCover 3: %system.ncover.v3.x64.path% (this is Windows 2008 Server R2 64-bit)
Additional NCover Arguments:
//ias LVK.*
//eas .*Tests
//eas LVK\.CodeQuality
//ea LVK\.CodeQuality\.IgnoreCodeCoverageAttribute
Additional NCover Reporting Arguments:
//or FullCoverageReport:Html:{teamcity.report.path}
NUnit Runner: NUnit 2.5.5
.NET Runtime Platform: auto (MSIL), Version: 4.0
Run tests from: **\bin\Debug*Tests.dll
As far as I know, NCover is not capable of covering tests for .NET 4.0. They announced a new version for .NET 4.0 but it is not there yet.
Find info on the subject here. Personally I have no experience with this task...
Update: They released NCover 3.4 with .NET 4.0 support. Perhaps You give it a try.
I've found your answer from this thread: NUnit isn't running Visual Studio 2010 code
To the nunit-console-x86.exe.config file, under <configuration> add:
<startup>
<requiredRuntime version="v4.0.30319" />
</startup>
Under <runtime> add
<loadFromRemoteSources enabled="true" />
Fixed it for me. I'm using TeamCity not FinalBuilder though.
I'm also using NCover 3.4.
Related
I have a small demo project, with unit tests.
using .net5, and 64 bit/any cpu.
I am having trouble with them running in my azure pipeline.
I have made a runsetting file, but cannot specify the 5.0 net framework
I've made a failing unit test, but it doenst get run.
when I 'debug' it I get this in my azure log..
2021-03-04T16:29:27.9652466Z /TestAdapterPath:"C:\DevOps\Agent 2_work\3\s"
2021-03-04T16:29:27.9652833Z /diag:"C:\DevOps\Agent 2_work_temp\c9f3dc90-7d06-11eb-9f3e-ab7c238ad233.txt"
2021-03-04T16:29:28.0475388Z Settings file provided does not conform to required format. An error occurred while loading the settings. Error: Invalid setting 'RunConfiguration'. Invalid value 'AnyCPU' specified for 'TargetPlatform'..
2021-03-04T16:29:28.2479067Z ##[debug]Exit code 1 received from tool 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\Extensions\TestPlatform\vstest.console.exe'
2021-03-04T16:29:28.2484150Z ##[debug]STDIO streams have closed for tool 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\Extensions\TestPlatform\vstest.console.exe'
2021-03-04T16:29:28.2516774Z ##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
my .runsettings are:
<?xml version="1.0" encoding="utf-8"?>
<!-- File name extension must be .runsettings -->
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<!-- Path relative to directory that contains .runsettings file-->
<ResultsDirectory>.\bin\TestResults</ResultsDirectory>
<!-- x86 or x64 -->
<!-- You can also change it from the Test menu;
choose "Processor Architecture for AnyCPU Projects" -->
<TargetPlatform>AnyCPU</TargetPlatform>
<!-- Framework35 | [Framework40] | Framework45 ... todo Framework50 er ikke suporteret-->
<TargetFrameworkVersion>Framework45</TargetFrameworkVersion>
<!-- true or false -->
<!-- Value that specifies the exit code when no tests are discovered -->
<TreatNoTestsAsError>false</TreatNoTestsAsError>
</RunConfiguration>
</RunSettings>
Does ms Azure not support .net50 for unit testing, with vs tests?
Thanks for you thoughts..
Kenneth
I have made a runsetting file, but cannot specify the 5.0 net
framework
TargetFrameworkVersion:
FrameworkCore10 for .NET Core sources, FrameworkUap10 for
UWP-based sources, Framework45 for .NET Framework 4.5 and
higher, Framework40 for .NET Framework 4.0, and Framework35 for
.NET Framework 3.5.
This setting specifies the version of the unit test framework used to
discover and execute the tests. It can be different from the version
of the .NET platform that you specify in the build properties of the
unit test project.
If you omit the TargetFrameworkVersion element from the .runsettings
file, the platform automatically determines the framework version
based on the built binaries.
For the error:
An error occurred while loading the settings. Error: Invalid setting
'RunConfiguration'. Invalid value 'AnyCPU' specified for
'TargetPlatform'
TargetPlatform : x86, x64
According to the elements that the RunConfiguration element can include, the TargetPlatform node does not provide the value of AnyCPU.
I want to run NUnit on TeamCity and generate report with dotCover. But for some reasons I cannot get the coverage report for all my project assemblies.
TeamCity config as below
Teamcity: 6.5.3
NUnit: 2.5.10
.NET Runtime: Platform: x86
.NET Runtime: Version: v4.0
dotCover: bundled with TC not customized
I have checked that all *.dll *.xml *.pdb files are there in the directory as expected as below (reference DLLs are not listed)
MY.PROJECT.A.dll
MY.PROJECT.A.pdb
MY.PROJECT.A.xml
MY.PROJECT.B.dll
MY.PROJECT.B.pdb
MY.PROJECT.B.xml
MY.PROJECT.C.dll
MY.PROJECT.C.pdb
MY.PROJECT.C.xml
MY.PROJECT.Test.dll
MY.PROJECT.Test.pdb
MY.PROJECT.Test.xml
MY.PROJECT.Test.dll is executed with NUnit and this assembly is excluded in the coverage report by using *Test* filter. But only MY.PROJECT.A is in the dotCover coverage report whereas MY.PROJECT.B and MY.PROJECT.C are not included.
I checked the log but not error is found.
Any thoughts is much appreciated.
Finally, I figured out what is going on, and hope this answer is useful for those who are still struggling with this or similar problem.
Basically, dotCover only includes those assemblies are actually used (more precisely, those assemblies loaded by CLR) by the tests in the code coverage report.
In my case, only MY.PROJECT.A is used by tests, MY.PROJECT.B and MY.PROJECT.C are not used by tests due to external dependencies. and even with using MY.PROJECT.B directives in the tests, it does not count as CLR's lazy loading.
One dummy workaround to show 0% coverage report for these two assemblies is that either use anything in the assembly in the tests, or force load these assemblies by calling System.Reflection.Assembly.Load("MY.PROJECT.B")
Related Question:
Visual Studio Code Coverage Not Showing All Assemblies
What constitutes full code coverage ...
I am in the middle of trying to get specflow UI tests working with TFS build and I am very very close.
I wanted to be able to run the same test on a number of browsers, and so after some research I found this:
http://www.baseclass.ch/blog/Lists/Beitraege/Post.aspx?ID=4&mobile=0
And it works like a dream for local test runs. the next step was to get this to call a selenium grid during a TFS 2010 build.
After getting the nunit community build task into my build template, I am now stuck with an error I can't seem to resolve. The build reports the following error:
Error: The system cannot find the file specified. Stack Trace: at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo) at System.Diagnostics.Process.Start() at TfsBuildExtensions.Activities.CodeQuality.NUnit.RunProcess(String fullPath, String workingDirectory, String arguments) in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Activities\CodeQuality\NUnit\NUnit.cs:line 339 at TfsBuildExtensions.Activities.CodeQuality.NUnit.PublishMSTestResults(String resultTrxFile, String collectionUrl, String buildNumber, String teamProject, String platform, String flavor) in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Activities\CodeQuality\NUnit\NUnit.cs:line 394 at TfsBuildExtensions.Activities.CodeQuality.NUnit.PublishTestResultsToTFS(ActivityContext context, String folder) in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Activities\CodeQuality\NUnit\NUnit.cs:line 387 at TfsBuildExtensions.Activities.CodeQuality.NUnit.InternalExecute() in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Activities\CodeQuality\NUnit\NUnit.cs:line 299 at TfsBuildExtensions.Activities.BaseCodeActivity.Execute(CodeActivityContext context) in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Common\BaseCodeActivity.cs:line 67.
however, this seems to be masking a deeper error as to me it simply says that nunit can't load a custom assembly. So I had a look through the binaries folder on the build server for the nunit logs, and sure enough in my test run .xml file I find another error:
System.Configuration.ConfigurationErrorsException : The type 'OpenQA.Selenium.Remote.RemoteWebDriver, Baseclass.Contrib.SpecFlow.Selenium.NUnit.SpecFlowPlugin' could not be found. It may require assembly qualification, e.g. "MyType, MyAssembly".
at Autofac.Configuration.ConfigurationRegistrar.LoadType(String typeName, Assembly defaultAssembly)
at Autofac.Configuration.ConfigurationRegistrar.RegisterConfiguredComponents(ContainerBuilder builder, SectionHandler configurationSection)
at Autofac.Configuration.ConfigurationRegistrar.RegisterConfigurationSection(ContainerBuilder builder, SectionHandler configurationSection)
at Autofac.Configuration.Core.ConfigurationModule.Load(ContainerBuilder builder)
at Autofac.Module.Configure(IComponentRegistry componentRegistry)
at Autofac.ContainerBuilder.Build(IComponentRegistry componentRegistry, Boolean excludeDefaultModules)
at Autofac.ContainerBuilder.Build(ContainerBuildOptions options)
at RegistrationForm.Tests.Acceptance.Features.UserRegistrationFeature.FeatureSetup() in c:\Builds\1\Testing\RegistrationForm - Nightly - Main\Sources\Testing\RegistrationForm\Main\RegistrationForm.Tests.Acceptance\Features\UserRegistration.feature.cs:line 0
From this I started to look at the path to the custom assembly. This is specified in the app.config file for the project (which is transformed during the build. I also checked to ensure the file is being transformed, and in the binaries directory on the build server it is). the relevant section of the file is this.
<specFlow>
<stepAssemblies>
<stepAssembly assembly="SpecFlow.Assist.Dynamic" />
<stepAssembly assembly="Baseclass.Contrib.SpecFlow.Selenium.NUnit.Bindings" />
</stepAssemblies>
<unitTestProvider name="SeleniumNUnit" />
<plugins>
<add name="Baseclass.Contrib.SpecFlow.Selenium.NUnit" path="..\packages\Baseclass.Contrib.SpecFlow.Selenium.NUnit.1.2.0\tools" />
</plugins>
Which works locally. I figured I just need to change the path in the transformed file. At first I assumed the working directory was the binaries directory so in the transform file I have this:
<specFlow>
<plugins xdt:Transform="Replace">
<add name="Baseclass.Contrib.SpecFlow.Selenium.NUnit" path="." />
</plugins>
The dll is in the same dir (binaries) so I tried ".", "", ".\" - none of these worked. So after thinking a little more and reading the errors more carefully I thought I needed to look in the Sources folder on the build server. The Dll is in the packages folder (nuget package restore), and so the path should be the same as I use locally. this didn't work either. So what about a path relative to the ".feature" file throwing the error? this simply needed an extra "..\" - still no luck.
So i am at a bit of a loss, I feel I have tried all the paths i can think of, but my knowledge of specflow plugins and TFS build is letting me down. can anyone give me any pointers?
But the initial error is basically saying the TFSs nunit support cant find a file when it calls CreateProcess, and as you say it works for local test runs, so is it as simple as NUnit isn't installed on the machine that is running the test for you?
I got there in the end!
So somewhere along the lines of setting this up I endded up with something like this in my app.config:
<component
name="IE"
type="Baseclass.Contrib.SpecFlow.Selenium.NUnit.RemoteWebDriver, Baseclass.Contrib.SpecFlow.Selenium.NUnit.SpecFlowPlugin"
service="OpenQA.Selenium.Remote.RemoteWebDriver, WebDriver"
instance-scope="per-dependency">
<parameters>
<parameter name="browser" value="InternetExplorer" />
<parameter name="url" value="http://192.168.1.3:4444/wd/hub" />
</parameters>
</component>
The problem (as it says in the error) is that it could not find OpenQA.Selenium.Remote.RemoteWebDriver. I belive I saw this in one of the examples I found on the bassclass page. After changing it to OpenQA.Selenium.IWebDriver everything worked fine, and nUnit was able to run my specFlow tests on my TFS 2010 build server. I did hit another snag in that when the build tried to parse the results, it tried to use MSTest 11 rather than 10 (prob because I was using VS 2012). However, this TFS machine was just a proof of concept and was a windows 2008 32bit edition running on Virtual PC on windows 7. (I could therfor not run 64bit, and so could not install VS2012 on my build server). I resolved this by creating a new environment variable for MStest 11 and pointing it at MSTest 10. The entire process now works perfectly.
I am trying to implement an automated build process. After the build, the unit tests on nunit-console.exe are run. The following error is displayed:
> c:\nunit_2.5.10\nunit-console.exe c:\builds\Output\bin\TDD.nunit /framework=4.0.30319 /nologo /trace=Off
ProcessModel: Default DomainUsage: Default
Execution Runtime: v4.0.30319
Unhandled Exception:
System.ArgumentException: NUnit components for version 4.0.30319 of the CLR are not installed
Parameter name: targetRuntime
at NUnit.Util.TestAgency.LaunchAgentProcess(RuntimeFramework targetRuntime, Boolean enableDebug)
at NUnit.Util.TestAgency.CreateRemoteAgent(RuntimeFramework framework, Int32 waitTime, Boolean enableDebug)
at NUnit.Util.TestAgency.GetAgent(RuntimeFramework framework, Int32 waitTime, Boolean enableDebug)
at NUnit.Util.ProcessRunner.Load(TestPackage package)
at NUnit.ConsoleRunner.ConsoleUi.Execute(ConsoleOptions options)
at NUnit.ConsoleRunner.Runner.Main(String[] args)
There is no nunit-agent.exe on the build machine. However, on my machine it is not even called, so I suppose it is not necessary.
Why is nunit-agent.exe required in some cases but not always required? What conditions should be satisfied so nunit-agent would not need to launch?
Edit: I have found one resource, which kind of describes how it works, but not quite well: http://www.nunit.org/index.php?p=nunit-agent&r=2.5.10. It says that it is launched when program needs to run under a different framework than the one being used by NUnit (which is the case, since NUnit is compiled for 2.0). However, on my machine the nunit-agent.exe does not run although conditions seem to be the same.
I ran into this same error and it was definitely solved by including nunit-agent.exe in the folder where nunit-console.exe was launched. The complete list of .exes and .dlls necessary to run a test successfully was:
nunit.core.dll
nunit.core.interfaces.dll
nunit.framework.dll
nunit.util.dll
nunit-agent.exe
nunit-console.exe
nunit-console-runner.dll
All files are packaged in the download available from nunit.org. As of this post, 2.6.3 is the current version. Documentation for the console runner can be found here. And the direct download for the zip file is here.
For a test assembly targeting .NET 4.5.1, the following statement will execute tests:
nunit-console.exe your-assembly.dll /framework=v4.5.1
Adding a "startup/supportedRuntime" configuration tag to nunit-console.exe.config solved it for me.
<startup useLegacyV2RuntimeActivationPolicy="true">
<supportedRuntime version="v4.0.30319" />
</startup>
Try using Fusion to see what assembly might be missing and where the .exe is looking.
http://msdn.microsoft.com/en-us/library/e74a18c4(v=vs.110).aspx
I am having problems running tests with the command line NUnit test runner.
I am using version 2.5.4 with .NET 4 on an x64 machine.
Using the following line results in a failure "Could not load file or assembly 'bar' or one of
its dependencies. The system cannot find the file specified."
nunit-console-x86 foo.dll bar.dll /framework=4.0.30319
If I reverse the dll file names it complains about not finding 'foo' instead...
It works if I run them separately like:
nunit-console-x86 foo.dll /framework=4.0.30319
Also the tests of the second file works if I run:
nunit-console-x86 bar.dll /framework=4.0.30319
Before upgrading our projects to 4.0 we used NUnit 2.5.2 and the same command line tool options and at that point the runner worked well with multiple assemblies. It seems like the ability to run tests on multiple files at the same time is broken...
Anyone that can see the same behavior or does it work indicating that my environment is somehow broken?
/Per
Assembly loading behaviour has changed between 2.5.4 and 2.5.3. It was causing problems for us, so we reverted to 2.5.3, since that still supports the 4.0 framework.
We use nunit-console.exe 2.5.3 with multiple assemblies in our msbuild script, which looks like this:
<Exec Command="%22$(NUnit_Install_Directory)bin\net-2.0\nunit-console.exe%22
/noshadow #(TestableAssemblies, ' ') /xml $(BuildFilesPath)\NUnit-Results.xml" />
On execution, it fills out like this (edited for readability):
"c:\Program Files\NUnit 2.5.3\bin\net-2.0\nunit-console.exe" /noshadow D:\BuildAgent\GojiSoft.Application.Test\bin\Release\GojiSoft.Application.Test.dll D:\BuildAgent\GojiSoft.Common.Test\bin\Release\GojiSoft.Common.Test.dll
/xml D:\BuildAgent\work\2f016459feee51ce\Build\NUnit-Results.xml
We don't use a 64bit machine for our unit tests, so YMMV.
I have a blog post about the addtional modification your should make to the nunit-console-x86.exe.config here: Getting .Net 4.0, Team City, MSBuild and Nunit to play nice.
I has the same problem. I got around it by creating an nunit project in the GUI that includes multiple dlls. Then I can run it with nunit-console.