Taurus NUnit runner not finding tests - nunit

I have a simple NUnit test that makes a simple WebAPI call:
[TestFixture]
public class PerformanceTests
{
private const string web_api = <myapiurl>;
[Test]
public async Task PerformanceTest()
{
var response = await client.GetAsync(web_api);
Assert.AreEqual(HttpStatusCode.OK, response.StatusCode);
}
}
The test runs fine in Visual Studio when run using the normal test runner and also via ReSharper; it is highlighted as a NUnit test.
I have Taurus installed and have created a simple yml file to run my test:
execution:
- executor: nunit
iterations: 500
scenario:
script: C:\Users\...\tests.dll # assembly with tests
When I run the yml file in Taurus:
bzt my-nunit-tests.yml
the tests do not run and I get the following output:
Target: C:\Users\...\tests.dll
15:36:51 ERROR: NUnitExecutor STDERR:
Unhandled Exception: System.ArgumentException: Nothing to run, no tests were loaded
at NUnitRunner.NUnitRunner.Main(String[] args)
It would seem that the custom Taurus NUnit test runner is not picking up the tests. I can run the tests with the standard dotnet test command (the project does have the Microsoft.NET.Test.Sdk as a dependency).
As my test project is .NET Core I am publishing the project to ensure all dependencies are included. This ensures all the files specified here are in the same directory as the test assembly.
Update: I have created the exact same test project but in .NET Framework and Taurus finds the tests. To me this suggests the custom Taurus NUnit Test Runner doesn't work with .NET Core projects but I can't confirm this.

As of June 2018 Taurus' Custom NUnit runner does not support .NET Core. It would seem that this is because NUnit does not allow .NET Core tests to be run via the .NET Framework engine.

Related

SpecFlow 2.1.0 nUnit report generation with nUnit 3.6.0 not working

The SpecFlow report generation using nUnit is not working as expected.
The expected report should display the results of the execution as well, whereas the execution result is not displayed at all. Instead, only the tests are listed.
I am using the below format to generate the report.
specflow.exe nunitexecutionreport "C:\Users\Paresh\Documents\Visual Studio 2015\Projects\SpecFlowDemoNUnit\SpecFlowDemoNUnit\SpecFlowDemoNUnit.csproj" /xmlTestResult:"C:\Tests.xml" /out:"C:\Tests.html"
The file Tests.xml is generated after executing the tests using 'nunit3-console.exe'.
The versions used:
SpecFlow.2.1.0
NUnit.3.6.0
I figured out that the SpecFlow report generation with nUnit is probably broken with SpecFlow.2.1.0 and NUnit.3.6.0.
But after using a parameter 'format=nunit2' while executing the tests via nUnit console, the report generation worked. So finally, the commands used will look like these:
nUnit test execution:
nunit3-console.exe --labels=All --out=TestResult.txt "-- result=TestResult.xml;format=nunit2" bin\Debug\SpecFlowDemoNUnit.dll
SpecFlow Report Generation:
specflow.exe nunitexecutionreport SpecFlowDemoNUnit.csproj /out:MyResult.html
Here is the reference link:
https://github.com/techtalk/SpecFlow/wiki/Reporting

Nunit 3 msbuild task fails in TeamCity 9.1.5

I'm trying to upgrade to Nunit 3.0.1 and I'm using TeamCity for the ci build.
We used to have this msbuild task to run the nunit tests with versions 2.x but now it fails with the following error with NUnit 3.
<Target Name="Test">
<NUnit Assemblies="#(TestAssembly)" NUnitVersion="NUnit-3.0.1"/>
</Target>
[NUnit] Failed to find plugin 'Test/NUnit-3.0.1'
at JetBrains.TeamCity.Utils.PluginManager.LoadExtensions(String prefix, String plugin) in c:\BuildAgent\work\ad31cec0a1b0f083\src\Utils\src\PluginManager.cs:line 50
at JetBrains.TeamCity.NUnitCommon.NUnitFactory.NUnitRunnerFactory.ContainsNUnitPlugin(ITestRunArguments myArguments) in c:\BuildAgent\work\ad31cec0a1b0f083\src\NUnitCommon\src\NUnitFactory\NUnitRunnerFactory.cs:line 34
at JetBrains.TeamCity.NUnitCommon.NUnitFactory.NUnitRunnerFactory.CreateRunner(ITestRunArguments myArguments) in c:\BuildAgent\work\ad31cec0a1b0f083\src\NUnitCommon\src\NUnitFactory\NUnitRunnerFactory.cs:line 43
at JetBrains.BuildServer.NAntLoggers.RunnerFactory.FindTestRunner(ITestRunArguments arguments) in c:\BuildAgent\work\ad31cec0a1b0f083\src\NUnitBootstrap\src\RunnerFactory.cs:line 46
at JetBrains.BuildServer.NAntLoggers.RunnerFactory.CreateTestRunner(ITestRunArguments arguments) in c:\BuildAgent\work\ad31cec0a1b0f083\src\NUnitBootstrap\src\RunnerFactory.cs:line 31
at JetBrains.BuildServer.NAntLoggers.NUnitLauncher2.Run2(String[] args) in c:\BuildAgent\work\ad31cec0a1b0f083\src\NUnitBootstrap\src\NUnitLauncher2.cs:line 100
at JetBrains.BuildServer.NAntLoggers.NUnitLauncher2.Run(String[] args) in c:\BuildAgent\work\ad31cec0a1b0f083\src\NUnitBootstrap\src\NUnitLauncher2.cs:line 56
[11:14:09][NUnit] D:\BuildAgent\work\3e91aa62371f2902\teamcity\tc_unittest.xml(22, 3): D:\BuildAgent\plugins\dotnetPlugin\bin\JetBrains.BuildServer.NUnitLauncher.exe "##" D:\BuildAgent\temp\buildTmp\tmp24F.tmp exited with code -42.
I tried also with version 3.0.0, 3.0 and 3 but the same error is returned.
From TeamCity documentation Nunit 3.0 should be supported. See:
https://confluence.jetbrains.com/display/TCD9/NUnit+for+MSBuild
Any idea of what I can be missing?
As stated by JetBrains support, the builtin msbuild nunit task is not and won't be supported from 3.0 and forward.
The only alternative is to install the nunit console on the agent (exactly what I wanted to avoid) and then run the test through either the teamcity nunit runner or a msbuild exec task.
For reference:
https://youtrack.jetbrains.com/issue/TW-43784
https://confluence.jetbrains.com/display/TCD9/Getting+Started+with+NUnit#GettingStartedwithNUnit-Case2.MSBuild

TeamCity SpecFlow NUnit Watin and ApartmentState STA

Im trying to get our specflow watin tests to run on our new teamcity server. The problem is that I cant get the built in nunit runner to run tests in appartmentstate STA.
We used this configuration earlier, which works with other testrunners:
<NUnit>
<TestRunner>
<add key="ApartmentState" value="STA" />
</TestRunner>
</NUnit>
But the TeamCity NUnit test runner doesnt seem to pick up on this config. We have other config sanity tests in place, so we know the testrunner reads the configuration for our test project atleast.
There's the alternative to use the RequiresSTA attribute like so:
[Test, RequiresSTA]
public void ShouldRunThreadApartmentStateSTAWith()
{
Assert.AreEqual(ApartmentState.STA, Thread.CurrentThread.GetApartmentState());
}
But since the NUnit tests are generated by SpecFlow, I don't have control over how these attributes are set.
How can I get this setup to work?
Found the answer.
Setting the RequireSTA attribute in AssemblyInfo.cs makes the configuration obsolete.
[assembly:RequiresSTA]
The NUnit tests generated by SpecFlow is a partial class definition, so simply create another partial class definition in another file that has the attributes on it.

Teamcity not DLL's for some NUnit Test projects

I get this error when running my Moq tests through Teamcity 5
Test(s) failed.
System.IO.FileNotFoundException :
Could not load file or assembly 'Moq,
Version=3.1.416.3, Culture=neutral,
PublicKeyToken=69f491c39445e920' or
one of its dependencies. The system
cannot find the file specified. at
MyCode.Tests.SomeHandlerTests.Setup()
The tests run fine on my local; they just fail on the build server.
I made sure the assemblies are in the Bin (looking at them now over RDP just be double sure).
So the issue was to do with the Test DLL search path under the nunit settings
It was:
..\Tests\**\*Test*.dll
But is now:
..\Tests\*\bin\Debug\*Test*.dll
And things work nicely
UPDATE
http://confluence.jetbrains.com/display/TCD8/NUnit
You can use this pattern
**\*.dll
as long as you add this pattern in the "Do not run tests from" field
**\obj\**\*.dll
I had a similar issue, but found that I had different version's of Moq between my 2 Test projects.
The issue that I had was that the correct version was not available.
Just do
Update-Package Moq
From the Package Manager command line

NUnit Test with WatiN, runs OK from Dev10, but when NUnit started from "C:\Program Files (x86)\NUnit 2.5.5\bin\net-2.0\nunit.exe"

I have the following code in an Nunit test ...
string url = "";
url = #"http://localhost/ClientPortalDev/Account/LogOn";
ieStaticInstanceHelper = new IEStaticInstanceHelper();
ieStaticInstanceHelper.IE = new IE(url);
ieStaticInstanceHelper.IE.TextField(Find.ById("UserName")).TypeText("abc");
ieStaticInstanceHelper.IE.TextField(Find.ById("Password")).TypeText("defg");
ieStaticInstanceHelper.IE.Button(Find.ById("submit")).Click();
ieStaticInstanceHelper.IE.Close();
On right-clicking the project in Dev10(visual studio 10) and choosing [Test With][NUnit 2.5], this test code runs with no problems. I have TestDriven installed.
When opening the NUnit from C:\Program Files (x86)\NUnit 2.5.5\bin\net-2.0\nunit.exe" and then opening my test dll, the following text is reported in NUnit Errors and failures
... LoginAsWellKnownUserShouldSucceed:
System.Runtime.InteropServices.COMException : Error HRESULT E_FAIL has been returned from a call to a COM component.
As an Aside ... Right-Clicking the source cs file in Dev10 and choosing Run Test, ... works as well.
The above test is actually part of TechTalk.SpecFlow 1.3 step, I have NUnit 2.5.5.10112, installed, I have Watin 20.20 installed,
I have the following App.config for my test dll
the start angle brackets have been removed ... how do you get xml to show up in
configuration>
configSections>
sectionGroup name="NUnit">
section name="TestRunner" type="System.Configuration.NameValueSectionHandler"/>
/sectionGroup>
/configSections>
NUnit>
TestRunner>
add key="ApartmentState" value="STA" />
/TestRunner>
/NUnit>
appSettings>
add key="configCheck" value="12345" />
/appSettings>
/configuration>
Anyone hit this before ?
The NUnit test obviously runs in NUnit 2.5.5 of TestDriven but not when running NUnit 2.5.5 from outside of Dev10 and TestDriven ?
Run the test in NUnit as admin.
I was running Dev10 as admin so I could attatch and debug w3ww, which was why the TestDriven test was working.
As soon as I started running NUnit as admin the COM Server isue issue goess away.
What makes it more confusing is that running the hello-world exmaple against google from the Watin site works even though NUnit is not run as admin.
To use NUNit + Watin + against local web server, running NUnit as admin solves the com server exception issue.
Try using RequiresSTA attribute in your test code instead of the configuration file.
I am using MSTest, call Refresh to avoid cached data, this worked for me:
browser.Refresh();
browser.TextField(Find.ById("username")).TypeText("user");
browser.TextField(Find.ById("password")).TypeText("pass");
browser.Button(Find.ByName("SUBMIT")).Click();