NUnit Test with WatiN, runs OK from Dev10, but when NUnit started from "C:\Program Files (x86)\NUnit 2.5.5\bin\net-2.0\nunit.exe" - nunit

I have the following code in an Nunit test ...
string url = "";
url = #"http://localhost/ClientPortalDev/Account/LogOn";
ieStaticInstanceHelper = new IEStaticInstanceHelper();
ieStaticInstanceHelper.IE = new IE(url);
ieStaticInstanceHelper.IE.TextField(Find.ById("UserName")).TypeText("abc");
ieStaticInstanceHelper.IE.TextField(Find.ById("Password")).TypeText("defg");
ieStaticInstanceHelper.IE.Button(Find.ById("submit")).Click();
ieStaticInstanceHelper.IE.Close();
On right-clicking the project in Dev10(visual studio 10) and choosing [Test With][NUnit 2.5], this test code runs with no problems. I have TestDriven installed.
When opening the NUnit from C:\Program Files (x86)\NUnit 2.5.5\bin\net-2.0\nunit.exe" and then opening my test dll, the following text is reported in NUnit Errors and failures
... LoginAsWellKnownUserShouldSucceed:
System.Runtime.InteropServices.COMException : Error HRESULT E_FAIL has been returned from a call to a COM component.
As an Aside ... Right-Clicking the source cs file in Dev10 and choosing Run Test, ... works as well.
The above test is actually part of TechTalk.SpecFlow 1.3 step, I have NUnit 2.5.5.10112, installed, I have Watin 20.20 installed,
I have the following App.config for my test dll
the start angle brackets have been removed ... how do you get xml to show up in
configuration>
configSections>
sectionGroup name="NUnit">
section name="TestRunner" type="System.Configuration.NameValueSectionHandler"/>
/sectionGroup>
/configSections>
NUnit>
TestRunner>
add key="ApartmentState" value="STA" />
/TestRunner>
/NUnit>
appSettings>
add key="configCheck" value="12345" />
/appSettings>
/configuration>
Anyone hit this before ?
The NUnit test obviously runs in NUnit 2.5.5 of TestDriven but not when running NUnit 2.5.5 from outside of Dev10 and TestDriven ?

Run the test in NUnit as admin.
I was running Dev10 as admin so I could attatch and debug w3ww, which was why the TestDriven test was working.
As soon as I started running NUnit as admin the COM Server isue issue goess away.
What makes it more confusing is that running the hello-world exmaple against google from the Watin site works even though NUnit is not run as admin.
To use NUNit + Watin + against local web server, running NUnit as admin solves the com server exception issue.

Try using RequiresSTA attribute in your test code instead of the configuration file.

I am using MSTest, call Refresh to avoid cached data, this worked for me:
browser.Refresh();
browser.TextField(Find.ById("username")).TypeText("user");
browser.TextField(Find.ById("password")).TypeText("pass");
browser.Button(Find.ByName("SUBMIT")).Click();

Related

specflow plugin with nunit and TFS build

I am in the middle of trying to get specflow UI tests working with TFS build and I am very very close.
I wanted to be able to run the same test on a number of browsers, and so after some research I found this:
http://www.baseclass.ch/blog/Lists/Beitraege/Post.aspx?ID=4&mobile=0
And it works like a dream for local test runs. the next step was to get this to call a selenium grid during a TFS 2010 build.
After getting the nunit community build task into my build template, I am now stuck with an error I can't seem to resolve. The build reports the following error:
Error: The system cannot find the file specified. Stack Trace: at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo) at System.Diagnostics.Process.Start() at TfsBuildExtensions.Activities.CodeQuality.NUnit.RunProcess(String fullPath, String workingDirectory, String arguments) in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Activities\CodeQuality\NUnit\NUnit.cs:line 339 at TfsBuildExtensions.Activities.CodeQuality.NUnit.PublishMSTestResults(String resultTrxFile, String collectionUrl, String buildNumber, String teamProject, String platform, String flavor) in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Activities\CodeQuality\NUnit\NUnit.cs:line 394 at TfsBuildExtensions.Activities.CodeQuality.NUnit.PublishTestResultsToTFS(ActivityContext context, String folder) in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Activities\CodeQuality\NUnit\NUnit.cs:line 387 at TfsBuildExtensions.Activities.CodeQuality.NUnit.InternalExecute() in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Activities\CodeQuality\NUnit\NUnit.cs:line 299 at TfsBuildExtensions.Activities.BaseCodeActivity.Execute(CodeActivityContext context) in d:\Projects\CodePlex\teambuild2010contrib\CustomActivities\Legacy\VS2010\Source\Common\BaseCodeActivity.cs:line 67.
however, this seems to be masking a deeper error as to me it simply says that nunit can't load a custom assembly. So I had a look through the binaries folder on the build server for the nunit logs, and sure enough in my test run .xml file I find another error:
System.Configuration.ConfigurationErrorsException : The type 'OpenQA.Selenium.Remote.RemoteWebDriver, Baseclass.Contrib.SpecFlow.Selenium.NUnit.SpecFlowPlugin' could not be found. It may require assembly qualification, e.g. "MyType, MyAssembly".
at Autofac.Configuration.ConfigurationRegistrar.LoadType(String typeName, Assembly defaultAssembly)
at Autofac.Configuration.ConfigurationRegistrar.RegisterConfiguredComponents(ContainerBuilder builder, SectionHandler configurationSection)
at Autofac.Configuration.ConfigurationRegistrar.RegisterConfigurationSection(ContainerBuilder builder, SectionHandler configurationSection)
at Autofac.Configuration.Core.ConfigurationModule.Load(ContainerBuilder builder)
at Autofac.Module.Configure(IComponentRegistry componentRegistry)
at Autofac.ContainerBuilder.Build(IComponentRegistry componentRegistry, Boolean excludeDefaultModules)
at Autofac.ContainerBuilder.Build(ContainerBuildOptions options)
at RegistrationForm.Tests.Acceptance.Features.UserRegistrationFeature.FeatureSetup() in c:\Builds\1\Testing\RegistrationForm - Nightly - Main\Sources\Testing\RegistrationForm\Main\RegistrationForm.Tests.Acceptance\Features\UserRegistration.feature.cs:line 0
From this I started to look at the path to the custom assembly. This is specified in the app.config file for the project (which is transformed during the build. I also checked to ensure the file is being transformed, and in the binaries directory on the build server it is). the relevant section of the file is this.
<specFlow>
<stepAssemblies>
<stepAssembly assembly="SpecFlow.Assist.Dynamic" />
<stepAssembly assembly="Baseclass.Contrib.SpecFlow.Selenium.NUnit.Bindings" />
</stepAssemblies>
<unitTestProvider name="SeleniumNUnit" />
<plugins>
<add name="Baseclass.Contrib.SpecFlow.Selenium.NUnit" path="..\packages\Baseclass.Contrib.SpecFlow.Selenium.NUnit.1.2.0\tools" />
</plugins>
Which works locally. I figured I just need to change the path in the transformed file. At first I assumed the working directory was the binaries directory so in the transform file I have this:
<specFlow>
<plugins xdt:Transform="Replace">
<add name="Baseclass.Contrib.SpecFlow.Selenium.NUnit" path="." />
</plugins>
The dll is in the same dir (binaries) so I tried ".", "", ".\" - none of these worked. So after thinking a little more and reading the errors more carefully I thought I needed to look in the Sources folder on the build server. The Dll is in the packages folder (nuget package restore), and so the path should be the same as I use locally. this didn't work either. So what about a path relative to the ".feature" file throwing the error? this simply needed an extra "..\" - still no luck.
So i am at a bit of a loss, I feel I have tried all the paths i can think of, but my knowledge of specflow plugins and TFS build is letting me down. can anyone give me any pointers?
But the initial error is basically saying the TFSs nunit support cant find a file when it calls CreateProcess, and as you say it works for local test runs, so is it as simple as NUnit isn't installed on the machine that is running the test for you?
I got there in the end!
So somewhere along the lines of setting this up I endded up with something like this in my app.config:
<component
name="IE"
type="Baseclass.Contrib.SpecFlow.Selenium.NUnit.RemoteWebDriver, Baseclass.Contrib.SpecFlow.Selenium.NUnit.SpecFlowPlugin"
service="OpenQA.Selenium.Remote.RemoteWebDriver, WebDriver"
instance-scope="per-dependency">
<parameters>
<parameter name="browser" value="InternetExplorer" />
<parameter name="url" value="http://192.168.1.3:4444/wd/hub" />
</parameters>
</component>
The problem (as it says in the error) is that it could not find OpenQA.Selenium.Remote.RemoteWebDriver. I belive I saw this in one of the examples I found on the bassclass page. After changing it to OpenQA.Selenium.IWebDriver everything worked fine, and nUnit was able to run my specFlow tests on my TFS 2010 build server. I did hit another snag in that when the build tried to parse the results, it tried to use MSTest 11 rather than 10 (prob because I was using VS 2012). However, this TFS machine was just a proof of concept and was a windows 2008 32bit edition running on Virtual PC on windows 7. (I could therfor not run 64bit, and so could not install VS2012 on my build server). I resolved this by creating a new environment variable for MStest 11 and pointing it at MSTest 10. The entire process now works perfectly.

Gallio with NCover shows 0% code coverage in Sonar UI

I am using sonar-runner to run tests and code coverage over my C# code with the help of gallio plugin. The tests are running fine, but I am not able to see any code coverage on the sonar web UI.
My Sonar settings are as follows:
sonar-project.properties
mentioning only relevant bits
sonar.gallio.coverage.tool = NCover
sonar.NCover.installDirectory = C:/Program Files/NCover
sonar.donet.visualstudio.testProjectPattern = .Test
sonar.dotnet.buildConfigurations = "Release|x86"
Any idea what coule be missing??
sonar.projectKey=XXX:XXX
sonar.projectVersion=trunk
sonar.projectName=XXX
sources=.
sonar.language=cs
sonar.dotnet.visualstudio.solution.file=Project.sln
sonar.dotnet.excludeGeneratedCode=false
sonar.dotnet.4.0.sdk.directory=C:/WIndows/Microsoft.NET/Framework/v4.0.30319
sonar.dotnet.version=4.0
# Gallio
sonar.gallio.mode=
sonar.gallio.coverage.tool=NCover
sonar.gallio.runner=IsolatedAppDomain
sonar.NCover.installDirectory=c:/Program Files/NCover
sonar.gallio.installDirectory=C:/Program Files/Gallio
sonar.dotnet.test.assemblies=$(SolutionDir)/../**/bin/**/*.Tests.Unit.dll
# FXCop
sonar.fxcop.mode=
#StyleCop
sonar.stylecop.mode=
#NDeps
sonar.ndeps.mode=skip
sonar-runner.properties
You said
sonar.dotnet.buildConfigurations = "Release|x86"
If that's true, your build likely isn't generating .pdb files, which are needed to figure out the mapping between the binaries and your source files.
Does it work if you try it with a Debug build?
I was seeing this same behavior with NCover in Sonar. I found that Sonar was generating invalid arguments for Gallio's NCover runner.
Try piping the output from Sonar's runner into a text file so that you can examine the arguments more easily (on the command line, you can just type sonar-runner > output.txt to do this).
You will likely see a line like this in your output:
INFO .u.c.CommandExecutor - Executing command: C:\Program Files\Gallio\bin\Gallio.Echo.exe /r:Local /report-directory:E:\Reports\.sonar /report-name-format:gallio-report /report-type:Xml E:\Projects\UnitTests\bin\Release\UnitTests.dll /runner-property:NCoverCoverageFile=E:\Reports\.sonar\coverage-report.xml /runner-property:NCoverArguments=//ias MyFirstAssembly;MySecondtAssembly;MyThirdAssembly
If you attempt to execute this manually via Gallio on the command line, you will get an error:
Cannot find file 'MyFirstAssembly;MySecondtAssembly;MyThirdAssembly'
If you edit this list manually down to a single entry such as MyFirstAssembly*, everything will work as expected.
This seems to indicate that Sonar is generating invalid command line arguments for Gallio. As much as I love NCover, the easiest solution was to use OpenCover instead.

NUnit components for version 4.0.30319 of the CLR are not installed

I am trying to implement an automated build process. After the build, the unit tests on nunit-console.exe are run. The following error is displayed:
> c:\nunit_2.5.10\nunit-console.exe c:\builds\Output\bin\TDD.nunit /framework=4.0.30319 /nologo /trace=Off
ProcessModel: Default DomainUsage: Default
Execution Runtime: v4.0.30319
Unhandled Exception:
System.ArgumentException: NUnit components for version 4.0.30319 of the CLR are not installed
Parameter name: targetRuntime
at NUnit.Util.TestAgency.LaunchAgentProcess(RuntimeFramework targetRuntime, Boolean enableDebug)
at NUnit.Util.TestAgency.CreateRemoteAgent(RuntimeFramework framework, Int32 waitTime, Boolean enableDebug)
at NUnit.Util.TestAgency.GetAgent(RuntimeFramework framework, Int32 waitTime, Boolean enableDebug)
at NUnit.Util.ProcessRunner.Load(TestPackage package)
at NUnit.ConsoleRunner.ConsoleUi.Execute(ConsoleOptions options)
at NUnit.ConsoleRunner.Runner.Main(String[] args)
There is no nunit-agent.exe on the build machine. However, on my machine it is not even called, so I suppose it is not necessary.
Why is nunit-agent.exe required in some cases but not always required? What conditions should be satisfied so nunit-agent would not need to launch?
Edit: I have found one resource, which kind of describes how it works, but not quite well: http://www.nunit.org/index.php?p=nunit-agent&r=2.5.10. It says that it is launched when program needs to run under a different framework than the one being used by NUnit (which is the case, since NUnit is compiled for 2.0). However, on my machine the nunit-agent.exe does not run although conditions seem to be the same.
I ran into this same error and it was definitely solved by including nunit-agent.exe in the folder where nunit-console.exe was launched. The complete list of .exes and .dlls necessary to run a test successfully was:
nunit.core.dll
nunit.core.interfaces.dll
nunit.framework.dll
nunit.util.dll
nunit-agent.exe
nunit-console.exe
nunit-console-runner.dll
All files are packaged in the download available from nunit.org. As of this post, 2.6.3 is the current version. Documentation for the console runner can be found here. And the direct download for the zip file is here.
For a test assembly targeting .NET 4.5.1, the following statement will execute tests:
nunit-console.exe your-assembly.dll /framework=v4.5.1
Adding a "startup/supportedRuntime" configuration tag to nunit-console.exe.config solved it for me.
<startup useLegacyV2RuntimeActivationPolicy="true">
<supportedRuntime version="v4.0.30319" />
</startup>
Try using Fusion to see what assembly might be missing and where the .exe is looking.
http://msdn.microsoft.com/en-us/library/e74a18c4(v=vs.110).aspx

Teamcity not DLL's for some NUnit Test projects

I get this error when running my Moq tests through Teamcity 5
Test(s) failed.
System.IO.FileNotFoundException :
Could not load file or assembly 'Moq,
Version=3.1.416.3, Culture=neutral,
PublicKeyToken=69f491c39445e920' or
one of its dependencies. The system
cannot find the file specified. at
MyCode.Tests.SomeHandlerTests.Setup()
The tests run fine on my local; they just fail on the build server.
I made sure the assemblies are in the Bin (looking at them now over RDP just be double sure).
So the issue was to do with the Test DLL search path under the nunit settings
It was:
..\Tests\**\*Test*.dll
But is now:
..\Tests\*\bin\Debug\*Test*.dll
And things work nicely
UPDATE
http://confluence.jetbrains.com/display/TCD8/NUnit
You can use this pattern
**\*.dll
as long as you add this pattern in the "Do not run tests from" field
**\obj\**\*.dll
I had a similar issue, but found that I had different version's of Moq between my 2 Test projects.
The issue that I had was that the correct version was not available.
Just do
Update-Package Moq
From the Package Manager command line

NUnit-console 2.5.4 not capable of running multiple assemblies?

I am having problems running tests with the command line NUnit test runner.
I am using version 2.5.4 with .NET 4 on an x64 machine.
Using the following line results in a failure "Could not load file or assembly 'bar' or one of
its dependencies. The system cannot find the file specified."
nunit-console-x86 foo.dll bar.dll /framework=4.0.30319
If I reverse the dll file names it complains about not finding 'foo' instead...
It works if I run them separately like:
nunit-console-x86 foo.dll /framework=4.0.30319
Also the tests of the second file works if I run:
nunit-console-x86 bar.dll /framework=4.0.30319
Before upgrading our projects to 4.0 we used NUnit 2.5.2 and the same command line tool options and at that point the runner worked well with multiple assemblies. It seems like the ability to run tests on multiple files at the same time is broken...
Anyone that can see the same behavior or does it work indicating that my environment is somehow broken?
/Per
Assembly loading behaviour has changed between 2.5.4 and 2.5.3. It was causing problems for us, so we reverted to 2.5.3, since that still supports the 4.0 framework.
We use nunit-console.exe 2.5.3 with multiple assemblies in our msbuild script, which looks like this:
<Exec Command="%22$(NUnit_Install_Directory)bin\net-2.0\nunit-console.exe%22
/noshadow #(TestableAssemblies, ' ') /xml $(BuildFilesPath)\NUnit-Results.xml" />
On execution, it fills out like this (edited for readability):
"c:\Program Files\NUnit 2.5.3\bin\net-2.0\nunit-console.exe" /noshadow D:\BuildAgent\GojiSoft.Application.Test\bin\Release\GojiSoft.Application.Test.dll D:\BuildAgent\GojiSoft.Common.Test\bin\Release\GojiSoft.Common.Test.dll
/xml D:\BuildAgent\work\2f016459feee51ce\Build\NUnit-Results.xml
We don't use a 64bit machine for our unit tests, so YMMV.
I have a blog post about the addtional modification your should make to the nunit-console-x86.exe.config here: Getting .Net 4.0, Team City, MSBuild and Nunit to play nice.
I has the same problem. I got around it by creating an nunit project in the GUI that includes multiple dlls. Then I can run it with nunit-console.