My partner and I have a suite of tests running nightly on a build server for our project. We use CruiseControl.NET to run the server, and the tests are written using WatiN and NUnit. We have CruiseControl.NET running as a service with access to interact with the desktop on a local system account. Every few times that we run a build, certain tests will fail with error messages such as the following:
Test: cfarmweb.tests.Views.GeneralRegressionTest.DuplicateUsernameTest
Type: Failure
Message: SetUp : System.Runtime.InteropServices.COMException : Creating an instance of the COM component with CLSID {0002DF01-0000-0000-C000-000000000046} from the IClassFactory failed due to the following error: 800704a6. TearDown : System.NullReferenceException : Object reference not set to an instance of an object.
at WatiN.Core.IE.CreateNewIEAndGoToUri(Uri uri, IDialogHandler logonDialogHandler, Boolean createInNewProcess)
at WatiN.Core.IE..ctor(String url)
at cfarmweb.tests.Navigator.SiteNavigator..ctor(String browserName, Boolean visible) in c:\ccworkdir\CFarm\builddir\cfarmweb.tests\Navigator\SiteNavigator.cs:line 35
at cfarmweb.tests.Views.GeneralRegressionTest.MakeNavigator() in c:\ccworkdir\CFarm\builddir\cfarmweb.tests\Views\GeneralRegressionTest.cs:line 34
--TearDown
at WatiN.Core.Browser.OnGetNativeDocument()
at WatiN.Core.DomContainer.get_NativeDocument()
at WatiN.Core.Document.ContainsText(String text)
at cfarmweb.tests.Navigator.SiteNavigator.HasText(String target) in c:\ccworkdir\CFarm\builddir\cfarmweb.tests\Navigator\SiteNavigator.cs:line 213
at cfarmweb.tests.Navigator.SiteNavigator.SignOut() in c:\ccworkdir\CFarm\builddir\cfarmweb.tests\Navigator\SiteNavigator.cs:line 110
at cfarmweb.tests.Views.GeneralRegressionTest.DisposeNavigator() in c:\ccworkdir\CFarm\builddir\cfarmweb.tests\Views\GeneralRegressionTest.cs:line 123
The success of the builds does not seem to be dependent on changes to the code itself, as we have had builds break or be fixed after changes to parts of the program that are unrelated to the tests.
We are both new to the field of software testing (and development in general), but nothing we've found online about this error seems to pertain to our situation. We've seen everything from a system reboot pending to compatibility issues with Internet Explorer 8 to JavaScript errors, but nothing we've tried has fixed the issues. One of the most difficult parts is that it's not consistently reproducible. How can we fix this problem?
Ben,
I had the exact same issue, surprisingly enough...I think I have the solution. It appears to be a threading issue. The [RequiresSTA] tag at the top of a test is meant to create each test to be single-threaded by implicitly placing the tag [STAThread] on each method. However, I am inclined to believe that these tags are not being called on the [SetUp] or [TearDown] methods, creating threading issues. I have hopefully resolved the issue by placing the [STAThread] tag on each method (including the [SetUp] and [TearDown] methods) explicitly. I will let you know of any further changes, but it's worth a shot.
There are some similar issues related to Watin and IE8.
Running Watin on TeamCity
failed due to the following error: 800704a6 while trying to read data from a text file in teamcity
https://serverfault.com/questions/179156/ie8-script-error-800704a6
From what i understand, make sure you windows is fully updated, make another restart just to make sure and check if problem persist.
If it does, try runnning ccnet not in service mode.
If still no good, try to play with internet options security to determine if this affect the problem.
HTH
Related
I was wondering if anybody could help with an issue I have come across, I have recently added to my test framework the sharing of a value using feature content
This has been placed in the [BeforeScenario] trigger
FeatureContext.Current.Add("CreatedUser", createdUser);
I then clean this up in the [AfterScenario] with
FeatureContext.Current.Remove("CreatedUser");
Since adding this I am unable to run more then 1 test, if I run the tests individually (1 by 1) then they execute with no issue.
If I select a few tests to run from visual studio, the first one runs with no issue and the next test will always fail with the following error.
Message: System.Collections.Generic.KeyNotFoundException : The given key was not present in the dictionary.
TearDown : System.NullReferenceException : Object reference not set to an instance of an object.
I added the remove line as i thought the feature context was being carried from test to test and causing an issue but with or without the removal i am getting this error.
Any help is appreciated
I eventually managed to fix this issue by removing the Allure NuGet packages.
I had previously installed
Specflow.Allure
Allure.Commons
This package was causing the issue, I am not sure of the reason but after removing this error stopped occurring.
I have a NancyFX web project using SimpleAuthentication that works fine when self-hosting from the terminal. When I self-host from unit tests (NUnit runner with Xamarin Studio), I receive exception "Failed to reflect on the current domain's Assemblies while searching for plugin" with error message "Could not load file or assembly or one of its dependencies." repeated many times.
Any idea why this would fail? This looks like the code throwing the exception is at https://github.com/SimpleAuthentication/SimpleAuthentication/blob/master/Code/SimpleAuthentication.Core/ReflectionHelpers.cs. I tried a IntPtr.Size measurement to make sure they are running in the same 32/64 bitness in both cases (it is running 32-bit).
UPDATE: Per recommendation, I tried to run some prerelease builds to the same resulting, ending up with these binaries:
./Nancy.1.3-Pre1524/lib/net40/Nancy.dll
./Nancy.Authentication.Forms.1.3-Pre1524/lib/net40/Nancy.Authentication.Forms.dll
./Nancy.FlashMessages.1.0.4.0/lib/net45/Nancy.FlashMessages.dll
./Nancy.FlashMessages.Razor.1.0.4.0/lib/net45/Nancy.FlashMessages.Razor.dll
./Nancy.Hosting.Self.1.3-Pre1524/lib/net40/Nancy.Hosting.Self.dll
./Nancy.SimpleAuthentication.0.3.14/lib/net40/Nancy.SimpleAuthentication.dll
./Nancy.Viewengines.Razor.1.3-Pre1524/BuildProviders/Nancy.ViewEngines.Razor.BuildProviders.dll
./Nancy.Viewengines.Razor.1.3-Pre1524/lib/net40/Nancy.ViewEngines.Razor.dll
UPDATE: Running the ReflectionHelpers.cs code in isolation shows that the only DLL failing to load is MonoDevelop.NUnit. The NUnit dll is unneeded for this part anyhow.
I suppose I should contemplate https://github.com/SimpleAuthentication/SimpleAuthentication/issues/144
I fixed this with GetLoadableTypes from this post from #haacked http://haacked.com/archive/2012/07/23/get-all-types-in-an-assembly.aspx/
And here it's the PR (https://github.com/SimpleAuthentication/SimpleAuthentication/pull/171)
I didn't had the problem with Nunit I had the problem with Microsoft.Owin.Security.DataProtection.DpapiDataProtector that it's not implemented in Mono....
I'm running acceptance tests on a project via SpecFlow, NUnit and Coypu (for browser automation, using the WatiN driver). The running of the tests are invoked via a powershell/psake script.
If I run these tests on my local box, they run fine. However, we have a build server on which a Jenkins job will automatically run these tests, and when run via this Jenkins job they don't execute -- they just hang.
Looking in task manager I can see there's two instances of iexplore.exe that are created when the Jenkins job runs. However after a certain point they just hang - no changes in memory usage or CPU.
nunit-agent-x86.exe and nunit-console-x86.exe are also running but mostly hung, just nunit-agent-x86.exe going up very slowly in memory.
If I kill one of the iexplore.exe processes things continue, but the SpecFlow specs all subsequently fail.
At the point of killing iexplore.exe, the following exception is in the log:
Unhandled Exception: System.Runtime.InteropServices.COMException: The remote procedure call failed. (Exception from HRESULT: 0x800706BE)
If I invoke the psake script manually when logged in to the server, the specs run OK.
This issue began to occur when I tried to use basic DI for the BrowserSession as in the gist here: https://gist.github.com/2301407
Before that I was sharing the BrowserSession via a static property of an NUnit [SetupFixture] class. Things were working mostly OK that way, except a small issue with a test involving a modal dialog not working correctly, but I wasn't sure I was doing it right so wanted to do the technique in the gist
I'm a bit lost as to what's causing the hang. Any ideas what it is or tips to track it down?
UPDATE: After switching to Firefox as the browser, and Selenium as the driver, the problem has gone away...
I got a couple of responses on the Coypu list. I've yet to test them out as everything is fine with Firefox at the moment, but in case they are of assistance to anyone else...
On Thursday, 2 August 2012 16:38:30 UTC+1, Adiel wrote:
[...]i believe that watin needs nunit to run in STA (single-threaded) which could possibly be related to your problem.
In other words, perhaps you made your tests thread safe with the static singleton browser session, but now via specflow's IOC you are
getting multiple instances due to the way nunit is running.
On Thursday, 2 August 2012 16:41:11 UTC+1, Matt Ellis wrote:
This sounds like it's Internet Explorer's protected mode getting in the way. IE runs different zones, such as Internet and Intranet
(and about:blank) in different processes, and IIRC WatiN doesn't
handle that very well. If you can disable protected mode on your
server, you should be fine.
I just recently ran into some trouble setting up a fully automated nightly build / testing / deployment environment. To explain the setup in short. We use Mercurial as source control system and TeamCity as build server. It currently only has one build agent on a separate machine to keep things simple.
The idea is as follows: each night a TeamCity build runs. Pulls latest from Mercurial repository, runs a build, runs unit tests, deploys to test server, runs acceptance tests (on build agent) against freshly deployed test server and if all of that passes deploys the lot to a staging server. I've got everything working except for acceptance tests. These are done with SpecFlow and WatiN. Everyhting runs nicely on development machines (Windows 7 32 bit). However when I try to run the acceptance tests (NUnit in TeamCity) the runner reports fail beacuse of step [Browser].ContainsText(browserText). All steps up to that ran correctly and if I replicate the steps in the same browser on build agent (got full admin access) I can see the text being present. So to my best knowledge the test should pass.
I found this stackoverflow thread that got me thinking I may be doing somwthing wrong with how I keep the browser instance around. To explain: I have a set of common steps such as "when I click X button" or "when I enter text X into field Y" and the one with an issue "then then I am shown X". Because I'm making as much as possible of tests reusable I've also generalized a #requires_browser scenario requirement which is then defined like this:
[BeforeScenario("requires_browser")]
public void RequiresBrowserBeforeFeature()
{
Settings.MakeNewIeInstanceVisible = false;
ScenarioContext.Current.Set<Browser>(new IE(true));
}
[AfterScenario("requires_browser")]
public void RequiresBrowserAfterFeature()
{
ScenarioContext.Current.Get<Browser>().Close();
ScenarioContext.Current.Get<Browser>().Dispose();
}
and then the common step:
[Then("I am shown \"(.*)\"")]
public void ThenIAmShown(string text)
{
Assert.IsTrue(ScenarioContext.Current.Get<Browser>().ContainsText(text));
}
From this I can define other common steps that enter text into text boxes, click buttons and so on. All this works perfectly on development environment, running in visual studio 2010 [as administrator].
Am I wrong in thinking that ScenarioContext should be preserved until the test is finished?
What I've tried so far:
Configuring build agent service to run as local system (SYSTEM). This causes the issue as described - the ThenIAmShown(string text) method will always fail. It has interaction with local desktop enabled.
Configured build agent service to run as network administrator (full admin rights). This setup won't even runs tests. The exception I get:
TearDown method failed. WatiN.Core.Exceptions.BrowserNotFoundException : Could not find an IE window matching constraint: Timeout while waiting to attach to newly created instance of IE.. Search expired after '30' seconds.
TearDown : System.Collections.Generic.KeyNotFoundException : The given key was not present in the dictionary.
at WatiN.Core.IE.CreateIEPartiallyInitializedInNewProcess()
at WatiN.Core.IE.CreateNewIEAndGoToUri(Uri uri, IDialogHandler logonDialogHandler, Boolean createInNewProcess)
...
Already added assemblies Interop.SHDocVw.dll and Microsoft.mshtml.dll to bin directory (Copy To Output in project references).
Running NUnit UI from the build agent. This runs the unit tests as expected. Leads me to believe there might be an issue with security. Runnint NUnit 2.0.5 on Windows Server 2008 R2 Enterprise.
Anyone had this issue?
I am getting WatiN.Core.Exceptions.TimeoutException:
Timeout while Internet Explorer busy error while executing my tests via CruiseControl.Net.
Any one have idea how to resolve this?
While we are using TeamCity, we had to disable IE protected mode.
Also, check that user, under which watiN tests are being run can interact with desktop.
I know this question is old and answered, but below are some of my observations.
It is possible to run watin tests under a service account
but the following restrictions/prerequisites apply:
service must run in desktop interactive mode. Only available if running as system.
tests must not create a new windows, even alert/confirm dialogs
Ie cannot create a new window, so watin fails when looking for/expecting it to appear.
ie may show its own warnings, e.g. Insecure content in a secure Page, this can cause tests to fail*
if the tests fail/timeout and the ie instance is forcefully closed, the next instance may try to restore the previous state. The tests then appear to fail*
this can be turned off in the advanced settings.
*from what I've experienced, usually because the prompt is halting the document from being reported as loading-finished.
Feel free to add with other restrictions /comments.