I am following MyFancordionRunner example from Fancordion v1.0.4 official documentation to test a BedSheet application, but the suiteSetup method (see below) is not being called and the server remains null, causing the fixture tests to fail with a NullPointerException.
override Void suiteSetup() {
super.suiteSetup
server = BedServer(AppModule#.pod).addModule(WebTestModule#).startup
}
Looking at FancordionRunner source code, the runFixture(Obj fixtureInstance) method should be invoking suiteSetup() the first time a Fixture is run as per this piece of code...
FixtureResult runFixture(Obj fixtureInstance) {
...
locals := Locals.instance
firstFixture := (locals.originalRunner == null)
if (firstFixture) {
locals.originalRunner = this
suiteSetup()
...
}
But for some reason in my case the condition (locals.originalRunner == null) must be returning false, causing the suiteSetup() invocation to be skipped. It seems that this piece of code uses Fantom Actors which I'm not familiar with.
I am manually invoking the suiteSetup within MyFancordionRunner like this:
override Void fixtureSetup(Obj fixtureInstance) {
if (server == null) suiteSetup
...
This workaround solves the NullPointerException issue and allows the fixtures to run successfully but I don't know if this workaround is defeating the purpose of the Actor logic, which I presume is meant to invoke suiteSetup only once.
Can anyone explain what could be going on here that is preventing the suiteSetup method from being called within runFixture(...), please?
I don't know what's going on here without seeing a lot more code.
The only part of Actor being used is Actor.locals() which is really just a pot to hold thread local variables - as it is assumed that all tests are run in the same thread.
As you shown, the logic in runFixture() is pretty simple, are you sure it is being called?
Related
I've got a base class I'm using with MSpec which provides convenience methods around AutoMock:
public abstract class SubjectBuilderContext
{
static AutoMock _container;
protected static ISubjectBuilderConfigurationContext<T> BuildSubject<T>()
{
_container = AutoMock.GetLoose();
return new SubjectBuilderConfigurationContext<T>(_container);
}
protected static Mock<TDouble> GetMock<TDouble>()
where TDouble : class
{
return _container.Mock<TDouble>();
}
}
Occasionally, I'm seeing an exception happen when attempting to retrieve a Mock like so:
It should_store_the_receipt = () => GetMock<IFileService>().Verify(f => f.SaveFileAsync(Moq.It.IsAny<byte[]>(), Moq.It.IsAny<string>()), Times.Once());
Here's the exception:
System.ObjectDisposedExceptionInstances cannot be resolved and nested
lifetimes cannot be created from this LifetimeScope as it has already
been disposed.
I'm guessing it has something to do with the way MSpec runs the tests (via reflection) and that there's a period of time when nothing actively has references to any of the objects in the underlying lifetime scope being used by AutoMock which causes the lifetimescope to get disposed. What's going on here, and is there some simple way for me to keep it from happening?
The AutoMock lifetime scope from Autofac.Extras.Moq is disposed when the mock itself is disposed. If you're getting this, it means the AutoMock instance has been disposed or has otherwise lost scope and the GC has cleaned it up.
Given that, there are a few possibilities.
The first possibility is that you've got some potential threading challenges around async method calls. Looking at the method that's being mocked, I see you're verifying the call to a SaveFileAsync method. However, I don't see any async related code in there, and I'm not entirely sure when/how the tests running are calling it given the currently posted code, but if there is a situation where an async call causes the test to run on one thread while the AutoMock loses scope or otherwise gets killed on the other thread, I could see this happening.
The second possibility is the mix of static and instance items in the context. You are storing the AutoMock as a static, but it appears the context class in which it resides is a base class that is intended to supply instance-related values. If two tests run in parallel, for example, the first test will set the AutoMock to what it thinks it needs, then the second test will overwrite the AutoMock and the first will go out of scope, disposing the scope.
The third possibility is multiple calls to BuildSubject<T> in one test. The call to BuildSubject<T> initializes the AutoMock. If you call that multiple times in one test, despite changing the T type, you'll be stomping the AutoMock each time and the associated lifetime scope will be disposed.
The fourth possibility is a test ordering problem. If you only see it sometimes but not other times, it could be that certain tests inadvertently assume that some setup, like the call to BuildSubject<T>, has already been done; while other tests may not make that assumption and will call BuildSubject<T> themselves. Depending on the order the tests run, you may sometimes get lucky and not see the exception, but other times you may run into the problem where BuildSubject<T> gets called at just the wrong time and causes you pain.
I have a GUI program with wxwidgets. When a Button is clicked, a new thread is supposed to start and run, so that it doesn't block the UI.
From the thread, a function is called that takes a callback function as argument.
When I run the program without threading (hence blocked UI), there's no Problem.
But when I use boost:thread as i do in the code below, the thread does not run properly.
The program stops in the callback function CallBack() when it tries to set the member variable m_intValue. This is succesful without a seperate thread, but hangs when I put it in a separate thread.
Any thoughts? I have tried to simplify the code so that it's more readable. Please excuse if it won't compile like this. I'm just asking, maybe someone knows right away where the problem is!
Code
class MainFrame : public wxFrame
{
protected:
boost:thread m_workerThread;
m_intValue;
WorkerThread(Foo& afoo, Bar& aBar)
{
afoo.doSmth(aBar, boost::bind(&MainFrame::CallBack, this, _1, _2, _3)
}
void OnButtonClick()
{
Foo oFoo;
Bar oBar;
m_workerThread = boost::thread(&MainFrame::WorkerThread, this, boost::ref(oFoo), boost::ref(oBar));
}
void CallBack(int arg, int arg1, int arg2 )
{
m_intValue = arg;
}
};
The code as shown is completely MT-unsafe as there is no synchronization between main thread (which presumably accesses m_intValue as well) and the worker thread, in context of which your callback is executing. This can be fixed by using a critical section or a mutex to protect access to it.
If your real code uses any wxWidgets UI functionality from the callback, it's not going to work at all however, as mentioned by #ravenspoint, wxWidgets UI can only be used from the main thread. The only solution to this problem is to execute the code doing any UI from the main thread.
And the best, because very simple, way to do it is to use CallAfter(), e.g.
void WorkerThread()
{
...
wxGetApp().CallAfter(Callback, arg1, arg2);
}
If you are using C++11 a very useful variant is to avoid defining Callback() at all and just do
wxGetApp().CallAfter([=]() { m_intValue = arg; });
Now this is perfectly safe as the assignment is done in the main thread.
I am guessing that it is problematical to have WorkerThread and CallBack as methods of your MainFrame. wxWidgets does not like running GUI code in a worker thread. While the posted code looks harmless, I wonder what you have in the real code ... ?
I suggest moving WorkerThread and CallBack into a new clas that does NOT inherit from any wx class - this will prevent you inadvertently executing GUI code from your worker thread and causing havoc.
I'm using GWTP, adding a Contract layer to abstract the knowledge between Presenter and View, and I'm pretty satisfied of the result with GWTP.
I'm testing my presenters with Mockito.
But as time passed, I found it was hard to maintain a clean presenter with its tests.
There are some refactoring stuff I did to improve that, but I was still not satisfied.
I found the following to be the heart of the matter :
My presenters need often asynchronous call, or generally call to objects method with a callback to continue my presenter flow (they are usually nested).
For example :
this.populationManager.populate(new PopulationCallback()
{
public void onPopulate()
{
doSomeStufWithTheView(populationManager.get());
}
});
In my tests, I ended to verify the population() call of the mocked PopulationManager object. Then to create another test on the doSomeStufWithTheView() method.
But I discovered rather quickly that it was bad design : any change or refactoring ended to broke a lot of my tests, and forced me to create from start others, even though the presenter functionality did not change !
Plus I didn't test if the callback was effectively what I wanted.
So I tried to use mockito doAnswer method to do not break my presenter testing flow :
doAnswer(new Answer(){
public Object answer(InvocationOnMock invocation) throws Throwable
{
Object[] args = invocation.getArguments();
((PopulationCallback)args[0]).onPopulate();
return null;
}
}).when(this.populationManager).populate(any(PopulationCallback.class));
I factored the code for it to be less verbose (and internally less dependant to the arg position) :
doAnswer(new PopulationCallbackAnswer())
.when(this.populationManager).populate(any(PopulationCallback.class));
So while mocking the populationManager, I could still test the flow of my presenter, basically like that :
#Test
public void testSomeStuffAppends()
{
// Given
doAnswer(new PopulationCallbackAnswer())
.when(this.populationManager).populate(any(PopulationCallback.class));
// When
this.myPresenter.onReset();
// Then
verify(populationManager).populate(any(PopulationCallback.class)); // That was before
verify(this.myView).displaySomething(); // Now I can do that.
}
I am wondering if it is a good use of the doAnswer method, or if it is a code smell, and a better design can be used ?
Usually, my presenters tend to just use others object (like some Mediator Pattern) and interact with the view. I have some presenter with several hundred (~400) lines of code.
Again, is it a proof of bad design, or is it normal for a presenter to be verbose (because its using others objects) ?
Does anyone heard of some project which uses GWTP and tests its presenter cleanly ?
I hope I explained in a comprehensive way.
Thank you in advance.
PS : I'm pretty new to Stack Overflow, plus my English is still lacking, if my question needs something to be improved, please tell me.
You could use ArgumentCaptor:
Check out this blog post fore more details.
If I understood correctly you are asking about design/architecture.
This is shouldn't be counted as answer, it's just my thoughts.
If I have followed code:
public void loadEmoticonPacks() {
executor.execute(new Runnable() {
public void run() {
pack = loadFromServer();
savePackForUsageAfter();
}
});
}
I usually don't count on executor and just check that methods does concrete job by loading and saving. So the executor here is just instrument to prevent long operations in the UI thread.
If I have something like:
accountManager.setListener(this);
....
public void onAccountEvent(AccountEvent event) {
....
}
I will check first that we subscribed for events (and unsubscribed on some destroying) as well I would check that onAccountEvent does expected scenarios.
UPD1. Probably, in example 1, better would be extract method loadFromServerAndSave and check that it's not executed on UI thread as well check that it does everything as expected.
UPD2. It's better to use framework like Guava Bus for events processing.
We are using this doAnswer pattern in our presenter tests as well and usually it works just fine. One caveat though: If you test it like this you are effectively removing the asynchronous nature of the call, that is the callback is executed immediately after the server call is initiated.
This can lead to undiscovered race conditions. To check for those, you could make this a two-step process: when calling the server,the answer method only saves the callback. Then, when it is appropriate in your test, you call sometinh like flush() or onSuccess() on your answer (I would suggest making a utility class for this that can be reused in other circumstances), so that you can control when the callback for the result is really called.
We use TeamCity as our CI server, and I've just started seeing "TestFixtureSetUp Failed" in the test failure window.
Any idea how I go about debugging this problem? The tests run fine on my workstation (R# test runner in VS2008).
It is a bit of a flaw in the implementation of TestFixtureSetUp (and TestFixtureTearDown) that any exceptions are not well reported. I wrote the first implementation of them and I never got it to work the way it was supposed to. At the time the concepts in the NUnit code were tightly coupled to the idea that actions were directly related to a single test. So the reporting of everything was related to a test result. There wasn't really a space for reporting something that happened at the suite level without a huge re-write (it isn't a refactoring when you change a sheep into an escalator).
Because of that bit of history it's hard to find out what really happened in a TestFixtureSetUp. There isn't a good place to attach the error. The TestFixtureSetUp call is a side effect of running a test instead of being directly related to it.
#TrueWill has the right idea. Check the logs and then modify the test to add more logging if necessary. You might want to put at try/catch inside the TestFixtureSetup and log a lot in the catch block. I just thought I could add some background to it (in other words it's kind of my fault).
I'd check the Build Log first.
If it's not obvious from that, you could try including Console.WriteLines in the tests - I'm not positive, but I think those are written to the Build Log. Alternately you could log to a file (even using log4net if you wanted to get fancy).
If you have Visual Studio installed on the CI server, you could try running the build/tests from there. If it's a connectivity issue, that might resolve it.
I've seen path issues, though, where relative paths to files were no longer correct or absolute paths were used. These are harder to debug, and might require logging the paths and then checking if they exist on the build server.
I ran into this today when creating some integration tests that have long running setup that I don't want to duplicate. I ended up wrapping all the test fixture setup logic in a try/catch. I then add a SetUp method whose sole purpose is to see if a failure occurred during fixture setup and provide better logging.
Exception testFixtureSetupException = null;
[TestFixtureSetUp]
public void FixtureSetup()
{
try
{
// DoTestFixtureSetup
}
catch (Exception ex)
{
testFixtureSetupException = ex;
}
}
[SetUp]
// NUnit doesn't support very useful logging of failures from a TestFixtureSetUp method. We'll do the logging here.
public void CheckForTestFixturefailure()
{
if (testFixtureSetupException != null)
{
string msg = string.Format("There was a failure during test fixture setup, resulting in a {1} exception. You should check the state of the storage accounts in Azure before re-running the RenewStorageAccountE2ETests. {0}Exception Message: {3}{0}Stack Trace:{4}",
Environment.NewLine, testFixtureSetupException.GetType(), accountNamePrefix, testFixtureSetupException.Message, testFixtureSetupException.StackTrace);
Assert.Fail(msg);
}
}
I was getting the same error while running any test with SpecFlow using Visual NUnit. When I tried doing the same from the Unit Test Explorer(provided by Resharper), it gave a little more helpful message: Binding methods with more than 10 parameters are not supported. I realized I can't have a SpecFlow method with more than 10 params, had to remove the test.
I was able to see that I was not creating my test database correctly by doing a quick switch to VS Unit Testing. In my Case it was able to return a better response to the reason why it failed. I usually use NUnit.
"Unable to create instance of class X. Error: System.Data.SqlClient.SqlException: A file activation error occurred. The physical file name '\DbTest.mdf' may be incorrect. Diagnose and correct additional errors, and retry the operation.
CREATE DATABASE failed. Some file names listed could not be created. Check related errors..
"
Run the unit test in debug mode. You may find a runtime error in the the setup.
If you are using SpecFlow and C# in Visual Studio, look at the auto-generated <whatever>.feature.cs file after the test fails. On the public partial class <whatever>Feature line, you should see a symbol which when hovered over will show the reason that the NUnit fixture setup failed. In my case, it was that some of my BeforeFeature methods in my TestHooks class were not static. All BeforeTestRun, AfterTestRun, BeforeFeature, and AfterFeature methods need to be static.
I had this issue and it was caused by adding a private readonly Dictionary in the class, much the same way that you add a private const string.
I tried to make the Dictionary a constant but you can't do that at compile time. I solved this by putting my Dictionary in a method that returns it.
I was troubled by this today. I did the following to get the actual error.
(1) Write another test in a separate fixture which initializes an instance of the troubling test fixture, explicitly calls setup methods such as TestFixtureSetUp and SetUp if any, and then executes the target test method.
(2) Add exception handling code for the new code above, and log / output the actual exception to somewhere.
You can catch the exception and write it in the console on the TearDown
Something like:
[SetUpFixture]
public class BaseTest
{
private Exception caughtException = null;
[SetUp]
public void RunBeforeAnyTests()
{
try
{
throw new Exception("On purpose");
}
catch (Exception ex)
{
caughtException = ex;
}
}
[TearDown]
public void RunAfterAnyTests()
{
if (caughtException != null)
{
Console.WriteLine(string.Format("TestFixtureSetUp failed in {0} - {1}", this.GetType(), caughtException.Message));
}
}
}
And the result will be:
TestFixtureSetUp failed in IntegratedTests.Services.BaseTest - On purpose
I had this symptom caused by an error during field initialization. If you initialize your fields in the [SetUp] method, you should see a better error message.
[TestFixture]
internal class CommandParserTest
{
// obscure error message
private CommandParser parser = new CommandParser(...);
...
}
[TestFixture]
internal class CommandParserTest
{
private CommandParser parser;
[SetUp]
public void BeforeTest()
{
// better error message
parser = new CommandParser(...);
}
...
}
When executing the second line of this code Rhino Mocks throws an InvalidOperationException with a message "This action is invalid when the mock object is in replay state"
var mockScanner = MockRepository.GenerateMock<PortScanner>(null);
mockScanner.Expect((scanner => { scanner.Scan(null, null); }));
Stepping through the code in a debugger one can see the debugger run the method defined in the class and directly after control leaves this method the exception occurs.
This similar code in another test does work without issue
var mockView = MockRepository.GenerateMock<IScanView>(null);
mockView.Expect(view => { view.Close(); });
var controller = new ScanController(mockView);
controller.Exit();
mockView.VerifyAllExpectations();
The only difference that I can think of that might be of any consequense between theese two tests is that Exit is a member on an interface while Scan is a virtual member on a class
What am I missing?
Update
Further exploration has indicated that this is related to the way Rhino handles virtual methods. I am focusing mmy study of the documentation here now
The exception was caused because Rhino Mocks did not have the required level of access to the type in order to mock it properly. Granting internal access to the Rhino Mocks assembly using InternalsVisibleTo solved the problem.
It's noteworthy that this does not affect interfaces. I believe the reason for this is because the mocking framework needs to override the implementation on a class where there is none on an interface.
What happens if you remove the extra set of parentheses from the first expression?
var mockScanner = MockRepository.GenerateMock<PortScanner>(null);
mockScanner.Expect( scanner => { scanner.Scan(null, null); } );