There's support for forkMode in Ant and Maven and occasionally we use it with value perTest. However, the JUnit-tests in Eclipse still fail when we run the tests on a class or on a project (Run As -> JUnit Test). Obviously JUnit uses default settings or behaviour and executes the tests in parallel causing some red crosses in the JUnit-view.
Is there a way to code something into the test-class that lets JUnit behave like the forkMode setting? We don't mind if there's an Eclipse-only solution for this.
Or can this be done with a Run Configuration in Eclipse?
EDIT:
I understand that the problems are based on data remaining after tests and further tests will fail due to that. While this makes sense, please understand that this doesn't answer my question. Think of my situation as being part of some sort of a Tiger Team. We have a bunch of issues and fixing that part of existing tests is just one of them. Trust me, we will try to cover everything... (I haven't heard that in a while)
Eclipse runs the JUnit test serially, in a single thread, in the same JVM. If you have tests that normally operate in parallel, this should not affect the test behavior. However, if you assume that you can change settings in the VM, like system properties, or class static variables, and the next test will not have those changes, that will break your tests.
The rule of thumb is that each test should leave the system (vm, database, filesystem) exactly as it found it so that each test can be run at any time, in any order.
Related
I have a bunch of fixture stuff that I want to do once for the test class but I also don't want the associated tests messing with it.
I don't really get the py.test fixture system yet so I'm not seeing how this is supposed to be done.
In vanilla Django this stuff is achieved with setUpTestData which lets you create some fixtures in the DB once for the test class. Then at the start of each test case it drops a transaction savepoint and at the end of each test it resets to that save point. (This is in addition to it transactioning around the entire class so as to leave a clean db at the end.)
I could get this functionality by inheriting off Django's TransactionTestCase but pytest-django seems to want to run without that and I've achieved everything else I need without doing so.
I've looked around for this quite a lot, and the best I could find is this pytest plugin, which I have not tested myself:
https://github.com/tipsi/pytest-tipsi-django
Per the pytest-django issue list, it looks like this has been raised before, but there is no movement towards a fix:
https://github.com/pytest-dev/pytest-django/issues/514
My NUnit tests fail unless the nunit runner is launched with /noshadow parameter.
But in CC.net, it seems to be impossible to supply this parameter in the <nunit> block.
I know I always can fall back to generic <exec> block, but is there really no way to configure the <nunit> block?
I would surmise that if this switch/flag isn't documented, then it isn't available in the that you mention.
The thing to keep in mind with these custom tasks, is that usually they are just friendly-wrappers for what eventually becomes a command-line call.
The task-author is just making things simpler for you. They take on the onus of creating the correct commandline, and pass that to the original .exe.
Now, it looks like somebody did address the command line of your interest here:
https://github.com/loresoft/msbuildtasks/blob/master/Source/MSBuild.Community.Tasks/NUnit.cs
Note the code:
if (DisableShadowCopy)
{
builder.AppendSwitch(c+"noshadow");
}
So I would see if you can get this task working.
In fact, I barely use any of the built in CC.NET tasks, except for source-code download and starting up msbuild.exe...and then the publishing. I leave the hard stuff to msbuild.
Aka, I pull source-code, which includes a MyBuild.proj file.
Then I have cc.net execute "msbuild.exe MyBuild.proj"
Then I have cc.net do some of the publishing.
Why?
If most of my logic is in a msbuild .proj file, then if I ever switch to another CI tool, the transition is much less traumatic. In fact, I recently learned that an old job of mine went to TFS, and because I wrote most of the build logic in msbuild (and not a lot of cc.net tasks)....the transition to TFS was fairly painless. If I had used cc.net tasks instead......every single one of those would have had to been translated to a corresponding tfs task.... :<
Anyways. Back to your question. Keep in mind...that somebody is basically (via a task) is usually just writing up a nice way to wire up things, and doing the command line arguments/syntax sugar for you. So they sometimes miss a flag, or a flag gets added later, but the original task is not updated.
So you'll either need to modify the source code yourself........ :< Or pick a library that keeps more up to date.
Good luck.
I'm using Eclipse and JUnit 4 while developing an application within a tomcat container. The container manages the connection to our Oracle database.
While testing with JUnit i've got the following problem: In the constructor of the test subject there is something like this:
public Subject() {
// stuff
FancySingleton.getInstance().getFancy("stuff");
}
Unfortunately the method getFancy() tries to execute a Query which it can't because JUnit does not run within the tomcat container and ends up in an endless loop.
My first idea was to out-commend the code. At second thought it appeared to be a bad idea. I could forget to remove the comments before committing.
My second idea was to highlight the code for eclipse so that it doesn't execute it while running a JUnit test. But it requires eclipse to support such a method.
At last i thought of something like preprocessor directives.
What is your idea? Just passing in a boolean to the constructor is imho not a clean way of dealing with such a circumstance.
You'd either mock FancySingleton, or you'd do it right and inject an implementation.
I have some NUnit tests which uses a TestCaseSource function. Unfortunately, the TestCaseSource function that I need takes a long time to initialize, because it scans a folder tree recursively to find all of the test images that would be passed into the test function. (Alternatively it could load from a file list XML every time it's run, but automatic discovery of new image files is still a requirement.)
Is it possible to specify an NUnit attribute together with TestCaseSource such that NUnit does not enumerate the test cases (does not call the TestCaseSource function) until either the user clicks on the node, or until the test suite is being run?
The need to get all test images stored in a folder is a project requirement because other people who do not have access to the test project will need to add new test images to the folder, without having to modify the test project's source code. They would then be able to view the test result.
Some dogmatic unit-testers may counter that I am using NUnit to do something it's not supposed to do. I have to admit that I have to meet a requirement, and NUnit is such a great tool with a great GUI that satisfies most of my requirements, such that I do not care about whether it is proper unit testing or not.
Additional info (from NUnit documentation)
Note on Object Construction
NUnit locates the test cases at the
time the tests are loaded, creates
instances of each class with
non-static sources and builds a list
of tests to be executed. Each source
object is only created once at this
time and is destroyed after all tests
are loaded.
If the data source is in the test
fixture itself, the object is created
using the appropriate constructor for
the fixture parameters provided on the
TestFixtureAttribute or the default
constructor if no parameters were
specified. Since this object is
destroyed before the tests are run, no
communication is possible between
these two phases - or between
different runs - except through the
parameters themselves.
It seems the purpose of loading the test cases up front is to avoid having communications (or side-effects) between TestCaseSource and the execution of the tests. Is this true? Is this the only reason to require test cases to be loaded up front?
Note:
A modification of NUnit was needed, as documented in http://blog.sponholtz.com/2012/02/late-binded-parameterized-tests-in.html
There are plans to introduce this option to later versions of NUnit.
I don't know of a way to delay-load test names in the GUI. My recommendation would be to move those tests to a separate assembly. That way, you can quickly run all of your other tests, and load the slower exhaustive tests only when needed.
Java compiler provides incremental build, so javac ant task as well. But most other processes don't.
Considering build processes, they transform some set of files (source) into another set of files (target).
I can distinct two cases here:
Transformator cannot take a subset of source files, only the whole set. Here we can only make lazy build - if no files from source was modified - we skip processing.
Transformator can take a subset of sources files and produce a partial result - incremental build.
What are ant internal, third-party extensions or other tools to implement lazy and incremental build?
Can you provide some widespread buildfile examples?
I am interested this to work with GWT compiler in particular.
The uptodate task is Ant's generic solution to this problem. It's flexible enough to work in most situations where lazy or incremental compilation is desirable.
I had the same problem as you: I have a GWT module as part of my code, and I don't want to pay the (hefty!) cost of recompiling it when I don't need to. The solution in my case looked something like this:
<uptodate property="gwtCompile.mymodule.notRequired"
targetfile="www/com.example.MyGwtModule/com.example.MyGwtModule.nocache.js">
<srcfiles dir="src" includes="**"/>
</uptodate>
<target name="compile-mymodule-gwt" unless="gwtCompile.mymodule.notRequired">
<compile-gwt-module module="com.example.MyGwtModule"/>
</target>
Related to GWT, it's not possible to do incremental builds because the GWT compiler looks at all the source code at once and optimizes and inlines code. This means code that wasn't changed could be evaluated differently, for example if you start using a method from a class that wasn't changed, the method was in the previous compilation step left out, but now needs to be compiled in.