I have set of Junit test suites. All are working fine in eclipse.
In the test suites one test suite we will pass some System properties.
But those modified system properties should not propagate to other test suites So i just put those only in setup method like below,
#BeforeClass
public static void setUp() {
System.setProperty("public", "publicfolder");
System.setProperty("private", "privatefolder");
}
But this is working fine in eclipse only. While running it outside all other test suites are working fine except the above one.
I know to pass system properties in gradle in build file. but how could i pass those system properties to only one test suite instead of all thats my question here.
You could add another Test task so there's two in total. Each could have a different filter to run separate test suites and each could pass different system properties.
See here for a similar solution
Related
I have three classes all containing multiple tests. Each class has the annotation of TestFixture. I need each class to run the tests in order. (e.g. TextFixture1 runs all of its tests, then TestFixture2 runs all of its tests, and finally TestFixture3 runs all of its tests.) How can I accomplish this?
Use the OrderAttribute on each fixture, specifying the order in which you want the fixtures to run. Use the same attribute on each test method, specifying the order the test should run within the fixture.
See OrderAttribute in the docs for more info.
I have a testng.xml file and i have three classes and all three have tests with priority from 1 to 4(#Test(priority=1 to 4)). On running it the tests with priority 1 from different classes run first and affect the flow of my test.What kind of testng annotation can I use to avoid this?
#Test(priority=1) given in multiple classes
use preserve order
<test name="Test" preserve-order="true">
So that all classes specified in testng.xml will run in specified order and then in each class methods will run as per priority.
In testng, the order of the classes mentioned in the testng.xml doesn't matter if you have the priority set in the tests inside those classes. The tests will run according to the priorities (priority=1 tests will run first and then priority=2 and further on).
To solve the above issue, you need to remove the priorities from the tests inside the classes and then put the classes in the testng in the order you want to execute those classes and put <preserve-order="true"> in the testng xml.
If you want to run the tests inside a class also in a particular order, then you can use dependsOnMethods between the tests mentioned inside the class like:
#Test(dependsOnMethods = {"parentTest"})
public void childTest() {
}
#Test
public void parentTest() {
}
In the above case, when the parentTest() pass only then the childTest() will execute.
I have several classes each containing several methods annotated with #Test. The classes are not extending TestCase. Now I want to
write a main that executes all these methods (for command line use)
create a class that can be "Run as -> JUnit Test" in Eclipse executing all these methods
Since the classes are no TestCases I can't just add them to a suite. Also extending TestCase is not an option because the test methods are just annotated and their names don't start with 'test'.
How can I do this?
Try this:
#RunWith(Suite.class)
#Suite.SuiteClasses({
Test1.class,
Test2.class,
Test3.class,
Test4.class
})
public class YourTestSuite {
}
You can set up run configurations for a project. Right click on your project. Then selecte 'Run as' -> 'Run configurations'. Within that you can select run all tests
What Eugene posted works if you have a small number of classes. If you want all the tests in a certain package pattern, Classpath Suite lets you specify this with patterns instead of listing them all out.
I am using junit with eclipse to write function tests.
When running an individual test it runs in the order that I have them set within the class.
Eg.
testCreateUser
testJoinUserToRoom
testVerify
testDeleteUser
However when I run this test as part of a suite, (so in a package) the order is random.
It will for example do the verify, then delete user then joinuserToRoom then Createuser.
My tests within the suite are not dependent on each other. However each individual test within a test is dependent on them being run in the correct order.
Is there any way I can achieve this?
Thanks.
You can't guarantee the order of execution of test methods in JUnit.
The order of execution of test classes within a suite is guaranteed (if you're using Suite), but the order of execution if the test classes are found by reflection isn't (for instance, if you're running a package in Eclipse, or a set of tests from maven or ant). This may be definable by ant or maven, but it isn't defined by JUnit.
In general, JUnit executes the test methods in the order in which they are defined in the source file, but not all JVMs guarantee this (particulary with JVM 7). If some of the methods are inherited from an abstract base test class, then this may not hold either. (This sounds like your case, but I can't tell from your description).
For more information on this, see my answer to Has JUnit4 begun supporting ordering of test? Is it intentional?.
So what can you do to fix your problem? There are two solutions.
In your original example, you've actually only got one test (verify), but you've got 4 methods, two setup (createUser, joinUserToRoom) and one teardown (deleteUser). So your first option is to better define your test cases, using a TestRule, in particular ExternalResource. ExternalResource allows you to define before/after behaviour for a test, similar to #Before/#After. However, the advantage of ExternalResource is that you can factor this out of your test.
So, you would create/delete the user in your external resource:
public class UsesExternalResource {
#Rule
public ExternalResource resource= new ExternalResource() {
#Override
protected void before() throws Throwable {
// create user
};
#Override
protected void after() {
// destroy user
};
};
#Test
public void testJoinUserToRoom() {
// join user to room
// verify all ok
}
}
For me, this is simpler and easier to understand, and you get independent tests, which is a good thing. This is what I would do, but you will need to refactor your tests a bit. You can also stack these Rules using RuleChain.
Your second option, if you really want to introduce dependencies between your test methods, is to look at TestNG, in which you can define dependencies from one test to another.
If they have a 'correct' order, then they are not multiple tests, but a single test that you have incorrectly annotated as being multiple independent tests.
Best practise would to rewrite them in approved junit style (setup - act - verify), supported by #Before or #BeforeClass methods that did any required common setup.
Quick workaround would be to have a single #Test-annotated method that called the other test methods in sequence. That becomes something like the preferred alternative if you are using Junit not to do strict unit testing, but something more like scenario-driven systems testing. It's not necessarily the best tool for such use, but it does work perfectly well in some cases.
Then, what you would have so far is have a single test:
#Test public void testUserNominalLifeCycle(...
which could then, if you are feeling virtuous, be supplemented by extra new tests like
#Test public void testUserWhoNeverJoinsARoom(...
I have some NUnit tests which uses a TestCaseSource function. Unfortunately, the TestCaseSource function that I need takes a long time to initialize, because it scans a folder tree recursively to find all of the test images that would be passed into the test function. (Alternatively it could load from a file list XML every time it's run, but automatic discovery of new image files is still a requirement.)
Is it possible to specify an NUnit attribute together with TestCaseSource such that NUnit does not enumerate the test cases (does not call the TestCaseSource function) until either the user clicks on the node, or until the test suite is being run?
The need to get all test images stored in a folder is a project requirement because other people who do not have access to the test project will need to add new test images to the folder, without having to modify the test project's source code. They would then be able to view the test result.
Some dogmatic unit-testers may counter that I am using NUnit to do something it's not supposed to do. I have to admit that I have to meet a requirement, and NUnit is such a great tool with a great GUI that satisfies most of my requirements, such that I do not care about whether it is proper unit testing or not.
Additional info (from NUnit documentation)
Note on Object Construction
NUnit locates the test cases at the
time the tests are loaded, creates
instances of each class with
non-static sources and builds a list
of tests to be executed. Each source
object is only created once at this
time and is destroyed after all tests
are loaded.
If the data source is in the test
fixture itself, the object is created
using the appropriate constructor for
the fixture parameters provided on the
TestFixtureAttribute or the default
constructor if no parameters were
specified. Since this object is
destroyed before the tests are run, no
communication is possible between
these two phases - or between
different runs - except through the
parameters themselves.
It seems the purpose of loading the test cases up front is to avoid having communications (or side-effects) between TestCaseSource and the execution of the tests. Is this true? Is this the only reason to require test cases to be loaded up front?
Note:
A modification of NUnit was needed, as documented in http://blog.sponholtz.com/2012/02/late-binded-parameterized-tests-in.html
There are plans to introduce this option to later versions of NUnit.
I don't know of a way to delay-load test names in the GUI. My recommendation would be to move those tests to a separate assembly. That way, you can quickly run all of your other tests, and load the slower exhaustive tests only when needed.