I'm currently creating a unit custom JUnit runner (which will precisely call custom code before/after each test method) e.g.
class MyRunner extends BlockJUnit4ClassRunner {
private MyApi api = new MyApi();
public MyRunner(Class<?> klass) throws InitializationError {
super(klass);
}
// todo
}
However, I would like to support other runners e.g. MockitoJunitRunner and SpringRunner, so instead of reinventing the wheel, I'd like to use my runner like the following (using a custom MyConfig annotation to specify existing test runners): -
#RunWith(MyRunner.class)
#MyConfig(testRunner=MockitoJUnitRunner.class)
public class MockitoRunnerTest {
}
... or ...
#RunWith(MyRunner.class)
#MyConfig(testRunner=SpringRunner.class)
public class MockitoRunnerTest {
}
This means the test runner is very light i.e. it's like a Junit rule and simply proxies to another existing Junit runner after calling it's own code.
My gut feeling is that this has already be implemented/solved - just having problems finding it.
NOTE: I want to avoid using rules due to these problems - see Apply '#Rule' after each '#Test' and before each '#After' in JUnit
Related
I'm running a set of integration tests and while most of them finish within reasonable timeline, there're two tests that are waiting for specific conditions (financial markets conditions to be precise) and they can last for 2-3 hours. So ideally I'd like to achieve two things:
Start those two tests after other tests are finished
Run them in parallel
Is there a way to achieve that in NUnit/XUnit (or other test runner)?
Start those two tests after other tests are finished
You could keep those two tests in a separate nunit test project, allowing you to run all the other tests separately.
For running tests in parallel, this blog has a nice article:
https://blog.sanderaernouts.com/running-unit-tests-in-parallel-with-nunit
Mark your test fixtures with the Parallelizable attribute and set the parallel scope to ParallelScope.All.
Create a private class called TestScope and implement IDisposable.
Put all startup and clean-up logic inside the TestScope constructor and .Dispose() method respectively.
Wrap your test code in a using (var scope = new TestScope) { ... } block
[TestFixture]
[Parallelizable(ParallelScope.All)]
public class MyClassTests {
[Test]
public void MyParallelTest() {
using(var scope = new TestScope()) {
scope.Sut.DoSomething();
scope.Repository.Received(1).Save();
}
}
private sealed class TestScope : IDisposable {
public IRepository Repository{get;}
public MyClass Sut {get;}
public TestScope() {
Repository = Substitute.For<IRepository>();
Sut = new MyClass(Repository);
}
public void Dispose() {
//clean-up code goes here
Repository?.Dispose()
}
}
}
You should take precautions to ensure that while running in parallel, your tests do not interfere with each other.
As the article states:
How to safely run tests in parallel
To allow tests to run in parallel
without them interfering with each other, I have been applying the
following pattern for a while:
Create a nested private TestScope class that implements IDisposable.
All initialization or startup code that would go into the SetUp method
goes into the constructor of the TestScope class.
Any clean-up or
teardown code that would go into the TearDown method goes into the
Dispose method All tests run inside a using block that handles the
creation and disposal of the TestScope.
[TestFixture]
[Parallelizable(ParallelScope.All)]
public class MyClassTests {
[Test]
public void MyParallelTest() {
using(var scope = new TestScope()) {
scope.Sut.DoSomething();
scope.Repository.Received(1).Save();
}
}
private sealed class TestScope : IDisposable {
public IRepository Repository{get;}
public MyClass Sut {get;}
public TestScope() {
Repository = Substitute.For<IRepository>();
Sut = new MyClass(Repository);
}
public void Dispose() {
//clean-up code goes here
Repository?.Dispose()
}
}
}
The article provides more valuable advice. I suggest reading it, and thanking the author.
Parallel test run depends on arguments of test runner, if you are using xUnit console test runner there is a -parallel argument or MSBuild Options, see: https://xunit.net/docs/running-tests-in-parallel. But in any case you have to split your long running tests on separate Test Classes.
It is harder to guarantee order of test running you could use TestCollection (however according to quide collection running sequentially). You could rename your long running test to place them at the end of list, i.e TestClass2 will be executed after TestClass1. You also could use category attribute parameter to separate tests and run them via 2 commands from dotnet test --filter=TestCategory=LongTests (one for long and another for others), see https://learn.microsoft.com/ru-ru/dotnet/core/testing/selective-unit-tests?pivots=mstest
I try to start unit testing a mid size Xtext project.
The generator currently relies on some external resources that I would like to mock inside my test. Thus, I inject the needed object via #Inject into the Generator class.
e.g in pseudocode:
class MyGenerator implements IGenerator{
#Inject
ExternalResourceInterface resourceInterface;
...
}
I create the actual binding inside the languages RuntimeModule:
class MyRuntimeModule{
...
#Override
public void configure(Binder binder) {
super.configure(binder);
binder.bind(ExternalResourceInterface .class).to(ExternalResourceProductionAcess.class);
}
...
}
This works fine for the production environment.
However, in the generator test case, I would like to replace the binding with my mocked version, so that the following call to the CompilationTestHelper uses the mock:
compiler.assertCompilesTo(dsl, expectedJava);
Question:
Where do I tell guice/Xtext to bind the injection to the mock?
If you annotate your test case with RunWith and InjectWith, your test class will be injected via a specific IInjectorProvider implementation.
If that injector provider uses a custom module (like you have shown), the test case gets injected using that configuration. However, you have to make sure you use this injector throughout the test code (e.g. you do not rely on a registered injector, etc.).
Look for the following code as an example (have not compiled it, but this is the base structure you have to follow):
#RunWith(typeof(XtextRunner))
#InjectWith(typeof(LanguageInjectorProvider))
public class TestClass {
#Inject
CompilationTestHelper compiler
...
}
I've got an odd question for which Google has proven barren:
I've got a project in .net with ~20 classes that all have tests in them. One of the classes has common test setup code, although a few of the classes have their own TestFixtureSetup that looks exactly like the common class (not my architecture choice - this predates my employment). I have my own test class for which I have some different code that runs prior to running a few particular tests within the class.
Some more info that's relevant: The custom setup code that I have enables data to be available for a few combinatorial tests I have in my own test class. As the value source for the combinatorial params, the List that is returned first initializes some data.
Alright, here's the question: When I try to run a test in ANOTHER test class, it's "building" the tests from every other class. In my case, it's building the combinatorial test that I have - and thus, triggering the custom setup method that I have.
How do I prevent NUnit from building tests in other classes? As in, I run a test in one class, all I'd like NUnit to do is build tests from that class ONLY.
I tried to remove any NDA-no-no language, but here's the combinatorial I have:
[Test, Combinatorial, Category("Regressive")]
public void Test05_CombiTestExample(
[ValueSource("ListA")] User user,
[ValueSource("ListB")] KeyValuePair<string, string> searchKvp,
[ValueSource("ListC")] string scope)
{
And here's one of the lists that is being reference:
public IEnumerable<KeyValuePair<string, string>> ListB
{
get
{
InitCustomData();
if ([Redacted] != null)
{
return new Dictionary<string, string>()
{
[Redacted]
};
}
return null;
}
}
The line in question is "InitCustomData();" which, because my combinatorial is being built prior to running any setup or anything, is being executed anyway. I want this to stay here - I just don't want NUnit to start building test cases from any other class besides the one it's currently running a test in.
I'm new to the test automation scene so forgive me if this is a stupid question but google has failed me this time. Or at least anything I've read has just confused me further.
I'm using JUnit 4 and Selenium Webdriver within Eclipse. I have several tests that I need to run as a suite and also individually. At the moment these tests run fine when run on their own. At the start of the test an input box is presented to the tester/user asking first what server they wish to test on (this is a string variable which becomes part of a URL) and what browser they wish to test against. At the moment when running the tests in a suite the user is asked this at the beginning of each test, because obviously this is coded into each of their #Before methods.
How do I take in these values once, and pass them to each of the test methods?
So if server = "server1" and browser = "firefox" then firefox is the browser I want selenium to use and the URL I want it to open is http://server1.blah.com/ for all of the following test methods. The reason I've been using seperate #Before methods is because the required URL is slightly different for each test method. i.e each method tests a different page, such as server1.blah.com/something and server1.blah.com/somethingElse
The tests run fine, I just don't want to keep inputting the values because the number of test methods will eventually be quiet large.
I could also convert my tests to testNG if there is an easier way of doing this in testNG. I thought the #BeforeSuite annotation might work but now I'm not sure.
Any suggestions and criticism (the constructive kind) are much appreciated
You can adapt the solution for setting a global variable for a suite in this answer to JUnit 4 Test invocation.
Basically, you extend Suite to create MySuite. This creates a static variable/method which is accessible from your tests. Then, in your tests, you check the value of this variable. If it's set, you use the value. If not, then you get it. This allows you to run a single test and a suite of tests, but you'll only ask the user once.
So, your suite will look like:
public class MySuite extends Suite {
public static String url;
/**
* Called reflectively on classes annotated with <code>#RunWith(Suite.class)</code>
*
* #param klass the root class
* #param builder builds runners for classes in the suite
* #throws InitializationError
*/
public MySuite(Class<?> klass, RunnerBuilder builder) throws InitializationError {
this(builder, klass, getAnnotatedClasses(klass));
// put your global setup here
MySuite.url = getUrlFromUser();
}
}
This would be used in your Suite like so:
#RunWith(MySuite.class)
#SuiteClasses({FooTest.class, BarTest.class, BazTest.class});
Then, in your test classes, you can either do something in the #Before/#After, or better look at TestRule, or if you want Before and After behaviour, look at ExternalResource. ExternalResource looks like this:
public static class FooTest {
private String url;
#Rule
public ExternalResource resource= new ExternalResource() {
#Override
protected void before() throws Throwable {
url = (MySuite.url != null) ? MySuite.url : getUrlFromUser();
};
#Override
protected void after() {
// if necessary
};
};
#Test
public void testFoo() {
// something which uses resource.url
}
}
You can of course externalize the ExternalResource class, and use it from multiple Test Cases.
I think the main functionality of TestNG that will be useful here is not just #BeforeSuite but #DataProviders, which make it trivial to run the same test with a different set of values (and won't require you to use statics, which always become a liability down the road).
You might also be interested in TestNG's scripting support, which makes it trivial to ask the user for some input before the tests start, here is an example of what you can do with BeanShell.
It might make sense to group tests so that the test suite will have the same #Before method code, so you have a test suite for each separate.
Another option might be to use the same base url for each test but navigate to the specific page by getting selenium to click through to where you want to carry out the test.
If using #RunWith(Suite.class), you can add static methods with #BeforeClass (and #AfterClass), which will run before (and after) the entire Suite you define. See this question.
This of course won't help if you are referring to the entire set of classes found dynamically, and are not using Suite runner.
Using Eclipse (Helios), I could create a JUnit test file ClassATest.java of the source file ClassA.java by using New -> JUnit Test Case -> Class under test..., then choose all the methods of ClassA to be tested.
If later we add some more methods to ClassA, how do we easily reflect this addition in ClassATest ? (No copy/paste plz).
One solution is to use MoreUnit
With MoreUnit installed to Eclipse, one can right click onto the newly added method (and not yet unit tested), and choose "Generate Test"
Of course, if one always follows the writing-test-before-writing-method style, then this solution is not needed. However in reality sometimes you don't have a clear idea of what you would want to do, in that case you would have to code up some method, play with it, then rethink and code again until you are satisfied with the code and want to make it stable by adding unit test.
You should look into creating a JUnit test suite which will execute all tests within the classes you specify. Thus, adding new test cases is as simple as creating a new class and adding it to the #Suite.SuiteClasses list (as seen below).
Here's an example.
Example JUnit Test Suite Class:
#RunWith(Suite.class)
#Suite.SuiteClasses({
TestClassFoo.class
})
public class ExampleTestSuite {}
Example Test Case class:
public class TestClassFoo {
#Test
public void testFirstTestCase() {
// code up test case
}
}