Having a test class like this
public class VerySimpleFactory {
#TestFactory
public Stream<? extends DynamicNode> someTests() {
DynamicContainer container1 = DynamicContainer.dynamicContainer("A",
Arrays.asList(t("A1"), t("A2"), t("A3"), t("A4"), t("A5")));
DynamicContainer container2 = DynamicContainer.dynamicContainer("B",
Arrays.asList(t("B1"), t("B2"), t("B3"), t("B4"), t("B5")));
DynamicContainer container3 = DynamicContainer.dynamicContainer("C",
Arrays.asList(t("C1"), t("C2"), t("C3"), t("C4"), t("C5")));
DynamicContainer container4 = DynamicContainer.dynamicContainer("D",
Arrays.asList(t("D1"), t("D2"), t("D3"), t("D4"), t("D5")));
return Arrays.asList(container1, container2, container3, container4).stream();
}
#Test
public void t1() throws Exception {
Thread.sleep(1000);
}
#Test
public void t2() throws Exception {
Thread.sleep(1000);
}
public DynamicTest t(String name) {
return DynamicTest.dynamicTest(name, () -> Thread.sleep(1000));
}
}
the Tests having a #Test annotaiton are discovered instantly by JUnit View, but the tests from TestFactory are discoverd at runtime, each after the last test was completely executed. This leads to a changing and "jumping" JUnit view. Also I cannot select a special test I'm interested in to be executed as single test, until all previous tests were executed.
It would be much nicer if all dynamic tests were shown in JUnit view also at beginning of test execution.
If this doesn't happen, is it a problem of JUnit 5, eclipse or my code?
Dynamic tests are dynamic. Not static.
It is not possible to know before-hand which and how many tests will be generated by #TestFactory annotated method ... actually, it may produce tests in an eternal loop.
Copied from https://junit.org/junit5/docs/current/user-guide/#writing-tests-dynamic-tests-examples
generateRandomNumberOfTests() implements an Iterator that generates
random numbers, a display name generator, and a test executor and then
provides all three to DynamicTest.stream(). Although the
non-deterministic behavior of generateRandomNumberOfTests() is of
course in conflict with test repeatability and should thus be used
with care, it serves to demonstrate the expressiveness and power of
dynamic tests.
Related
1) Contextualization:
In order, to have a complete test-isolation-state in all test of my Test-Class;
I would like to have a new-instance-repository(DAO) for each individual test;
My Repository is a Interface, thats the why I can not simply instantiate that.
My Goal is:
Run all tests 'Parallelly', meaning 'at the same time';
That's the why, I need individual/multiple instances of Repository(DAO) in each test;
Those multiple instances will make sure that the tests' conclusion would not interfere on those that still is running.
Below is the code for the above situation:
1.1) Code:
Current working status: working, BUT with ths SAME-REPOSITORY-INSTANCE;
Current behaviour:
The tests are not stable;
SOMETIMES, they interfere in each other;
meaning, the test that finalize early, destroy the Repository Bean that still is being used, for the test that is still running.
public class ServiceTests2 extends ConfigTests {
private List<Customer> customerList;
private Flux<Customer> customerFlux;
#Lazy
#Autowired
private ICustomerRepo repo;
private ICustomerService service;
#BeforeEach
public void setUp() {
service = new CustomerService(repo);
Customer customer1 = customerWithName().create();
Customer customer2 = customerWithName().create();
customerList = Arrays.asList(customer1,customer2);
customerFlux = service.saveAll(customerList);
}
#Test
#DisplayName("Save")
public void save() {
StepVerifier.create(customerFlux)
.expectNextSequence(customerList)
.verifyComplete();
}
#Test
#DisplayName("Find: Objects")
public void find_object() {
StepVerifier
.create(customerFlux)
.expectNext(customerList.get(0))
.expectNext(customerList.get(1))
.verifyComplete();
}
}
2) The ERROR happening:
This ERROR happens in the failed-Tests:
3) Question:
How Can I create multiple instances of Repository
Even if, it being a Interface(does not allow instantation)?
In order, to have a COMPLETE TEST-ISOLATION
Meaning: ONE different instance of Repository in each test?
Thanks a lot for any help or idea
You can use the #DirtiesContext annotation on the test class that modifies the application context.
Java Doc
Spring documentation
By default, this will mark the application context as dirty after the entire test class is run. If you would like to mark the context as dirty after a single test method, then you can either annotate the test method instead or set the classMode property to AFTER_EACH_TEST_METHOD at your class level annotation.
#DirtiesContext(classMode = ClassMode.AFTER_EACH_TEST_METHOD)
When an application context is marked dirty, it is removed from the
testing framework's cache and closed; thus the underlying Spring
container is rebuilt for any subsequent test that requires a context
with the same set of resource locations.
I am very new to C# and nunit. Pls bear with me if this is basic and has been already been asked here.
We have a global setup,defined by [SetupFixture] class,which is expected to be run only once. The private variables are defined in it's [setup]. We wish to use the same variables in all our testfixtures,hence inheriting the testbase class in all our testfixtures.
But, while executing Testcase1, i observe that globalSetup() is called more than once. Can anyone point me the issue? sample code is as below.
namespace CTB
{
[SetupFixture]
public class Testbase
{
private byte val1;
private byte val2;
[setup]
public void globalSetup
{
val1 = 5;
val2 = 10;
}
[Teardown]
public void globalTeardown
{
//
}
}
}
namespace CTB.Testcase
{
public class TestCase : Testbase
{
[Setup]
public void Setup()
{
}
[Teardown]
public void Teardown()
{
}
[Test]
public void Testcase1()
{
byte val3 = val1 + val2; // Expect 15
}
}
}
I'm assuming that the answer to my comment is "No" and that you are using a current version of NUnit 3. Please correct me if I'm wrong. :-)
You have made the class TestBase serve two functions:
It's the base class for your TestFixture and therefore it's a TestFixture itself.
It's marked as a SetUpFixture so it also serves that function - a completely different function, by the way.
To be clear, you should never do this. It's a sort of "trick" that almost seems designed to confuse NUnit - not your intention of course. Your test fixtures should have no inheritance relationship with any SetUpFixture. Use different classes for the test fixture base and the setup fixture.
With that out of the way, here is the longer story of what is happening...
Before your tests even execute, the SetUpFixture is first "run" - in quotes because it actually does nothing. That's because it doesn't contain any methods marked with [OneTimeSetUp] or '[OneTimeTearDown]`.
NOTE: As an alternate explanation, if you are using a pretty old version of NUnit, the [SetUp] and [TearDown] methods are actually called at this point. Nnit V2 used those attributes with different meanings when encountered in a SetUpFixture versus a TestFixture.
Next your tests execute. Before each test, the inherited [SetUp] and [TearDown] methods are run. Of course, these are actually the same methods as in step 1. NUnit has been tricked into doing this!
Here is some general guidance for the future...
If you want multiple fixtures to use the same data, a base class is useful. Any public or protected fields or properties will be shared by the inheriting fixtures.
If you want to do some common setup or teardown for a group of unrelated test fixtures, use a SetUpFixture. Note that the only way to pass data from a SetUpFixture to the test fixtures is through static fields or properties. Generally, you use a SetUpFixture to set up the environment in which the test is run, not to provide data.
Never use the same class for both purposes.
Consider this Test
[TestFixture]
class Sample
{
[Test]
public void Test()
{
Thread.CurrentThread.Name = "Foo";
}
}
If I debug this test, it passes without error.
If I run this test, it fails with the following exception
System.InvalidOperationException : This property has already been set and cannot be modified.
In run mode, the test's thread's name is "NonParallelWorker".
In debug mode, the test's thread's name is null
As a constraint, assume the code-under-test is not allowed to change, and attempts to set the thread's name, without checking for null first.
E.g.
public void SampleMethodUnderTest()
{
// It is important that this method gets to set this field.
Thread.CurrentThread.Name = "Important Value";
}
My search through the documentation and other's posts has come up dry...
Question
Is there any way to disable/modify NUnit's thread-naming behavior?
Try adding the RequiresThreadAttribute.
[TestFixture]
class Sample
{
[Test, RequiresThread]
public void Test()
{
Thread.CurrentThread.Name = "Foo";
}
}
I think this will work currently, although the fact that this creates an unnamed thread may be an implementation detail, and not something that will necessarily work reliably going forward, I'm not sure. The alternative of course is to create your own user-controlled thread in the test, and pass any exceptions back to NUnit.
I'm using NUnit 3.0 and TestFixtureSource to run test cases inside a fixture multiple times with different parameters/configurations (I do want to do this at TestFixture level). Simple example:
[TestFixtureSource(typeof (ConfigurationProvider))]
public class Fixture
{
public Fixture(Configuration configuration)
{
_configuration = configuration;
}
private Configuration _configuration;
[Test]
public void Test()
{
//do something with _configuration
Assert.Fail();
}
}
Let's say Test() fails for one of the configurations and succeeds for another. In the run report file and in Visual Studio's Test Explorer the name for both the failed and the succeeded runs will be displayed as just Test(), which doesn't tell me anything about which setup caused issues.
Is there a way to affect the test cases names in this situation (i.e. prefix its name per fixture run/configuration)? As a workaround I'm currently printing to the results output before each test case fires but I would rather avoid doing that.
Since NUnit 3.0 is in beta and this feature is fairly new I wasn't able to find anything in the docs. I found TestCaseData but I don't think it's tailored to be used with fixtures just yet (it's designed for test cases).
I can't find a way to change the testname, but it should not be neccessary, because NUnit3 constructs the testname by including a description of the testfixture.
The example class Fixture from the question can be used unchanged if the Configuration and ConfigurationProvider has an implementation like this:
public class Configuration
{
public string Description { get; }
public Configuration(string description)
{
Description = description;
}
public override string ToString()
{
return Description;
}
}
public class ConfigurationProvider : IEnumerable
{
public IEnumerator GetEnumerator()
{
yield return new Configuration("Foo");
yield return new Configuration("Bar");
yield return new Configuration("Baz");
}
}
The 'trick' is to make sure the constructor-parameter to the fixture is a string or has a ToString-method that gives a sensible description of the fixture.
If you are using NUnit 3 Test Adapter in Visual Studio, then the testfixtures will be displayed as Fixture(Foo), Fixture(Bar) and Fixture(Baz) so you can easily distinguish between their tests. The xml-output from nunit3-console.exe also uses descriptive names, fx: fullname=MyTests.Fixture(Bar).Test
<test-case id="0-1003" name="Test" fullname="MyTests.Fixture(Bar).Test" methodname="Test" classname="MyTests.Fixture" runstate="Runnable" result="Failed" ... >
<failure>
<message><![CDATA[]]></message>
<stack-trace><![CDATA[at MyTests.Fixture.Test() in ... ]]></stack-trace>
</failure>
...
</test-case>
One way to perform such actions is to have find and replace tokens in source code and dynamically build test libraries before execution using command line msbuild. High level steps are
Define test case names as sometest_TOKEN in source then using command line tools like fnr.exe replce _TOKEN with whatever you like. For example sometest_build2145.
Compile the dll with using msbuild for example msbuild /t:REbuild mytestproj.sln. Thereafter execute all test cases in mytestproj.dll.
I have a nunit 2.5.10 parameterized test:
[TestFixture(parameter1)]
[TestFixture(parameter2)]
public class MyTest : BaseTest
{
var param="";
public MyTest(string arg)
{
param=arg;
}
[Test]
public Test()
{
//... test stuff
}
}
In VS 12 with Resharper 7.0.1 I run only one fixture but! the constructor is called twice - once before test execution (for one parameter) and then after test execution but for another parameter. Why? Only constructor is called twice, test itsself is running as expected - only one time.
it's a known problem, unfortunately, see RSRP-336641 "When calling certain case on Parametrized TestFixture all cases constructors and TestFixtureSetUp are called."