Suppose I have a simple test like so:
public class SimpleTestFactory {
private int instanceNumber;
#Factory(dataProvider="provideTestData")
public SimpleTestFactory(int instanceNumber) {
this.instanceNumber = instanceNumber;
}
#DataProvider
public static Object[][] provideTestData() { return new Object[][] {{1},{2}}; }
#Test
public void testOne() { System.out.printf("Test 1, Instance %d%n", instanceNumber); }
}
When I run this in Eclipse, I get only one entry instead of two in the "all tests" tab; both test instances are "grouped together" under SimpleTestFactory -> testOne(). Furthermore, say that there is an if-block in testOne() that fails instance 2. Then, the results in the "all tests" tab, it may show (depending on whether instance 1 or instance 2 is first) a test passing with a stack trace or a test failing with no stack trace, as if the result of one instance is overwriting another.
How can I get it so that the two instances are shown separately? That is, I get something like SimpleTestFactory[1] -> testOne() and SimpleTestFactory[2] -> testOne()? What if I add in a second test, testTwo()?
Have your test implement org.testng.ITest and override getName() to return the name of the instance.
Related
1) Contextualization:
In order, to have a complete test-isolation-state in all test of my Test-Class;
I would like to have a new-instance-repository(DAO) for each individual test;
My Repository is a Interface, thats the why I can not simply instantiate that.
My Goal is:
Run all tests 'Parallelly', meaning 'at the same time';
That's the why, I need individual/multiple instances of Repository(DAO) in each test;
Those multiple instances will make sure that the tests' conclusion would not interfere on those that still is running.
Below is the code for the above situation:
1.1) Code:
Current working status: working, BUT with ths SAME-REPOSITORY-INSTANCE;
Current behaviour:
The tests are not stable;
SOMETIMES, they interfere in each other;
meaning, the test that finalize early, destroy the Repository Bean that still is being used, for the test that is still running.
public class ServiceTests2 extends ConfigTests {
private List<Customer> customerList;
private Flux<Customer> customerFlux;
#Lazy
#Autowired
private ICustomerRepo repo;
private ICustomerService service;
#BeforeEach
public void setUp() {
service = new CustomerService(repo);
Customer customer1 = customerWithName().create();
Customer customer2 = customerWithName().create();
customerList = Arrays.asList(customer1,customer2);
customerFlux = service.saveAll(customerList);
}
#Test
#DisplayName("Save")
public void save() {
StepVerifier.create(customerFlux)
.expectNextSequence(customerList)
.verifyComplete();
}
#Test
#DisplayName("Find: Objects")
public void find_object() {
StepVerifier
.create(customerFlux)
.expectNext(customerList.get(0))
.expectNext(customerList.get(1))
.verifyComplete();
}
}
2) The ERROR happening:
This ERROR happens in the failed-Tests:
3) Question:
How Can I create multiple instances of Repository
Even if, it being a Interface(does not allow instantation)?
In order, to have a COMPLETE TEST-ISOLATION
Meaning: ONE different instance of Repository in each test?
Thanks a lot for any help or idea
You can use the #DirtiesContext annotation on the test class that modifies the application context.
Java Doc
Spring documentation
By default, this will mark the application context as dirty after the entire test class is run. If you would like to mark the context as dirty after a single test method, then you can either annotate the test method instead or set the classMode property to AFTER_EACH_TEST_METHOD at your class level annotation.
#DirtiesContext(classMode = ClassMode.AFTER_EACH_TEST_METHOD)
When an application context is marked dirty, it is removed from the
testing framework's cache and closed; thus the underlying Spring
container is rebuilt for any subsequent test that requires a context
with the same set of resource locations.
Having a test class like this
public class VerySimpleFactory {
#TestFactory
public Stream<? extends DynamicNode> someTests() {
DynamicContainer container1 = DynamicContainer.dynamicContainer("A",
Arrays.asList(t("A1"), t("A2"), t("A3"), t("A4"), t("A5")));
DynamicContainer container2 = DynamicContainer.dynamicContainer("B",
Arrays.asList(t("B1"), t("B2"), t("B3"), t("B4"), t("B5")));
DynamicContainer container3 = DynamicContainer.dynamicContainer("C",
Arrays.asList(t("C1"), t("C2"), t("C3"), t("C4"), t("C5")));
DynamicContainer container4 = DynamicContainer.dynamicContainer("D",
Arrays.asList(t("D1"), t("D2"), t("D3"), t("D4"), t("D5")));
return Arrays.asList(container1, container2, container3, container4).stream();
}
#Test
public void t1() throws Exception {
Thread.sleep(1000);
}
#Test
public void t2() throws Exception {
Thread.sleep(1000);
}
public DynamicTest t(String name) {
return DynamicTest.dynamicTest(name, () -> Thread.sleep(1000));
}
}
the Tests having a #Test annotaiton are discovered instantly by JUnit View, but the tests from TestFactory are discoverd at runtime, each after the last test was completely executed. This leads to a changing and "jumping" JUnit view. Also I cannot select a special test I'm interested in to be executed as single test, until all previous tests were executed.
It would be much nicer if all dynamic tests were shown in JUnit view also at beginning of test execution.
If this doesn't happen, is it a problem of JUnit 5, eclipse or my code?
Dynamic tests are dynamic. Not static.
It is not possible to know before-hand which and how many tests will be generated by #TestFactory annotated method ... actually, it may produce tests in an eternal loop.
Copied from https://junit.org/junit5/docs/current/user-guide/#writing-tests-dynamic-tests-examples
generateRandomNumberOfTests() implements an Iterator that generates
random numbers, a display name generator, and a test executor and then
provides all three to DynamicTest.stream(). Although the
non-deterministic behavior of generateRandomNumberOfTests() is of
course in conflict with test repeatability and should thus be used
with care, it serves to demonstrate the expressiveness and power of
dynamic tests.
I have a test that takes in test data. When using nunit console app to run the test, is there a way I can specify the data to be used?
Eg:
[Test, TestCaseSource(typeof(TestData))]
public void ATest(string param1, int param2)
public class TestData : IEnumerable
{
public IEnumerator GetEnumerator()
{
yield return new object[] { "blah1 blah1", 1};
yield return new object[] { "blah2 blah2", 2};
}
}
I want to be able to run ATest with test data ["blah2 blah2", 2] only. If I run as follows:
nunit3-console.exe Tests.dll --test=ATest --workers=1 --noresult
it will run twice.
Just run...
nunit3-console.exe Tests.dll --test ATest("blah2 blah2", 2)
or
nunit3-console.exe Tests.dll --where "test~=blah2"
If that string is unique to all your tests.
Note that the first one may require some escaping of the quotes, depending on your operating system.
One way to do this would be through returning a TestCaseData object instead.
Something like this: (untested, so syntax might be a little off!)
[Test, TestCaseSource(typeof(TestData))]
public void ATest(string param1, int param2)
public IEnumerator GetEnumerator()
{
yield return new TestCaseData("blah1 blah1", 1).SetName("FirstTest");
yield return new TestCaseData("blah2 blah2", 2).SetName("SecondTest");
}
To run the first test, you would then use the command line:
nunit3-console.exe Tests.dll --test=YourNameSpace.ATest.FirstTest --workers=1 --noresult
Depending what you're doing, setting the category may be more suitable than the name. The docs page shows what's available: https://github.com/nunit/docs/wiki/TestCaseData
I'm using NUnit 3.0 and TestFixtureSource to run test cases inside a fixture multiple times with different parameters/configurations (I do want to do this at TestFixture level). Simple example:
[TestFixtureSource(typeof (ConfigurationProvider))]
public class Fixture
{
public Fixture(Configuration configuration)
{
_configuration = configuration;
}
private Configuration _configuration;
[Test]
public void Test()
{
//do something with _configuration
Assert.Fail();
}
}
Let's say Test() fails for one of the configurations and succeeds for another. In the run report file and in Visual Studio's Test Explorer the name for both the failed and the succeeded runs will be displayed as just Test(), which doesn't tell me anything about which setup caused issues.
Is there a way to affect the test cases names in this situation (i.e. prefix its name per fixture run/configuration)? As a workaround I'm currently printing to the results output before each test case fires but I would rather avoid doing that.
Since NUnit 3.0 is in beta and this feature is fairly new I wasn't able to find anything in the docs. I found TestCaseData but I don't think it's tailored to be used with fixtures just yet (it's designed for test cases).
I can't find a way to change the testname, but it should not be neccessary, because NUnit3 constructs the testname by including a description of the testfixture.
The example class Fixture from the question can be used unchanged if the Configuration and ConfigurationProvider has an implementation like this:
public class Configuration
{
public string Description { get; }
public Configuration(string description)
{
Description = description;
}
public override string ToString()
{
return Description;
}
}
public class ConfigurationProvider : IEnumerable
{
public IEnumerator GetEnumerator()
{
yield return new Configuration("Foo");
yield return new Configuration("Bar");
yield return new Configuration("Baz");
}
}
The 'trick' is to make sure the constructor-parameter to the fixture is a string or has a ToString-method that gives a sensible description of the fixture.
If you are using NUnit 3 Test Adapter in Visual Studio, then the testfixtures will be displayed as Fixture(Foo), Fixture(Bar) and Fixture(Baz) so you can easily distinguish between their tests. The xml-output from nunit3-console.exe also uses descriptive names, fx: fullname=MyTests.Fixture(Bar).Test
<test-case id="0-1003" name="Test" fullname="MyTests.Fixture(Bar).Test" methodname="Test" classname="MyTests.Fixture" runstate="Runnable" result="Failed" ... >
<failure>
<message><![CDATA[]]></message>
<stack-trace><![CDATA[at MyTests.Fixture.Test() in ... ]]></stack-trace>
</failure>
...
</test-case>
One way to perform such actions is to have find and replace tokens in source code and dynamically build test libraries before execution using command line msbuild. High level steps are
Define test case names as sometest_TOKEN in source then using command line tools like fnr.exe replce _TOKEN with whatever you like. For example sometest_build2145.
Compile the dll with using msbuild for example msbuild /t:REbuild mytestproj.sln. Thereafter execute all test cases in mytestproj.dll.
Is it possible to do Inter Type Declarations with AspectJ on Compiled Class Files at Load Time Weaving?
As an example: I compile some Groovy code and want to add fields or methods with IDT.
Update:
Oh my goodness, you do not need reflection to access members or execute methods. Eclipse shows errors in the editor, but you may just ignore them, the code compiles and runs fine anyway. So the aspect is really much more strightforward and simple:
public aspect LTWAspect {
public static String Application.staticField = "value of static field";
public String Application.normalField = "value of normal field";
public void Application.myMethod() {
System.out.println(normalField);
}
void around() : execution(void Application.main(..)) {
System.out.println("around before");
proceed();
System.out.println("around after");
System.out.println(Application.staticField);
new Application().myMethod();
}
}
Original answer:
Yes, but you have a hen-and-egg problem there, i.e. you cannot just reference the newly introduced fields from your LTW aspect code without reflection. (The last sentence is not true, see update above.) Plus, in order to make your LTW aspect compile, you need the classes to be woven on the project's build path so as to be able to reference them. Example:
Java project
public class Application {
public static void main(String[] args) {
System.out.println("main");
}
}
AspectJ project
import org.aspectj.lang.SoftException;
public aspect LTWAspect {
public static String Application.staticField = "value of static field";
public String Application.normalField = "value of normal field";
public void Application.myMethod() {
try {
System.out.println(Application.class.getDeclaredField("normalField").get(this));
} catch (Exception e) {
throw new SoftException(e);
}
}
void around() : execution(void Application.main(..)) {
System.out.println("around before");
proceed();
System.out.println("around after");
try {
System.out.println(Application.class.getDeclaredField("staticField").get(null));
Application.class.getDeclaredMethod("myMethod", null).invoke(new Application());
} catch (Exception e) {
throw new SoftException(e);
}
}
}
So, e.g. in Eclipse you need to put the Java project on the AspectJ project's build path under "Projects" because only then it can see Java class Application on which you want to declare members. After compilation you just start the Java project and do LTW on the aspect project (don't forget an aop-ajc.xml referencing LTWAspect).
In my example above I declare a static member, a non-static ("normal") member and a non-static method. My advice prints the static member and calls the non-static method, both via reflection. The non-static method then prints the non-static member, again via reflection. This is not nice, but it works and proves the ITD in combination with LTW is possible. There might be a more elegant way, but if so I am unaware of it. (Update: There is a more elegant way: Just ignore the errors marked by Eclipse IDE, see above.)
Program output
around before
main
around after
value of static field
value of normal field