We use NUnit to execute integration tests. These tests are very time consuming. Often the only way to detect a failure is on a timeout.
I would like the tests to stop executing as soon as a single failure is detected.
Is there a way to do this?
Using nunit-console, you can achieve this by using the /stoponerror command line parameter.
See here for the command line reference.
For nunit-console v3, it changes to --stoponerror (see here for the command line reference).
I'm using NUnit 3 and the following code works for me.
public class SomeTests {
private bool stop;
[SetUp]
public void SetUp()
{
if (stop)
{
Assert.Inconclusive("Previous test failed");
}
}
[TearDown]
public void TearDown()
{
if (TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Failed)
{
stop = true;
}
}
}
Alternatively you could make this an abstract class and derive from it.
This is probably not the ideal solution, but it does what you require i.e. Ignore remaining tests if a test has failed.
[TestFixture]
public class MyTests
{
[Test]
public void Test1()
{
Ascertain(() => Assert.AreEqual(0, 1));
}
[Test]
public void Test2()
{
Ascertain(() => Assert.AreEqual(1, 1));
}
private static void Ascertain( Action condition )
{
try
{
condition.Invoke();
}
catch (AssertionException ex)
{
Thread.CurrentThread.Abort();
}
}
}
Since TestFixtureAttribute is inheritable, so you could potentially create a base class with this attribute decorated on it and have the Ascertain protected Method in it and derive all TestFixture classes from it.
The only downside being, you'll have to refactor all your existing Assertions.
Related
The NUnit docs describe how to use SetUpFixture so a chunk of code will execute only once. I'm failing to get this working.
I have read Is it possible to have a [OneTimeSetup] for ALL tests? and https://docs.nunit.org/articles/nunit/writing-tests/attributes/setupfixture.html
I have the following structure for my tests
The actual tests (note the inheritence)
namespace net.UiTests
{
[TestFixture]
[NonParallelizable]
internal class SanityTests : TestBase
{
[SetUp]
public void Initialize()
{
//do stuff
}
[TearDown]
public void TestCleanUp()
{
//do stuff
}
[Test]
public void SomeTest()
{
//do stuff
}
}
}
And the base class, which is where the SetUpFixture lives
namespace net.UiTests
{
[NonParallelizable]
[SetUpFixture]
internal class TestBase
{
[OneTimeSetUp]
public static void AssemblyInit()
{
//do stuff
}
[OneTimeTearDown]
public static void AssemblyCleanup()
{
//do stuff
}
}
}
I am unsure why the AssemblyInit() method executes multiple times. The stack trace gives me no clues other than [External Code]
You have made your base class a SetUpFixture, which is causing the problem.
The SetUpFixture runs once before and once after all the fixtures in your UiTests namespace.
But due to inheritance, each TestFixture now "contains" a OneTimeSetUp and a OneTimeTearDown method. Those methods, when found in a TestFixture, are supposed to run once per fixture class and that's what they do.
To solve the problem, you have to stop using the SetUpFixture as a base class.
I'm new to Junit.
If I execute something in the #before part, and my code fails. e.g. I want to make a server connection, but it fails for some reason.
Will Junit execute my Test in the #Test section even if the #before fails? Because then my test would fail because I haven't a server connection. How can I handle it if its the case?
You can try this out with a basic junit.
public class SamplePlayTest {
#Before
public void setup() {
System.out.println("Before called");
throw new RuntimeException();
}
#Test
public void test() {
System.out.println("Test case called");
}
}
This gives output as
Before called
And fails with error java.lang.RuntimeException. Test case called is not printed.
I have run into issues where after #Around skips the method execution and returns a value right away, it also skips rest of Aspect advices. See my example code:
public class MyService {
...
#AnnotationAround
#AnnotationAfterReturning
public MyResult get() {
...
}
...
}
#Aspect
public class AroundInjector {
#Pointcut(value="execution(* *(..))")
public void anyPublicMethod() {
}
#Around("anyPublicMethod() && #annotation(around)")
public Object aroundThis(ProceedingJoinPoint pjp, AnnotationRound around) throws Throwable {
return new MyResult();
}
}
#Aspect
public class AfterReturningInjector {
#Pointcut(value="execution(* *(..))")
public void anyPublicMethod() {
}
#AfterReturning(pointcut="#annotation(afterReturn)", returning="result")
public void permissionCheckOwnership(JoinPoint jp, AnnotationAfterReturning afterReturn, Object result) throws Throwable {
System.out.println(">>>>>>>>>>>>>> never here");
}
}
So, after #Around advice returns directly, the #AfterReturning is never executed. Please help!
Your method never returns anything because its execution is prevented by the around advice, so the advice for its return joinpoint is not executed.
To solve this either call the method inside your around advice (and take care to have the correct aspect precedence), or make its content part of the around advice.
I've used NUnit with VS2008, and now am adapting to MSTest on VS2010. I used to be able to create an object in TestSetup() and dispose of it in TestCleanup(), and have the object created each time a test method was run in NUnit, preventing me from duplicating the code in each test method.
Is this not possible with MSTest? The examples I am finding using the ClassInitialize and ClassCleanup and TestInitialize and TestCleanup attributes only show how to write to the console. None show any more detailed use of these attributes.
Here is a simple example using TestInitialize and TestCleanup.
[TestClass]
public class UnitTest1
{
private NorthwindEntities context;
[TestInitialize]
public void TestInitialize()
{
this.context = new NorthwindEntities();
}
[TestMethod]
public void TestMethod1()
{
Assert.AreEqual(92, this.context.Customers.Count());
}
[TestCleanup]
public void TestCleanup()
{
this.context.Dispose();
}
}
I wish to launch the GUI application 2 times from Java test. How should we use #annotation in this case?
public class Toto {
#BeforeClass
public static void setupOnce() {
final Thread thread = new Thread() {
public void run() {
//launch appli
}
};
try {
thread.start();
} catch (Exception ex) { }
}
}
public class Test extends toto {
#Test
public void test() {
setuptonce();
closeAppli();
}
#test
public void test2()
{
setuptonce();
}
}
To launch it a second time, which annotation should I use? #afterclass?
Method annotated with #BeforeClass means that it is run once before any of the test methods are run in the test class. Method annotated with #Before is run once before every test method in the class. The counterparts for these are #AfterClass and #After.
Probably you are aiming for something like the following.
#BeforeClass
public static void setUpClass() {
// Initialize stuff once for ALL tests (run once)
}
#Before
public void setUp() {
// Initialize stuff before every test (this is run twice in this example)
}
#Test
public void test1() { /* Do assertions etc. */ }
#Test
public void test2() { /* Do assertions etc. */ }
#AfterClass
public static void tearDownClass() {
// Do something after ALL tests have been run (run once)
}
#After
public void tearDown() {
// Do something after each test (run twice in this example)
}
You don't need to explicitly call the #BeforeClass method in your test methods, JUnit does that for you.
The #BeforeClass annotation is used to run something once, before test actually runs.
So, depending on what do you want to get (and why), you can simply wrap launch code in a cycle, move launch code in other method and call it from somewhere else or write separate test case.