Can a TestActionAttribute in NUnit run BeforeTest before the fixture's own SetUp method? - nunit

I have some old MbUnit code which looks like this:
public class MyFixture {
[SetUp]
public void SetUp() {
// Add data to database
}
[Test, Rollback]
public void DoTest() {
// Tests with the data
}
}
My new NUnit Rollback attribute looks a bit like this:
public class RollbackAttribute : TestActionAttribute
{
public override void BeforeTest(TestDetails testDetails)
{
// Begin transaction
}
public override void AfterTest(TestDetails testDetails)
{
// Abort transaction
}
}
The Rollback should roll back the new data added in the SetUp method as well as any modifications during the test itself. Unfortunately, it seems that NUnit's BeforeTest runs after the fixture's SetUp method, so the data added during SetUp is not rolled back.
Is there a way to run BeforeTest before SetUp?
One option would be a base class, and replace the existing Rollback attributes with additional code in SetUp and TearDown, however some of my tests require running outside a transaction (they create multiple transactions themselves during the test run), so adding transactions around all test cases would require a bit of care. I'd rather find a solution which can re-use the existing Rollback attributes.

Is there a way to run BeforeTest before SetUp?
I don't think so, see e.g. this related discussion on google groups. Issue being discussed there is very similar, as you can see, code in SetUp method would run even prior to BeforeTest method used on test fixture level (you have it on test level).
Workaround from my point of view would be to remove the SetUpAttribute from the SetUp method and call the SetUp method explicitly at the beginning of the each test, i.e.:
public class MyFixture
{
public void SetUp()
{
// Add data to database
}
[Test, Rollback]
public void DoTest()
{
SetUp();
// Tests with the data
}
}
Your question also reminded me of question that marc_s raised in this SO thread. Question is unrelated to your problem, but he used the same construct as I am proposing above so it is perhaps not that bad idea.
EDIT:
Here is an opened issue on NUnit's github. But still, requested order there is:
BeforeTest (BaseFixture)
BaseSetUp BeforeTest (Fixture)
SetUp
BeforeTest (Test)
Test AfterTest (Test)
TearDown AfterTest (Fixture)
BaseTearDown AfterTest (BaseFixture)
So not exactly what you desire, "BeforeTest (Test)" would be executed after SetUp.

Related

Is there any alternative to [OneTimeSetup] in Nunit?

In my existing [OneTimeSetup] method, I want to check some preconditions before running any test. But I can't do so as the object which I'll be needing to check preconditions is initialized in Base class [Setup] method. I can't initialize this earlier due to some project limitations.
So, Is there any way where I can execute some code after Base [Setup] method (to check some preconditions) and before any suite execution? I want to execute this once per suite.
[SetUpFixture]
Class GlobalSetup
{
[OneTimeSetUp]
public void OneTimeSetUp(){
setup();
CheckIfDataIsPresent(); // I can't do this here as this code needs Obj O to be initialized. which will be initialized in Base class's [Setup] methed
}
}
Public class Base
{
[Setup]
public void setUp()
{
//some code where we initialize obj O;
}
}
[TestFixture]
public class Test : Base
{
// tests to be executed
}
You already did a nice job of explaining why what you want to do won't work, so I don't have to. :-)
The problem is that each your tests needs a fresh instance of that object, so you properly create it in a [SetUp] method. You would like to ensure that it's possible to create such an object once before you run any tests.
I can only give you a non-specific answer, since you haven't given a lot of info in your example code. If you update your question, I may be able to update my answer. Here goes...
Both your tests and the check you want to perform require an instance of object o. So one approach would be to initialize o one more time in the OneTimeSetup, perform the check and then throw it away. Since you are initializing o in every test, I assume it's not expensive to do so. Say you have 100 tests. You are setting up o 100 times. So make it 101 and be done!
Alternatively, determine what is required for o to be initialized successfully and check that. For example, if it needs a file to be present, check that the file is present. If the file has to have 100 records in some format, check that it's so. Perhaps you might give us more detail about what those prerequisites are.
Finally, you might reconsider whether you really need a new instance per test. Since you suggest you would be willing to make a check once per fixture (I assume that's what you mean by suite) then perhaps you really only need one instance per fixture rather than a new one for each test.

Run long-running NUnit/xUnit tests so that they are not blocking other tests

I'm running a set of integration tests and while most of them finish within reasonable timeline, there're two tests that are waiting for specific conditions (financial markets conditions to be precise) and they can last for 2-3 hours. So ideally I'd like to achieve two things:
Start those two tests after other tests are finished
Run them in parallel
Is there a way to achieve that in NUnit/XUnit (or other test runner)?
Start those two tests after other tests are finished
You could keep those two tests in a separate nunit test project, allowing you to run all the other tests separately.
For running tests in parallel, this blog has a nice article:
https://blog.sanderaernouts.com/running-unit-tests-in-parallel-with-nunit
Mark your test fixtures with the Parallelizable attribute and set the parallel scope to ParallelScope.All.
Create a private class called TestScope and implement IDisposable.
Put all startup and clean-up logic inside the TestScope constructor and .Dispose() method respectively.
Wrap your test code in a using (var scope = new TestScope) { ... } block
[TestFixture]
[Parallelizable(ParallelScope.All)]
public class MyClassTests {
[Test]
public void MyParallelTest() {
using(var scope = new TestScope()) {
scope.Sut.DoSomething();
scope.Repository.Received(1).Save();
}
}
private sealed class TestScope : IDisposable {
public IRepository Repository{get;}
public MyClass Sut {get;}
public TestScope() {
Repository = Substitute.For<IRepository>();
Sut = new MyClass(Repository);
}
public void Dispose() {
//clean-up code goes here
Repository?.Dispose()
}
}
}
You should take precautions to ensure that while running in parallel, your tests do not interfere with each other.
As the article states:
How to safely run tests in parallel
To allow tests to run in parallel
without them interfering with each other, I have been applying the
following pattern for a while:
Create a nested private TestScope class that implements IDisposable.
All initialization or startup code that would go into the SetUp method
goes into the constructor of the TestScope class.
Any clean-up or
teardown code that would go into the TearDown method goes into the
Dispose method All tests run inside a using block that handles the
creation and disposal of the TestScope.
[TestFixture]
[Parallelizable(ParallelScope.All)]
public class MyClassTests {
[Test]
public void MyParallelTest() {
using(var scope = new TestScope()) {
scope.Sut.DoSomething();
scope.Repository.Received(1).Save();
}
}
private sealed class TestScope : IDisposable {
public IRepository Repository{get;}
public MyClass Sut {get;}
public TestScope() {
Repository = Substitute.For<IRepository>();
Sut = new MyClass(Repository);
}
public void Dispose() {
//clean-up code goes here
Repository?.Dispose()
}
}
}
The article provides more valuable advice. I suggest reading it, and thanking the author.
Parallel test run depends on arguments of test runner, if you are using xUnit console test runner there is a -parallel argument or MSBuild Options, see: https://xunit.net/docs/running-tests-in-parallel. But in any case you have to split your long running tests on separate Test Classes.
It is harder to guarantee order of test running you could use TestCollection (however according to quide collection running sequentially). You could rename your long running test to place them at the end of list, i.e TestClass2 will be executed after TestClass1. You also could use category attribute parameter to separate tests and run them via 2 commands from dotnet test --filter=TestCategory=LongTests (one for long and another for others), see https://learn.microsoft.com/ru-ru/dotnet/core/testing/selective-unit-tests?pivots=mstest

Why Nunit3 OneTimeSetUp() is called after [Test] and not before

I have my unit tests written in Nunit 2.6 but planning to upgrade to Nunit 3.6.1 , however I noticed a weird problem with Nunit 3.6.1 (or may be I did not understood it correctly). The problem is around OneTimeSetUp().
In Nunit 2.6.3, I had SetUpFixtureAttribute [SetUpFixture] and inside that SetUpAttribute [SetUp] and it worked as expected for me, the flow was
SetUpFixture.Setup
TestFixture.Setup
TestFixture.Test
TestFixture.TearDown
TestFixture.Setup
TestFixture.Test
TestFixture.TearDown
SetUpFixture.TearDown
When I upgraded to Nunit 3, I replaced the SetUp() inside SetUpFixture with OneTimeSetUp, and after running my code I got following flow
TestFixture.Setup
TestFixture.Test
TestFixture.TearDown
SetUpFixture.OneTimeSetUp
SetUpFixture.OneTimeTearDown
Following is the sample code which I tried on my machine and also the command line output
[SetUpFixture]
public class TestBase
{
[OneTimeSetUp]
//[SetUp]
public static void MyTestSetup()
{
Console.WriteLine(" ---------- Calling OneTimeSetUp ----------");
}
}
[TestFixture]
class TestClass : TestBase
{
[Test]
public void test()
{
Console.WriteLine("\n ....I'm inside TestClass.test() ....");
}
}
Console Output
=> TestSample.TestClass.test
....I'm inside TestClass.test() ....
=> TestSample.TestClass
---------- Calling OneTimeSetUp ----------
=> TestSpecflow.TestBase
---------- Calling OneTimeSetUp ----------
Can someone please suggest what am i missing here ?
I'm running the test via nunit-console
The issue is that the output is misleading and not in the order the code is executed. Because NUnit 3 supports parallel execution, it captures output and displays it on the console when that level of test execution is completed.
In your case, the fixture setup wraps the tests, so it finishes executing after the tests and outputs the captured text afterward.
If you debug your tests, or switch your Console.WriteLine calls to TestContext.Progress.WriteLine which outputs immediately, you will see that the code is executed in the order you expect.
If it is not in the order you expect, look at the namespaces. Remember that a [SetupFixture] is used to setup at the namespace level. If your tests are in a different namespace, they may be called in a different order. If you want a setup for all of your tests, put the class in your top level namespace, or if you have multiple namespaces, in no namespace.
Here is some test code,
namespace NUnitFixtureSetup
{
[SetUpFixture]
public class SetupClass
{
[OneTimeSetUp]
public void MyTestSetup()
{
TestContext.Progress.WriteLine("One time setup");
}
}
[TestFixture]
public class TestClass
{
[Test]
public void TestMethod()
{
TestContext.Progress.WriteLine("Test Method");
}
}
}
And here is the output from running with nunit3-console.exe
=> NUnitFixtureSetup.SetupClass
One time setup
=> NUnitFixtureSetup.TestClass.TestMethod
Test Method
Rob's answer is the fundamental reason you have a problem, but there is an additional one, which is present in your code although not in Rob's.
In your code, you are using TestBase twice: as a SetUpFixture and as the base class for your TestFixture.
That means the OneTimeSetUp method will be used twice... Once before all the fixtures in the namespace and once before any test fixture that inherits from it. Using a SetUpFixture in this way defeats it's purpose, which is to have some code that runs only once before all the fixtures in a namespace.
Use separate classes as a base class (if you need one) and as a setup fixture (if you need one of those).

Making sure JPA commits my transaction before another service accesses the data

It feels so simple:
I have a ViewScoped bean (JPA2 + EE6 + Seam3, if that matters) where the user of the web application can invoke a method like this:
public void save() {
doEntityManagerStuff(); // manipulates data in the database
callRemoteWebservice(); // which is to read said data and propagate it to other systems
}
Unfortunately, save() starts a transaction at the opening curly bracket and doesn't commit it before the closing bracket, meaning that the new data is not available to the remote web service to read.
I have tried to explicitly extract and annotate the database work:
#TransactionAttribute(REQUIRES_NEW)
private void doEntityManagerStuff() {
blabla(); // database stuff
}
But that didn't have any impact at all. (Maybe because that's EJB stuff and I'm running on seam...?)
The only thing that worked for me so far was to inject #UserTransaction and force commit the transaction at the end of either save() or doEntityManagerStuff() but that felt incredibly dirty and dangerous.
The other alternative would be to turn off container-managed transactions for the entire project, but that means I'd have to make all my beans manage their transactions manually, just so I can make this one case work.
Is there a better way?
To answer my own question:
I only went half-way, and that's why it didn't work. I didn't know enough about EJBs and their boudaries, and naively though just annotating the doEntityManagerStuff(...) method with a transaction attribute in my view-scoped CDI/Seam bean would be enough.
It isn't.
When I moved said method into a separate, stateless EJB, injected that into my CDI/Seam bean and called it from there, everything worked as expected.
#Stateless
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public class MyPersister {
...
public void doEntityManagerStuff() {
blabla(); // database stuff
}
...
}
and
#ViewScoped
public class MyWebsiteBean {
...
#Inject MyPersister persister;
...
public void save() {
persister.doEntityManagerStuff(); //uses its own transaction
callRemoteWebService();
}
...
}

Integration Testing Entity Framework CRD operations

I'm struggling with Integration Tests when using Entity Framework.
I seed my database with Test data in my Test project, but I am wondering how you manange to test the Create, Update and Delete operations.
Basicly I have my Test data which e.g. contains 5 customer entries... I can now write some unit tests to get the data based on these 5 entries. (e.g. get all will return a collection containing 5 items).
But what if I have a test which deletes 1 customer, this means the GetAll test will expect 5 customers, but only return 4 (if this test is executed after the delete test) and fails.
How do you work around this issue? Do you give a certain order to your tests or reseed the database before every test (but this sounds so bad?)...
Thanks !
An effective way to do this is to use Transaction Scope. This basically wraps all sql calls and rolls back changes if the scope is disposed without calling the Complete method. The basic test will look like this.
public class TransactionalTestsBase
{
private TransactionScope _scope;
[TestInitialize]
public void Initialize()
{
_scope = new TransactionScope();
}
[TestCleanup]
public void TestCleanup()
{
_scope.Dispose();
}
[TestMethod]
public void CrudAction()
{
var repo = new YourRepo();
var client = ; // Make client
repo.DeleteClient(client);
Assert.AreEqual(4,repo.GetClients().Count());
}
}
Ideally you would inherit from this base test class not write your tests in it.
There is some new heat in beta still that I think will help with this greatly in the future. Have a look at Effort