I've been testing my business logic in ServiceStack 3.9.38 project, and faced a problem when running unit tests separatly leads to success, and running tests all together leads to fail of one of them. After several hours I've made a reproducible unit test. If you run the whole fixture, the second test will fail. Running tests separatly makes them pass.
using Funq;
using NUnit.Framework;
[TestFixture]
public class Test
{
interface IBar {}
class Bar : IBar {}
class TestFoo { public IBar Bar { get; set; } }
[Test]
public void Test1()
{
var container = new Container();
var m = new TestFoo();
container.Register<IBar>(new Bar());
Assert.NotNull(container.Resolve<IBar>(), "Resolve");
container.AutoWire(m);
Assert.NotNull(m.Bar, "Autowire");
}
[Test]
public void Test2()
{
var container = new Container();
var m = new TestFoo();
container.AutoWire(m);
Assert.Throws<ResolutionException>(() => container.Resolve<IBar>());
Assert.IsNull(m.Bar); // FAILS HERE
}
}
Is that an issue of Funq.Container configuration? Or this is a bug? Any workarounds?
EDIT: I've posted an issue on GitHub: https://github.com/ServiceStack/ServiceStack/issues/521
There is a private static dictionary autoWireCache at AutoWireHelpers.cs that is caching your resolutions When you run the test them twice the value is pulled from the cache and your test fails. I believe the caching is a feature of ServiceStack's customized funq for performance gains.
There is no public interface for clearing that cache so I see no quick solution to the way you have the tests setup.
Related
1) Contextualization:
In order, to have a complete test-isolation-state in all test of my Test-Class;
I would like to have a new-instance-repository(DAO) for each individual test;
My Repository is a Interface, thats the why I can not simply instantiate that.
My Goal is:
Run all tests 'Parallelly', meaning 'at the same time';
That's the why, I need individual/multiple instances of Repository(DAO) in each test;
Those multiple instances will make sure that the tests' conclusion would not interfere on those that still is running.
Below is the code for the above situation:
1.1) Code:
Current working status: working, BUT with ths SAME-REPOSITORY-INSTANCE;
Current behaviour:
The tests are not stable;
SOMETIMES, they interfere in each other;
meaning, the test that finalize early, destroy the Repository Bean that still is being used, for the test that is still running.
public class ServiceTests2 extends ConfigTests {
private List<Customer> customerList;
private Flux<Customer> customerFlux;
#Lazy
#Autowired
private ICustomerRepo repo;
private ICustomerService service;
#BeforeEach
public void setUp() {
service = new CustomerService(repo);
Customer customer1 = customerWithName().create();
Customer customer2 = customerWithName().create();
customerList = Arrays.asList(customer1,customer2);
customerFlux = service.saveAll(customerList);
}
#Test
#DisplayName("Save")
public void save() {
StepVerifier.create(customerFlux)
.expectNextSequence(customerList)
.verifyComplete();
}
#Test
#DisplayName("Find: Objects")
public void find_object() {
StepVerifier
.create(customerFlux)
.expectNext(customerList.get(0))
.expectNext(customerList.get(1))
.verifyComplete();
}
}
2) The ERROR happening:
This ERROR happens in the failed-Tests:
3) Question:
How Can I create multiple instances of Repository
Even if, it being a Interface(does not allow instantation)?
In order, to have a COMPLETE TEST-ISOLATION
Meaning: ONE different instance of Repository in each test?
Thanks a lot for any help or idea
You can use the #DirtiesContext annotation on the test class that modifies the application context.
Java Doc
Spring documentation
By default, this will mark the application context as dirty after the entire test class is run. If you would like to mark the context as dirty after a single test method, then you can either annotate the test method instead or set the classMode property to AFTER_EACH_TEST_METHOD at your class level annotation.
#DirtiesContext(classMode = ClassMode.AFTER_EACH_TEST_METHOD)
When an application context is marked dirty, it is removed from the
testing framework's cache and closed; thus the underlying Spring
container is rebuilt for any subsequent test that requires a context
with the same set of resource locations.
I am very new to C# and nunit. Pls bear with me if this is basic and has been already been asked here.
We have a global setup,defined by [SetupFixture] class,which is expected to be run only once. The private variables are defined in it's [setup]. We wish to use the same variables in all our testfixtures,hence inheriting the testbase class in all our testfixtures.
But, while executing Testcase1, i observe that globalSetup() is called more than once. Can anyone point me the issue? sample code is as below.
namespace CTB
{
[SetupFixture]
public class Testbase
{
private byte val1;
private byte val2;
[setup]
public void globalSetup
{
val1 = 5;
val2 = 10;
}
[Teardown]
public void globalTeardown
{
//
}
}
}
namespace CTB.Testcase
{
public class TestCase : Testbase
{
[Setup]
public void Setup()
{
}
[Teardown]
public void Teardown()
{
}
[Test]
public void Testcase1()
{
byte val3 = val1 + val2; // Expect 15
}
}
}
I'm assuming that the answer to my comment is "No" and that you are using a current version of NUnit 3. Please correct me if I'm wrong. :-)
You have made the class TestBase serve two functions:
It's the base class for your TestFixture and therefore it's a TestFixture itself.
It's marked as a SetUpFixture so it also serves that function - a completely different function, by the way.
To be clear, you should never do this. It's a sort of "trick" that almost seems designed to confuse NUnit - not your intention of course. Your test fixtures should have no inheritance relationship with any SetUpFixture. Use different classes for the test fixture base and the setup fixture.
With that out of the way, here is the longer story of what is happening...
Before your tests even execute, the SetUpFixture is first "run" - in quotes because it actually does nothing. That's because it doesn't contain any methods marked with [OneTimeSetUp] or '[OneTimeTearDown]`.
NOTE: As an alternate explanation, if you are using a pretty old version of NUnit, the [SetUp] and [TearDown] methods are actually called at this point. Nnit V2 used those attributes with different meanings when encountered in a SetUpFixture versus a TestFixture.
Next your tests execute. Before each test, the inherited [SetUp] and [TearDown] methods are run. Of course, these are actually the same methods as in step 1. NUnit has been tricked into doing this!
Here is some general guidance for the future...
If you want multiple fixtures to use the same data, a base class is useful. Any public or protected fields or properties will be shared by the inheriting fixtures.
If you want to do some common setup or teardown for a group of unrelated test fixtures, use a SetUpFixture. Note that the only way to pass data from a SetUpFixture to the test fixtures is through static fields or properties. Generally, you use a SetUpFixture to set up the environment in which the test is run, not to provide data.
Never use the same class for both purposes.
Having a test class like this
public class VerySimpleFactory {
#TestFactory
public Stream<? extends DynamicNode> someTests() {
DynamicContainer container1 = DynamicContainer.dynamicContainer("A",
Arrays.asList(t("A1"), t("A2"), t("A3"), t("A4"), t("A5")));
DynamicContainer container2 = DynamicContainer.dynamicContainer("B",
Arrays.asList(t("B1"), t("B2"), t("B3"), t("B4"), t("B5")));
DynamicContainer container3 = DynamicContainer.dynamicContainer("C",
Arrays.asList(t("C1"), t("C2"), t("C3"), t("C4"), t("C5")));
DynamicContainer container4 = DynamicContainer.dynamicContainer("D",
Arrays.asList(t("D1"), t("D2"), t("D3"), t("D4"), t("D5")));
return Arrays.asList(container1, container2, container3, container4).stream();
}
#Test
public void t1() throws Exception {
Thread.sleep(1000);
}
#Test
public void t2() throws Exception {
Thread.sleep(1000);
}
public DynamicTest t(String name) {
return DynamicTest.dynamicTest(name, () -> Thread.sleep(1000));
}
}
the Tests having a #Test annotaiton are discovered instantly by JUnit View, but the tests from TestFactory are discoverd at runtime, each after the last test was completely executed. This leads to a changing and "jumping" JUnit view. Also I cannot select a special test I'm interested in to be executed as single test, until all previous tests were executed.
It would be much nicer if all dynamic tests were shown in JUnit view also at beginning of test execution.
If this doesn't happen, is it a problem of JUnit 5, eclipse or my code?
Dynamic tests are dynamic. Not static.
It is not possible to know before-hand which and how many tests will be generated by #TestFactory annotated method ... actually, it may produce tests in an eternal loop.
Copied from https://junit.org/junit5/docs/current/user-guide/#writing-tests-dynamic-tests-examples
generateRandomNumberOfTests() implements an Iterator that generates
random numbers, a display name generator, and a test executor and then
provides all three to DynamicTest.stream(). Although the
non-deterministic behavior of generateRandomNumberOfTests() is of
course in conflict with test repeatability and should thus be used
with care, it serves to demonstrate the expressiveness and power of
dynamic tests.
I have a bunch of test where I use the new UseInMemory function in EF7. When I run them all some of them fail. When I run them single they all pass.
My best guess it is a conflict in EF7 because of the fact that every test runs in its own thread and they all kind of using the same DbContext class.
Here one of my Tests:
[Fact]
public void Index()
{
DbContextOptionsBuilder<DatabaseContext> optionsBuilder = new DbContextOptionsBuilder<DatabaseContext>();
optionsBuilder.UseInMemoryDatabase();
db = new DatabaseContext(optionsBuilder.Options);
AdminController controller = new AdminController(db);
var result = controller.Index() as ViewResult;
Assert.Equal("Index", result.ViewName);
}
I remake the dbContext object in every test but it seem not to make any different.
Would be greatful for any input. Thanks :)
The problem is, that the memory storage in InMemoryDatabase is registered as Singleton so you actually share the data between DbContexts even you think you don't.
You have to create your DbContexts like this:
public abstract class UnitTestsBase
{
protected static T GetNewDbContext<T>() where T : DbContext
{
var services = new ServiceCollection();
services
.AddEntityFramework()
.AddInMemoryDatabase()
.AddDbContext<T>(options => options.UseInMemoryDatabase());
var serviceProvider = services.BuildServiceProvider();
var dbContext = serviceProvider.GetRequiredService<T>();
dbContext.Database.EnsureDeleted();
return dbContext;
}
}
var newTestDbContext = GetNewDbContext<TestDbContext>()
I also was led to beleive that .UseInMemoryDatabase() has no persistence, but that does not seem to be the case (at least with the latest versions)!
As noted in How can I reset an EF7 InMemory provider between unit tests? you want to do a db.Database.EnsureDeleted() BUT I also noticed that this does NOT reset auto increment ids.
This is more a solution / work around than an actual question. I'm posting it here since I couldn't find this solution on stack overflow or indeed after a lot of Googling.
The Problem:
I have an MVC 3 webapp using EF 4 code first that I want to write unit tests for. I'm also using NCrunch to run the unit tests on the fly as I code, so I'd like to avoid backing onto an actual database here.
Other Solutions:
IDataContext
I've found this the most accepted way to create an in memory datacontext. It effectively involves writing an interface IMyDataContext for your MyDataContext and then using the interface in all your controllers. An example of doing this is here.
This is the route I went with initially and I even went as far as writing a T4 template to extract IMyDataContext from MyDataContext since I don't like having to maintain duplicate dependent code.
However I quickly discovered that some Linq statements fail in production when using IMyDataContext instead of MyDataContext. Specifically queries like this throw a NotSupportedException
var siteList = from iSite in MyDataContext.Sites
let iMaxPageImpression = (from iPage in MyDataContext.Pages where iSite.SiteId == iPage.SiteId select iPage.AvgMonthlyImpressions).Max()
select new { Site = iSite, MaxImpressions = iMaxPageImpression };
My Solution
This was actually quite simple. I simply created a MyInMemoryDataContext subclass to MyDataContext and overrode all the IDbSet<..> properties as below:
public class InMemoryDataContext : MyDataContext, IObjectContextAdapter
{
/// <summary>Whether SaveChanges() was called on the DataContext</summary>
public bool SaveChangesWasCalled { get; private set; }
public InMemoryDataContext()
{
InitializeDataContextProperties();
SaveChangesWasCalled = false;
}
/// <summary>
/// Initialize all MyDataContext properties with appropriate container types
/// </summary>
private void InitializeDataContextProperties()
{
Type myType = GetType().BaseType; // We have to do this since private Property.Set methods are not accessible through GetType()
// ** Initialize all IDbSet<T> properties with CollectionDbSet<T> instances
var DbSets = myType.GetProperties().Where(x => x.PropertyType.IsGenericType && x.PropertyType.GetGenericTypeDefinition() == typeof(IDbSet<>)).ToList();
foreach (var iDbSetProperty in DbSets)
{
var concreteCollectionType = typeof(CollectionDbSet<>).MakeGenericType(iDbSetProperty.PropertyType.GetGenericArguments());
var collectionInstance = Activator.CreateInstance(concreteCollectionType);
iDbSetProperty.SetValue(this, collectionInstance,null);
}
}
ObjectContext IObjectContextAdapter.ObjectContext
{
get { return null; }
}
public override int SaveChanges()
{
SaveChangesWasCalled = true;
return -1;
}
}
In this case my CollectionDbSet<> is a slightly modified version of FakeDbSet<> here (which simply implements IDbSet with an underlying ObservableCollection and ObservableCollection.AsQueryable()).
This solution works nicely with all my unit tests and specifically with NCrunch running these tests on the fly.
Full Integration Tests
These Unit tests test all the business logic but one major downside is that none of your LINQ statements are guaranteed to work with your actual MyDataContext. This is because testing against an in memory data context means you're replacing the Linq-To-Entity provider but a Linq-To-Objects provider (as pointed out very well in the answer to this SO question).
To fix this I use Ninject within my unit tests and setup InMemoryDataContext to bind instead of MyDataContext within my unit tests. You can then use Ninject to bind to an actual MyDataContext when running the integration tests (via a setting in the app.config).
if(Global.RunIntegrationTest)
DependencyInjector.Bind<MyDataContext>().To<MyDataContext>().InSingletonScope();
else
DependencyInjector.Bind<MyDataContext>().To<InMemoryDataContext>().InSingletonScope();
Let me know if you have any feedback on this however, there are always improvements to be made.
As per my comment in the question, this was more to help others searching for this problem on SO. But as pointed out in the comments underneath the question there are quite a few other design approaches that would fix this problem.