Using NUnit I wish to run all the tests in a certain project against multiple cultures.
The project deals with parsing data that should be culture neutral, to ensure this I would like to run every test against multiple cultures.
The current solution I have is
public abstract class FooTests {
/* tests go here */
}
[TestFixture, SetCulture ("en-GB")] public class FooTestsEN : FooTests {}
[TestFixture, SetCulture ("pl-PL")] public class FooTestsPL : FooTests {}
Ideally, I shouldn't have to create these classes and instead use something like:
[assembly: SetCulture ("en-GB")]
[assembly: SetCulture ("pl-PL")]
Unfortunatelly this isn't possible now but is planned for future.
You can also do this.
public class AllCultureTests
{
private TestSomething() {...}
[Test]
[SetCulture("pl-PL")]
public void ShouldDoSomethingInPoland()
{
TestSomething();
}
}
Maybe that's something you would prefer?
NUnit's SetCultureAttribute applies one culture to a test, multiple cultures are not (yet) supported.
You can work around this by using the TestCaseAttribute with language codes and setting the culture manually:
[Test]
[TestCase("de-DE")]
[TestCase("en-US")]
[TestCase("da-DK")]
public void YourTest(string cultureName)
{
var culture = CultureInfo.GetCultureInfo(cultureName);
Thread.CurrentThread.CurrentCulture = culture;
Thread.CurrentThread.CurrentUICulture = culture;
var date = new DateTime(2012, 10, 14);
string sut = date.ToString("dd/MM/yyyy");
Assert.That(sut, Is.EqualTo("14/10/2012"));
}
Note that this unit test will fail for de and da - testing for different cultures is really important :)
If you don't mind switching, MbUnit has had this feature for nearly five years now.
You can apply the MultipleCulture attribute at the test, fixture and assembly levels.
Related
I am very new to C# and nunit. Pls bear with me if this is basic and has been already been asked here.
We have a global setup,defined by [SetupFixture] class,which is expected to be run only once. The private variables are defined in it's [setup]. We wish to use the same variables in all our testfixtures,hence inheriting the testbase class in all our testfixtures.
But, while executing Testcase1, i observe that globalSetup() is called more than once. Can anyone point me the issue? sample code is as below.
namespace CTB
{
[SetupFixture]
public class Testbase
{
private byte val1;
private byte val2;
[setup]
public void globalSetup
{
val1 = 5;
val2 = 10;
}
[Teardown]
public void globalTeardown
{
//
}
}
}
namespace CTB.Testcase
{
public class TestCase : Testbase
{
[Setup]
public void Setup()
{
}
[Teardown]
public void Teardown()
{
}
[Test]
public void Testcase1()
{
byte val3 = val1 + val2; // Expect 15
}
}
}
I'm assuming that the answer to my comment is "No" and that you are using a current version of NUnit 3. Please correct me if I'm wrong. :-)
You have made the class TestBase serve two functions:
It's the base class for your TestFixture and therefore it's a TestFixture itself.
It's marked as a SetUpFixture so it also serves that function - a completely different function, by the way.
To be clear, you should never do this. It's a sort of "trick" that almost seems designed to confuse NUnit - not your intention of course. Your test fixtures should have no inheritance relationship with any SetUpFixture. Use different classes for the test fixture base and the setup fixture.
With that out of the way, here is the longer story of what is happening...
Before your tests even execute, the SetUpFixture is first "run" - in quotes because it actually does nothing. That's because it doesn't contain any methods marked with [OneTimeSetUp] or '[OneTimeTearDown]`.
NOTE: As an alternate explanation, if you are using a pretty old version of NUnit, the [SetUp] and [TearDown] methods are actually called at this point. Nnit V2 used those attributes with different meanings when encountered in a SetUpFixture versus a TestFixture.
Next your tests execute. Before each test, the inherited [SetUp] and [TearDown] methods are run. Of course, these are actually the same methods as in step 1. NUnit has been tricked into doing this!
Here is some general guidance for the future...
If you want multiple fixtures to use the same data, a base class is useful. Any public or protected fields or properties will be shared by the inheriting fixtures.
If you want to do some common setup or teardown for a group of unrelated test fixtures, use a SetUpFixture. Note that the only way to pass data from a SetUpFixture to the test fixtures is through static fields or properties. Generally, you use a SetUpFixture to set up the environment in which the test is run, not to provide data.
Never use the same class for both purposes.
Let's say I have the following test case that has been written using xUnit:
public static IEnumerable<object[]> testValues = new List<object[]>
{
new object[] {new double?[] {0.0}, 0.0, 0.0},
};
[Theory]
[MemberData(nameof(testValues))]
public void Test1(double?[] values, double expectedQ1, double expectedQ3)
{
// Test code
}
How could I express the same unit test in nUnit instead of xUnit?
Note: The main problem here seems to be the use of MemberData, which for so far, I haven't been able to find an nUnit equivalent. What would be the correct way of expressing such unit test cases in nUnit?
Like this:
public static IEnumerable<object[]> testValues = new List<object[]>
{
new object[] {new double?[] {0.0}, 0.0, 0.0},
};
[TestCaseSource(nameof(testValues))]
public void Test1(double?[] values, double expectedQ1, double expectedQ3)
{
// Test code
}
Note that NUnit has TheoryAttribute but you don't want it here. In NUnit, a Theory is a bit more than just a parameterized test. You should read the docs to understand what it is before deciding if you need it. Of course, you should read up on TestCaseSourceAttribute as well. :-)
Other attributes in NUnit that allow data to be specified for a test case include TestCaseAttribute, ValuesAttribute, ValueSourceAttribute, RandomAttribute and RangeAttribute.
I'm using NUnit 3.0 and TestFixtureSource to run test cases inside a fixture multiple times with different parameters/configurations (I do want to do this at TestFixture level). Simple example:
[TestFixtureSource(typeof (ConfigurationProvider))]
public class Fixture
{
public Fixture(Configuration configuration)
{
_configuration = configuration;
}
private Configuration _configuration;
[Test]
public void Test()
{
//do something with _configuration
Assert.Fail();
}
}
Let's say Test() fails for one of the configurations and succeeds for another. In the run report file and in Visual Studio's Test Explorer the name for both the failed and the succeeded runs will be displayed as just Test(), which doesn't tell me anything about which setup caused issues.
Is there a way to affect the test cases names in this situation (i.e. prefix its name per fixture run/configuration)? As a workaround I'm currently printing to the results output before each test case fires but I would rather avoid doing that.
Since NUnit 3.0 is in beta and this feature is fairly new I wasn't able to find anything in the docs. I found TestCaseData but I don't think it's tailored to be used with fixtures just yet (it's designed for test cases).
I can't find a way to change the testname, but it should not be neccessary, because NUnit3 constructs the testname by including a description of the testfixture.
The example class Fixture from the question can be used unchanged if the Configuration and ConfigurationProvider has an implementation like this:
public class Configuration
{
public string Description { get; }
public Configuration(string description)
{
Description = description;
}
public override string ToString()
{
return Description;
}
}
public class ConfigurationProvider : IEnumerable
{
public IEnumerator GetEnumerator()
{
yield return new Configuration("Foo");
yield return new Configuration("Bar");
yield return new Configuration("Baz");
}
}
The 'trick' is to make sure the constructor-parameter to the fixture is a string or has a ToString-method that gives a sensible description of the fixture.
If you are using NUnit 3 Test Adapter in Visual Studio, then the testfixtures will be displayed as Fixture(Foo), Fixture(Bar) and Fixture(Baz) so you can easily distinguish between their tests. The xml-output from nunit3-console.exe also uses descriptive names, fx: fullname=MyTests.Fixture(Bar).Test
<test-case id="0-1003" name="Test" fullname="MyTests.Fixture(Bar).Test" methodname="Test" classname="MyTests.Fixture" runstate="Runnable" result="Failed" ... >
<failure>
<message><![CDATA[]]></message>
<stack-trace><![CDATA[at MyTests.Fixture.Test() in ... ]]></stack-trace>
</failure>
...
</test-case>
One way to perform such actions is to have find and replace tokens in source code and dynamically build test libraries before execution using command line msbuild. High level steps are
Define test case names as sometest_TOKEN in source then using command line tools like fnr.exe replce _TOKEN with whatever you like. For example sometest_build2145.
Compile the dll with using msbuild for example msbuild /t:REbuild mytestproj.sln. Thereafter execute all test cases in mytestproj.dll.
I've been testing my business logic in ServiceStack 3.9.38 project, and faced a problem when running unit tests separatly leads to success, and running tests all together leads to fail of one of them. After several hours I've made a reproducible unit test. If you run the whole fixture, the second test will fail. Running tests separatly makes them pass.
using Funq;
using NUnit.Framework;
[TestFixture]
public class Test
{
interface IBar {}
class Bar : IBar {}
class TestFoo { public IBar Bar { get; set; } }
[Test]
public void Test1()
{
var container = new Container();
var m = new TestFoo();
container.Register<IBar>(new Bar());
Assert.NotNull(container.Resolve<IBar>(), "Resolve");
container.AutoWire(m);
Assert.NotNull(m.Bar, "Autowire");
}
[Test]
public void Test2()
{
var container = new Container();
var m = new TestFoo();
container.AutoWire(m);
Assert.Throws<ResolutionException>(() => container.Resolve<IBar>());
Assert.IsNull(m.Bar); // FAILS HERE
}
}
Is that an issue of Funq.Container configuration? Or this is a bug? Any workarounds?
EDIT: I've posted an issue on GitHub: https://github.com/ServiceStack/ServiceStack/issues/521
There is a private static dictionary autoWireCache at AutoWireHelpers.cs that is caching your resolutions When you run the test them twice the value is pulled from the cache and your test fails. I believe the caching is a feature of ServiceStack's customized funq for performance gains.
There is no public interface for clearing that cache so I see no quick solution to the way you have the tests setup.
This is more a solution / work around than an actual question. I'm posting it here since I couldn't find this solution on stack overflow or indeed after a lot of Googling.
The Problem:
I have an MVC 3 webapp using EF 4 code first that I want to write unit tests for. I'm also using NCrunch to run the unit tests on the fly as I code, so I'd like to avoid backing onto an actual database here.
Other Solutions:
IDataContext
I've found this the most accepted way to create an in memory datacontext. It effectively involves writing an interface IMyDataContext for your MyDataContext and then using the interface in all your controllers. An example of doing this is here.
This is the route I went with initially and I even went as far as writing a T4 template to extract IMyDataContext from MyDataContext since I don't like having to maintain duplicate dependent code.
However I quickly discovered that some Linq statements fail in production when using IMyDataContext instead of MyDataContext. Specifically queries like this throw a NotSupportedException
var siteList = from iSite in MyDataContext.Sites
let iMaxPageImpression = (from iPage in MyDataContext.Pages where iSite.SiteId == iPage.SiteId select iPage.AvgMonthlyImpressions).Max()
select new { Site = iSite, MaxImpressions = iMaxPageImpression };
My Solution
This was actually quite simple. I simply created a MyInMemoryDataContext subclass to MyDataContext and overrode all the IDbSet<..> properties as below:
public class InMemoryDataContext : MyDataContext, IObjectContextAdapter
{
/// <summary>Whether SaveChanges() was called on the DataContext</summary>
public bool SaveChangesWasCalled { get; private set; }
public InMemoryDataContext()
{
InitializeDataContextProperties();
SaveChangesWasCalled = false;
}
/// <summary>
/// Initialize all MyDataContext properties with appropriate container types
/// </summary>
private void InitializeDataContextProperties()
{
Type myType = GetType().BaseType; // We have to do this since private Property.Set methods are not accessible through GetType()
// ** Initialize all IDbSet<T> properties with CollectionDbSet<T> instances
var DbSets = myType.GetProperties().Where(x => x.PropertyType.IsGenericType && x.PropertyType.GetGenericTypeDefinition() == typeof(IDbSet<>)).ToList();
foreach (var iDbSetProperty in DbSets)
{
var concreteCollectionType = typeof(CollectionDbSet<>).MakeGenericType(iDbSetProperty.PropertyType.GetGenericArguments());
var collectionInstance = Activator.CreateInstance(concreteCollectionType);
iDbSetProperty.SetValue(this, collectionInstance,null);
}
}
ObjectContext IObjectContextAdapter.ObjectContext
{
get { return null; }
}
public override int SaveChanges()
{
SaveChangesWasCalled = true;
return -1;
}
}
In this case my CollectionDbSet<> is a slightly modified version of FakeDbSet<> here (which simply implements IDbSet with an underlying ObservableCollection and ObservableCollection.AsQueryable()).
This solution works nicely with all my unit tests and specifically with NCrunch running these tests on the fly.
Full Integration Tests
These Unit tests test all the business logic but one major downside is that none of your LINQ statements are guaranteed to work with your actual MyDataContext. This is because testing against an in memory data context means you're replacing the Linq-To-Entity provider but a Linq-To-Objects provider (as pointed out very well in the answer to this SO question).
To fix this I use Ninject within my unit tests and setup InMemoryDataContext to bind instead of MyDataContext within my unit tests. You can then use Ninject to bind to an actual MyDataContext when running the integration tests (via a setting in the app.config).
if(Global.RunIntegrationTest)
DependencyInjector.Bind<MyDataContext>().To<MyDataContext>().InSingletonScope();
else
DependencyInjector.Bind<MyDataContext>().To<InMemoryDataContext>().InSingletonScope();
Let me know if you have any feedback on this however, there are always improvements to be made.
As per my comment in the question, this was more to help others searching for this problem on SO. But as pointed out in the comments underneath the question there are quite a few other design approaches that would fix this problem.