NUnit extension - nunit

Hi All i have a question regarding NUnit Extension (2.5.10).
What i am trying to do is write some additional test info to the
database. For that i have created NUnit extension using Event
Listeners.
The problem i am experiencing is that public void
TestFinished(TestResult result) method is being called twice at
runtime. And my code which writes to the database is in this method
and that leaves me with duplicate entries in the database. The
question is: Is that the expected behaviour? Can i do something about
it?
The extension code is below. Thanks.
using System;
using NUnit.Core;
using NUnit.Core.Extensibility;
namespace NuinitExtension
{
[NUnitAddinAttribute(Type = ExtensionType.Core,
Name = "Database Addin",
Description = "Writes test results to the database.")]
public class MyNunitExtension : IAddin, EventListener
{
public bool Install(IExtensionHost host)
{
IExtensionPoint listeners = host.GetExtensionPoint("EventListeners");
if (listeners == null)
return false;
listeners.Install(this);
return true;
}
public void RunStarted(string name, int testCount){}
public void RunFinished(TestResult result){}
public void RunFinished(Exception exception){}
public void TestStarted(TestName testName){}
public void TestFinished(TestResult result)
{
// this is just sample data
SqlHelper.SqlConnectAndWRiteToDatabase("test", test",
2.0, DateTime.Now);
}
public void SuiteStarted(TestName testName){}
public void SuiteFinished(TestResult result){}
public void UnhandledException(Exception exception){}
public void TestOutput(TestOutput testOutput){}
}
}

I have managed to fix the issue by simply removing my extension
assembly from NUnit 2.5.10\bin\net-2.0\addins folder. At the moment
everything works as expected but i am not sure how. I thought that you
have to have the extension/addin assembly inside the addins folder.
I am running tests by opening a solution via NUnit.exe. My extension
project is part of the solution i am testing. I have also raised this issue with NUnit guys and got the following explanation:
Most likely, your addin was being loaded twice. In order to make it easier to test addins, NUnit searches each test assembly for addins to be loaded, in addition to searching the addins directory. Normally, when you are confident that your addin works, you should remove it from the test assembly and install it in the addins folder. This makes it available to all tests that are run using NUnit. OTOH, if you really only want the addin to apply for a certain project, then you can leave it in the test assembly and not install it as a permanent addin.
http://groups.google.com/group/nunit-discuss/browse_thread/thread/c9329129fd803cb2/47672f15e7cc05d1#47672f15e7cc05d1

Not sure this answer is strictly relevant but might be useful.
I was having a play around with the NUnit library recently to read NUnit tests in so they could easily be transfered over to our own in-house acceptance testing framework.
It turns out we probably wont stick with this but thought it might be useful to share my experiences figuring out how to use the NUnit code:
It is different in that it doesn't get run by the NUnit console or Gui Runner but just by our own console app.
public class NUnitTestReader
{
private TestHarness _testHarness;
public void AddTestsTo(TestHarness testHarness)
{
_testHarness = testHarness;
var package = new TestPackage(Assembly.GetExecutingAssembly().Location){AutoBinPath = true};
CoreExtensions.Host.InitializeService();
var testSuiteBuilder = new TestSuiteBuilder();
var suite = testSuiteBuilder.Build(package);
AddTestsFrom(suite);
}
private void AddTestsFrom(Test node)
{
if (!node.IsSuite)
AddTest(node);
else
{
foreach (Test test in node.Tests)
AddTestsFrom(test);
}
}
private void AddTest(Test node)
{
_testHarness.AddTest(new WrappedNUnitTest(node, TestFilter.Empty));
}
}
The above reads NUnit tests in from the current assembly wraps them up and then adds them to our inhouse test harness. I haven't included these classes but they're not really important to understanding how the NUnit code works.
The really useful bit of information here is the static to "InitialiseService" this took quite a bit of figuring out but is necessary to get the basic set of test readers loaded in NUnit. You need to be a bit careful when looking at the tests in NUnit aswell as it includes failing tests (which I assume dont work because of the number of statics involved) - so what looks like useful documentation is actually misleading.
Aside from that you can then run the tests by implementing EventListener. I was interested in getting a one to one mapping between our tests and NUnit tests so each test is run on it's own. To achieve this you just need to implement TestStarted and TestFinished to do logging:
public void TestStarted(TestName testName)
{
}
public void TestFinished(TestResult result)
{
string text;
if (result.IsFailure)
text = "Failure";
else if (result.IsError)
text = "Error";
else
return;
using (var block = CreateLogBlock(text))
{
LogFailureTo(block);
block.LogString(result.Message);
}
}
There are a couple of problems with this approach: Inherited Test base classes from other assemblies with SetUp methods that delegate to ones in the current assembly dont get called. It also has problems with TestFixtureSetup methods which are only called in NUnit when TestSuites are Run (as opposed to running test methods on their own).
These both seem to be problems with NUnit although if you dont want to construct wrapped tests individually I think you could just put in a call to suite.Run with the appropriate parameters and this will fix the latter problem

Related

How to run test methods individually with Citrus Framework?

I have the next code
#Test
public class ApiTestIT extends TestNGCitrusTestDesigner {
#CitrusTest(name = "testApi2IT")
public void testApi1IT() {
//TO-DO here
}
#CitrusTest(name = "testApi2IT")
public void testApi2IT() {
echo("Hello Citrus!");
}
}
How can i run test methods individually?
I trying using -Dtest and -Dit.test bad no work; always run the test at same time.
Thanks
To execute single test methods, you have to specify them in the -Dit.test specification like -Dit.test=ApiTestIT#testApi2IT.
Nevertheless, this functionality is not provided by citrus but by the maven failsafe plugin. For more information, have a look at the documentation for Running a Single Test
Some examples with citrus can be found in the Citrus samples repository.

How to run code before/after cucumber suite?

I'm trying to figure out how to run some code before and after all my cucumber tests run.
I've been tracking down a bug for a few days where some our processes create jobs on a server, and don't properly clean it up. It's easy to miss so ideally I don't want engineers to have to manually add a check to every test.
I was hoping there'd be a way to put a hook in before any tests ran to cache how many jobs exist on the server, then a hook at the end to ensure that the value hasn't changed.
I know this isn't really the best way to use cucumber, as that is more of a system test type thing to do, but doing it this way would be the best way to fit it into the existing infrastructure.
Use #BeforeClass and #AfterClass annotations in your run file.
#RunWith(Cucumber.class)
#Cucumber.Options(
format = {"json", "<the report file>"},
features = {"<the feature file>"},
strict = false,
glue = {"<package with steps classes>"})
public class TestRunFile {
#BeforeClass
public static void getJobNumbersOnServerBeforeStarting() {
//Implement logic
}
#AfterClass
public static void getJobNumbersOnServerAfterCompletion() {
//Implement logic
}
}
How about using tagged hooks.
#Before("#jobCheck")
public void beforeScenario() {
// actions
}
#After("#jobCheck")
public void afterScenario() {
// actions
}
And then for each scenario that requires this check, add #jobCheck before the Scenario definition as below.
Feature: Some feature description
#jobCheck
Scenario: It should process a sentence
// The steps
More on JVM hooks here: https://zsoltfabok.com/blog/2012/09/cucumber-jvm-hooks/

Configuration settings make tests unstable with NCrunch

I have a problem with tests becoming unstable under NCrunch. It looks like it has to do with some shadow copying issue. My test goes something like this
class SaveViewSettings : ISaveSettings
{
public void SaveSettings()
{
Properties.View.Default.Save();
}
}
[TestFixture]
// ReSharper disable once InconsistentNaming
class SaveViewSettings_Should
{
[Test]
public void Save_Settings()
{
var ctx = Properties.View.Default;
var sut = new SaveViewSettings();
ctx.LeftSplitter = 12.34;
sut.SaveSettings();
ctx.Reload();
ctx.LeftSplitter.Should().Be(12.34);
}
}
When reloading the settings using ctx.Reload() i get
System.Configuration.ConfigurationErrorsException : ...
----> System.Configuration.ConfigurationErrorsException...
(C:\...\AppData\Local\Remco_Software_Ltd\nCrunch.TestRunner.AppDom_Url_q2piuozo0uftcc2pz5zv15hpilzfpoqk\[version]\user.config...)
A similar problem has been raised on the NCrunch forum about 3 months ago: Unrecognized configuration section userSettings
You might get similar errors with NCrunch when working on multiple solutions with application settings.
I think this might root down to NCrunch always using the same product and user name when shadow building so that all configuration settings are mapped to the same user.config file path.
It seems that by now there is no known solution to this. A work around is to manually delete the user config in
%LOCALAPPDATA%\Remco_Software_Ltd\nCrunch.TestRunner.AppDom_...\user.config`.
Note that the usual way to do this ctx.Reset() might fail as well, so you really have to locate and delete the user.config yourself using ConfigurationManager.
I have automated this work around by adding the following code which stabilizes the test with NCrunch
[Test]
public void Save_Settings()
{
#if NCRUNCH
// Every once in a while NCrunch throws ConfigurationErrorException, cf.:
// - http://forum.ncrunch.net/yaf_postsm7538_Unrecognized-configuration-section-userSettings.aspx
// - http://www.codeproject.com/Articles/30216/Handling-Corrupt-user-config-Settings
// - http://stackoverflow.com/questions/2903610/visual-studio-reset-user-settings-when-debugging
// - http://stackoverflow.com/questions/9038070/how-do-i-get-the-location-of-the-user-config-file-in-programmatically
var config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.PerUserRoamingAndLocal);
if (File.Exists(config.FilePath))
File.Delete(config.FilePath);
#endif
...
Over the past few years i came to consider .NET configuration settings as a kind of legacy feature. It was introduced with .NET 2.0 and was great at the time but it has some issues that you need to be aware of. Maybe it is a good idea to look for alternatives or abstractions like e.g. HumbleConfig which makes it easy to switch.

How to write a custom Junit runner that will log results

I am working with JUnit and I have this idea which I want to implement.
I want to write a runner that will log the results of each test to an excel or a text file so that i can attach it in my reports.
What do i need to learn to get started
Two alternatives
Write a RunListener and use it like:
public void main(String... args) {
JUnitCore core= new JUnitCore();
core.addListener(new MyRunListener());
core.run(MyTestClass.class);
}
Write a RunListener again. But this time extend an org.junit.runner.Runner implementation and override its run method like
#Override
public void run(RunNotifier notifier) {
notifier.addListener(new MyRunNotifier());
super.run(notifier);
}
Second approach can also be used in tests with #RunWith(MyRunner.class) annotation.

Entity Framework 4.1 "Code First" SetInitializer not being called again after Database.Delete

Trying to do some unit testing with EF 4.1 code first. I have my live db (SQL Server) and my unit test DB( Sql CE). After fighting (and losing) with EF, Sql CE 4.0 and Transaction support I decided the simplest way to run my test was to:
Create Db
Run Test
Delete Db
Rinse and repeat
I have my [Setup] and [TearDown] functions:
[SetUp]
public void Init()
{
System.Data.Entity.Database.SetInitializer(new MyTestContextInitializer());
_dbContext = ContainerFactory.Container.GetInstance<IContext>();
_testConnection = _dbContext.ConnectionString;
}
[TearDown]
public void Cleanup()
{
_dbContext.Dispose();
System.Data.Entity.Database.Delete(_testConnection);
}
Issue is that System.Data.Entity.Database.SetInitializer does not call MyTestContextInitializer after the first test.
Hence the 2nd test then fails with:
System.Data.EntityException : The
underlying provider failed on Open.
----> System.Data.SqlServerCe.SqlCeException
: The database file cannot be found.
Check the path to the database
TIA for any pointers
I got around this by calling 'InitializeDatabase' manually. Like so:
[SetUp]
public void Init()
{
var initializer = new MyTestContextInitializer();
System.Data.Entity.Database.SetInitializer(initializer);
_dbContext = ContainerFactory.Container.GetInstance<IContext>();
initializer.InitializeDatabase((MyTestContext)_dbContext);
_testConnection = _dbContext.ConnectionString;
}
[TearDown]
public void Cleanup()
{
System.Data.Entity.Database.Delete(_testConnection);
_dbContext.Dispose();
}
I think it may be a bug with EF 4.1 RC.
It's not a bug, the initializer set with
System.Data.Entity.Database.SetInitializer
is only called when the context is created for the first time in the AppDomain. Hence, since you're running all your tests in a single AppDomain, it's only called when the first test is ran.
It took me almost a day to find out what caused my strange unittest behaviour: the database connection stayed open or the database was not created with a every new test. I searched everywhere for the root of the cause: MSTest (no Admin rights or where working copies of files somehow deleted?), SQL Server Express/CE (login failure?), Unity (objects not disposed?) or Entity Framework (no proper database initialization?). It turned out to be EF. Thanks a lot for the answer!