Nunit Test Order - Single Test - Multiple Orders - nunit

For Nunit Test order, is it possible to have multiple order positions for a test, such as [Test, Order(3, 10, #...)]?
Scenario - I am automating a multi-page online application using end-to-end ordered tests (with Asserts verifying entered info), and I want to verify info on previous pages is still there - saved - when I click Back button (basically, reverse-check the end-to-end starting from last page of application using Back button to navigate). I want to reuse Asserts already there instead of rewriting the same ones again.
I have tried the following as an experiment, which generate build error CS1729 - 'type' does not contain a constructor that takes 'number' arguments:
[Test, Order(66, 67)]
**
[Test, Order(66)(67)]
**
[Test, Order(66), Order(67)]
**
[Test, Order(66)]
[Test, Order(67)]
I Googled this, and I don't see anything that does the above, so I assume it is not possible, but want to confirm. I cannot remove the Order attribute as that's part of the requirement for this test.

No, that's not possible. The OrderAttribute sets a property, which is a simple integer.
To simulate the effect you want, you may do the following:
Extract all the code of the test in question to a separate method, called by the test. It should have the same signature as the test.
Replicate the test you want to repeat, using a different name.
Assign a different order attribute to each test.
Assuming a signature of void TestMethod() it will look something like this...
[Test, Order(66)]
public void MyTest66()
{
MyRealTest();
}
[Test, Order(67)]
public void MyTest67()
{
MyRealTest();
}
private void MyRealTest()
{
// Your actual test code goes here
}
Note... this is a really ugly hack but the only guaranteed way I can think of to do what you indicate is required.

Related

How to run code before/after cucumber suite?

I'm trying to figure out how to run some code before and after all my cucumber tests run.
I've been tracking down a bug for a few days where some our processes create jobs on a server, and don't properly clean it up. It's easy to miss so ideally I don't want engineers to have to manually add a check to every test.
I was hoping there'd be a way to put a hook in before any tests ran to cache how many jobs exist on the server, then a hook at the end to ensure that the value hasn't changed.
I know this isn't really the best way to use cucumber, as that is more of a system test type thing to do, but doing it this way would be the best way to fit it into the existing infrastructure.
Use #BeforeClass and #AfterClass annotations in your run file.
#RunWith(Cucumber.class)
#Cucumber.Options(
format = {"json", "<the report file>"},
features = {"<the feature file>"},
strict = false,
glue = {"<package with steps classes>"})
public class TestRunFile {
#BeforeClass
public static void getJobNumbersOnServerBeforeStarting() {
//Implement logic
}
#AfterClass
public static void getJobNumbersOnServerAfterCompletion() {
//Implement logic
}
}
How about using tagged hooks.
#Before("#jobCheck")
public void beforeScenario() {
// actions
}
#After("#jobCheck")
public void afterScenario() {
// actions
}
And then for each scenario that requires this check, add #jobCheck before the Scenario definition as below.
Feature: Some feature description
#jobCheck
Scenario: It should process a sentence
// The steps
More on JVM hooks here: https://zsoltfabok.com/blog/2012/09/cucumber-jvm-hooks/

Configuration settings make tests unstable with NCrunch

I have a problem with tests becoming unstable under NCrunch. It looks like it has to do with some shadow copying issue. My test goes something like this
class SaveViewSettings : ISaveSettings
{
public void SaveSettings()
{
Properties.View.Default.Save();
}
}
[TestFixture]
// ReSharper disable once InconsistentNaming
class SaveViewSettings_Should
{
[Test]
public void Save_Settings()
{
var ctx = Properties.View.Default;
var sut = new SaveViewSettings();
ctx.LeftSplitter = 12.34;
sut.SaveSettings();
ctx.Reload();
ctx.LeftSplitter.Should().Be(12.34);
}
}
When reloading the settings using ctx.Reload() i get
System.Configuration.ConfigurationErrorsException : ...
----> System.Configuration.ConfigurationErrorsException...
(C:\...\AppData\Local\Remco_Software_Ltd\nCrunch.TestRunner.AppDom_Url_q2piuozo0uftcc2pz5zv15hpilzfpoqk\[version]\user.config...)
A similar problem has been raised on the NCrunch forum about 3 months ago: Unrecognized configuration section userSettings
You might get similar errors with NCrunch when working on multiple solutions with application settings.
I think this might root down to NCrunch always using the same product and user name when shadow building so that all configuration settings are mapped to the same user.config file path.
It seems that by now there is no known solution to this. A work around is to manually delete the user config in
%LOCALAPPDATA%\Remco_Software_Ltd\nCrunch.TestRunner.AppDom_...\user.config`.
Note that the usual way to do this ctx.Reset() might fail as well, so you really have to locate and delete the user.config yourself using ConfigurationManager.
I have automated this work around by adding the following code which stabilizes the test with NCrunch
[Test]
public void Save_Settings()
{
#if NCRUNCH
// Every once in a while NCrunch throws ConfigurationErrorException, cf.:
// - http://forum.ncrunch.net/yaf_postsm7538_Unrecognized-configuration-section-userSettings.aspx
// - http://www.codeproject.com/Articles/30216/Handling-Corrupt-user-config-Settings
// - http://stackoverflow.com/questions/2903610/visual-studio-reset-user-settings-when-debugging
// - http://stackoverflow.com/questions/9038070/how-do-i-get-the-location-of-the-user-config-file-in-programmatically
var config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.PerUserRoamingAndLocal);
if (File.Exists(config.FilePath))
File.Delete(config.FilePath);
#endif
...
Over the past few years i came to consider .NET configuration settings as a kind of legacy feature. It was introduced with .NET 2.0 and was great at the time but it has some issues that you need to be aware of. Maybe it is a good idea to look for alternatives or abstractions like e.g. HumbleConfig which makes it easy to switch.

Jbehave : GivenStories in the end of execution

I'm using a GivenStories for executing Login scenario which is located in different story.
I was wondering if there is a way to use something similar in order to execute a logout story which is also located in different story than one I actually executing.
I know that I can do some tricks with #before/after annotations , but the question is if I can execute a "post" story
Thanks
Based on the jBehave annotation documentation a post story step can be implemented by annotating a step class method with #AfterStory (or #AfterStories if you want to execute only after all stories complete). The #AfterStory method will execute regardless of whether your executing story contains a step from the related step class (i.e. is guaranteed to execute after every story - see below for restricting to given stories).
The #BeforeStory and #AfterStory annotations allow the corresponding
methods to be executed before and after each story, either a
GivenStory or not:
#AfterStory // equivalent to #AfterStory(uponGivenStory=false)
public void afterStory() {
// ...
}
#AfterStory(uponGivenStory=true)
public void afterGivenStory() {
// ...
}
This is the answer I got from the jbehave dev channel.
Hi,
there is no such mechanism, but you could:
use the Lifecycle to execute steps (not stories) after the execution
of a scenario (executed after each scenario) have a final scenario
which invokes the given stories

How do I get Swashbuckle to have Swagger UI to group by Version?

I am playing with the new Azure API Apps (template in Visual Studio 2013 w/ the new SDK bits from 3/24/15) and I'd like have my Swagger UI group my calls by Version #. In my case, I'm currently versioning by URI (I realize REST purists will tell me not to do this - please don't try to "correct my error" here). For instance, I may have these calls:
http://example.com/api/Contacts <-- "latest"
http://example.com/api/1/Contacts
http://example.com/api/2/Contacts
http://example.com/api/Contacts{id} <-- "latest"
http://example.com/api/1/Contacts/{id}
http://example.com/api/2/Contacts/{id}
Functionally, this works great! (Yes, I know some of you will cringe. Sorry this hurts your feelings.) However, my problem is w/ Swagger UI organization. By default, Swagger UI groups these by the Controller Name (Contacts in this case). I see in the SwaggerConfig.cs file that I can change this:
// Each operation be assigned one or more tags which are then used by consumers for various reasons.
// For example, the swagger-ui groups operations according to the first tag of each operation.
// By default, this will be controller name but you can use the "GroupActionsBy" option to
// override with any value.
//
//c.GroupActionsBy(apiDesc => apiDesc.HttpMethod.ToString());
What I don't understand is how I can tweak this to group all of the "latest" together and then all of v1 together and then all of v2 together, etc.
How can I do this? If it absolutely must require that I add the word "latest" (or equiv) into the path in place of the version number, then I can do that but I'd prefer not have to do that.
I believe what you're looking to do is to uncomment a line a few lines below that one in SwaggerConfig.cs
c.OrderActionGroupsBy(new DescendingAlphabeticComparer());
except you'd change the name of the class to something like ApiVersionComparer() and then implement it as a new class:
public class ApiVersionComparer : IComparer<string>
{
public int Compare(string x, string y)
{
// Write whatever comparer you'd like to here.
// Yours would likely involve parsing the strings and having
// more complex logic than this....
return -(string.Compare(x, y));
}
}
If you've gotten far enough to ask this question, I'm sure I can leave the sort implementation for you. :-)

testNG: sending report via mail within the #AfterSuite section

I'd like to send the report generated by testNG ( java+eclipse+testNG) within the #AfterSuite section.
It's not a problem to send it, but the point is that the report is generated after the #AfterSuite section, so , basically, i send the previous one instead of the last one !
Any idea about how can I solve it ?
As you are seeing, #AfterSuite runs before the report is generated.
Have you though about implementing a TestNG IReporter listener ?
public class MyReporter implements IReporter {
#Override
public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> iSuites, String s) {
//Create your bespoke results
//Email results
}
}
Obviously you can see a flaw in that you have to generate your own results from the raw results data (which may be advantageous if you just want to email a subset of data).
The ideal solution would to be able to extend the default report generator, but I am not sure this can be done. However there is an existing listener provided by http://reportng.uncommons.org/, which actually provides a much nicer report output.
If you extend this this class, and call their code, and then add email generator code afterwards, it may work
public class MyReporter extends HTMLReporter {
#Override
public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> iSuites, String s) {
super.generateReport(xmlSuites, iSuites, s);
//Email results
}
}
You can attach a listener to a test suite in several ways, as explained on the TEstNG website (http://testng.org/doc/documentation-main.html#listeners-testng-xml)
An alternative to all of this woudl be to use a build tool like Maven to run your tests, then have a post test event to email the results.
I copied the answer from Krishnan.
It works for me.
By the way, in my test environment, I need to extends the org.testng.reporters.EmailableReporter2 instead of EmailableReporter to make sure the correct count.
See below for your reference:
Krishnan Mahadevan Krishnan Mahadevan at Jul 31, 2012 at 8:58 am I am guessing that you are referring to the TestNG generated
"emailable-report.html" which you would want to mail.
With that assumption here's how you should be able to do it.
Extend org.testng.reporters.EmailableReporter
Override org.testng.reporters.EmailableReporter.generateReport(List,
List, String) and have it do something as below :
#Override
public void generateReport(List xml, List suites, String
outdir) {
super.generateReport(xml, suites, outdir);
SendFileEmail e= new SendFileEmail();
e.sendEmail();
}
Now add up this listener of yours into your suite file using
tag.