I've managed to tune the output from my SpecFlow tests so that it reads nicely, with just the steps reported plus failures. But it's still pretty unreadable without the Feature and Scenario names also being reported.
Looking at the generated code, it appears that the Feature and Scenario names are encoded as NUnit DescriptionAttributes.
Can I configure SpecFlow or NUnit to also report these to stdout, so I get a nicely flowing "story-like" output?
If you define an extra method in your step definition class as follows then NUnit will report the feature and scenario text.
[BeforeScenario]
public void OutputScenario()
{
Console.WriteLine("Feature: " + FeatureContext.Current.FeatureInfo.Title);
Console.WriteLine(FeatureContext.Current.FeatureInfo.Description);
Console.WriteLine("\r\nScenario: " + ScenarioContext.Current.ScenarioInfo.Title);
}
I hope this helps.
Related
I have newly started working on BDD Cucumber. I am using scala for writing test cases. I am trying to use Scenario Outline and passing parameters in step definitions. My code is as follows.
Scenario Outline: Data is parsed and persisted
Given Portal is running
When A data of <type> is received
Then The data of <type> with <Id> should be parsed and persisted
Examples:
| type | Id |
| Personal | 1 |
|Professional | 2 |
Now in my when condition I am trying to get these parameters as follows
When("""^A data of \"([^\"]*)\" is received$""") {
(type: String) =>
//My code
}
Now on running my code I am getting following error everytime.
io.cucumber.junit.UndefinedStepException: The step "A data of Personal is received" is undefined. You can implement it using the snippet(s) below:
When("""A data of Personal is received""") { () =>
// Write code here that turns the phrase above into concrete actions
throw new io.cucumber.scala.PendingException()
}
Though I have my code in when. Also If I don't use Scenario Outline then it works fine but I want to use Scenario Outline for my code.
I am using tags in my feature file to run my test cases. When I run my test cases with command sbt test #tag1, test cases executes fine but when all test cases are finished running on cmd I am getting following error:
[error] Expected ';'
[error] #tag1
I tried putting ";" after tag but still getting same error
What is this issue and how I can resolve it?
I have 4-5 feature files in my application. That means 4-5 tags. As of now the test case which I want to run I give path of feature file and "glue" it with step deinition in my Runner Class. How I can provide all the tags in my Runner class so that my application runs all the test cases one by one when started?
You are missing the double quotes around <type>:
When A data of "<type>" is received
Just some general advice.
When cuking keep things as simple as possible, focus on clarity and simplicity, do not worry about repetition.
Your task would be much simpler if you wrote 2 simple scenarios
Scenario: Personal data
Given Portal is running
When personal data is received
Then personal data should be persisted
Scenario: Professional data
...
Secondly don't use tags to run your features, you don't need tags yet.
You can cuke much more effectively if you avoid scenario outlines, regex's, tags, transforms etc. etc.. The main power in Cucumber is using natural language to express yourself clearly. Focus on that and keep it simple ...
I have allure reporting setup for my c# selenium framework, and everything is working fine, but I have noticed something that bothers me that I'd like to change. In every single test, there is always an attachment called "console output" that is empty and 0kb in size. My question is, Is there any way to remove/disable this?
You can see what I mean in the picture below:
I'm guessing this is the confluence of two minor bugs, one in nunit and one in allure.
On the NUnit side, the XML that is created for a test result contains an <output> element to hold the text output by the test. It sounds as if an empty element is produced when there is no output. You can check whether this is the case with your version of NUnit by examining the XML output.
On the allure side, an empty element could be ignored, but apparently, it isn't.
Either or both of these should be reported to the respective projects.
I've configured my karma.conf.js with enabled preprocessing to get a report about the code coverage of my tests. I've added this line to the preprocessors section.
preprocessors: {
'public/js/app.js': ['coverage'],
'public/js/filters.js': ['coverage'],
'public/js/directives.js': ['coverage'],
'public/js/services/*.js': ['coverage'],
'public/js/controllers/*.js': ['coverage'],
},
What I'm get is a report that is totally wrong. I know that I've written tests for each modules and the function within. But the coverage report shows me only the tests for the services correctly.
For instance the tests for directives. I know that I've written some tests and the tests will also be executed. But the report shows me that I've just tests for 36% of my code lines.
What could be the reason for this strange behavior?
Update:
I see this output from the spec reporter:
Directives:
bsTooltip:
when the element was created:
PASSED - should call the popup function
bsSwitchtext:
when the model isBusy changes to true:
PASSED - should call the button method with loading
when the model isBusy changes to false changes:
PASSED - should call the button method with loading
So I think that my test will all be executed.
Looks like there's issue with Typescript & Jasmine which is used by Angular. Enabling source map for test build appears to fix this issue.
I enabled source map in Angular 6.1 as follows,
Go to angular.json and in main project, find test, and add sourceMap:true to enable source map for the test run.
to enable that in CLI, run with command --source-map or --sm=true
Github issue links
Code coverage report issue with branch coverage (if path not taken)
ng test --code-coverage in 6.1 improperly detecting branches
I'm obligated to write this answer as I had the same issue, and this was the first question on Google search.
Try adding a console.log('testing123'); at one of the points which shows as not covered. If it shows up when you run the tests you know that something is going wrong with Istanbul.
However my guess would be that either there is something wrong with your configuration, and those tests are not running at al, or the tests are not executing the code as you think they are.
Try changing what you have in preprocessors to:
preprocessors: {
'**/public/js/**/*.js': ['coverage']
},
I was not able to the report to work unless I followed the specific syntax with **/ before directors in the preprocessors object.
The karma-coverage documentation includes the preceding **/ before directories listed in preprocessors.
Based on this SO answer.
I am running some NUnit tests automatically when my nightly build completes. I have a console application which detects the new build, and then copies the built MSI's to a local folder, and deploys all of my components to a test server. After that, I have a bunch of tests in NUnit dll's that I run by executing "nunit-console.exe" using Process/ProcessStartInfo. My question is, how can programmatically I get the numbers for Total Success/Failed tests?
Did you consider using a continous integration server like CruiseControl.NET?
It builds and runs the tests for you and displays the results in a web page. If you just want a tool, let the nunit-console.exe output the results in XML and parse/transform it with an XSLT script like the ones coming from cruise control.
Here is an example of such an XSL file if you run the transformation on the direct output of nunit-console.exe then you will have to adapt the select statements and remove cruisecontrol.
However it sounds like you might be interested in continuous integration.
We had a similar requirement and what we did was to read into the Test Result XML file that is generated by NUnit.
XmlDocument testresultxmldoc = new XmlDocument();
testresultxmldoc.Load(this.nunitresultxmlfile);
XmlNode mainresultnode = testresultxmldoc.SelectSingleNode("test-results");
this.MachineName = mainresultnode.SelectSingleNode("environment").Attributes["machine-name"].Value;
int ignoredtests = Convert.ToInt16(mainresultnode.Attributes["ignored"].Value);
int errors = Convert.ToInt16(mainresultnode.Attributes["errors"].Value);
int failures = Convert.ToInt16(mainresultnode.Attributes["failures"].Value);
int totaltests = Convert.ToInt16(mainresultnode.Attributes["total"].Value);
int invalidtests = Convert.ToInt16(mainresultnode.Attributes["invalid"].Value);
int inconclusivetests = Convert.ToInt16(mainresultnode.Attributes["inconclusive"].Value);
We recently had a similar requirement, and wrote a small open source library to combine the results files into one aggregate set of results (as if you had run all of the tests with a single run of nunit-console).
You can find it at https://github.com/15below/NUnitMerger
I'll quote from the release notes for nunit 2.4.3:
The console runner now uses negative return codes for errors encountered in trying to run the test. Failures or errors in the test themselves give a positive return code equal to the number of such failures or errors.
(emphasis mine). The implication here is that, as is usual in bash, a return of 0 indicates success, and non-zero indicates failure or error (as above).
HTH
I've setup some Nunit tests for validating my statistical formulas within my .net v2 application, for company records i need to have a printed copy of this output. Is anyone aware of any commands in NUnit to automatically print the XML to default printer?
If printing isn't possible saving to a folder may work for us.
thanks in advance
The NUnit console automatically gives the results as xml. To state your own name on the xml file, this is what you need to do:
nunit-console /xml:someFileNameHere.xml yourFileWithNUnitTestsHere.dll