We have a Protractor-CucumberJS framework using BDD feature files and need to create a suite of sanity tests by tagging the relevant scenarios as '#sanity' in feature files and run only the scenarios with this tag.
With this setup, Protractor launches the browser for each feature file one by one, checks if it contains the '#sanity' tag and closes the browser if tag not found and so on.
This will take up a lot of time as there are close to 200 feature files and hence we want the the browser to launch only if the feature file contains the required tag and not otherwise.
Is there a way to achieve that.
Config file -
exports.config = {
seleniumAddress: 'http://127.0.0.1:4444/wd/hub',
framework: 'custom',
frameworkPath: require.resolve('protractor-cucumber-framework'),
specs: [../features/*.feature]
cucumberOpts: {
require: [
'../step-definitions/**/*_Steps.js',
'generateJsonReport.js'
],
format: 'pretty',
tags: ['#sanity'],
},
multiCapabilities: [{
browserName: 'chrome',
shardTestFiles: true,
}
],
onPrepare: function () {
browser.driver.manage().window().setSize(1600, 800);
}
};
This is not possible due to the fact that you are using the option shardTestFiles:true, see the config.
If you disable it, or remove it, it will first search all the files (without parsing all files and opening a browser for it) for your specific tag and only run the featurefiles that have that tag.
Use tags:'#sanity', in Conf file, This will call only Sanity features
Related
I want to run feature files in a desired order or sequence, for example:
tags:"`#ProtractorScenario` or #CucumberScenario"
But cucumber scenario is getting executed first. Can someone guide me on this?
Note: Cucumber is executing scenario based on alphabetical order of feature file in folder
Also, in cases with more than 50+ feature files, what would be the best way to define sequencing of cucumber feature files?
In order to have reliable tests, your tests should be independent and not rely on the order they are run in. The reason being that your test shouldn't depend on the system being in a certain state, as this will lead to flaky tests. Each of your tests should set up the expected state (and teardown!), so they can be run independently.
Below is how protractor executes cucumber feature files:
Protractor finds out all feature files specified in specs, save the absolute file path into an array, let's call it feature_list.
Protractor starts a session (start a browser instance)
Protractor generates a Cucumber CLI as below, and execute the CLI to hand over the running control cucumber:
./node_modules/bin/cucumber --require xxx --format xxx feature1,feature2,....featureN
feature1,feature2,....featureN calculated by feature_list.join(',')
From above, we can learn the only opportunity to change the order
is given an order-done feature_list to protractor specs.
Note: every member of the feature_list should be absolute/relative
path of single feature file. folder and wildcard are not recommended to appear in the path.
You can get a solution code from my github: spec.filter.js, which implements:
filter feature file by cucumberOpts.tags
order filter result of above step 1 by priority
Guide to use spec.filter.js:
// protractor conf file
const specFilter = require('./spec.filter.js');
var config = {
seleniumAddress: 'xxxxx',
capabilities:'xxxx',
framework: 'custom',
frameworkPath: require.resolve('protractor-cucumber-framework'),
ignoreUncaughtExceptions: true,
specs: [
'./aa/**/*.feature',
'./bb/**/*.feature'
],
cucumberOpts: {
require: [
'xxx'
],
priorities: {
// feature has tag #SPGC-21542 or #SPGC-21944 or #SPGC-21946
// will has priority 1
'1': ['#SPGC-21542 or #SPGC-21944', '#SPGC-21946'],
// feature has tag #SPGC-22055 will has priority 2,
// feature has heighest priority will put ahead at
// the `specs` list and get executed firstly.
'2': ['#SPGC-22055']
}
tags: ""
}
....
};
exports.config = specFilter(config);
I'm trying to run unit tests using karma and i'm getting the error You need to include some adapter that implements __karma__.start method!. I tried running with grunt and karma start commands. I did googling and all the solutions didn't work out. Not sure what i'm doing wrong. I included the right adapter which comes with karma-jasmine, which has the __karma__.start method, under plugins in karma.conf.js file. Here's my configuration file :-
module.exports = function(config){
config.set({
// root path location that will be used to resolve all relative paths in files and exclude sections
basePath : '../',
files : [
'bower_components/angular/angular.js',
'bower_components/angular-mocks/angular-mocks.js',
'node_modules/requirejs/require.js',
'node_modules/karma-jasmine/lib/adapter.js',
'app.js',
'mainapp/mainapp.js',
'mainapp/notes/notes.js',
'mainapp/notes/partial/create/create.js',
'mainapp/notes/partial/create/create-spec.js'
],
// files to exclude
exclude: [
'bower_components/angular/angular/*.min.js'
],
// karma has its own autoWatch feature but Grunt watch can also do this
autoWatch : false,
// testing framework, be sure to install the correct karma plugin
frameworks: ['jasmine', 'browserify', 'requirejs'],
// browsers to test against, be sure to install the correct browser launcher plugins
browsers : ['PhantomJS'],
// map of preprocessors that is used mostly for plugins
preprocessors: {
'mainapp/notes/partial/create/create-spec.js' : ['browserify']
},
reporters: ['progress'],
// list of karma plugins
plugins : [
'karma-teamcity-reporter',
'karma-chrome-launcher',
'karma-phantomjs-launcher',
'karma-babel-preprocessor',
'karma-requirejs',
'karma-jasmine',
'karma-browserify'
],
singleRun: true
})}
Using the requirejs framework turns off the automatic calling of __karma__.start. You need to create a file that a) configures RequireJS and b) calls __karma__.start to kick of the tests. Here's an example. It scans through the files that Karma is serving to find the files that contains tests. This is based on a naming convention that any file that ends with spec.js or test.js is a test file. It converts the file names to module names. It then configures RequireJS. One thing it does is pass all the test modules as deps so that all test modules are loaded right away. It sets __karma__.start as callback so that when all modules passed in deps are loaded, the tests start.
var allTestFiles = [];
var TEST_REGEXP = /(spec|test)\.js$/i;
Object.keys(window.__karma__.files).forEach(function(file) {
if (TEST_REGEXP.test(file)) {
// Normalize paths to RequireJS module names.
allTestFiles.push(file);
}
});
require.config({
baseUrl: '/base',
paths: {
'chai': 'node_modules/chai/chai',
'chai-jquery': 'node_modules/chai-jquery/chai-jquery',
'jquery': 'node_modules/jquery/dist/jquery.min',
'underscore': '//cdnjs.cloudflare.com/ajax/libs/underscore.js/1.6.0/underscore-min',
'sn/sn-underscore': 'static/scripts/sn/sn-underscore',
'vendor/jquery-ui': '//cdnjs.cloudflare.com/ajax/libs/jqueryui/1.11.4/jquery-ui.min'
},
deps: allTestFiles,
callback: window.__karma__.start
});
You need to include this file in your files parameter in your karma.conf.js file. Since you use the requirejs framework, you just need to put it first in the list. For instance, if you call the file test-main.js (as suggested in Karma's documentation):
files: [
'test-main.js',
...
]
If you load RequireJS yourself by listing it in files, you need to put test-main.js after RequireJS.
I am successfully able to run the protractor scripts. Below is the protractor.config.js file which I am using to run the protractor scripts.
var Jasmine2Reporter = require('protractor-jasmine2-screenshot-reporter');
var HtmlReporter = require('protractor-html-screenshot-reporter');
var jReporter=new Jasmine2Reporter({
dest: './protractor-result',
fileName: 'protractor-demo-tests-report.html'
});
var reporter=new HtmlReporter({
baseDirectory: './protractor-result', // a location to store screen shots.
docTitle: 'Protractor Demo Reporter',
docName: 'protractor-demo-tests-report.html'
});
exports.config = {
allScriptsTimeout: 11000,
specs: [
'testCaseOne.spec.js' // Hardcoded to run single script.
'*.spec.js' // to run all scripts.
],
capabilities: {
'browserName': 'chrome'
},
baseUrl: 'http://localhost:8000/app/',
framework: 'jasmine2',
};
I am successfully able to run the protractor scripts. Below is the protractor.config.js file which I am using to run the protractor scripts
To run above file, I used below command.
$ npm run protractor
My Expectation:
Now, I would like to run the single protractor script from command prompt. How this can be achieved? This will be useful when I will try to run the protractor test cases from any test management tool.
Can anyone please help me on this.
Try this:
protractor protractor.conf.js --specs='specs/run-just-this-spec.js'
If you want to run a specific test you need use jasmine2 and pass the grep option. https://github.com/angular/protractor/blob/19139272d190dd9c1888d9c3fc2f480f7c6c8edb/docs/jasmine-upgrade.md
Additionally to the given answers, you can use suites, which are sets of specs:
You can have suites which consist only of one spec.
You can run particular spec like this:
protractor --suite=my-suite-name
Also you can temporarily exclude suite or spec in Jasmine using xdescribe and xit (just type x before describe or it).
Also you can focus on particular suite or spec in Jasmin using fdescribe and fit (just type f before describe or it).
Use the node.js process.env object.
var w00t = process.env.TESTED || '*';
exports.config = {
allScriptsTimeout: 11000,
specs: [
w00t + '.spec.js'
],
Prepend TESTED=testCaseOn when you start protractor to execute the desired spec. To execute all scripts add nothing so that *.spec.js will be called.
I have two environments I'm running my tests in (locally, and travic ci). And I need to make a few tweaks in my tests if I'm running them locally.
Is it possible to do it using Karma without having two separate configuration files?
You can programmatically call karma and pass it a configuration object, then listen the callback to close the server:
karma.server.start(config, function (exitCode){
if(exitCode){
console.err('Error in somewhere!');
}
});
The config object is basically an object that contains some properties and you can use it to enrich a skeleton configuration file you already have.
Imagine to have a configuration file like the following in 'path/to/karma.conf.js':
// Karma configuration
module.exports = function(config) {
config.set({
// base path, that will be used to resolve files and exclude
basePath: '../',
// frameworks to use
frameworks: ['mocha'],
files: [ ... ].
// test results reporter to use
// possible values: 'dots', 'progress', 'junit', 'growl', 'coverage'
// choose it before starting Karma
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: false,
browsers: ['PhantomJS'],
// If browser does not capture in given timeout [ms], kill it
captureTimeout: 60000,
// Continuous Integration mode
// if true, it capture browsers, run tests and exit
singleRun: true,
plugins: [
'karma-mocha',
'karma-phantomjs-launcher'
]
});
};
Now I want to tweak it a bit before starting karma:
function enrichConfig(path){
var moreConfig = {
// say you want to overwrite/choose the reporter
reporters: ['progress'],
// put here the path for your skeleton configuration file
configFile: path
};
return moreConfig;
}
var config = enrichConfig('../path/to/karma.conf.js');
Currently with this technique we're generating several configuration for all our environment.
I guess you can configure your TravisCI configuration file to pass some arguments to the wrapper in order to activate some particular property in the enrichConfig function.
Update
If you want to pass parameters (e.g. the configuration file path) to your script, then just look up in the arguments array to pick it up.
Assume your script above it saved in a startKarma.js file, change your code to this:
var args = process.argv;
// the first two arguments are 'node' and 'startKarma.js'
var pathToConfig = args[2];
var config = enrichConfig(pathToConfig);
then:
$ node startKarma.js ../path/to/karma.conf.js
I'm using the Karma Test Runner and I've configured it to use Chrome and PhantomJS like this:
browsers = ['Chrome', 'PhantomJS'];
How can I configure Karma to open these browsers with certain command line arguments, like --diable-web-security in the case of Chrome, and --web-security=no in the case of PhantomJS?
I suppose one option would be to write a custom browser script, but that seems like overkill if there is some feature in Karma I don't know about that handles this case.
Something like this should work:
// karma.conf.js
module.exports = function(config) {
config.set({
browsers: ['Chrome_without_security','PhantomJS_without_security'],
// you can define custom flags
customLaunchers: {
Chrome_without_security: {
base: 'Chrome',
flags: ['--disable-web-security']
},
PhantomJS_without_security: {
base: 'PhantomJS',
flags: ['--web-security=no']
}
}
});
};
More information here: https://github.com/karma-runner/karma-chrome-launcher#configuration
One solution I found, which may not be the most elegant, is to actually modify Karma's launcher scripts for each browser. This has solved the problem.