Selective test execution in Karma Jasmine using pattern matching - command-line

I'm having trouble invoking the command line option on karma-jasmine which allows for the execution of only those tests which match a given pattern. My spec reads as follows:
/path/to/single-test/main.spec.js
describe('my first test suite', function() {
it('always passes', function() {
expect(true).toBe(true);
});
it('still always passes', function() {
expect(true).toBe(true);
});
});
I'm assuming the description (for example "still always passes") is the item against which the pattern specified by the grep command line option is matched. When I attempt to run the second example based on the fact that its description is the only example containing the word "still", both examples are executed instead of just the one:
$ karma start -- --grep=still
INFO [karma]: Karma v0.12.35 server started at http://localhost:9876/
INFO [launcher]: Starting browser PhantomJS
INFO [PhantomJS 1.9.8 (Linux 0.0.0)]: Connected on socket 7Dn7Ez1Reap7ch0Uzsb0 with id 44623726
PhantomJS 1.9.8 (Linux 0.0.0): Executed 2 of 2 SUCCESS (0.002 secs / 0.001 secs)
How do I execute just this one example based on a pattern? The official documentation doesn't give a sample of the usage of the pattern matching option.
I read in the discussion of a pull request, that the grep option can be used in conjunction with "fit" and "fdescribe." This works when tested. However, in the case of using grep with "fit", what's the purpose of the pattern argument to the grep option? (It would be nice to be able to execute tests selectively without the need to augment source code!)
Here is the remainder of the files in my project for reference:
/path/to/single-test/karma.conf.js
module.exports = function(config) {
config.set({
basePath: '',
frameworks: ['jasmine'],
files: ['*.spec.js'],
exclude: [],
preprocessors: {},
reporters: ['progress'],
port: 9876,
colors: true,
logLevel: config.LOG_INFO,
autoWatch: false,
browsers: ['PhantomJS'],
singleRun: true
});
};
/path/to/single-test/package.json
{
"name": "single-test",
"version": "1.0.0",
"description": "",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "MIT",
"devDependencies": {
"jasmine-core": "^2.3.4",
"karma": "^0.12.35",
"karma-jasmine": "^0.3.5",
"karma-phantomjs-launcher": "^0.2.0",
"phantomjs": "^1.9.17"
}
}

You have to start a Karma server, then specify the --grep option in a Karma runner. I.e. something along the lines of:
karma start path/to/karma.conf.js
Then in another terminal:
karma run path/to/karma.conf.js -- --grep=still
It is important that you set singleRun: false in the configuration options.

There is karma-jasmine-spec-tags plugin which helps to filter running tests by tags in their names.
Example usage:
$ karma start --tags smoke
$ karma start --skip-tags slow,bench
$ karma start --tags bench --skip-tags slow
$ karma start --tag-prefix 'scope:' --tags critical
Where a spec is following:
describe('Example test', () => {
it('should be a #smoke test', () => {
// ...
});
it('#slow test', () => {
// ...
});
})
describe('Performance test suite #bench', () => {
it('#fast #smoke test', () => {
// ...
});
it('#slow test', () => {
// ...
});
})
describe('Custom tag prefix', () => {
it('test scope:critical', () => {
// ...
});
})

Related

how can Parametrize protractor-config file for different browsers and test suites

I am running my Protractor Test from command npm run e2e
I want a way so that if I pass npm run e2e firefox then my test will get executed on Firefox browser.
Or if I run npm run e2e chrome then it should run on chrome
if I pass both npm run e2e firefox chrome then my test should run on both the browser in parallel.
Is it possible to parametrize protractor config file?
Similarly if I can pass test suite name via command and it should execute only tests under that particular test suite.
Here is my config file and this is what I want to achieve:
`//var HtmlReporter = require('protractor-html-screenshot-reporter');
exports.config = {
allScriptsTimeout: 30000,
//Add parameters for browser names
params:{
pass: {
browserName : 'chrome',
testSuitName : 'e2e/TestSuites/_BVT/*.js',
}
},
suites: {
//Define here List of Sanity Test Scenarios:
BVT : testSuitName,
},
// configure multiple browsers to run tests
multiCapabilities: [
shardTestFiles: true,
maxInstances: 2
{'browserName': browserName}
],
baseUrl: 'http://mytestUrl/',
framework: 'jasmine2',
jasmineNodeOpts: {
defaultTimeoutInterval: 30000
},
onPrepare: function() {
var jasmineReporters = require('jasmine-reporters');
browser.driver.manage().window().maximize();
return browser.getProcessedConfig().then(function(config) {
var browserName = config.capabilities.browserName;
var junitReporter = new jasmineReporters.JUnitXmlReporter({
consolidateAll: true,
savePath: 'tests/test-results',
filePrefix: browserName + '-xmloutput',
modifySuiteName: function(generatedSuiteName, suite) {
return browserName + '.' + generatedSuiteName;
}
});
jasmine.getEnv().addReporter(junitReporter);
});
},
resultJsonOutputFile: 'tests/test-results/output.json'
};`
Would appreciate any help on this.
Thanks in advance.
I know this post is a bit old now, but this solution may help people having a similar problem.
Use multiple config files, one for each browser type, setup a base config file, and require that into each of the other config files, and then create an npm script for each config file. No need for parameters and everything stacks nicely.
so the base config (called something like "protractor_base.js")would look something like:
exports.config = {
seleniumAddress: 'http://localhost:4444/wd/hub',
rootElement: '[ng-app]',
allScriptsTimeout: 60000,
framework: 'jasmine',
specs: ['example-spec.js']
};
And then your other configs (ie "protractor_chrome.conf.js") would look something like:
protractor = require('./protractor_base.js');
var config = protractor.config;
config.capabilities: {
'browserName': 'chrome'
};
exports.config = config;
You can then specify a multi browser config, just a chrome one, etc.
--capabilities.chromeOptions.args=headless --capabilities.chromeOptions.args=disable-gpu --capabilities.chromeOptions.args=window-size=1200,600
The above code should work for you.
I was looking for passing parameters from command line to protractor config file and found a way to do this:
npm run e2e -- --baseUrl=http://testurl:8080 --suite=suite_name_defined_in_config --capabilities.browserName=browser_Name
where my npm package.json :
"e2e": "protractor tests/protractor-conf.js",
and config file contains :
suites: {
BVT: 'e2e/TestSuites/_BVT/*.js',
Full: 'e2e/TestSuites/Full/**/*.js',
},
capabilities: {
'browserName': 'chrome'
},
baseUrl: 'http://localhost:8080/',

Karma: browser idle

I am trying Karma for the first time, and after several hours I still can't get it to work.
When I run the test by typing karma start karma.conf.js in the terminal, the browser window opens and displays the following (I have also tried with Chrome with the same result):
This is the terminal output:
29 07 2015 16:27:12.835:INFO [karma]: Karma v0.13.3 server started at http://localhost:9876/
29 07 2015 16:27:12.852:INFO [launcher]: Starting browser Firefox
29 07 2015 16:27:15.866:INFO [Firefox 33.0.0 (Windows 7 0.0.0)]: Connected on socket HA1RSN-QsWuAO7NIAAAA with id 26755366
My karma.conf.js file is located at the root of my Node.js project and looks like this:
module.exports = function(config) {
config.set({
basePath: '',
frameworks: ['jasmine'],
files: [
'tests/unit/test.js'
],
exclude: [],
preprocessors: {},
reporters: ['progress'],
port: 9876,
colors: true,
logLevel: config.LOG_INFO,
autoWatch: false,
browsers: ['Firefox'],
singleRun: false
})
}
My test.js file looks like this (example test from a book, my actual tests will the Angular.js tests):
describe("First Test", function () {
var counter;
beforeEach(function () {
counter = 0;
});
it("increments value", function () {
counter++;
expect(counter).toEqual(1);
});
it("decrements value", function () {
counter--;
expect(counter).toEqual(0);
});
});
I'm using Node.js version 0.12.05.
I appreciate any help as I feel really lost here.
You need to trigger a test run, if you want it to execute. There are two ways to do this
Run karma run karma.conf.js in a second terminal window in the same working directory
Change the option singleRun to true, this way it will start, execute the test and then exit.

browserify/karma/angular.js "TypeError: Cannot read property '$injector' of null" when second test uses "angular.mock.inject", "currentSpec" is null

I have an Angular.js app and am experimenting with using it with Browserify. The app works, but I want to run tests too, I have two jasmine tests that I run with karma. I user browserify to give me access to angular.js and angular-mocks.js and other test fixtures within the tests.
Versions are:-
"angular": "^1.4.0",
"angular-mocks": "^1.4.0",
"browserify": "^10.2.3",
"karma": "^0.12.32",
"karma-browserify": "^4.2.1",
"karma-chrome-launcher": "^0.1.12",
"karma-coffee-preprocessor": "^0.2.1",
"karma-jasmine": "^0.3.5",
"karma-phantomjs-launcher": "^0.1.4",
If I run the tests individually (by commenting one or the other from the karma.conf file) they both work OK. (yey!)
But if I run them both I get this error
TypeError: Cannot read property '$injector' of null
at Object.workFn (/tmp/3efdb16f2047e981872d82fd8db9c0a8.browserify:2272:22 <- node_modules/angular-mocks/angular-mocks.js:2271:0)
looking at line 2271 of the angular.mocks.js It reads
if (currentSpec.$injector) {
So clearly currentSpec is somehow now null.
I have isolated the problem to when I call "angular.mock.inject" in the second test.
beforeEach(angular.mock.inject(function (_GridUtilService_) {
gridUtilService = _GridUtilService_;
}));
If I comment this out it works, but obviously I can't then run a test n my gridUtilService.
Does anyone know how to run two (or more :-) angular-mock jasmine tests with karma and browserify?
below are my tests, karma.conf file. the Angular services work when deployed but for this purpose they can simply be dumb services that do nothing.
karma.conf:-
// Karma configuration
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['browserify', 'jasmine'],
// list of files / patterns to load in the browser
files: [
'src/main/assets/js/**/*.js',
// 'src/test/**/*.js'
'src/test/services/SettingUtil*.js',
'src/test/services/GridUtil*.js'
],
// list of files to exclude
exclude: [
'src/main/assets/js/**/app-config.js'
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
'src/main/assets/js/**/*.js': ['browserify'],
'src/test/**/*.js': ['browserify']
},
browserify: {
debug: true,
extensions: ['.js', '.coffee', '.hbs']
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['progress'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_DEBUG,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: true,
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
// browsers: ['PhantomJS', 'Chrome'],
// browsers: ['PhantomJS'],
browsers: ['Chrome'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: false
});
};
src/test/services/SettingUtilServiceTest.js:
'use strict';
describe("SettingUtilServiceTest.", function() {
it("contains spec with an expectation", function() {
expect(true).toBe(true);
});
require('angular');
require('angular-mocks');
// can't do below see error at https://github.com/xdissent/karma-browserify/issues/10
//beforeEach(module('dpServices'));
//so need todo this
beforeEach(angular.mock.module('dpServices'));
var fixtures = require('./serviceFixtures.js');
var sus = fixtures.settingUtilServiceTestFixtures;
var ts1 = sus.tablesetting1;
var ts2 = sus.tablesetting2;
var settingUtilService;
beforeEach(angular.mock.inject(function (_settingUtilService_) {
settingUtilService = _settingUtilService_;
}));
it('should return an object containing mins and maxs from function minMaxes()', function() {
expect(ts1).toBeDefined();
expect(ts2).toBeDefined();
var minMaxs = settingUtilService.minMaxs(ts1);
var mins = minMaxs.mins;
expect(mins).toBeDefined();
var maxs = minMaxs.maxs;
expect(maxs).toBeDefined();
});
});
src/test/services/GridUtilServiceTest.js:
'use strict';
describe("GridUtilServiceTest.", function() {
it("is a set of tests to test GridUtilService.", function() {
expect(true).toBe(true);
});
require('angular');
require('angular-mocks');
// can't do below see error at https://github.com/xdissent/karma-browserify/issues/10
// beforeEach(module('dpServices'));
//so need todo this
beforeEach(angular.mock.module('dpServices'));
var fixtures = require('./gridFixtures.js');
var gridFix = fixtures.gridUtilServiceTestFixtures;
var ts1 = gridFix.tablesetting1;
var ts2 = gridFix.tablesetting2;
var ts3 = gridFix.tablesetting3;
var gridUtilService;
beforeEach(angular.mock.inject(function (_GridUtilService_) {
gridUtilService = _GridUtilService_;
}));
it('should return an object containing mins and maxs from function minMaxes()', function() {
expect(ts1).toBeDefined();
expect(ts2).toBeDefined();
expect(ts3).toBeDefined();
});
});
If you need access to the angular setup I can provide it (split into multiple files using browserify's require() function and built with gulp... but as I say the app runs ok and the tests only fail when there are two tests, so I think the issue is with karma-jasmine and angular-mocks or overwriting the currentSpec variable.
If anyone knows how to split my angular tests into multiple tests (I don't want a monolithic angular test) without the error message all help is appreciated. thanks.
I had the same issue, the issue for me was because I was requiring angular and angular-mocks exactly as you were inside each describe block. I moved the two lines
require('angular');
require('angular-mocks');
above the describe blocks and made sure to only call them once.

Running protractor tests on multiple browsers with browser stack

I am trying to run E2E tests on multiple browsers on browser stack, I took the reference from
E2E testing on multiple/parallel browsers in Protractor?
and
Internet Explorer Selenium protractor e2e tests
but the error I get every time I try to run the tests -
ERROR - Unable to start a WebDriver session.
C:\MrWhiteMVP\whitemvp-integrationtests_develop\node_modules\gulp-protractor\node_modules\protractor\node_modules\selenium-webdriver\lib\atoms\error.js:113
var template = new Error(this.message);
^
UnknownError: Authorization required
at new bot.Error (C:\MrWhiteMVP\whitemvp-integrationtests_develop\node_modules\gulp-protractor\node_modules\protractor\node_modules\selenium-webdriver\lib\atoms\error.js:113:18)
at Object.bot.response.checkResponse (C:\MrWhiteMVP\whitemvp-integrationtests_develop\node_modules\gulp-protractor\node_modules\protractor\node_modules\selenium-webdriver\lib\atoms\response.js:103:11)
But if I run the tests on 1 browser, then it works perfectly fine. this is how my conf file looks like
'use strict';
exports.config = {
seleniumAddress: 'http://hub.browserstack.com/wd/hub',
multicapabilities: [{
'browserstack.user': 'testuser',
'browserstack.key': 'testkey',
// Needed for testing localhost
// 'browserstack.local': 'true',
// Settings for the browser you want to test
'browserName': 'chrome',
'version': '36.0',
'os': 'OS X',
'os_version': 'Mavericks',
'resolution': '1024x768'
},
{
'browserstack.user': 'testuser',
'browserstack.key': 'testkey',
// Needed for testing localhost - 'browserstack.local': 'true',
// Settings for the browser
'browserName': 'firefox',
'os': 'windows'
}],
baseUrl: 'http://origin-develop.mvp.livebranches.com',
// The require statement must be down here, since jasmine-reporters
// needs jasmine to be in the global and protractor does not guarantee
// this until inside the onPrepare function.
onPrepare: function()
{
require('jasmine-reporters');
jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter('xmloutput', true, true));
},
},
};
could anyone please tell me what am I doing wrong here, also we use gulp ti run specs but my problem is it is saying not even going beyond authentication.
I think that first of all, you have an extra comma on your config.
onPrepare: function()
{
require('jasmine-reporters');
jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter('xmloutput', true, true));
},
}, <-----
};

Testing async with karma

I am trying to set up some async tests using karma and jasmine. I am clearly making a very stupid mistake but I need it pointing out to me. After simplifying as much as possible I have the following:
package.json
{
"name": "newtest",
"version": "0.0.0",
"scripts": {
"test": "karma start karma.conf.js"
},
"devDependencies": {
"karma": "^0.12.28",
"karma-chrome-launcher": "^0.1.5",
"karma-jasmine": "^0.2"
}
}
karma.conf.js
module.exports = function(config) {
config.set({
basePath: '',
frameworks: ['jasmine'],
files: [
'tests/**/*.js'
],
reporters: ['progress'],
port: 9876,
colors: true,
logLevel: config.LOG_INFO,
autoWatch: false,
browsers: ['Chrome'],
singleRun: true
});
};
tests/dummy.spec.js
describe("Testing async", function() {
it('should fail this test', function(done) {
setTimeout(function(){
expect(1).toBe(2);
done();
}, 1000);
});
it('should not fail this test', function(done) {
done();
});
});
and I am getting the following:
npm test
> newtest#0.0.0 test /home/mark/Projects/newtest
> karma start karma.conf.js
INFO [karma]: Karma v0.12.16 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 39.0.2171 (Linux)]: Connected on socket T7j6LvNAwvS89wUdymCb with id 16891024
Chrome 39.0.2171 (Linux) Testing async should not fail this test FAILED
TypeError: undefined is not a function
at null.<anonymous> (/home/mark/Projects/newtest/tests/dummy.spec.js:12:5)
Chrome 39.0.2171 (Linux): Executed 2 of 2 (1 FAILED) (0.007 secs / 0.005 secs)
npm ERR! Test failed. See above for more details.
npm ERR! not ok code 0
So the test that I think should fail is passing fine, and vice-versa. Can someone point me to my error(s)?
I guess the first one fail because when the timeout is reach the test is allready finish, so both doesn't work.
It's has you'r not using jasmine 2.
The syntax seems good to me, what I can tell you is the difference with my configuration (wich is working) :
I have put karma-jasmine in dependencies with a ~ :
"dependencies": {
...
"karma-jasmine": "~0.2.0"
},
I'm using PhantomJS :
browsers:['PhantomJS'],
I don't know why I am having the problem, but I have tried on another machine and it works as expected.