We're running Protractor with Selenium Grid, with (for now) 2 browsers. When tests fail I see that they fail, and I see how many tests fail on each browser but somehow I cannot see on which browser the tests fail.
Is this a Selenium or a Protractor shortcoming or am I doing something wrong?
Do you use jasmine reporters? I find the console indicates clear enough in which browser the error originates. And of course the html report is very clear.
Add these in the protractor.conf.js in the onprepare block like this.
onPrepare: function () {
// The require statement must be down here, since jasmine-reporters
// needs jasmine to be in the global and protractor does not guarantee
// this until inside the onPrepare function.
require('jasmine-reporters');
jasmine.getEnv().addReporter(new jasmine.ConsoleReporter());
jasmine.getEnv().addReporter(new HtmlReporter({
baseDirectory: 'reports/screenshots' ,
takeScreenShotsOnlyForFailedSpecs: true
}));
},
Be sure to use jasmine-reporters ~0.4.0 ,the 1.0.0 does not work yet with protractor because it work with a newer jasmine version then protractor until now uses.
Related
I'm planning to setup env for testing in Visual Studio Code IDE.
The problem that I have is that VS Code in any XXX.test.js file don't know what is browser or page variable. I'm receiving such errors:
'page' is not defined.eslint(no-undef)
Is this something related only to IDE that I'm using? Some plugin from this IDE? Or maybe it is normal for jest-puppeteer and I need to get used to it?
The whole test when it runs works ok but it seems that IDE only don't know what is page.
I'm working right now on this simple example and I'm just learning this but I'm trying to setup env for it in the same time. My Network.test.js file looks like this. No imports or require at all at the top of the file.
describe("Google", () => {
beforeAll(async () => {
await page.goto("https://google.com");
});
it('should display "google" text on page', async () => {
await expect(page).toMatch("google");
});
});
Later I need to switch it to Typescript which I think will be also a challenge. Now I just want to focus on pure javascript approach.
How to get rid of this IDE errors that are not errors in reality?
Add the Puppeteer globals, Page and Browser, to your ESLint config.
Then, add the following to your .eslintrc file:
env: {
jest: true,
},
// Puppeteer globals
globals: {
page: true,
browser: true,
},
See Here : Link
Should Karma launch a browser that with my front-end app loaded?
I should see a landing page of my app if Karma is correctly loaded, is this correct?
If not loading my app landing page, why Karma launch a visible browser?
This is my karma.config.js
I am learning Karma, not sure if I did everything alright.
var webpackConfig = require('./webpack.config.js');
webpackConfig.entry = {};
/*
The entry point from the referenced Webpack configuration
has to be removed or tests will fail in weird and inscrutable
ways. Easy enough, just define an empty entry object.
*/
module.exports = function(config) {
config.set({
bathPath: './dist/',
webpack: webpackConfig,
/* reference the Webpack configuration in the Karma configuration */
preprocessors: {
// './dist/bundle.js': ['webpack'],
'./test/*.js': ['webpack']
/* application’s entry point, karama-webpack plugin requires this */
/* you have to tell Karma that you want the karma-webpack plugin
to process these files. That’s what ['webpack'] does.
*/
},
/*
The plugins section is missing from my karma.config.js file.
When missing, Karma will load any plugins it can find in the
node-modules folder. Much simpler in my opinion.
*/
frameworks: ['jasmine'],
files: [ {pattern: 'index.html', }, './test/*.js'],
/* The files array determines which files are included in the
browser and which files are watched and served by Karma
*/
browsers: ['Chrome'],
reporters: ['progress'],
port: 9876
});
};
This is correct. Karma loads a page which includes all your javascript and all your specs. It then runs any specs that might be in there. This is not full integration testing. This is javascript unit testing.
If you wanted to debug in browser, you can click the debug button and you'll see the page that it runs. Hit that page and check out the console and you'll see your spec output.
You'll need something more for full integration tests like protractor or capybara or ...
Probably depends on what you're using to build your site.
I managed to get a couple of EAK/grunt based Ember apps upgraded to 1.11 with HTMLBars, and then got them migrated to Ember CLI/Brocolli. The unit tests were setup for karma test runner so I'm looking at how to get those running in the CLI projects now, but I didn't write the tests and really have no experience with unit testing javascript modules.
Searching around the iNet, I can see that others have also used karma becasue of its coverage output and are trying to get it to work with Ember CLI, but that Ember Core isn't supporting it, though they say anyone should be able to get it set up with a custom addon. I'm also trying to use the 'testem' runner to see what sticks with that.
The Ember site does have an 'automating tests with runners' page for v1.10, with sections on 'testem' and 'karma', but it doesn't appear for v1.11 so I can't tell from that site what is or isn't relevant. But it seems like I should be able to work out a solution for the karma test runner, so I added the old devDependencies to the project package.json:
"karma": "^0.12.31",
"karma-chai": "~0.1.0",
"karma-chrome-launcher": "~0.1.2",
"karma-coverage": "~0.2.1",
"karma-firefox-launcher": "~0.1.3",
"karma-junit-reporter": "~0.2.1",
"karma-mocha": "~0.1.3",
"karma-phantomjs-launcher": "~0.1.2",
"karma-sinon-chai": "~0.1.5"
I also dropped the old 'karma.conf.js' (along with a few other karma confs) in the project and updated the paths inside (from 'vendor' to 'bower_components'). I did find a 'ember-cli-karma' node mode and installed it, but it seems to just have a 'package.json'. It has no docs and seems like just a stubbed out starter project with no implementation. I also installed 'karma', 'karma-cli' and 'testem' node modules.
The testem docs say to add you src and test files to 'testem.json', but with out examples I don't know what that means; a list of every src and test file? With what path; relative, absolute? Forward slashes, backslashes? preceded with / or ./ or ../? I just left them out because I think the system just finds the src and tests by convention.
When I run 'karma init' I get:
readline.js:529
this.line = this.line.slice(this.cursor);
^
TypeError: Cannot read property 'slice' of undefined
When I run 'testem' I get:
TEST'EM 'SCRIPTS!
Open the URL below in a browser to connect.
http://localhost:7357/aN;0faN;NaNf
...then the project's '../tests/index.html' loads in a browser, but is not able to 'find' any of the asset files (css, js) so nothing executes or renders correctly. I just see template expressions ({{content-for 'head'}}, etc).
When I run 'ember test' I get:
Building...BuildingBuilding.Building..Building...Built project successfully.
1..0
# tests 0
# pass 0
# fail 0
# ok
No tests were run, please check whether any errors occurred in the page (ember test --server) and ensure that you have a test launcher (e.g. PhantomJS) enabled.
When I run 'ember test --server' I get:
The test index.html loaded in a browser with a test report. When I uncheck 'hide passed tests' the report indicates '29 passed, 28 failed'. It has 11 sections where a particular test may have 3 problems such as 'could not load', 'failed', 'could not find module', 'attempting to register an unknown factory' or 'died'.
With this, I'm obviously running testem and not karma, so may as well work on getting testem working and figure out karma later. If there were more examples and migration troubleshooting docs I might have a systematic way to work through some of these problems.
I ran into "No tests were run,..." problem recently after a node upgrade. I fixed it with a:
npm install -g phantomjs
This provides some additional options as well:
https://github.com/ember-cli/ember-cli/issues/3969
I had the Cannot read property 'slice' of undefined error on MS Windows, running via MSys2. I have solved it by using karma init from an ordinary cmd prompt.
I have a project that uses "coffee-script": "^1.7.1" in its package.json.
The code has this line in it:
[{id: id, name: name}, ...] = result.rows
This compiles fine using coffeescript version 1.7.1
The problem is that I am trying to use mocha for unit tests and it gives me an error on this line:
Parse error on line xyz: Unexpected '...'
Apparently mocha uses an older coffeescript. Is there a way to make it work without adjusting the source for mocha?
EDIT:
my Gruntfile.coffee:
'use strict'
module.exports = ->
#initConfig
cafemocha:
src: ['test/*.coffee']
options:
reporter: 'spec'
ui: 'bdd'
coffee:
compile:
files:
'lib/mylib.js': ['src/*.coffee']
#loadNpmTasks 'grunt-cafe-mocha'
#loadNpmTasks 'grunt-contrib-coffee'
#registerTask 'default', ['coffee', 'cafemocha']
I added mocha.opts to the test directory:
--require coffee-script/register
--compilers coffee:coffee-script/register
--reporter spec
--ui bdd
but, still, when I run grunt, it gives me the same error. I am new to this environment, and I find it too complicated, please help.
Starting from version 1.7.x CoffeeScript compiler should be explicitly registered (see change log for version 1.7.0).
So, the problem is that CoffeeScript compiler is not registered when you're running your mocha tests, so node.js treats all your .coffee files as .js files.
The best possible solution is to specify --compilers option for your mocha tests:
--compilers coffee:coffee-script/register
If you don't want to include it to every mocha call, you could set it up using mocha.opts file.
Here are some useful links:
issue about it on github
reference in mocha docs
the reason behind this breaking change in CoffeeScript engine
Update
Looks like your issue is much deeper then I thought.
First, grunt-cafe-mocha doesn't respect mocha.opts because it's running tests by requireing mocha as a dependency, instead of calling mocha test runner.
So, it would've been enough to add require('coffee-script/register') to the top of your gruntfile, if not for this old grunt issue.
In short, grunt uses coffee-script 1.3.x, forcing all its tasks to use the same version of coffee. I had the same problem with grunt-contrib-connect, being unable to use latest coffee-script in my express app.
So, the only help I can offer you is a small grunt task I wrote to solve similar problem in one of my projects. It runs mocha in a separate child process, thus completely isolating it from grunt.
N.B. I had a thought about releasing this task to npm, but considered it too minor.
I'm having troubles to add test code coverage, I'm using Karma and files added to Karma are already bundled with browserify, so in karma.conf.coffee it looks like that:
files: [
{ pattern:'bin/public/client/app.js', served:yes: included:yes }
{ pattern:'src/lib/vendor/angular-mocks/angular-mocks.js', served:yes: included:yes }
{ pattern:'bin/tests.js', served:yes: included:no }
]
And that works for running the test, but not coverage
I'm using karma-coverage npm package, and this:
preprocessors: 'bin/public/client/app.js':['coverage']
reporters: ['progress','coverage']
Actually does create coverage stat files, but those are completely wrong, because it reddens parts that browserify brought from node_modules (because I don't have tests to cover those)
Ideally I have to gather source maps that browserify generates, and run coverage against those, but browserify embeds source maps into .js files. Using karma-sourcemap-loader lets me see original coffeescript files of tests, when debugging (for some reason it works only in ChromeCanary, nevertheless it works)
I tried to do preprocessors: 'src/client/**/*.coffee':['coverage'], but that yields no stats at all saying 'No data to display'
Do you have any ideas?
upd:
I've figured by running browserify-istanbul transform right after coffeeify and that gave me nice diagram like this:
Now, I need somehow to remove app.js from it, because really it doesn't matter and really confuses
upd:
Oh, instead of javascript I have to supply coffee files:
preprocessors : {
'bin/tests.js': ['sourcemap']
'src/client/**/*.coffee': ['coverage']
}
Seems I answered my own question. Also it seems there's a bug in current version of karma-coverage - it throws an error when coverageReporter.type is html (which is by default html). I'm glad I've figured it out. It's always nice to see how much code covered by tests