I have a problem, again...
I try to make a test report from my continuous integration tests in Azure DevOps. I have written unit tests as described in:
https://code.visualstudio.com/api/working-with-extensions/testing-extension
I have written yml mostly as described here:
https://code.visualstudio.com/api/working-with-extensions/continuous-integration
Now I want to "publish" my test results...
I think to publish them, I have to create an XML(or TRX) in one of the following formats: JUnit, NUnit 2, NUnit 3, Visual Studio Test (TRX) and xUnit 2.
It seems I am limited how to create a reporter/testrunner or whatever... I don't get it.
The provided API of vscode looks like:
testRunner.configure({
ui: "tdd",
useColors: true
});
module.exports = testRunner;
The expected type for the API is:
interface MochaSetupOptions {
//milliseconds to wait before considering a test slow
slow?: number;
// timeout in milliseconds
timeout?: number;
// ui name "bdd", "tdd", "exports" etc
ui?: string;
//array of accepted globals
globals?: any[];
// reporter instance (function or string), defaults to `mocha.reporters.Dot`
reporter?: any;
// bail on the first test failure
bail?: boolean;
// ignore global leaks
ignoreLeaks?: boolean;
// grep string or regexp to filter tests with
grep?: any;
// colored output from test results
useColors?: boolean;
// causes test marked with only to fail the suite
forbidOnly?: boolean;
}
I think my best try was using that module https://www.npmjs.com/package/mocha-junit-reporter:
testRunner.configure({
reporter: 'mocha-junit-reporter',
reporterOptions: {
mochaFile: './path_to_your/file.xml'
}
});
I know it fits not the API as described, but when you have a look at the source code from the vscode-module:
function configure(opts) {
mocha = new Mocha(opts);
}
exports.configure = configure;
So it fits the documentation of the "mocha-junit-reporter" module
let a: any = {
ui: "tdd",
reporter: "xunit",
reporterOptions: {
output: normalize(join(getExtensionRootPath(), "..", "TestResults", "unit-tests", os + ".xml")),
}
};
testRunner.configure(a);
this did it for me except for Linux I will edit this answer if get managed it for Linux too.
Related
In my unit tests for my Roslyn analyzer, I need to use Microsoft.AspNetCore.Mvc in the test code string. I believe that means I need the long form of testing, so I did the following.
var expected = VerifyCS.Diagnostic(MyAnalyzer.DiagnosticId)
.WithLocation(0)
.WithArguments("Method");
var assemblies = ReferenceAssemblies
.Default
.AddPackages(ImmutableArray.Create(new PackageIdentity("Microsoft.AspNetCore.Mvc.Core", "6.0.0.0")));
await new VerifyCS.Test {
ReferenceAssemblies = assemblies,
TestState = {
Sources = {
test
},
ExpectedDiagnostics = {
expected
}
}
}.RunAsync();
When running the test, it tells me:
Unable to find package 'Microsoft.AspNetCore.Mvc.Core'. Existing packages must be restored before performing an install or update.
I'm not sure how to resolve that in the unit test.
I have been trying to automate a case in which i have to create a group of Urls. So after executing the below script all of the groups that are required are added. After completing all of its task , it is throwing timeout error. But the same objects when used in other specs works perfectly.
describe('Test for ToolbarExpandField',function(){
it('Creating a new url group',function(){
emulator.createNewURLGroup(URLGroupName,URLGroupList);
})
})
createNewURLGroup:function(URLGroupName,URLGroupList){
base.click(base.byElement(base.getLocator(emulatorObjects.dropUpBodyOption,['New URL Group'])));
emulatorObjects.uRLGroupNameField.sendKeys(URLGroupName);
browser.waitForAngular();
base.click(emulatorObjects.confirmButton);
expect(base.byElement(base.byCss("option[value = '"+URLGroupName+"']")).getText()).toEqual(URLGroupName);
for(var i = 1; i<URLGroupList.length ; i++){
tsHelper.checkPresence(emulatorObjects.addNewUrlDiv,true);
base.click(emulatorObjects.addNewUrlDiv);
emulatorObjects.urlNameField.sendKeys(URLGroupList[i].name);
emulatorObjects.urlLinkField.sendKeys(URLGroupList[i].link);
base.click(emulatorObjects.saveUrlDetails);
}
tsHelper.checkPresence(emulatorObjects.addNewUrlDiv,false);
base.click(emulatorObjects.confirmButton);// Errors occur here
}
The purpose of testing is to check and test something. So, each test case should have some expectation and it's result. That's why when you used it in some other test cases, it worked, because those test cases must be already having some expectation.
You can add expectation to Creating a new url group test case after calling createNewURLGroup function or if you don't have anything to check, then you can just add expectation which is always true (Not a good way):
Example:
it('Creating a new url group',function(){
emulator.createNewURLGroup(URLGroupName,URLGroupList);
expect(true).toBeTruthy();
})
I am trying to figure out what criteria should be for a decision to 'inline' some work as a set of calls directly in a lets say Does clause (using aliases), or have a set of separate tasks with proper dependencies. It seems like it can be done in either way
For example
var target = Argument ("target", "build");
Task ("build")
.Does (() =>
{
NuGetRestore ("./source/solution.sln");
DotNetBuild ("./source/solution.sln", c => c.Configuration = "Release");
CopyFiles ("./**/*.dll", "./output/");
});
Task ("pack")
.IsDependentOn ("build")
.Does (() =>
{
NuGetPack ("./solution.nuspec");
});
RunTarget (target);
I could 'inline' all of this right into 'pack' task, and I could have a separate task for each of nuget restore, dotnetbuild and copy files actions
Unfortunately, the main answer to this is, it depends. It depends on your own preferences, and how you want to work.
Personally, I break Tasks into a concrete piece of functionality, or unit of work. So, in the above example, I would have a Task for:
NuGetRestore
DotNetBuild
CopyFiles
NuGetPack
The thought process here being that depending on what I wanted to do, I might want to only run one of those tasks, and I wouldn't want to do everything that was required. Breaking the Tasks into individual ones, gives me the option to piece these Tasks together as required.
If you put all the aliases into a single Task, you no longer have the option of doing that.
Best practice is to have one task per step in your build process, an example flow could be:
Clean
Restore
Build
Test
Pack
Publish
Then it'll be much more clear what takes time and what's the cause of any failure.
Cake will abort on any failure so the flow will be the same, but it'll give you more granular control and insight.
There's a simple example solution at github.com/cake-build/example
Convertng your script according to that example would look something like this:
var target = Argument("target", "Pack");
var configuration = Argument("configuration", "Release");
FilePath solution = File("./source/solution.sln");
Task("Clean")
.Does(() =>
{
CleanDirectories(new [] {
"./source/**/bin/" + configuration,
"./source/**/obj/" + configuration
});
});
Task("Restore")
.IsDependentOn("Clean")
.Does(() =>
{
NuGetRestore(solution);
});
Task("Build")
.IsDependentOn("Restore")
.Does(() =>
{
if(IsRunningOnWindows())
{
// Use MSBuild
MSBuild(solution, settings =>
settings.SetConfiguration(configuration));
}
else
{
// Use XBuild
XBuild(solution, settings =>
settings.SetConfiguration(configuration));
}
});
Task("Pack")
.IsDependentOn("Build")
.Does(() =>
{
NuGetPack("./solution.nuspec", new NuGetPackSettings {});
});
RunTarget(target);
Which will give you a nice step by step summary report like this
Task Duration
--------------------------------------------------
Clean 00:00:00.3885631
Restore 00:00:00.3742046
Build 00:00:00.3837149
Pack 00:00:00.3851542
--------------------------------------------------
Total: 00:00:01.5316368
If any step fails, it'll be much more clear which.
Say you have a file with:
AddReactImport();
And the plugin:
export default function ({types: t }) {
return {
visitor: {
CallExpression(p) {
if (p.node.callee.name === "AddReactImport") {
// add import if it's not there
}
}
}
};
}
How do you add import React from 'react'; at the top of the file/tree if it's not there already.
I think more important than the answer is how you find out how to do it. Please tell me because I'm having a hard time finding info sources on how to develop Babel plugins. My sources right now are: Plugin Handbook,Babel Types, AST Spec, this blog post, and the AST explorer. It feels like using an English-German dictionary to try to speak German.
export default function ({types: t }) {
return {
visitor: {
Program(path) {
const identifier = t.identifier('React');
const importDefaultSpecifier = t.importDefaultSpecifier(identifier);
const importDeclaration = t.importDeclaration([importDefaultSpecifier], t.stringLiteral('react'));
path.unshiftContainer('body', importDeclaration);
}
}
};
}
If you want to inject code, just use #babel/template to generate the AST node for it; then inject it as you need to.
Preamble: Babel documentation is not the best
I also agree that, even in 2020, information is sparse. I am getting most of my info by actually working through the babel source code, looking at all the tools (types, traverse, path, code-frame etc...), the helpers they use, existing plugins (e.g. istanbul to learn a bit about basic instrumentation in JS), the webpack babel-loader and more...
For example: unshiftContainer (and actually, babel-traverse in general) has no official documentation, but you can find it's source code here (fascinatingly enough, it accepts either a single node or an array of nodes!)
Strategy #1 (updated version)
In this particular case, I would:
Create a #babel/template
prepare that AST once at the start of my plugin
inject it into Program (i.e. the root path) once, only if the particular function call has been found
NOTE: Templates also support variables. Very useful if you want to wrap existing nodes or want to produce slight variations of the same code, depending on context.
Code (using Strategy #1)
import template from "#babel/template";
// template
const buildImport = template(`
import React from 'react';
`);
// plugin
const plugin = function () {
const importDeclaration = buildImport();
let imported = false;
let root;
return {
visitor: {
Program(path) {
root = path;
},
CallExpression(path) {
if (!imported && path.node.callee.name === "AddMyImport") {
// add import if it's not there
imported = true;
root.unshiftContainer('body', importDeclaration);
}
}
}
};
};
Strategy #2 (old version)
An alternative is:
use a utility function to generate an AST from source (parseSource)
prepare that AST once at the start of my plugin
inject it into Program (i.e. the root path) once, only if the particular function call has been found
Code (using Strategy #2)
Same as above but with your own compiler function (not as efficient as #babel/template):
/**
* Helper: Generate AST from source through `#babel/parser`.
* Copied from somewhere... I think it was `#babel/traverse`
* #param {*} source
*/
export function parseSource(source) {
let ast;
try {
source = `${source}`;
ast = parse(source);
} catch (err) {
const loc = err.loc;
if (loc) {
err.message +=
"\n" +
codeFrameColumns(source, {
start: {
line: loc.line,
column: loc.column + 1,
},
});
}
throw err;
}
const nodes = ast.program.body;
nodes.forEach(n => traverse.removeProperties(n));
return nodes;
}
Possible Pitfalls
When a new node is injected/replaced etc, babel will run all plugins on them again. This is why your first instrumentation plugin is likely to encounter an infinite loop right of the bet: you want to remember and not re-visit previously visited nodes (I'm using a Set for that).
It gets worse when wrapping nodes. Nodes wrapped (e.g. with #babel/template) are actually copies, not the original node. In that case, you want to remember that it is instrumented and skip it in case you come across it again, or, again: infinite loop 💥!
If you don't want to instrument nodes that have been emitted by any plugin (not just yours), that is you want to only operate on the original source code, you can skip them by checking whether they have a loc property (injected nodes usually do not have a loc property).
In your case, you are trying to add an import statement which won't always work without the right plugins enabled or without program-type set to module.
I believe there's an even better way now: babel-helper-module-imports
For you the code would be
import { addDefault } from "#babel/helper-module-imports";
addDefault(path, 'React', { nameHint: "React" })
I've created few tests (Specs) in Protractor - that every test works fine when running alone. the problem is that when executing them as one suite - the tests break.
I'd like to add some operation in between the tests - such as timeout or logout.
is there an option to do it?
I've tried looking here:
https://github.com/angular/protractor/blob/master/docs/referenceConf.js
this is my conf.js file (the specs part):
suites:{
sanity: ['*/AccountSettingsTest.js','*/createApptest.js']
},
specs: ['*/AccountSettingsTest.js'],
thanks
I believe you can use jasmine afterAll function. It should run after the describe block in your test. Just your logout/timeout function into an afterAll block inside of your describe block, and for that spec file, it will run after the describe. Since you have multiple specs, I imagine you'll want it in each spec file, as the order the files run in may vary.
From the jasmine docs:
describe("A spec using beforeAll and afterAll", function() {
var foo;
beforeAll(function() {
foo = 1;
});
afterAll(function() {
foo = 0;
});
it("sets the initial value of foo before specs run", function() {
expect(foo).toEqual(1);
foo += 1;
});
it("does not reset foo between specs", function() {
expect(foo).toEqual(2);
});
});
You can indeed try to add the below line in your conf.js file to restart your browser between tests -
restartBrowserBetweenTests: false,
In order to keep a default timeout add below line in jasmineNodeOpts object -
defaultTimeoutInterval: 30000
A detailed explanation is given in the referenceConf.js file