Commands not recognised in GitHub Action Mocha tests - visual-studio-code

Question

My unit tests pass when run locally. When executed by a GitHub Action when a branch is committed, this action
on:
push:
branches:
- '*' # matches every branch that doesn't contain a '/'
- '*/*' # matches every branch containing a single '/'
- '**' # matches every branch
- '!master' # excludes master
workflow_dispatch:
inputs:
build:
description: Unit tests
type: boolean
required: true
default: "true"
jobs:
build:
strategy:
matrix:
os: [windows-latest]
# os: [ubuntu-latest, windows-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Install Node.js
uses: actions/setup-node#v1
with:
node-version: 16.x
- run: npm install
- run: npm i gulp gulp-cli -g
if: runner.os == 'MacOS'
- run: gulp
- run: xvfb-run -a npm test
if: runner.os == 'Linux'
- run: npm test
if: runner.os != 'Linux'
basically runs npm test.
When I do that locally, it works fine:
npm test
> vscode-print#0.9.25 test
> node ./out/test/runTest.js
Debugger listening on ws://127.0.0.1:63067/252bd7f9-c748-45b4-b9e6-8ba21e9fc909
For help, see: https://nodejs.org/en/docs/inspector
Debugger attached.
Found existing install in C:\Users\Peter\source\vsc-print\.vscode-test\vscode-win32-x64-archive-1.68.1. Skipping download
[16792:0704/000228.095:ERROR:node_bindings.cc(286)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[16792:0704/000228.095:ERROR:node_bindings.cc(286)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[main 2022-07-03T14:02:28.455Z] update#ctor - updates are disabled by the environment
[main 2022-07-03T14:02:29.679Z] Starting extension host with pid 32 (fork() took 15 ms).
Loading development extension at c:\Users\Peter\source\vsc-print
Decryption failed
Print Extension Test Suite
✔ Check platform browser launch command on win32 (200ms)
✔ Check platform alternate browser launch command on win32 (112ms)
✔ Print active editor (304ms)
Could not find the language 'jsonc', did you forget to load/include a language module?
✔ Print folder (812ms)
4 passing (2s)
[main 2022-07-03T14:02:32.887Z] Extension host with pid 32 exited with code: 0, signal: null.
Exit code: 0
Done
Waiting for the debugger to disconnect...
When the Action runs it, this is the output.
Downloaded VS Code into D:\a\vsc-print\vsc-print\.vscode-test\vscode-win32-x64-archive-1.68.1
[main 2022-07-03T13:06:32.496Z] update#ctor - updates are disabled by the environment
workbench#open()
[main 2022-07-03T13:06:37.868Z] Calling fork to start extension host...
[main 2022-07-03T13:06:37.879Z] Starting extension host with pid 776 (fork() took 11 ms).
IExtensionHostStarter.start() took 42 ms.
Loading development extension at d:\a\vsc-print\vsc-print
Print Extension Test Suite
1) Check platform browser launch command on win32
Error: 4 tests failed.
2) Check platform alternate browser launch command on win32
3) Print active editor
at d:\a\vsc-print\vsc-print\out\test\suite\index.js:25:27
4) Print folder
at done (d:\a\vsc-print\vsc-print\node_modules\mocha\lib\mocha.js:1009:7)
0 passing (518ms)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
4 failing
1) Print Extension Test Suite
Check platform browser launch command on win32:
Error: command 'extension.test.browserLaunchCommand' not found
at v._tryExecuteCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3532)
at v.executeCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3414)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
2) Print Extension Test Suite
Check platform alternate browser launch command on win32:
Error: command 'extension.test.browserLaunchCommand' not found
at v._tryExecuteCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3532)
at v.executeCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3414)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
3) Print Extension Test Suite
Print active editor:
Error: command 'extension.test.flags' not found
at v._tryExecuteCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3532)
at v.executeCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3414)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
4) Print Extension Test Suite
Print folder:
Error: command 'extension.test.flags' not found
at v._tryExecuteCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3532)
at v.executeCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3414)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
[main 2022-07-03T13:06:41.294Z] Waiting for extension host with pid 776 to exit.
[main 2022-07-03T13:06:41.330Z] Extension host with pid 776 exited with code: 1, signal: null.
Exit code: 1
Failed
Failed to run tests
Error: Process completed with exit code 1.
The unrecognised commands are registered near the start of the extension's activation event.
export function activate(context: vscode.ExtensionContext) {
let ecmPrint = vscode.workspace.getConfiguration("print", null).editorContextMenuItemPosition,
etmButton = vscode.workspace.getConfiguration("print", null).editorTitleMenuButton,
disposable: vscode.Disposable;
vscode.commands.executeCommand("setContext", "ecmPrint", ecmPrint);
vscode.commands.executeCommand("setContext", "etmButton", etmButton);
context.subscriptions.push(vscode.workspace.onDidChangeConfiguration(checkConfigurationChange));
context.subscriptions.push(vscode.commands.registerCommand("extension.print", printCommand));
context.subscriptions.push(vscode.commands.registerCommand("extension.printFolder", printFolderCommand));
context.subscriptions.push(vscode.commands.registerCommand("extension.test.flags", () => testFlags));
context.subscriptions.push(vscode.commands.registerCommand("extension.test.browserLaunchCommand", PrintSession.getLaunchBrowserCommand));
This suggests that the activation method hasn't been called. Is this something I must do explicitly?

Answer 1

I just noticed this up the top of extension.test.ts
// You can import and use all API from the 'vscode' module
// as well as import your extension to test it
import * as vscode from 'vscode';
// import * as myExtension from '../../extension';
That's not how my tests are written. They don't import the code and call it directly, they assume the extension is installed and active and they use registered commands. Some of these commands were added for no other reason than to allow fine grained unit tests.
The problem I describe in the question is due to the fact that the extension is not installed.
Discovering I can import from the extension doesn't solve all the problems. Some of my tests are strictly speaking integration tests. They need the extension to be installed and active.
To meet that requirement, gulp package rather than gulp build will also produce the VSIX. The only real trick is obtaining the VSIX name to install it in the test environment.
const vsixName = fs.readdirSync(extensionDevelopmentPath)
.filter(p => path.extname(p) === ".vsix")
.sort((a, b) => a < b ? 1 : a > b ? -1 : 0)[0];
const launchArgs = [
path.resolve(__dirname, '../../src/test/test-docs'),
"--install-extension",
vsixName
];
A hybrid approach is possible; some of the registered commands exist only for use by unit tests. These could be junked in favour of direct calls.