Deployment failing using Google Cloud Build on to GCS - google-cloud-storage

I have a reactjs static app running on Google Cloud Storage. Using Cloud Build, I'm trying to automate the build and deployment process.
Following is the cloudbuild.yaml file:
steps:
# Install
- name: 'gcr.io/cloud-builders/npm'
entrypoint: 'npm'
args: ['install']
# Test
#- name: 'gcr.io/cloud-builders/npm'
#entrypoint: 'npm'
#args: ['test', 'test:ci']
# Build
- name: 'gcr.io/cloud-builders/npm'
entrypoint: 'npm'
args: ['build']
# Syncing to Static Hosting on GCS
- name: 'gcr.io/cloud-builders/gsutil'
args: ["-m", "rsync", "-r", "-c", "-d", "./build", "gs://mybucket.com"]
The last step which is Syncing to GCS is failing. I'm getting below error:
Starting Step #0
Step #0: Already have image (with digest): gcr.io/cloud-builders/npm
Step #0: npm WARN saveError ENOENT: no such file or directory, open '/workspace/package.json'
Step #0: npm notice created a lockfile as package-lock.json. You should commit this file.
Step #0: npm WARN enoent ENOENT: no such file or directory, open '/workspace/package.json'
Step #0: npm WARN workspace No description
Step #0: npm WARN workspace No repository field.
Step #0: npm WARN workspace No README data
Step #0: npm WARN workspace No license field.
Step #0:
Step #0: up to date in 0.711s
Step #0: found 0 vulnerabilities
Step #0:
Finished Step #0
Starting Step #1
Step #1: Already have image (with digest): gcr.io/cloud-builders/npm
Finished Step #1
Starting Step #2
Step #2: Already have image (with digest): gcr.io/cloud-builders/gsutil
Step #2: CommandException: arg (./build) does not name a directory, bucket, or bucket subdir.
Finished Step #2
ERROR
ERROR: build step 2 "gcr.io/cloud-builders/gsutil" failed: step exited with non-zero status: 1
My project structure:
RootProject
-> front-end-webapp
-> src/
-> cloudbuild.yaml
-> build/
-> public/
-> package.json
-> backend-service1
-> backend-service2
The trigger which I have written points to cloudbuild.yaml file inside the front-end-webapp sub-folder.
When I manually execute
npm run build or yarn build
then I get production ready files inside build folder under front-end-webapp which I want to put on Google Cloud Storage. Only files inside the build folder and not the build folder itself containing files.
I tried a lot but not able to get it. If I do . instead of './build' in the args like args: ["-m", "rsync", "-r", "-c", "-d", "./build", "gs://mybucket.com"] then cloud build executes successfully but it also copies entires files and sub-folders present in the root project.

The build job is triggered by a file committed into the front-end-webapp and you run the cloudbuild.yaml file which is into front-end-webapp.
BUT, the root of the repo, is / and not front-end-webapp. Thereby, add the dir: 'front-end-webapp' attribute on each step. That force the steps to start into the correct folder!
EDIT
When I'm stuck and lost in the directory management, I introduce this step for having a view of what exist in my workspace.
- name: gcr.io/cloud-builders/gcloud:latest
entrypoint: "ls"
args: ["-lah","/workspace"]

Related

Pipeline failed if scanning scala files

I created Gitlab CI which sends all files in project folder to scan in SonarQube, and it's working perfect with python files, but if I add scala files it's failed. My Gitlab CI:
image: testimage
variables:
SONARQUBE_URL: https://sonarqube.com
stages:
- PyLint
pylint:
stage: PyLint
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
script:
- apt-get install scala -y
- cd project
- scalac *.scala
- ls
- cd ..
- sed -i 's/PROJECT-NAME/'"$CI_PROJECT_NAME"'/g' sonar-project.properties
- sonar-scanner -Dsonar.login=$SONAR_TOKEN -Dsonar.qualitygate.wait=true -Dsonar.projectVersion=${CI_PIPELINE_ID}
- echo 'Repository Link:' "$SONARQUBE_URL${CI_PROJECT_NAME}"
Some lines from logs:
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 13.545s
INFO: Final Memory: 16M/136M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarQube Scanner execution
java.lang.IllegalStateException: Can not execute Findbugs
at org.sonarsource.scanner.cli.Main.main(Main.java:61)
Caused by: java.lang.IllegalStateException: One (sub)project contains Java source files that are not compiled (/builds/scala/myrepo).
Property sonar.java.binaries was not set, it is required to locate the compiled .class files. For instance set the property to: sonar.java.binaries=target/classes
Sonar JavaResourceLocator.classpath was empty
Sonar JavaResourceLocator.classFilesToAnalyze was empty
at org.sonar.plugins.findbugs.FindbugsConfiguration.buildMissingCompiledCodeException(FindbugsConfiguration.java:154)
at org.sonar.plugins.findbugs.FindbugsConfiguration.initializeFindbugsProject(FindbugsConfiguration.java:124)
at org.sonar.plugins.findbugs.FindbugsExecutor.execute(FindbugsExecutor.java:117)
... 31 more
ERROR:
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.
Cleaning up file based variables
00:01
ERROR: Job failed: command terminated with exit code 1
My solution was to add this line in sonar-project.properties file:
sonar.java.binaries=.

Commands not recognised in GitHub Action Mocha tests

My unit tests pass when run locally. When executed by a GitHub Action when a branch is committed, this action
on:
push:
branches:
- '*' # matches every branch that doesn't contain a '/'
- '*/*' # matches every branch containing a single '/'
- '**' # matches every branch
- '!master' # excludes master
workflow_dispatch:
inputs:
build:
description: Unit tests
type: boolean
required: true
default: "true"
jobs:
build:
strategy:
matrix:
os: [windows-latest]
# os: [ubuntu-latest, windows-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Install Node.js
uses: actions/setup-node#v1
with:
node-version: 16.x
- run: npm install
- run: npm i gulp gulp-cli -g
if: runner.os == 'MacOS'
- run: gulp
- run: xvfb-run -a npm test
if: runner.os == 'Linux'
- run: npm test
if: runner.os != 'Linux'
basically runs npm test.
When I do that locally, it works fine:
npm test
> vscode-print#0.9.25 test
> node ./out/test/runTest.js
Debugger listening on ws://127.0.0.1:63067/252bd7f9-c748-45b4-b9e6-8ba21e9fc909
For help, see: https://nodejs.org/en/docs/inspector
Debugger attached.
Found existing install in C:\Users\Peter\source\vsc-print\.vscode-test\vscode-win32-x64-archive-1.68.1. Skipping download
[16792:0704/000228.095:ERROR:node_bindings.cc(286)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[16792:0704/000228.095:ERROR:node_bindings.cc(286)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[main 2022-07-03T14:02:28.455Z] update#ctor - updates are disabled by the environment
[main 2022-07-03T14:02:29.679Z] Starting extension host with pid 32 (fork() took 15 ms).
Loading development extension at c:\Users\Peter\source\vsc-print
Decryption failed
Print Extension Test Suite
✔ Check platform browser launch command on win32 (200ms)
✔ Check platform alternate browser launch command on win32 (112ms)
✔ Print active editor (304ms)
Could not find the language 'jsonc', did you forget to load/include a language module?
✔ Print folder (812ms)
4 passing (2s)
[main 2022-07-03T14:02:32.887Z] Extension host with pid 32 exited with code: 0, signal: null.
Exit code: 0
Done
Waiting for the debugger to disconnect...
When the Action runs it, this is the output.
Downloaded VS Code into D:\a\vsc-print\vsc-print\.vscode-test\vscode-win32-x64-archive-1.68.1
[main 2022-07-03T13:06:32.496Z] update#ctor - updates are disabled by the environment
workbench#open()
[main 2022-07-03T13:06:37.868Z] Calling fork to start extension host...
[main 2022-07-03T13:06:37.879Z] Starting extension host with pid 776 (fork() took 11 ms).
IExtensionHostStarter.start() took 42 ms.
Loading development extension at d:\a\vsc-print\vsc-print
Print Extension Test Suite
1) Check platform browser launch command on win32
Error: 4 tests failed.
2) Check platform alternate browser launch command on win32
3) Print active editor
at d:\a\vsc-print\vsc-print\out\test\suite\index.js:25:27
4) Print folder
at done (d:\a\vsc-print\vsc-print\node_modules\mocha\lib\mocha.js:1009:7)
0 passing (518ms)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
4 failing
1) Print Extension Test Suite
Check platform browser launch command on win32:
Error: command 'extension.test.browserLaunchCommand' not found
at v._tryExecuteCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3532)
at v.executeCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3414)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
2) Print Extension Test Suite
Check platform alternate browser launch command on win32:
Error: command 'extension.test.browserLaunchCommand' not found
at v._tryExecuteCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3532)
at v.executeCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3414)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
3) Print Extension Test Suite
Print active editor:
Error: command 'extension.test.flags' not found
at v._tryExecuteCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3532)
at v.executeCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3414)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
4) Print Extension Test Suite
Print folder:
Error: command 'extension.test.flags' not found
at v._tryExecuteCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3532)
at v.executeCommand (vscode-file://vscode-app/d:/a/vsc-print/vsc-print/.vscode-test/vscode-win32-x64-archive-1.68.1/resources/app/out/vs/workbench/workbench.desktop.main.js:1706:3414)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
[main 2022-07-03T13:06:41.294Z] Waiting for extension host with pid 776 to exit.
[main 2022-07-03T13:06:41.330Z] Extension host with pid 776 exited with code: 1, signal: null.
Exit code: 1
Failed
Failed to run tests
Error: Process completed with exit code 1.
The unrecognised commands are registered near the start of the extension's activation event.
export function activate(context: vscode.ExtensionContext) {
let ecmPrint = vscode.workspace.getConfiguration("print", null).editorContextMenuItemPosition,
etmButton = vscode.workspace.getConfiguration("print", null).editorTitleMenuButton,
disposable: vscode.Disposable;
vscode.commands.executeCommand("setContext", "ecmPrint", ecmPrint);
vscode.commands.executeCommand("setContext", "etmButton", etmButton);
context.subscriptions.push(vscode.workspace.onDidChangeConfiguration(checkConfigurationChange));
context.subscriptions.push(vscode.commands.registerCommand("extension.print", printCommand));
context.subscriptions.push(vscode.commands.registerCommand("extension.printFolder", printFolderCommand));
context.subscriptions.push(vscode.commands.registerCommand("extension.test.flags", () => testFlags));
context.subscriptions.push(vscode.commands.registerCommand("extension.test.browserLaunchCommand", PrintSession.getLaunchBrowserCommand));
This suggests that the activation method hasn't been called. Is this something I must do explicitly?
I just noticed this up the top of extension.test.ts
// You can import and use all API from the 'vscode' module
// as well as import your extension to test it
import * as vscode from 'vscode';
// import * as myExtension from '../../extension';
That's not how my tests are written. They don't import the code and call it directly, they assume the extension is installed and active and they use registered commands. Some of these commands were added for no other reason than to allow fine grained unit tests.
The problem I describe in the question is due to the fact that the extension is not installed.
Discovering I can import from the extension doesn't solve all the problems. Some of my tests are strictly speaking integration tests. They need the extension to be installed and active.
To meet that requirement, gulp package rather than gulp build will also produce the VSIX. The only real trick is obtaining the VSIX name to install it in the test environment.
const vsixName = fs.readdirSync(extensionDevelopmentPath)
.filter(p => path.extname(p) === ".vsix")
.sort((a, b) => a < b ? 1 : a > b ? -1 : 0)[0];
const launchArgs = [
path.resolve(__dirname, '../../src/test/test-docs'),
"--install-extension",
vsixName
];
A hybrid approach is possible; some of the registered commands exist only for use by unit tests. These could be junked in favour of direct calls.

So i‘m trying to deploy a function on openfaas based on arm architecture,but i failed.Can someone tell me how to solve it?

my kubernetes is based on arm,with kubernetes1.22.4
my faas-cli command:
faas-cli build -f ./add.yml
the response code :
[0] > Building add.
Clearing temporary build folder: ./build/add/
Preparing: ./add/ build/add/function
Building: myw/add:latest with python template. Please wait..
Sending build context to Docker daemon 8.192kB
Step 1/31 : FROM --platform=${TARGETPLATFORM:-linux/amd64} ghcr.io/openfaas/classic-watchdog:0.2.0 as watchdog
---> 6f97aa96da81
Step 2/31 : FROM --platform=${TARGETPLATFORM:-linux/amd64} python:2.7-alpine
---> 8579e446340f
Step 3/31 : ARG TARGETPLATFORM
---> Using cache
---> a75f5a062540
Step 4/31 : ARG BUILDPLATFORM
---> Using cache
---> c90a8309e851
Step 5/31 : ARG ADDITIONAL_PACKAGE
---> Using cache
---> 4ee3e6fab2a3
Step 6/31 : COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog
---> Using cache
---> 33d972637c65
Step 7/31 : RUN chmod +x /usr/bin/fwatchdog
---> Running in 6204e8546454
standard_init_linux.go:219: exec user process caused: exec format error
The command '/bin/sh -c chmod +x /usr/bin/fwatchdog' returned a non-zero code: 1
[0] < Building add done in 0.54s.
[0] Worker done.
Total build time: 0.54s
Errors received during build:
- [add] received non-zero exit code from build, error: The command '/bin/sh -c chmod +x /usr/bin/fwatchdog' returned a non-zero code: 1
then can someone tell me the currect way to build a function with arm in openfaas,and why this happened,did i do someting wrong?

docker-compose selectively thinks config.yml is a folder

I am running docker-compose 1.25.5 on a ubuntu 20 box and I have a github repo working "fine" in its home folder... I can docker-compose build and docker-compose up with no problem, and the container does what is expected. The github repo is current with the on-disk files.
As a test, however, I created a new folder, pulled the repo, and ran docker-compose build with no problem but when I tried to run docker-compose up, I get the following error:
Starting live_evidently_1 ... done
Attaching to live_evidently_1
evidently_1 | Traceback (most recent call last):
evidently_1 | File "app.py", line 14, in <module>
evidently_1 | with open('config.yml') as f:
evidently_1 | IsADirectoryError: [Errno 21] Is a directory: 'config.yml'
live_evidently_1 exited with code 1
config.yml on my host is a file (of course) and the docker-compose.yml file is unremarkable:
version: "3"
services:
evidently:
build: ../
volumes:
- ./data:/data
- ./config.yml:/app/config.yml
etc...
...
So, I am left with two inter-related problems. 1) Why does the test version of the repo fail and the original version is fine (git status is unremarkable, all the files I want on github are up to date), and 2) Why does docker-compose think that config.yml is a folder when it is clearly a file? I would welcome suggestions.
You need to use bind mount type. To do this you have to use long syntax.
Like this.
volumes:
- type: bind
source: ./config.yml
target: /app/config.yml

Run QEMU from within Eclipse External tools

I'd like to ask since I am setting up the Eclipse IDE for Yocto app development and I got stuck to start the QEMU from within Eclipse.
I got working QEMU image OK such as
ubuntu#ubuntu:~/work/community/build-x11$ runqemu qemuarm
tmp/deploy/images/qemuarm/zImage-qemuarm.bin
tmp/deploy/images/qemuarm/fsl-image-multimedia-full-qemuarm.ext4
Within the Eclipse I follow
https://www.yoctoproject.org/docs/2.5/sdk-manual/sdk-manual.html#oxygen-starting-qemu-in-user-space-nfs-mode
But by configuring "External Tools" and try to run QEMU I got following
runqemu - INFO - Running MACHINE=qemuarm bitbake -e...
ERROR: Unable to find conf/bblayers.conf or conf/bitbake.conf. BBAPTH is unset and/or not in a build directory?
runqemu - WARNING - Couldn't run 'bitbake -e' to gather environment information:
runqemu - WARNING - Can't find qemuboot conf file, DEPLOY_DIR_IMAGE is NULL!
runqemu - INFO - Running MACHINE=qemuarm bitbake -e...
ERROR: Unable to find conf/bblayers.conf or conf/bitbake.conf. BBAPTH is unset and/or not in a build directory?
runqemu - WARNING - Couldn't run 'bitbake -e' to gather environment information:
runqemu - INFO - Setting STAGING_DIR_NATIVE to OECORE_NATIVE_SYSROOT (/home/ubuntu/work/community/build-x11/tmp/work/armv5e-fslc-linux-gnueabi/meta-ide-support/1.0-r3/recipe-sysroot-native)
runqemu - INFO - Setting STAGING_BINDIR_NATIVE to /home/ubuntu/work/community/build-x11/tmp/work/armv5e-fslc-linux-gnueabi/meta-ide-support/1.0-r3/recipe-sysroot-native/usr/bin
runqemu - INFO - QB_MEM is not set, use 512M by default
runqemu - INFO - Continuing with the following parameters:
KERNEL: [/home/ubuntu/work/community/build-x11/tmp/deploy/images/qemuarm/zImage-qemuarm.bin]
MACHINE: [qemuarm]
FSTYPE: [nfs]
NFS_DIR: [/home/ubuntu/work/community/build-x11/MY_QEMU_ROOTFS]
CONFFILE: []
/bin/sh: 1: stty: not found
Traceback (most recent call last):
File "/home/ubuntu/work/community/sources/poky/scripts/runqemu", line 1270, in main
config.setup_network()
File "/home/ubuntu/work/community/sources/poky/scripts/runqemu", line 997, in setup_network
self.saved_stty = subprocess.check_output("stty -g", shell=True).decode('utf-8')
File "/usr/lib/python3.5/subprocess.py", line 626, in check_output
**kwargs).stdout
File "/usr/lib/python3.5/subprocess.py", line 708, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'stty -g' returned non-zero exit status 127
Cleanup
Command 'lesspipe' is available in the following places
* /bin/lesspipe
* /usr/bin/lesspipe
The command could not be located because '/bin:/usr/bin' is not included in the PATH environment variable.
lesspipe: command not found
Command 'dircolors' is available in '/usr/bin/dircolors'
The command could not be located because '/usr/bin' is not included in the PATH environment variable.
dircolors: command not found
ubuntu#ubuntu:~/eclipse/cpp-oxygen/eclipse$
I wonder if any has experience such a problem on setting it up "External Tools" with Eclipse?
Thank you
Navigate to the build path and then trigger this command will work.
OR
Run source oe-init-build-env build path.