I am trying to use a plugin in my eks based k8s cluster,
I am using a Github Action controller that spawns on demand Container as Self Hosted runner
When the Github action start this plugin or any other that needs to build itself as a docker image fails with below error, any thoughts or ideas ?
This is my self hosted runner image Link
FYI : If i run a standalone alpine container in the cluster all typical cmd works, and this also works with default ubuntu based self hosted runner, so i dont think its the cluster
/usr/local/bin/docker build -t 60e226:1b6fc15462134e6fb8520b7df48cf7fd -f "/runner/_work/_actions/aquasecurity/trivy-action/master/Dockerfile" "/runner/_work/_actions/aquasecurity/trivy-action/master"
Sending build context to Docker daemon 644.6kB
Step 1/5 : FROM ghcr.io/aquasecurity/trivy:0.[3](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:3)7.1
0.37.1: Pulling from aquasecurity/trivy
c158987b0551: Pulling fs layer
67a7d067ef7d: Pulling fs layer[6]Download complete
67a7d067ef7d: Pull complete
2ec1cdd48f38: Verifying Checksum
2ec1cdd48f38: Download complete
2ec1cdd48f38: Pull complete
fe56e6aa700e: Pull complete
Digest: sha256:7c[16](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:16)7f7f3002948f1ec099555aa968bd8b8b097780603a38cc801fe965da0a69
Status: Downloaded newer image for ghcr.io/aquasecurity/trivy:0.37.1
---> c3e68408cd24
Step 2/5 : COPY entrypoint.sh /
---> 1f1da443ea86
Step 3/5 : RUN apk --no-cache add bash curl npm
---> Running in 647f7f479cac
fetch https://dl-cdn.alpinelinux.org/alpine/v3.[17](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:17)/main/x86_64/APKINDEX.tar.gz
48ABC73BEB7F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:[18](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:18)89:
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/main: Permission denied
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/x86_64/APKINDEX.tar.gz
48ABC73BEB7F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1889:
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/community: Permission denied
ERROR: unable to select packages:
bash (no such package):
required by: world[bash]
curl (no such package):
required by: world[curl]
npm (no such package):
required by: world[npm]
The command '/bin/sh -c apk --no-cache add bash curl npm' returned a non-zero code: 3
Warning: Docker build failed with exit code 3, back off 6.807 seconds before retry.
It was expected to build the docker image and proceed with the github action workflow
Tried different flavors of image and nothing worked except for ubunut-latest
the plugin in question
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action#master
with:
image-ref: 'test:latest'
format: 'table'
exit-code: '1'
ignore-unfixed: true
vuln-type: 'os,library'
severity: 'CRITICAL,HIGH'
So I cloned their repository, but how do I actually build the tasks?
Here is my scenario. We use Azure DevOps Server 2020 (on prem). All of our build pipelines run the Index Sources & Publish Symbols task
However, it has a bug https://github.com/microsoft/azure-pipelines-tasks/issues/14852. Luckily a fix was merged to master. However, we are not going to see it until it is propagated to the Azure DevOps Server edition and only All Mighty knows when it would happen.
So, I would like to build that task locally and upload to our Azure DevOps server. But I cannot find instructions on how to do it.
So, how can I build and consume it in our Azure DevOps Server?
EDIT 1
Tried to follow the procedure in https://github.com/microsoft/azure-pipelines-tasks/blob/master/ci/build-all-steps.yml, but it did not work.
The first step that seems relevant to me is https://github.com/microsoft/azure-pipelines-tasks/blob/6ab084f52e582370880127132d1c449634c9bfbc/ci/build-all-steps.yml#L39:
- script: |
cd ci
cd verifyMinAgentDemands
npm install
node index.js
displayName: Verify all min agent demands are valid
And it is fine:
C:\work\azure-pipelines-tasks\ci\verifyMinAgentDemands [master ≡]> npm install
npm WARN verifyminagentdemands#1.0.0 No description
npm WARN verifyminagentdemands#1.0.0 No repository field.
added 52 packages from 78 contributors and audited 52 packages in 1.743s
found 2 low severity vulnerabilities
run `npm audit fix` to fix them, or `npm audit` for details
C:\work\azure-pipelines-tasks\ci\verifyMinAgentDemands [master ≡]> node .\index.js
##vso[task.debug]agent.TempDirectory=undefined
##vso[task.debug]agent.workFolder=undefined
##vso[task.debug]loading inputs and endpoints
##vso[task.debug]loaded 0
##vso[task.debug]Agent.ProxyUrl=undefined
##vso[task.debug]Agent.CAInfo=undefined
##vso[task.debug]Agent.ClientCert=undefined
##vso[task.debug]Agent.SkipCertValidation=undefined
Verifying min agent demands.
Latest version of the Agent that's fully rolled out is 2.195.0.
The next step (https://github.com/microsoft/azure-pipelines-tasks/blob/6ab084f52e582370880127132d1c449634c9bfbc/ci/build-all-steps.yml#L47) seems to be relevant too:
- script: node make.js build --task "$(task_pattern)"
displayName: Build
condition: ne(variables['numTasks'], 0)
But:
C:\work\azure-pipelines-tasks [master ≡]> node make.js build --task PublishSymbolsV2
> prepending PATH C:\work\azure-pipelines-tasks\node_modules\.bin
tsc tool:
Version 2.3.4
C:\work\azure-pipelines-tasks\node_modules\.bin\tsc
npm tool:
6.14.12
C:\Program Files\nodejs\npm
------------------------------------------------------------
Building: PublishSymbolsV2
------------------------------------------------------------
> getting task externals
Downloading file: https://vstsagenttools.blob.core.windows.net/tools/symstore/2/symbol.zip
Could not use "nc", falling back to slower node.js method for sync requests.
C:\work\azure-pipelines-tasks\node_modules\sync-request\index.js:77
throw new Error(res.stderr.toString());
^
Error
at doRequestWith (C:\work\azure-pipelines-tasks\node_modules\sync-request\index.js:77:11)
at doRequest (C:\work\azure-pipelines-tasks\node_modules\sync-request\index.js:20:10)
at downloadFile (C:\work\azure-pipelines-tasks\make-util.js:411:22)
at downloadArchive (C:\work\azure-pipelines-tasks\make-util.js:451:27)
at C:\work\azure-pipelines-tasks\make-util.js:644:33
at Array.forEach (<anonymous>)
at getExternals (C:\work\azure-pipelines-tasks\make-util.js:639:25)
at C:\work\azure-pipelines-tasks\make.js:193:13
at Array.forEach (<anonymous>)
at Function.target.build (C:\work\azure-pipelines-tasks\make.js:158:14)
C:\work\azure-pipelines-tasks [master ≡]>
What is now?
I am using kubebuilder to create kubernetes operator project. After running the project init command described in quickstart guide
kubebuilder init --domain k8s.io --license apache2 --owner "The Kubernetes Authors"
dep ensure returns with error log given below.
Solving failure: No versions of k8s.io/client-go met constraints:
v8.0.0: Could not introduce k8s.io/client-go#v8.0.0, as it is not allowed by constraints from the following projects:
kubernetes-1.10.1 from (root)
kubernetes-1.10.1 from sigs.k8s.io/controller-runtime#master
v7.0.0: Could not introduce k8s.io/client-go#v7.0.0, as it is not allowed by constraints from the following projects:
kubernetes-1.10.1 from (root)
kubernetes-1.10.1 from sigs.k8s.io/controller-runtime#master
v6.0.0: Could not introduce k8s.io/client-go#v6.0.0, as it is not allowed by constraints from the following projects:
kubernetes-1.10.1 from (root)
kubernetes-1.10.1 from sigs.k8s.io/controller-runtime#master
Try using the latest kubebuilder from here. It's likely that the dependencies for the version in the quick start are out of date.
It works fine for me with v1.0.3
~/go/src/github.com $ kubebuilder init --domain k8s.io --license apache2 --owner "The Kubernetes Authors"
Run `dep ensure` to fetch dependencies (Recommended) [y/n]?
y
dep ensure
Running make...
make
go generate ./pkg/... ./cmd/...
go fmt ./pkg/... ./cmd/...
go vet ./pkg/... ./cmd/...
go run vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go all
CRD manifests generated under '/root/go/src/github.com/config/crds'
RBAC manifests generated under '/root/go/src/github.com/config/rbac'
go test ./pkg/... ./cmd/... -coverprofile cover.out
? github.com/pkg/apis [no test files]
? github.com/pkg/controller [no test files]
ok github.com/pkg/errors 0.207s coverage: 100.0% of statements
? github.com/cmd/manager [no test files]
go build -o bin/manager github.com/cmd/manager
Next: Define a resource with:
$ kubebuilder create api
I have a conflict between a number of install files.
I am getting the below error:
Transaction Summary
================================================================================
Install 612 Packages
Total size: 110 M Installed size: 403 M Downloading Packages: Running
transaction check Transaction check succeeded. Running transaction
test Error: Transaction check error: file /etc/iproute2/rt_protos
conflicts between attempted installs of
base-files-3.0.14-r89.nexbox_a95x_s905x and iproute2-4.14.1-r0.aarch64
file /etc/iproute2/rt_tables conflicts between attempted installs of
base-files-3.0.14-r89.nexbox_a95x_s905x and iproute2-4.14.1-r0.aarch64
file /etc/sysctl.conf conflicts between attempted installs of
base-files-3.0.14-r89.nexbox_a95x_s905x and procps-3.3.12-r0.aarch64
Error Summary
-------------
ERROR: amlogic-image-headless-sd-1.0-r0 do_rootfs: Function failed:
do_rootfs ERROR: Logfile of failure stored in:
/home/user/amlogic-bsp/build/tmp/work/nexbox_a95x_s905x-poky-linux/amlogic-image-headless-sd/1.0-r0/temp/log.do_rootfs.29264
ERROR: Task
(/home/user/amlogic-bsp/meta-meson/recipes-core/images/amlogic-image-headless-sd.bb:do_rootfs)
failed with exit code '1' NOTE: Tasks Summary: Attempted 3131 tasks of
which 3130 didn't need to be rerun and 1 failed.
I have seen somewhere that I should pin a file, but how do I do this? I can't find a tutorial or any reference to what that means.
I am also getting the below warning. Is this related?
WARNING: Layer meson should set LAYERSERIES_COMPAT_meson in its
conf/layer.conf file to list the core layer names it is compatible
with.
I'm new to OE coming over from OpenWRT.
For bitbake, I've added the layers for the packages below:
meta-openwrt:- OE/Yocto metadata layer for OpenWRT
superna9999/meta-meson:- Upstream Linux Amlogic Meson Yocto/OpenEmbedded Layer
And tried compiling the nexbox-a95x-s905x image
I think the problem is that /etc/iproute2/rt_protos is provided by base-files which is coming from meta-openwrt as well as from iproute2 package which is coming from other OE layers. its not clear for the image builder which one to use and hence the conflict
You can solve it via defining a iproute2_%.bbappend file in meta-openwrt where this file gets deleted from iproute2 package and preference is given to the one openwrt provides
do_install_append() {
rm -rf ${D}${sysconfdir}/iproute2/rt_protos
}
should help.
We are using distillery to create an executable build release in our
phoenix application. We are using docker to deploy our application on Digital Ocean. Since we don't want to share code with the other machine we want to execute the build compiled file as executable which we can run on some remote machine through command line.
So after some research we found that distillery plugin build executable release which creates a build using
mix release --executable
We are able to create the release. It provides three command to run these
punitjain#apple:project$ _build/dev/rel/project/bin/project foreground
punitjain#apple:project$ _build/dev/rel/project/bin/project start
punitjain#apple:project$ _build/dev/rel/project/bin/project console
I am getting following error after running foreground command
> $ _build/dev/rel/project/bin/project foreground [info] Application
> project exited: Project.start(:normal, []) returned an error:
> shutdown: failed to start child: Project.Endpoint
> ** (EXIT) shutdown: failed to start child: Phoenix.CodeReloader.Server
> ** (EXIT) an exception was raised:
> ** (UndefinedFunctionError) function Mix.Project.config/0 is undefined (module Mix.Project is not available)
> Mix.Project.config()
> (phoenix) lib/phoenix/code_reloader/server.ex:29: Phoenix.CodeReloader.Server.init/1
> (stdlib) gen_server.erl:328: :gen_server.init_it/6
> (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3 {"Kernel pid
> terminated",application_controller,"{application_start_failure,project,{{shutdown,{failed_to_start_child,'Elixir.Project.Endpoint',{shutdown,{failed_to_start_child,'Elixir.Phoenix.CodeReloader.Server',{undef,[{'Elixir.Mix.Project',config,[],[]},{'Elixir.Phoenix.CodeReloader.Server',init,1,[{file,\"lib/phoenix/code_reloader/server.ex\"},{line,29}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,328}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,247}]}]}}}}},{'Elixir.Project',start,[normal,[]]}}}"}
>
> Crash dump is being written to: erl_crash.dump...done Kernel pid
> terminated (application_controller)
> ({application_start_failure,project,{{shutdown,{failed_to_start_child,'Elixir.Project.Endpoint',{shutdown,{failed_to_start_child,'Elixir.Phoenix.Code
Can you please help me how to resolve this error?
I would also be very happy to hear if anyone is having any better approach to create an executable release file. Which I can use in any Linux based system to deploy without exposing my source code using Docker.
Please let me know if I need to provide more information or do any amendments in question asked.
thanks
The problem is that you are building a release with MIX_ENV=dev, with Phoenix this will mean that the code reloader is included, and the code reloader does not work within a release. You should either build the release with MIX_ENV=prod or disable the code reloader in dev.