Helm - Can't install dependencies from private Github repository - kubernetes-helm

I'm trying to install packages from a private repository I've deployed using chart-releases, but I'm not being able to do it.
Here is what I've done:
I've created a new private repository, added a sample chart to it and ran the following commands:
helm package charts/* --destination .deploy
cr upload -o odelucca -r helm-charts -p .deploy -t $MY_TOKEN
I've created the index.yaml with the following command:
cr index --config .cr.yaml -t $MY_TOKEN
# My .cr.yaml file:
# owner: odelucca
# git-repo: helm-charts
# package-path: .deploy
# index-path: index.yaml
# charts-repo: https://github.com/odelucca/helm-charts/
I've commit the index.yaml to the repo
I've added the remote helm repo with the following command:
helm repo add helm-charts https://raw.githubusercontent.com/odelucca/helm-charts/master --username $MY_EMAIL --password $MY_TOKEN
The repo was added, then I've added the following dependency to a local chart:
dependencies:
- name: serverless-common
version: 1.0.0
repository: "#helm-charts"
Now, I've tried to run the following:
helm dep update
I get the following errors:
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts):
Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: connect: connection refused
...Successfully got an update from the "helm-charts" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete.
Saving 1 charts
Downloading serverless-common from repo https://raw.githubusercontent.com/odelucca/helm-charts/master
Save error occurred: could not download https://github.com/odelucca/helm-charts/releases/download/serverless-common-1.0.0/serverless-common-1.0.0.tgz: Failed to fetch https://github.com/odelucca/helm-charts/releases/download/serverless-common-1.0.0/serverless-common-1.0.0.tgz : 404 Not Found
Deleting newly downloaded charts, restoring pre-update state
Error: could not download https://github.com/odelucca/helm-charts/releases/download/serverless-common-1.0.0/serverless-common-1.0.0.tgz: Failed to fetch https://github.com/odelucca/helm-charts/releases/download/serverless-common-1.0.0/serverless-common-1.0.0.tgz : 404 Not Found
Anyone can help me? I've tried a lot of different approaches, and none of them fix it

Have you tried editing the repository config file?
You should have something like this:
vim ${HOEM}/.config/helm/repositories.yaml
apiVersion: ""
generated: "0001-01-01T00:00:00Z"
repositories:
- caFile: ""
certFile: ""
insecure_skip_tls_verify: false
keyFile: ""
name: helm-charts
password: ""
url: "https://raw.githubusercontent.com/odelucca/helm-charts/master"
username: ""
Edit it and put your username and password to connect to your private registry.
For me, this works fine.
Maybe you also need to check your repo path.
You should be able to download with an URL like this:
${repository}/${name}-${version}.tgz

Related

Github Action failing to Build Images for the plugins being used in workflow

I am trying to use a plugin in my eks based k8s cluster,
I am using a Github Action controller that spawns on demand Container as Self Hosted runner
When the Github action start this plugin or any other that needs to build itself as a docker image fails with below error, any thoughts or ideas ?
This is my self hosted runner image Link
FYI : If i run a standalone alpine container in the cluster all typical cmd works, and this also works with default ubuntu based self hosted runner, so i dont think its the cluster
/usr/local/bin/docker build -t 60e226:1b6fc15462134e6fb8520b7df48cf7fd -f "/runner/_work/_actions/aquasecurity/trivy-action/master/Dockerfile" "/runner/_work/_actions/aquasecurity/trivy-action/master"
Sending build context to Docker daemon 644.6kB
Step 1/5 : FROM ghcr.io/aquasecurity/trivy:0.[3](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:3)7.1
0.37.1: Pulling from aquasecurity/trivy
c158987b0551: Pulling fs layer
67a7d067ef7d: Pulling fs layer[6]Download complete
67a7d067ef7d: Pull complete
2ec1cdd48f38: Verifying Checksum
2ec1cdd48f38: Download complete
2ec1cdd48f38: Pull complete
fe56e6aa700e: Pull complete
Digest: sha256:7c[16](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:16)7f7f3002948f1ec099555aa968bd8b8b097780603a38cc801fe965da0a69
Status: Downloaded newer image for ghcr.io/aquasecurity/trivy:0.37.1
---> c3e68408cd24
Step 2/5 : COPY entrypoint.sh /
---> 1f1da443ea86
Step 3/5 : RUN apk --no-cache add bash curl npm
---> Running in 647f7f479cac
fetch https://dl-cdn.alpinelinux.org/alpine/v3.[17](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:17)/main/x86_64/APKINDEX.tar.gz
48ABC73BEB7F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:[18](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:18)89:
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/main: Permission denied
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/x86_64/APKINDEX.tar.gz
48ABC73BEB7F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1889:
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/community: Permission denied
ERROR: unable to select packages:
bash (no such package):
required by: world[bash]
curl (no such package):
required by: world[curl]
npm (no such package):
required by: world[npm]
The command '/bin/sh -c apk --no-cache add bash curl npm' returned a non-zero code: 3
Warning: Docker build failed with exit code 3, back off 6.807 seconds before retry.
It was expected to build the docker image and proceed with the github action workflow
Tried different flavors of image and nothing worked except for ubunut-latest
the plugin in question
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action#master
with:
image-ref: 'test:latest'
format: 'table'
exit-code: '1'
ignore-unfixed: true
vuln-type: 'os,library'
severity: 'CRITICAL,HIGH'

JFrog Helm repository always returns 404

I've got 3 helm repositories in my JFrog account:
helm-local - intended to store private helm charts
helm-remote - intended to proxy public charts
helm - virtual one, unites both above
I configure my helm client like this: helm repo add %reponame% https://%name%.jfrog.io/artifactory/helm --username %username% --password %password% as it's descrbed in https://www.jfrog.com/confluence/display/RTF6X/Helm+Chart+Repositories.
Then I do helm repo update.
But when I try to do helm pull %reponame%/%chartname%, it returns 404
Error: failed to fetch https://%name%.jfrog.io/helm-local/%chartname%-%chartversion%.tgz : 404 Not Found
helm.go:88: [debug] failed to fetch https://%name%.jfrog.io/helm-local/%chartname%-%chartversion%.tgz : 404 Not Found
helm.sh/helm/v3/pkg/getter.(*HTTPGetter).get
helm.sh/helm/v3/pkg/getter/httpgetter.go:90
helm.sh/helm/v3/pkg/getter.(*HTTPGetter).Get
helm.sh/helm/v3/pkg/getter/httpgetter.go:42
helm.sh/helm/v3/pkg/downloader.(*ChartDownloader).DownloadTo
helm.sh/helm/v3/pkg/downloader/chart_downloader.go:99
helm.sh/helm/v3/pkg/action.(*Pull).Run
helm.sh/helm/v3/pkg/action/pull.go:129
main.newPullCmd.func2
helm.sh/helm/v3/cmd/helm/pull.go:75
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra#v1.1.3/command.go:852
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra#v1.1.3/command.go:960
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra#v1.1.3/command.go:897
main.main
helm.sh/helm/v3/cmd/helm/helm.go:87
runtime.main
runtime/proc.go:225
runtime.goexit
runtime/asm_amd64.s:1371
It seems fetching an index, and gets the latest version of the chart, but can't get a correct download URL of it.
What am I doing wrong?
I see you are trying to pull from helm-local directly; use the helm virtual and try to pull. Also, perform a zap cache on the virtual repo and recalculate the index on the local repo and test the pull.

jenkins -github build issue

I have been facing issues while trying to build a project in jenkins which fetches the files from github .
Below is the error log i got :
p.s: for security reason i have removed the ssh url of git .I can able to clone from my locally but when im trying it from jenkins it fails . I have generated ssh key in linux server and its able to connect to github .Can anyone please help me on this
Building in workspace /var/lib/jenkins/jobs/tr-data-events-testing/workspace
> /usr/local/bin/git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> /usr/local/bin/git config remote.origin.url git#github.url---------git # timeout=10
Fetching upstream changes from git#github.url-----------s.git
> /usr/local/bin/git --version # timeout=10
using GIT_SSH to set credentials jenkins dxc key
> /usr/local/bin/git fetch --tags --progress git#----url--------------.git +refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git#---url-------.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:766)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1022)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1053)
at hudson.scm.SCM.checkout(SCM.java:485)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1282)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532)
at hudson.model.Run.execute(Run.java:1741)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:529)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:381)
Caused by: hudson.plugins.git.GitException: Command "/usr/local/bin/git fetch --tags --progress git#github.-------events.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: error: The requested URL returned error: 503 while accessing https://--url-----------/info/refs
fatal: HTTP request failed
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1693)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1441)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:62)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:313)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:764)
... 11 more
ERROR: null
Started calculate disk usage of build
Finished Calculation of disk usage of build in 0 seconds
Started calculate disk usage of workspace
Finished Calculation of disk usage of workspace in 0 seconds
Finished: FAILURE
Help us localize this page
i resolved this issue by un setting the proxy in .gitconfig file . thank you for your answers
First, make sure Jenkins is running with the same user account you have created your ssh keys with.
It it does not, it would search for ~/.ssh/id_rsa in its own $HOME.
Second, to debug, you can launch Jenkins with this environment variable set first:
export GIT_SSH_COMMAND='ssh -Tv'
That way, any time it will try a git command involving ssh, you will see exactly where Jenkins/git tries to look for ssh keys, and what is the error.
Make sure you Git is recent enough to support this (Git 2.3+, Q4 2015).
But your logs show:
The requested URL returned error: 503 while accessing https://
So this has nothing to do with ssh. As mentioned, a proxy can interfere.

How to install yum repository key with ansible?

I tried it two ways:
- name: Add repository
yum_repository:
# from https://oss-binaries.phusionpassenger.com/yum/definitions/el-passenger.repo
name: passenger
description: Passenger repository
baseurl: https://oss-binaries.phusionpassenger.com/yum/passenger/el/$releasever/$basearch
repo_gpgcheck: 1
gpgcheck: 0
enabled: 1
gpgkey: https://packagecloud.io/gpg.key
sslverify: 1
sslcacert: /etc/pki/tls/certs/ca-bundle.crt
- name: Add repository key (option 1)
rpm_key:
key: https://packagecloud.io/gpg.key
- name: Add repository key (option 2)
command: rpm --import https://packagecloud.io/gpg.key
- name: Install nginx with passenger
yum: name={{ item }}
with_items: [nginx, passenger]
But for it to work, I need to ssh to the machine, confirm importing the key (by running any yum command, e.g. yum list installed), and then continue provisioning. Is there a way to do it automatically?
UPD here's what ansible says:
TASK [nginx : Add repository key] **********************************************
changed: [default]
TASK [nginx : Install nginx with passenger] ************************************
failed: [default] (item=[u'nginx', u'passenger']) => {"failed": true, "item": ["nginx", "passenger"], "msg": "Failure talking
to yum: failure: repodata/repomd.xml from passenger: [Errno 256] No more mirrors to try.\nhttps://oss-binaries.phusionpassen
ger.com/yum/passenger/el/7/x86_64/repodata/repomd.xml: [Errno -1] repomd.xml signature could not be verified for passenger"}
So, the key is indeed imported in both cases, but to be used it must be confirmed.
Fixed it by running yum directly with -y switch (and using rpm_key module, if anything):
- name: Install nginx with passenger
command: yum -y install {{ item }}
with_items: [nginx, passenger]
After adding the repository and the repository key, just update that repo's metadata with:
- name: update repo cache for the new repo
command: yum -q makecache -y --disablerepo=* --enablerepo=passenger
Then proceed with yum: name=... as before.

hudson.plugins.git.GitException: Could not fetch from any repository

I want to configure hudson+git and want to create a build. But I am getting following exception:
Started by user nitish
Checkout:workspace / /root/.hudson/jobs/NightlyGit/workspace - hudson.remoting.LocalChannel#dd7efd
Using strategy: Default
Checkout:workspace / /root/.hudson/jobs/NightlyGit/workspace - hudson.remoting.LocalChannel#dd7efd
Fetching changes from the remote Git repository
Fetching upstream changes from https://github.com/test/eng.git
ERROR: Problem fetching from origin / origin - could be unavailable. Continuing anyway
ERROR: (Underlying report) : Error performing command: /usr/bin/git fetch -t https://github.com/test/eng.git +refs/heads/nightly:refs/remotes/origin/nightly
Command "/usr/bin/git fetch -t https://github.com/test/engile.git +refs/heads/nightly:refs/remotes/origin/nightly" returned status code 128: error: The requested URL returned error: 401 while accessing https://github.com/test/eng.git/info/refs
fatal: HTTP request failed
ERROR: Could not fetch from any repository
FATAL: Could not fetch from any repository
hudson.plugins.git.GitException: Could not fetch from any repository
at hudson.plugins.git.GitSCM$3.invoke(GitSCM.java:887)
at hudson.plugins.git.GitSCM$3.invoke(GitSCM.java:845)
at hudson.FilePath.act(FilePath.java:758)
at hudson.FilePath.act(FilePath.java:740)
at hudson.plugins.git.GitSCM.gerRevisionToBuild(GitSCM.java:845)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:622)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1483)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:507)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:424)
at hudson.model.Run.run(Run.java:1366)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:145)
I am newbie to the git ,I dont know what is the exact problem is there any creadentials required while using git with hudson?. Right now I have created the hudson job and providing the git repository link but it throws above exception. can somebody help me to sort out this issue.
Try this:
Go to the job's "Configure" page.
Under Source Code Management → Git → Additional Behaviors, add a behavior "Wipe out repository & force clone":
Click on Save button.
Try to Build again.
I have created the SSH key on hudson server and added it to my github account that solved my issue.
Sometimes the permission of the workspace also can cause the same error.
sudo chown -R jenkins: <workspace>