I am trying to automate my helm installation. while trying to run the below command in GCP, the external URL is not accessible based on our organization standard.
helm repo add appdynamics-charts https://ciscodevnet.github.io/appdynamics-charts
I am getting below error:
Error: Looks like "https://ciscodevnet.github.io/appdynamics-charts" is not a valid chart repository or cannot be reached: Get https://ciscodevnet.github.io/appdynamics-charts/index.yaml: Forbidden
May I know what alternative options do I have? we have Jfrog and gitlab as repository.
Related
Disclaimer: I'm new to Kubernetes and Helm.
I am trying to install a Helm chart using the brand new Helm Hub and for the life of me I can't figure out how this is supposed to work.
A new version of Helm (3.0) was released only a few months ago with significant changes, one of them is that it doesn't come with any repositories configured. Helm released the Helm Hub which is supposed to be a centralized service to find charts.
I am trying to install a CloudBees Jenkins chart. This is what I get when I search the hub:
[me#localhost tmp]$ helm search hub cloudbees -o yaml
- app_version: 2.222.1.1
description: The Continuous Delivery Solution for Enterprises
url: https://hub.helm.sh/charts/cloudbees/cloudbees-core
version: 3.12.0+80c17a044bc4
- app_version: 9.2.0.139827
description: A Helm chart for CloudBees Flow
url: https://hub.helm.sh/charts/cloudbees/cloudbees-flow
version: 1.1.1
- app_version: 9.2.0.139827
description: A Helm chart for CloudBees Flow Agent
url: https://hub.helm.sh/charts/cloudbees/cloudbees-flow-agent
version: 1.1.1
- app_version: 2.204.3.7
description: CloudBees Jenkins Distribution provides development teams with a highly
dependable, secure, Jenkins environment curated from the most recent supported
Jenkins release. The distribution comes with a recommended catalog of tested plugins
available through the CloudBees Assurance Program.
url: https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution
version: 2.204.307
- app_version: 2.0.2
description: Helm chart for sidecar injector webhook deployment
url: https://hub.helm.sh/charts/cloudbees/cloudbees-sidecar-injector
version: 2.0.2
So it looks like the chart I am looking for is available: cloudbees-jenkins-distribution.
However, I can't find any way to install from the hub or to add a repository based on the hub output. Some of the things I've tried:
[me#localhost tmp]$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈
[me#localhost tmp]$ helm install myJenkins cloudbees-jenkins-distribution
Error: failed to download "cloudbees-jenkins-distribution" (hint: running `helm repo update` may help)
[me#localhost tmp]$ helm repo add cbRepo https://hub.helm.sh/charts/cloudbees
Error: looks like "https://hub.helm.sh/charts/cloudbees" is not a valid chart repository or cannot be reached: error converting YAML to JSON: yaml: line 8: mapping values are not allowed in this context
[me#localhost tmp]$ helm repo add cbRepo https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution
Error: looks like "https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution" is not a valid chart repository or cannot be reached: error converting YAML to JSON: yaml: line 8: mapping values are not allowed in this context
The documentation really doesn't say much about how I'm supposed to go from the Helm Hub to an installed chart. What am I missing here?
Helm Hub is NOT like a repo that you can add and install from it helm charts. It doesn't expose valid repos urls either. That's why you're getting the error message like below:
Error: looks like "https://hub.helm.sh/charts/cloudbees" is not a valid chart repository ...
when you're trying to run helm repo add on https://hub.helm.sh based urls.
I know it may seem pretty confusing but it just works like that, by its very design. Please refer to this discussion on Github. Specifically this comment explains it a bit more and I hope it also answers your question:
hub.helm.sh is not the helm repo, so it will not work the you trying,
it is only meant to view and search for charts. check in there for
chart repository and it that way, then you will be able to install the
charts.
Unfortunatelly the official helm documentation doesn't explain it well enough. It mentions only:
helm search hub searches the Helm Hub, which comprises helm charts
from dozens of different repositories.
But it shows "no explanation how to get from helm search repo which shows hub.helm.sh to helm repo add which magically shows the a new url to use." - as one user wrote in the thread mentioned above.
Despite that Helm hub doesn't have a convenient way to get a repository url, it has a github repo that contains all vendors repository in one file.
We can use that fact to create a workaround, that you can improve using automation tools like bash,awk, sed, perl, python, etc.
Let's imagine we want to get the helm chart url using helm v3 and command line tools only.
(helm3 is just a symlink for helm v3.1.2)
Let's get the repository name for jenkins distribution for cloudbees vendor
$ helm3 search hub jenkins --max-col-width 1000 | grep cloudbees | tr "\t" "\n"
https://hub.helm.sh/charts/cloudbees/cloudbees-core
3.12.0+80c17a044bc4
2.222.1.1
The Continuous Delivery Solution for Enterprises
https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution
2.204.307
2.204.3.7
CloudBees Jenkins Distribution provides development teams with a highly dependable, secure, Jenkins environment curated from the most recent supported Jenkins
release. The distribution comes with a recommended catalog of tested plugins available through the CloudBees Assurance Program.
As we can see the chart page on Helm hub is
# https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution
# ^^^^^^^^^
so we can assume that repository name is also cloudbees
Let's find the url for cloudbees helm repository
$ curl -s https://raw.githubusercontent.com/helm/hub/master/config/repo-values.yaml | grep cloudbees
- name: cloudbees
url: https://charts.cloudbees.com/public/cloudbees
Now we can add this repository using helm repo command:
$ helm3 repo add cloudbees https://charts.cloudbees.com/public/cloudbees
"cloudbees" has been added to your repositories
Don't forget to update:
$ helm3 repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "cloudbees" chart repository
Update Complete. ⎈ Happy Helming!⎈
To find the chart name let's repeat search in repo
$ helm3 search repo jenkins
NAME CHART VERSION APP VERSION DESCRIPTION
cloudbees/cloudbees-jenkins-distribution 2.204.307 2.204.3.7 CloudBees Jenkins Distribution provides develop...
cloudbees/cloudbees-core 3.12.0+80c17a044bc4 2.222.1.1 The Continuous Delivery Solution for Enterprises
And finally we can install the chart:
$ helm3 install cloudbees/cloudbees-jenkins-distribution --version 2.204.307
Happy Helming3 ! :)
True, helm search hub cloudbees does not return the url of the repo.
What you can do from the result of helm search hub cloudbees is helm install $urlFromBefore:
$ helm search hub cloudbees
- app_version: 2.222.1.1
description: The Continuous Delivery Solution for Enterprises
url: https://hub.helm.sh/charts/cloudbees/cloudbees-core
...
$ helm install cloudbees https://hub.helm.sh/charts/cloudbees/cloudbees-core
from helm/helm#7419
To get the repo url, you can do the search on https://hub.helm.sh or
do a work around when you want the repo url from console:
$ curl https://hub.helm.sh/api/chartsvc/v1/charts/search?q=rancher | json_pp
The repo url is in key url in repo (data[...].attributes.repo.url).
(json_pp is optional. It pretty prints json, which helps me to read it.)
Now continue with
# variables names show the place in json output from above
helm repo add $data_attributes_repo_name $data_attributes_repo_url
helm repo update
helm install $data_attributes_repo_name/$data_attributes_name
I have been trying to publish an artifact created by helm package command to artifactory.
I setup the environment on my local Windows 10 laptop by installing the helm client.
I am able to create chart archives through helm package command.
After the package creation I tried to publish the package to a artifactory repo(local artifactory pro instance).
It is required by Helm that the repository where the package needs to be pushed be added as a repo via helm repo add command. But before executing the helm repo add command, the artifactory repo should be configured as a Helm Chart Repository at the time of repo creation.
After the successful execution of helm repo add I tried to push the archive to that artifactory location.
https://github.com/belitre/helm-push-artifactory-plugin is the plugin I am using to publish the archive to artifactory. However, I keep getting error once I execute command helm push-artifactory :
Error: 400: could not properly parse response JSON:
HTTP Status 400 – Bad Request
Type Status Report
Message Invalid URI
Description The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).
Please guide. let me know in case I missed out on any important details.
EDIT 1 : START
Command for adding repo :
helm repo add helm-chart-artifcatory-repo http://localhost:8081/artifactory/helm-chart-artifactory-repo/ --username myadminusername --password myadminpass
"helm-chart-artifcatory-repo" has been added to your repositories
Command for pushing to artifactory :
helm push-artifactory xyz-demofile-0.1.1.tgz http://localhost:8081/artifactory/helm-chart-artifactory-repo --username myadminusername --password myadminpass
EDIT 1 : END
I have an existing github project. I want to create/add a helm folder to the project to store the helm yaml files. I want to reference this github project/folder to act like a helm repo in my local/dev environment. I know I can add the charts to my local/default helm repo. The use case is if another developer checks out the code in github and he needs to work on the charts then he can run helm install directly from the working folder. The helm.sh website has instructions of adding a gh-pages branch but I am wondering if I can avoid it.
Can I use an existing github project and it via the helm repo add command?
Unfortunately, I wasn't able to find a way to publish helm charts via GitHub using private repositories. On a theoretical level, it might work using GitHub token and 2nd (raw URLs method), but I haven't tried it. Since you're using docker registry anyway, it might be worth trying using OCI (docker) registry to store the charts.
If that doesn't work, or you have public repos, it is possible to either use GitHub Pages, or use GitHub raw URLs. Both of the solutions require public repository.
To use GitHub pages:
Setup github pages to publish docs folder as github pages (you can use a different name, just substitue later)
Package the helm repo as .tgz (using helm package): helm package charts/mychart -d docs/. Substitute charts/mychart with a path to a chart root folder
Include an index.yaml -- an index file for the repository helm repo index ./docs --url https://<YOUR_ORG_OR_USERNAME>.github.io/<REPO_NAME>
Now you can add the repo: helm repo add <INTERNAL_NAME> https://<YOUR_ORG_OR_USERNAME>.github.io/<REPO_NAME>
To use Raw URLs:
Place index.yaml and chart TGZs into a folder called docs, just like above
Now you can add a repo: helm repo add <INTERNAL_NAME> https://raw.githubusercontent.com/<YOUR_ORG_OR_USERNAME>/<REPO_NAME>/<BRANCH_USUALLY_MASTER>/docs
Firstly make sure that you have have fully functional helm repository. The tricky part is to access it as if it was simple HTTP server hosting raw files. Fortunately Github provides such feature using raw.githubusercontent.com. In order for helm to be able to pull files from such repository you need to provide it with Github username and token (Personal Access Token):
> helm repo add - username <your_github_username> - password <your_github_token> my-github-helm-repo 'https://raw.githubusercontent.com/my_organization/my-github-helm-repo/master/'
> helm repo update
> helm repo list
NAME URL
stable https://kubernetes-charts.storage.googleapis.com
local http://127.0.0.1:8879/charts
my-github-helmrepo https://raw.githubusercontent.com/my_organization/my-github-helm-repo/master/
> helm search my-app
NAME CHART VERSION APP VERSION DESCRIPTION
my-github-helmrepo/my-app-chart 0.1.0 1.0 A Helm chart for Kubernetes
These are steps for adding new packages to existing repository
If you want to add new package to existing repository simply:
1. Place new package in your local repository root
2. Execute: helm repo index .. This will detect new file/folder and make updates.
3. Commit and push your new package
4. Finally execute command: helm repo update
Security ascpect
It is important to realize where does helm actually store your Github token. It is stored as plain text in ~/.helm/repository/repositories.yaml. In this case it will be good to generate token with as few permissions as possible.
Take a look here: hosting helm private repository.
In case of Rancher the Private Catalogs get added and the private catalogs are also displayed but the helm charts associated to a private catalog can't be accessed. If I select a catalog I don't find the templates files listed.
In case if we put the same helm chart on a public catalog the templates file get listed. The issue so clearly is with Rancher and not with the helm charts.
I tried to put the helm charts on different private repository like ACR and git private repo and the issue still persists so also the issue with the registry is ruled out.
Steps to reproduce:
1) Create a Private App Catalog (any, but I used ACR)
2) Add the app catalog to Rancher by providing the correct Credentials.
3) Go and Launch the app
4) The helm chart(pushed in prev steps) gets listed.
5) Try to go and Launch the App.
Result:
You find that there is no template files listed (Values.yml,Deployment.yml etc are not listed )
Logs of Rancher Server
[ERROR] Failed to load chart: Error fetching helm URLs: [Error in HTTP
GET of [_blobs/.tgz], error: Get //user:*#_blobs/**-0.1.0.tgz:
unsupported protocol scheme ""]
I get a unsupported Protocol schema error when the chart tries to read the index.yml and then ries to get the *.tar.gz file
The issue seems to be linked to other issues like : https://github.com/rancher/rancher/issues/15671
We need to use Rancher Charts for charts to be correctly listed in the app catalog of Rancher.
Rancher Chart and Helm chart have some differences which are listed here : https://rancher.com/docs/rancher/v2.x/en/catalog/custom/creating/
There are two ways to use charts in Rancher:
Helm chart way that requires the GIT server responding to GET request ( charts are stored as tar.gz file along with index.yml file).
Rancher chart way where the charts are stored as normal files ( store the whole helm chart folder as it is , no need to gunzip it as in helm chart way) on GIT server.
In my case i had the tar.gz file that had the helm chart and index.yml file that renders the chart. This way is supported by Rancher only if there is some external server responding to the GET request which finds the chart from the index.yaml. Github pages support this feature that is why I was able to use the helm chart in Rancher.
Solution : I unzipped the tar and directly uploaded folder on GIT and use this GIT repo in Rancher to get the chart correctly listed under app catalog.
Do remember to use .git at the end of the url defined in the app catalog.
I am installing Kubernetes in a non-internet environment. I want to use Helm and want to set up a custom chart repository.
helm init barfs after creating ~/.helm/repository/repositories.yaml as it can't reach the default Google repo, so I will end up installing manually via kubectl - what is the format of this chart repository if I want to set up my own?
I will run helm init --dry-run --debug in order to get the manifest and amend this to point at a Docker registry that I have access to then install via kubectl.
I didn't see the section in the docs here: https://github.com/kubernetes/helm/blob/master/docs/chart_repository.md
It's a web server.
A custom helm repository doesnt have to do anything with your kubernetes. Setting up the custom repo can be done without connecting even to any kubernetes cluster.
helm init --client-only
Next step is to set up your custom repository somewhere like a storage account or whatever, then add that custom repo Url to your helm. So far this is all client side; you didnt touch your cluster yet.
helm repo add <HELM-REPO-NAME> <HEM-REPO-URL>
The final step which is where you will need to connect to a real kubernetes cluster to install the helm chart using Tiller which lives inside your cluster.