I have a helm chart and I want to add it to my gitlab repository. But when I run:
helm repo add repo_name url
I am getting the following error:
Error: looks like "https://gitlab.<domain>.com/group/infra/repo/helm/charts/" is not a valid chart repository or cannot be reached: error converting YAML to JSON: yaml: line 3: mapping values are not allowed in this context
Linter shows it is a valid chart.
Here is index.yaml:
apiVersion: v1
entries:
helloworld:
- apiVersion: v2
appVersion: 1.0.0
created: "2021-06-28T14:05:53.974207+01:00"
description: This Helm chart will be used to create hello world
digest: f290432f0280fe3f66b126c28a0bb21263d64fd8f73a16808ac2070b874619e7
name: helloworld
type: application
urls:
- https://gitlab.<domain>.com/group/infra/repo/helm/charts/helloworld-0.1.0.tgz
version: 0.1.0
generated: "2021-06-28T14:05:53.973549+01:00"
Not sure what is missing here.
It looks like you want to use the helm chart that is hosted on the gitlab. Unfortunately, it won't work as you want it to. As Lei Yang mentioned well in the comment:
helm repo and git repo are different things.
In the official documentation of Helm, you can find The Chart Repository Guide.
You can find it also a guide how to create a chart repository:
A chart repository is an HTTP server that houses an index.yaml file and optionally some packaged charts. When you're ready to share your charts, the preferred way to do so is by uploading them to a chart repository.
Here you can find section, how to properly host chart repos. There are several ways to do this - for example you can use a Google Cloud Storage (GCS) bucket, Amazon S3 bucket, GitHub Pages, or even create your own web server.
You can also use the ChartMuseum server to host a chart repository from a local file system.
ChartMuseum is an open-source Helm Chart Repository server written in Go (Golang), with support for cloud storage backends, including Google Cloud Storage, Amazon S3, Microsoft Azure Blob Storage, Alibaba Cloud OSS Storage, Openstack Object Storage, Oracle Cloud Infrastructure Object Storage, Baidu Cloud BOS Storage, Tencent Cloud Object Storage, DigitalOcean Spaces, Minio, and etcd.
Alternatively it could be also possible to host helm charts in JFrog.
You can host your own Public Helm repository on git.I have done it on Github and the process is very easy and straightforward.
You can follow this link
https://medium.com/#mattiaperi/create-a-public-helm-chart-repository-with-github-pages-49b180dbb417
You will have to package the chart and create an index.yaml file.You will also have to host your repository branch as Github pages.
I am not sure if gitlab also supports this but worth a shot.
Related
My situation is as follows:
have a kubernetes cluster with a couple of nodes
have argocd installed on the cluster and working great
using gitlab for my repo and build pipelines
have another repo for storing my helm charts
have docker images being built in gitlab and pushed to my gitlab registry
have argocd able to point to my helm chart repo and sync the helm chart with my k8s cluster
have helm chart archive files pushed to my gitlab repo
While this is a decent setup, it's not ideal.
The first problem i faced with using a helm chart git repo is that I can't (or don't know) how to differentiate my staging environment with my production environment. Since I have a dev environment and prod environment in my cluster, argocd syncs both environments with the helm chart repo. I could get around this with separate charts for each environment but that isn't a valid solution.
The second problem i faced, while trying to get around the above problem, is that I can't get argocd to pull helm charts from a gitlab oci registry. I made it so that my build pipeline pushed the helm chart archive file to my gitlab container registry with the tag dev-latest or prod-latest, which is great, just what I want. The problem is that argocd, as far as I can tell, can't pull from gitlab's container registry.
How do I go about getting my pipeline automated with gitlab as my repo and build pipeline, helm for packaging my application, and argocd for syncing my helm application with my k8s cluster?
is that I can't get argocd to pull helm charts from a gitlab oci registry.
You might be interested by the latest Jul. 2021 GitLab 14.1:
Build, publish, and share Helm charts
Helm defines a chart as a Helm package that contains all of the resource definitions necessary to run an application, tool, or service inside of a Kubernetes cluster.
For organizations that create and manage their own Helm charts, it’s important to have a central repository to collect and share them.
GitLab already supports a variety of other package manager formats.
Why not also support Helm? That’s what community member and MVP from the 14.0 milestone Mathieu Parent asked several months ago before breaking ground on the new GitLab Helm chart registry. The collaboration between the community and GitLab is part of our dual flywheel strategy and one of the reasons I love working at GitLab. Chapeau Mathieu!
Now you can use your GitLab project to publish and share packaged Helm charts.
Simply add your project as a remote, authenticating with a personal access, deploy, or CI/CD job token.
Once that’s done you can use the Helm client or GitLab CI/CD to manage your Helm charts.
You can also download the charts using the API or the user interface.
What’s next? First, we’d like to present additional metadata for charts.
Then we’ll start dogfooding the feature by using it as a replacement for https://charts.gitlab.io/.
So, try out the feature and let us know how it goes by commenting in the epic GitLab-#6366.
See Documentation and issue.
Disclaimer: I'm new to Kubernetes and Helm.
I am trying to install a Helm chart using the brand new Helm Hub and for the life of me I can't figure out how this is supposed to work.
A new version of Helm (3.0) was released only a few months ago with significant changes, one of them is that it doesn't come with any repositories configured. Helm released the Helm Hub which is supposed to be a centralized service to find charts.
I am trying to install a CloudBees Jenkins chart. This is what I get when I search the hub:
[me#localhost tmp]$ helm search hub cloudbees -o yaml
- app_version: 2.222.1.1
description: The Continuous Delivery Solution for Enterprises
url: https://hub.helm.sh/charts/cloudbees/cloudbees-core
version: 3.12.0+80c17a044bc4
- app_version: 9.2.0.139827
description: A Helm chart for CloudBees Flow
url: https://hub.helm.sh/charts/cloudbees/cloudbees-flow
version: 1.1.1
- app_version: 9.2.0.139827
description: A Helm chart for CloudBees Flow Agent
url: https://hub.helm.sh/charts/cloudbees/cloudbees-flow-agent
version: 1.1.1
- app_version: 2.204.3.7
description: CloudBees Jenkins Distribution provides development teams with a highly
dependable, secure, Jenkins environment curated from the most recent supported
Jenkins release. The distribution comes with a recommended catalog of tested plugins
available through the CloudBees Assurance Program.
url: https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution
version: 2.204.307
- app_version: 2.0.2
description: Helm chart for sidecar injector webhook deployment
url: https://hub.helm.sh/charts/cloudbees/cloudbees-sidecar-injector
version: 2.0.2
So it looks like the chart I am looking for is available: cloudbees-jenkins-distribution.
However, I can't find any way to install from the hub or to add a repository based on the hub output. Some of the things I've tried:
[me#localhost tmp]$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈
[me#localhost tmp]$ helm install myJenkins cloudbees-jenkins-distribution
Error: failed to download "cloudbees-jenkins-distribution" (hint: running `helm repo update` may help)
[me#localhost tmp]$ helm repo add cbRepo https://hub.helm.sh/charts/cloudbees
Error: looks like "https://hub.helm.sh/charts/cloudbees" is not a valid chart repository or cannot be reached: error converting YAML to JSON: yaml: line 8: mapping values are not allowed in this context
[me#localhost tmp]$ helm repo add cbRepo https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution
Error: looks like "https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution" is not a valid chart repository or cannot be reached: error converting YAML to JSON: yaml: line 8: mapping values are not allowed in this context
The documentation really doesn't say much about how I'm supposed to go from the Helm Hub to an installed chart. What am I missing here?
Helm Hub is NOT like a repo that you can add and install from it helm charts. It doesn't expose valid repos urls either. That's why you're getting the error message like below:
Error: looks like "https://hub.helm.sh/charts/cloudbees" is not a valid chart repository ...
when you're trying to run helm repo add on https://hub.helm.sh based urls.
I know it may seem pretty confusing but it just works like that, by its very design. Please refer to this discussion on Github. Specifically this comment explains it a bit more and I hope it also answers your question:
hub.helm.sh is not the helm repo, so it will not work the you trying,
it is only meant to view and search for charts. check in there for
chart repository and it that way, then you will be able to install the
charts.
Unfortunatelly the official helm documentation doesn't explain it well enough. It mentions only:
helm search hub searches the Helm Hub, which comprises helm charts
from dozens of different repositories.
But it shows "no explanation how to get from helm search repo which shows hub.helm.sh to helm repo add which magically shows the a new url to use." - as one user wrote in the thread mentioned above.
Despite that Helm hub doesn't have a convenient way to get a repository url, it has a github repo that contains all vendors repository in one file.
We can use that fact to create a workaround, that you can improve using automation tools like bash,awk, sed, perl, python, etc.
Let's imagine we want to get the helm chart url using helm v3 and command line tools only.
(helm3 is just a symlink for helm v3.1.2)
Let's get the repository name for jenkins distribution for cloudbees vendor
$ helm3 search hub jenkins --max-col-width 1000 | grep cloudbees | tr "\t" "\n"
https://hub.helm.sh/charts/cloudbees/cloudbees-core
3.12.0+80c17a044bc4
2.222.1.1
The Continuous Delivery Solution for Enterprises
https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution
2.204.307
2.204.3.7
CloudBees Jenkins Distribution provides development teams with a highly dependable, secure, Jenkins environment curated from the most recent supported Jenkins
release. The distribution comes with a recommended catalog of tested plugins available through the CloudBees Assurance Program.
As we can see the chart page on Helm hub is
# https://hub.helm.sh/charts/cloudbees/cloudbees-jenkins-distribution
# ^^^^^^^^^
so we can assume that repository name is also cloudbees
Let's find the url for cloudbees helm repository
$ curl -s https://raw.githubusercontent.com/helm/hub/master/config/repo-values.yaml | grep cloudbees
- name: cloudbees
url: https://charts.cloudbees.com/public/cloudbees
Now we can add this repository using helm repo command:
$ helm3 repo add cloudbees https://charts.cloudbees.com/public/cloudbees
"cloudbees" has been added to your repositories
Don't forget to update:
$ helm3 repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "cloudbees" chart repository
Update Complete. ⎈ Happy Helming!⎈
To find the chart name let's repeat search in repo
$ helm3 search repo jenkins
NAME CHART VERSION APP VERSION DESCRIPTION
cloudbees/cloudbees-jenkins-distribution 2.204.307 2.204.3.7 CloudBees Jenkins Distribution provides develop...
cloudbees/cloudbees-core 3.12.0+80c17a044bc4 2.222.1.1 The Continuous Delivery Solution for Enterprises
And finally we can install the chart:
$ helm3 install cloudbees/cloudbees-jenkins-distribution --version 2.204.307
Happy Helming3 ! :)
True, helm search hub cloudbees does not return the url of the repo.
What you can do from the result of helm search hub cloudbees is helm install $urlFromBefore:
$ helm search hub cloudbees
- app_version: 2.222.1.1
description: The Continuous Delivery Solution for Enterprises
url: https://hub.helm.sh/charts/cloudbees/cloudbees-core
...
$ helm install cloudbees https://hub.helm.sh/charts/cloudbees/cloudbees-core
from helm/helm#7419
To get the repo url, you can do the search on https://hub.helm.sh or
do a work around when you want the repo url from console:
$ curl https://hub.helm.sh/api/chartsvc/v1/charts/search?q=rancher | json_pp
The repo url is in key url in repo (data[...].attributes.repo.url).
(json_pp is optional. It pretty prints json, which helps me to read it.)
Now continue with
# variables names show the place in json output from above
helm repo add $data_attributes_repo_name $data_attributes_repo_url
helm repo update
helm install $data_attributes_repo_name/$data_attributes_name
In case of Rancher the Private Catalogs get added and the private catalogs are also displayed but the helm charts associated to a private catalog can't be accessed. If I select a catalog I don't find the templates files listed.
In case if we put the same helm chart on a public catalog the templates file get listed. The issue so clearly is with Rancher and not with the helm charts.
I tried to put the helm charts on different private repository like ACR and git private repo and the issue still persists so also the issue with the registry is ruled out.
Steps to reproduce:
1) Create a Private App Catalog (any, but I used ACR)
2) Add the app catalog to Rancher by providing the correct Credentials.
3) Go and Launch the app
4) The helm chart(pushed in prev steps) gets listed.
5) Try to go and Launch the App.
Result:
You find that there is no template files listed (Values.yml,Deployment.yml etc are not listed )
Logs of Rancher Server
[ERROR] Failed to load chart: Error fetching helm URLs: [Error in HTTP
GET of [_blobs/.tgz], error: Get //user:*#_blobs/**-0.1.0.tgz:
unsupported protocol scheme ""]
I get a unsupported Protocol schema error when the chart tries to read the index.yml and then ries to get the *.tar.gz file
The issue seems to be linked to other issues like : https://github.com/rancher/rancher/issues/15671
We need to use Rancher Charts for charts to be correctly listed in the app catalog of Rancher.
Rancher Chart and Helm chart have some differences which are listed here : https://rancher.com/docs/rancher/v2.x/en/catalog/custom/creating/
There are two ways to use charts in Rancher:
Helm chart way that requires the GIT server responding to GET request ( charts are stored as tar.gz file along with index.yml file).
Rancher chart way where the charts are stored as normal files ( store the whole helm chart folder as it is , no need to gunzip it as in helm chart way) on GIT server.
In my case i had the tar.gz file that had the helm chart and index.yml file that renders the chart. This way is supported by Rancher only if there is some external server responding to the GET request which finds the chart from the index.yaml. Github pages support this feature that is why I was able to use the helm chart in Rancher.
Solution : I unzipped the tar and directly uploaded folder on GIT and use this GIT repo in Rancher to get the chart correctly listed under app catalog.
Do remember to use .git at the end of the url defined in the app catalog.
I want to use Artifactory as private Helm repository and deploy my app using helm chart to Kubernetes everytime a new version of helm chart gets uploaded to Artifactory.
Yes, you can do it. You can find more information about it here
The Helm charts repo is a web server that serves files, it could also be something like Amazon S3 or Google Cloud Storage bucket and then you can add an index.yaml file and then all your charts. So you could also integrate that with Spinnaker.
Is this even possible with helm?
What options for security are possible (BasicAuth? Client Certs? OAuth2?)
Is there any example / documentation about this (both server and client side)?
Helm Private Repository using private GitHub repositories.
Steps:
Create a private GitHub repository named private-helm-registry or any.
Place your Raw Helm Charts.
Create a charts folder and place index.yaml and packaged charts in .tgz format inside this folder.
Create a GitHub Personal Access Token with read-only access.
Add your Repository to helm using the following command:
$ helm repo add helm-registry 'https://<git_hub_personal_access_token>#raw.githubusercontent.com/myGitHubAccountName/private-helm-registry/master/charts/'
"helm-registry" has been added to your repositories
Note:
1. Enclose the Url with single quotes ' '.
2. The trailing / is mandatory.
To add development or other branches as helm repository, use branch name:
$ helm repo add helm-registry-dev 'https://<git_hub_personal_access_token>#raw.githubusercontent.com/myGitHubAccountName/private-helm-registry/<branch>/charts/'
"helm-registry-dev" has been added to your repositories
Explore more at: Using a private github repo as helm chart repo.
If you want a private helm repository, there are not many options today, at least to my knowledge.
Use internal network where you deploy your repository, so it will be not accessible from the outer world.
Use helm plugins or write your own.
So basically there are no built-in helm private repositories, but you can achieve required functionality with helm plugins.
Plugin examples:
GitHub - it is for using GitHub Pages as a chart repository. Not a private repo, but it shows an idea of a plugin, it is very small and simple.
Keybase - similar to GitHub plugin, but stores charts in Keybase instead. Still not private, but can be modified to use keybase /private directories.
App Registry - uses app registry (quay.io) to store charts.
AWS S3 - provides a way to use Amazon S3 as a private repo for helm charts. I am the author of this plugin, I wrote it because all options above were not suitable for my purposes.
Azure Blob Stoage - This blog post contains a way to use Azure Blob storage as a private repo for helm charts
I assume you mean a Helm Repo rather than securing the docker images.
My understanding is that it's just a basic webserver
You could just use GCS or S3 and set up the IAM rules to secure them?