I do have multiple repositories like Project1, Project2, Project3.
I do have 1 repository where Helm charts are managed (deploy1).
I do this on Azure DevOps.
I now added a build Pipepline to Project1, which is working as expected.
Now i went into project deploy1 and wanted to create a new Release Pipeline, which is to be triggerd from Project1 build.
Now i would want to use the Helm chart from deploy1 to deploy to my kubernetes Cluster based on the published build from Project1.
Is this possible, is this even the correct approach?
Some might suggest that i keep the Helm Chart within Project1, but isnt that counter intuitive?
I also do not want to keep a copy of the same Helm Chart in every Projectn repository.
As i understood a Helm chart is used to manage a set of kubernetes Resources.
And if possible i would like to be able to remove my entire applicationstack, Project1, Project2 & Project3 with the unstill command from 1 Helm file.
well, I'd suggest using Azure Container Registry (acr) to store helm charts. That way you can use repo1 as source for all helm charts, when you'd build the charts, you'd package them and push to the acr. Then in other releases you'd use the same acr to pull those charts and apply them.
this can be done with az cli:
helm package --version $(build.buildId) --destination $(build.artifactStagingDirectory) %name%
az acr helm push %name%.tgz
you can pull them with az as well
Related
I have delved into the documentations of helm and still it is unclear what is the difference between the two. Here's what I understand so far
helm install -> install a helm chart
helm repo add -> add a repo from the internet
You can see Helm as a templating tool, which reads files from the templates directory, fills them with values from values.yaml, and deploys them into the Kubernetes cluster. These is all done by the helm install command. So, Helm install takes your chart and deploys it into the Kubernetes cluster.
One of the feature of Helm is helm package, which packages your chart into a single *.tgz file and then you can store it in the Helm registry. A lot of Helm charts are stored that way, you can look, e.g., into Artifact Hub. If you find a chart you'd like to install from the Helm registry, you can add that remote repo into your local Helm registry using helm repo add. Then, helm repo update downloads a Helm chart to your local registry. Downloading a repo just downloads the Helm chart into your local registry, but it does not deploy anything into the Kubernetes cluster. To do that, you need to use helm install.
We have around 50 micro services. Each micro service has its own repo in git. And in its repo we have a chart folder for the helm chart.
We also have an umbrella chart/parent chart for those 50 sub charts.
To differentiate dev, qa, production helm packages. We used different name for the umbrella chart and different versioning.
For example, all our development charts all have versions like version 0.0.0.xxxxx and production charts all have versions like 1.0.0.xxxxx.
The purpose of the different versioning strategy is so that we can pull down sub charts from different branches when building the parent chart.
When we run the pipeline from development branch, it will create helm chart with version prefix 0.0.0, and when from master branch, it creates chart version with prefix 1.0.0. And to make it simple, we are not using AppVersion, on Chart version and every time we build a new docker image, we bump up the Chart version.
For example, we have the following requirements.yaml in our development parent chart.
dependencies:
- name: subchart1
repository: 'chartmuseum'
version: ~0.0.0
- name: subchart2
repository: 'chartmusuem'
version: ~0.0.0
With this, when we run the pipeline of the development parent chart, it will pull down the dependencies that are built from development branch.
This works well.
The first problem we are having now is when multiple developers work on different micro services, they would include each other's changes when building the parent chart.
The second problem is with updating the sub charts. The yaml templates of all the charts are very similar(deployment, configmap, ingress). Sometimes, when we need to update an ingress setting for all the charts, we have to go to different git repos to update them and merge them to different branches. I am now considering creating one single dedicated git repo for all the charts. But I would like to hear some advice on the management of Chart. What are the best practices for managing helm charts and docker repositories in large scale.
My situation is as follows:
have a kubernetes cluster with a couple of nodes
have argocd installed on the cluster and working great
using gitlab for my repo and build pipelines
have another repo for storing my helm charts
have docker images being built in gitlab and pushed to my gitlab registry
have argocd able to point to my helm chart repo and sync the helm chart with my k8s cluster
have helm chart archive files pushed to my gitlab repo
While this is a decent setup, it's not ideal.
The first problem i faced with using a helm chart git repo is that I can't (or don't know) how to differentiate my staging environment with my production environment. Since I have a dev environment and prod environment in my cluster, argocd syncs both environments with the helm chart repo. I could get around this with separate charts for each environment but that isn't a valid solution.
The second problem i faced, while trying to get around the above problem, is that I can't get argocd to pull helm charts from a gitlab oci registry. I made it so that my build pipeline pushed the helm chart archive file to my gitlab container registry with the tag dev-latest or prod-latest, which is great, just what I want. The problem is that argocd, as far as I can tell, can't pull from gitlab's container registry.
How do I go about getting my pipeline automated with gitlab as my repo and build pipeline, helm for packaging my application, and argocd for syncing my helm application with my k8s cluster?
is that I can't get argocd to pull helm charts from a gitlab oci registry.
You might be interested by the latest Jul. 2021 GitLab 14.1:
Build, publish, and share Helm charts
Helm defines a chart as a Helm package that contains all of the resource definitions necessary to run an application, tool, or service inside of a Kubernetes cluster.
For organizations that create and manage their own Helm charts, it’s important to have a central repository to collect and share them.
GitLab already supports a variety of other package manager formats.
Why not also support Helm? That’s what community member and MVP from the 14.0 milestone Mathieu Parent asked several months ago before breaking ground on the new GitLab Helm chart registry. The collaboration between the community and GitLab is part of our dual flywheel strategy and one of the reasons I love working at GitLab. Chapeau Mathieu!
Now you can use your GitLab project to publish and share packaged Helm charts.
Simply add your project as a remote, authenticating with a personal access, deploy, or CI/CD job token.
Once that’s done you can use the Helm client or GitLab CI/CD to manage your Helm charts.
You can also download the charts using the API or the user interface.
What’s next? First, we’d like to present additional metadata for charts.
Then we’ll start dogfooding the feature by using it as a replacement for https://charts.gitlab.io/.
So, try out the feature and let us know how it goes by commenting in the epic GitLab-#6366.
See Documentation and issue.
So, what I'm trying to do is use helm to install an application to my kubernetes cluster. Let's say the image tag is 1.0.0 in the chart.
Then, as part of a CI/CD build pipeline, I'd like to update the image tag using kubectl, i.e. kubectl set image deployment/myapp...
The problem is if I subsequently make any change to the helm chart (e.g. number of replicas), and I helm upgrade myapp this will revert the image tag back to 1.0.0.
I've tried passing in the --reuse-values flag to the helm upgrade command but that hasn't helped.
Anyone have any ideas? Do I need to use helm to update the image tag? I'm trying to avoid this, as the chart is not available at this stage in the pipeline.
When using CI/CD to build and deploy, you should use a single source-of-truth, that means a file versioned in e.g. Git and you do all changes in that file. So if you use Helm charts, they should be stored in e.g. Git and all changes (e.g. new image) should be done in your Git repository.
You could have a build pipeline that in the end commit the new image to a Kubernetes config repository. Then a deployment pipeline is triggered that use Helm or Kustomize to apply your changes and possibly execute tests.
I have a private helm repo using apache, after migrating to helm3 I cannot install/search charts anymore.
Using helm v3
helm repo list
NAME URL
mas http://localhost:8080/charts/
helm search repo mas/devops-openshift
No results found
Using helm 2.*
helm search -r mas/devops-openshift
NAME CHART VERSION APP VERSION DESCRIPTION
mas/devops-openshift 7.0.0 Devops (OpenShift)
Same happens when using "helm install" command, it cannot find the charts.
I guess it could be something related to the helm repo index file. Maybe helmv3 is expecting a different structure? But same happen when generating index file from helmv3.
Thanks all for the answers but I've found the issue.
My repository were using development version of the charts so I had something like this 1.0.0-pre.dev (Semantic Versioning 2.0.0).
By default helm 3 does not look at non production charts.
You have to set the flag -devel. something like:
helm search repo mas/devops-openshift --devel
While migrating from helm 2 to helm 3 remove private repo and add it after migration, then run helm repo update to refresh repository file.
If the chart is available locally, run helm repo index <DIR> --url <your_repo_url> to create new index.yaml for this repository.
Running helm env will show you the directory where the repository.yamlis located so check if the file is generated correctly.