I'm currently writing a Helm Chart for my multi-service application. In the application, I depend on CustomResources, which I apply before everything else with helm via the "helm.sh/hook": crd-install hook.
Now I want to upgrade the application. Helm fails because the CRDs are already installed.
In some GH issues, I read about the builtin .Capabilities variable in Helm templates. I want to wrap my CRDs with an "if" checking if the CRD is already installed:
{{- if (not (.Capabilities.APIVersions.Has "virtualmachineinstancepresets.kubevirt.io")) }}
Unfortunately, I misunderstood the APIVersions property.
So my question is, does Helm provide a way of checking whether a CustomAPI is already installed, so I can exclude it from my Helm pre-hook install?
The simple answer for Helm v2 is manually choose --no-crd-hook flag when running helm install.
The workaround by using builtin .Capabilities variable can be a workaround. E.g., using this:
{{- if not (.Capabilities.APIVersions.Has "virtualmachineinstancepresets.kubevirt.io/v1beta1/MyResource") }}
apiVersion: ...
{{- end}}
However, it also means you will never be able to manage the installed CRDs by Helm again.
Checkout a long answer from blog post Helm V2 CRD Management which explained different approaches. However, I quote this:
CRD management in helm is, to be nice about it, utterly horrible.
personally I suggest managing CRDs via a separate chart out of app/library charts that depends on it, since they have totally different lifecycle.
Related
I'm using ArgoCD for gitops deployments right now and I have k8s manifest files that require configMap values to be built (via helm right now). However, ArgoCD does not allow me to use helm lookup feature so I can't have that in my helm chart.
I have a helm chart that uses helm lookup to pull values from a configMap to build a k8s manifest file (pretty common). The problem is that argocd does not allow me to use the helm lookup functionality when it is deploying the helm chart. Has anyone come across this situation where you need to build a manifest file using values in a configMap while using ArgoCD? How did you accomplish that? Any help is greatly appreciated.
I think this is a well-known issue for ArgoCD to work with helm, I think the below thread will be helpful for you.
https://github.com/argoproj/argo-cd/issues/5202
I am trying to set up a helmfile deployment for my local kubernetes cluster which is running using 'kind' (a lightweight alternative to minikube). I have charts set up for my app which are all deploying correctly, however I require an nginx-ingress controller. Luckily 'kind' provides one, which I am currently applying with the command:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
It seems perverse that I should have everything else set up to deploy at the touch of a button, but still have to 'remember' (and also train my colleagues to remember...) to run this additional command.
I realise I could copy and paste and create my own version, but I would like to keep up to date with any changes made at source. Is it possible to create a chart that makes a reference to an external template?
I am looking at solutions using either helm or helmfile.
Your linked YAML file seems to have been generated from the ingress-nginx chart.
Subchart
You can include ingress-nginx as a subchart in Helm 3 by adding it as a dependency to your own chart. In Helm 3, this is done with the dependencies field in Chart.yaml, e.g.:
apiVersion: v2
name: my-chart
version: 0.1.0
dependencies:
- name: ingress-nginx
version: ~4.0.6
repository: https://kubernetes.github.io/ingress-nginx
condition: ingress-nginx.enabled
This may be problematic, however, if you need to install multiple versions of your own chart in the same cluster. To handle this, you'd need to consider the implications of multiple Ingress controllers.
Chart
Ingress controllers are capable of handling ingresses from various releases across multiple namespaces. Therefore, I would recommend maintaining ingress-nginx separately from your own releases that depend on it. This would mean installing ingress-nginx like you already are or as a separate chart (guide).
If you go this route, there are tools that help make it easier for devs to take a hands-off approach for setting up their K8s environments. Some popular ones include Skaffold, DevSpace, Tilt, and Helmfile.
In the company where I currently work we have RBAC Helm chart that is in-house developed that among other things has namespace resource in the template folder, thus the namespaces are now part of the Helm release.
We now realised that it is not such a good idea to include the namespace in Helm, because there may be number of cases (insert any reason here) where Helm may try to recreate given resource and if this happens on the namespace object it will delete (in order to recreate) the namespace along with everything in it.
My question: Is it possibly to make Helm stop tracking the namespace without actually deleting it, so that the namespace is no longer part of the Helm release?
As per Helm documentation, you can ask Helm not to delete the resource.
annotations:
"helm.sh/resource-policy": keep
The annotation "helm.sh/resource-policy": keep instructs Helm to skip deleting this resource when a helm operation (such as helm uninstall, helm upgrade or helm rollback) would result in its deletion. However, this resource becomes orphaned. Helm will no longer manage it in any way. This can lead to problems if using helm install --replace on a release that has already been uninstalled, but has kept resources.
I've been developing a prototype chart that depends on some custom resource definitions that are defined in one of the child charts.
To be more specific, I'm trying to create the resources defined in the strimzi-kafka-operator within my helm chart and would like the dependency to be explicitly installed first. I followed the helm documentation and added the following to my Chart.yaml
dependencies:
- name: strimzi-kafka-operator
version: 0.16.2
repository: https://strimzi.io/charts/
I ran:
$ helm dep up ./prototype-chart
$ helm install ./prototype-chart
> Error: unable to build Kubernetes objects from release manifest: unable to recognize "": no matches for kind "KafkaTopic" in version "kafka.strimzi.io/v1beta1"
which shows that it's trying to deploy my chart before my dependency. What is the correct way to install dependencies first and then my parent chart?
(For reference, here is the question I opened on GitHub directly with Strimzi where they informed me they aren't sure how to use their helm as a dependency:
https://github.com/strimzi/strimzi-kafka-operator/issues/2552
)
Regarding CRD's: the fact that Helm by default won't manage those1 is a feature, not a bug. It will still install them if not present; but it won't modify or delete existing CRD's. The previous version of Helm (v2) does, but (speaking from experience) that can get you into all sorts of trouble if you're not careful. Quoting from the link you referenced:
There is not support at this time for upgrading or deleting CRDs using Helm. This was an explicit decision after much community discussion due to the danger for unintentional data loss. [...] One of the distinct disadvantages of the crd-install method used in Helm 2 was the inability to properly validate charts due to changing API availability (a CRD is actually adding another available API to your Kubernetes cluster). If a chart installed a CRD, helm no longer had a valid set of API versions to work against. [...] With the new crds method of CRD installation, we now ensure that Helm has completely valid information about the current state of the cluster.
The idea here is that Helm should operate only at the level of release data (adding/removing deployments, storage, etc.); but with CRD's, you're actually modifying an extension to the Kubernetes API itself, potentially inadvertently breaking other releases that use the same definitions. Consider if you're on a team that has a "library" of CRDs shared between several charts, and you want to uninstall one — formerly, With v2, Helm would happily let you modify or even delete those at will, with no checks on if/how they were used in other releases. Changes to CRDs are changes to your control plane / core API, and should be treated as such — you're modifying global resources.
In short: with v3, Helm positions itself more as a "developer" tool to define, template, and manage releases; CRDs, however, are meant to be managed independently e.g. by a "cluster administrator". At the end of the day, it's a win for all sides, since developers can setup/teardown deployments at will, with confidence that it's not going to break functionality elsewhere... and whoever's on call won't have to deal with alerts if/when you accidentally delete/modify a CRD and break things in production :)
See also the extensive discussion here for more context behind this decision.
Hope this helps!
I am just wondering if anyone has figured out a declarative way to have helm charts installed/configured as part of a cluster initiation and that could be checked into source control. Using Kuberenetes I have very much gotten used to the "everything as code" type of workflow and I realize that installing and configuring helm is based mostly on imperative workflows via the CLI.
The reason I am asking is because currently we have our cluster in development and will be recreating it in production. Most of our configuration has been done declaratively via the deployment.yaml file. However we have spent a significant amount of time installing and configuring certain helm charts (e.g. prometheus, grafana etc.)
There a tools like helmfile or helmsman which allow you to declare to be installed Helm releases as code.
Here is an example from a helmfile.yaml doing so:
releases:
# Published chart example
- name: promnorbacxubuntu # name of this release
namespace: prometheus # target namespace
chart: stable/prometheus # the chart being installed to create this release, referenced by `repository/chart` syntax
set: # values (--set)
- name: rbac.create
value: false
Running helmfile charts will then ensure that all listed releases are installed
My team had a similar kind of problem and we solved it with Operators. And the best part about of Operators is that there are 3 kinds and one of them is Helm based.
So you could use a Helm Based Operator , create an associated CRD and then declare your configurations there. Those configurations are then ported directly to the Helm chart without you, as the user, having to do anything.