Can I add arbitrary config to a pod spec deployed with a helm chart without modifying the helm chart? - kubernetes

Im using this helm chart to deploy: https://github.com/helm/charts/tree/master/stable/atlantis
It deploys this stateful set: https://github.com/helm/charts/blob/master/stable/atlantis/templates/statefulset.yaml
Is there a way I can add arbitrary config values to a pod spec that was deployed with a helm chart without having to modify the chart? For example I want to add an env: var that gets its value from a secret to the pod spec of the stateful set this chart deploys
Can I create my own helm chart that references this helm chart and add to the config of the pod spec? again without modifying the original chart?
EDIT: what Im talking about is adding an env var like this:
env:
- name: GET_THIS_VAR_IN_ATLANTIS
valueFrom:
secretKeyRef:
name: my-secret
key: abc
Maybe I can create another chart as a parent of this chart and override the entire env: block?

Is there a way I can add arbitrary config values to a pod spec that was deployed with a helm chart without having to modify the chart?
You can only make changes that the chart itself supports.
If you look at the StatefulSet definition you linked to, there are a lot of {{ if .Values.foo }} knobs there. This is an fairly customizable chart and you probably can change most things. As a chart author, you'd have to explicitly write all of these conditionals and macro expansions in.
For example I want to add an env: var that gets its value from a secret to the pod spec of the stateful set this chart deploys
This very specific chart contains a block
{{- range $key, $value := .Values.environment }}
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
so you could write a custom Helm YAML values file and add in
environment:
arbitraryKey: "any fixed value you want"
and then use the helm install -f option to supply that option when you install the chart.
This chart does not support injecting environment values from secrets, beyond a half-dozen specific values it supports by default (e.g., GitHub tokens).
As I say, this isn't generic at all: this is very specific to what this specific chart supports in its template expansions.

Should have marked the previous answer as the answer but things have changed in helm3.
While there is still no built-in way of patching a chart there is now builtin support for a "post renderer" https://helm.sh/docs/topics/advanced/
So, calling kustomize as a post renderer would probably be what most would suggest now with helm3

Related

helm dependencies with different namespaces

Right now, I have to install multiple helm charts in different namespaces for my product to work. I am trying to create a super helm chart in which I am planning to add the helm charts (of my tools, as mentioned above) and install them in one shot. My problem is, as these tools are in different namespaces I am not sure where to specify the namespace key where I want that particular dependency (chart) to be installed. For e.g. if below is the Charts.yaml of my super helm chart
dependencies:
- name: first_chart
version: 1.2.3
repository: https://firstchart.repo
- name: second_chart
version: 1.5.6
repository: https://secondchart.repo
I want my first chart to be installed in namespace foo and the second chart to be installed in namespace bar.
I was looking at using conditions but I believe conditions will only take a boolean as a value.
I stumbled upon this link (https://github.com/helm/helm/issues/2060) which says the we can do it in Helm 3 but mostly on how to keep releases between different namespaces. It does not specifically answer my question.
There is no builtin way to do this with pure Helm, but there is with helmfile.
Your example as helmfile.yaml:
releases:
- name: chart1 # name of the release (helm install <...> first_chart)
chart: repo1/first_chart
version: 1.2.3
namespace: foo
- name: chart2
chart: repo2/second_chart
version: 1.5.6
namespace: bar
# in case you want helmfile to automatically update repos
repositories:
- name: repo1
url: https://firstchart.repo
- name: repo2
url: https://secondchart.repo
Then, run:
helmfile sync => run helm install/upgrade on all releases, or
helmfile apply => same as sync, but do a diff first to only upgrade/install releases that changed
There is way more to helmfile, but this is the gist.
PS: if you struggle with values or want to have something similar to how umbrella Chart values are handled, have a look at helmfile: a simple trick to handle values intuitively
The way I solved this for my clusters with with ArgoCD's App of Apps cluster bootstrapping model. Of course, it requires that ArgoCD is install the cluster. However, for many reasons not relevant to this answer I would highly encourage installing ArgoCD regardless of the easy of bootstrapping capabilities.
Assuming ArgoCD is in place the structure is a single Helm chart containing templates for each of the child charts it will deploy and managed via Argo's Application CRD. You will notice there is a definition as part of the CRD, spec.destination.namespace, which governs where the chart will be deployed.
An example Application template which governs my cert-manager chart deployment to the cert-manager namespace looks like:
{{- if .Values.certManager.enabled }}
# ref: https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/#applications
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: cert-manager
# You'll usually want to add your resources to the argocd namespace.
namespace: argocd
# Add a this finalizer ONLY if you want these to cascade delete.
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
# The project the application belongs to.
project: cluster-configs
# Source of the application manifests
source:
repoURL: https://github.com/yourOrg/Helm
targetRevision: {{ .Values.targetRevision }}
path: charts/cert-manager-chart
# helm specific config
helm:
# Helm values files for overriding values in the helm chart
# The path is relative to the spec.source.path directory defined above
valueFiles:
{{- range .Values.certManager.valueFiles }}
- {{ . }}
{{- end }}
# Optional Helm version to template with. If omitted it will fall back to look at the 'apiVersion' in Chart.yaml
# and decide which Helm binary to use automatically. This field can be either 'v2' or 'v3'.
version: v3
# Destination cluster and namespace to deploy the application
destination:
server: https://kubernetes.default.svc
namespace: cert-manager
{{- end }}
With a corresponding values.yaml file for this parent chart which may look something like the following with the path to desired value file(s) in that child chart's directory specified.
targetRevision: v1.11.0
certManager:
enabled: true
valueFiles:
- "values.yaml"
clusterAutoScaler:
valueFiles:
- "envs/dev-account/saas/values.yaml"
clusterResourceLimits:
valueFiles:
- "values.yaml"
externalDns:
valueFiles:
- "envs/dev-account/saas/values.yaml"
ingressNginx:
enabled: true
valueFiles:
- "values.yaml"
Below is a screenshot of one of my app of apps directory to complete the example.

Using same spec across different deployment in argocd

I am currently using Kustomize. We are have multiple deployments and services. These have the same spec but different names. Is it possible to store the spec in individual files & refer them across all the deployments files?
Helm is a good fit for the solution.
However, since we were already using Kustomize & migration to Helm would have needed time, we solved the problem using namePrefix & label modifiers in Kustomize.
Use Helm, in ArgoCD create a pipeline with helm:3 container and create a helm-chart directory or repository. Pull the chart repository, deploy with helm. Use values.yaml for the dynamic values you want to use. Also, you will need to add kubeconfig file to your pipeline but that is another issue.
This is the best offer I can give. For further information I need to inspect ArgoCD.
I was faced with this problem and I resolved it using Helm3 charts:
I have a chart. Yaml file where I indicated my release name and version
values. Yam where I define all variable to use for a specific environment.
Values-test. Yaml a file to use, for example, in a test environment where you should only put the variable that must be changed from an environment to another.
I hope that can help you to resolve your issue.
I would also suggest using Helm. However a restriction of Helm is that you cannot create dynamic values.yaml files (https://github.com/helm/helm/issues/6699) - this can be very annoying, especially for multi-environment setups. However, ArgoCD provides a very nice way to do this with its Application type.
The solution is to create a custom Helm chart for generating your ArgoCD applications (which can be called with different config for each environment). The templates in this helm chart will generate ArgoCD Application types. This type supports a source.helm.values field where you can dynamically set the values.yaml.
For example, the values.yaml for HashiCorp Vault can be highly complex and this is a scenario where a dynamic values.yaml per environment is highly desirable (as this prevents having multiple values.yaml files for each environment which are large but very similar).
If your custom ArgoCD helm chart is my-argocd-application-helm, then the following are example values.yaml and the template which generates your Vault application i.e.
values.yaml
server: 1.2.3.4 # Target kubernetes server for all applications
vault:
name: vault-dev
repoURL: https://git.acme.com/myapp/vault-helm.git
targetRevision: master
path: helm/vault-chart
namespace: vault
hostname: 5.6.7.8 # target server for Vault
...
templates/vault-application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: {{ .Values.vault.name }}
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
destination:
namespace: 'vault'
server: {{ .Values.server }}
project: 'default'
source:
path: '{{ .Values.vault.path }}'
repoURL: {{ .Values.vault.repoURL }}
targetRevision: {{ .Values.vault.targetRevision }}
helm:
# Dynamically generate `values.yaml`
values: |
vault:
server:
ingress:
activeService: true
hosts:
- host: {{ required "Please set 'vault.hostname'" .Values.vault.hostname | quote }}
paths:
- /
ha:
enabled: true
config: |
ui = true
...
These values will then override any base configuration residing in the values.yaml specified by {{ .Values.vault.repoURL }} which can contain config which doesn't change for each environment.

How to easily duplicate helm charts

10 microservices on kubernetes with helm3 charts, and saw that all of them have similar structure standard, deployment, service, hpa, network policies etc. and basically the <helm_chart_name>/templates directory is 99% same on all with some if statements on top of file whether we want to deploy that resource,
{{ if .Values.hpa.create }}
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: {{ .Values.deployment.name }}
...
spec:
scaleTargetRef:
...
{{ end }}
and in values passing yes/no whether we want it - Is there some tool to easily create template for the helm charts ? To create Helm chart with this 5 manifests pre-populated with the reference to values as above ?
What you need is the Library Charts:
A library chart is a type of Helm chart that defines chart primitives
or definitions which can be shared by Helm templates in other charts.
This allows users to share snippets of code that can be re-used across
charts, avoiding repetition and keeping charts DRY.
You can find more details and examples in the linked documentation.
I think closest thing to the thing I want is https://helm.sh/docs/topics/library_charts/

helm - programmatically override subchart values.yaml

I'm writing a helm chart that uses the stable/redis chart as a subchart.
I need to override the storage class name used for both microservices within my chart, and within the redis chart.
I'm using helm 2.12.3
I would like to be able to specify redis.master.persistence.storageClass in terms of a template, like so
storage:
storageClasses:
name: azurefile
redis:
usePassword: false
master:
persistence:
storageClass: {{ $.Values.storage.storageClasses.name }}
Except, as I understand, templates aren't supported within values.yaml
As this is a public chart, I'm not able to modify it to depend on a global value as described here in the documentation
I considered using {{ $.Values.redis.master.persistence.storageClass }} elsewhere in my chart rather than {{ $.Values.storage.storageClasses.name }}, but this would:
Not hide the complexity of the dependencies of my chart
Not scale if I was to add yet another subchart dependency
In my values.yaml file I have:
storage:
storageClasses:
name: azurefile
redis:
master:
persistence:
storageClass: azurefile
I would like to specify a single value in values.yaml that can be overwritten at chart deploy time.
e.g. like this
helm install --set storage.storageClasses.name=foo mychart
rather than
helm install --set storage.storageClasses.name=foo --set redis.master.persistence.storageClass mychart
As you correctly mentioned, helm value files are plain yaml files which cannot contain any templates. For your use case, you'd need to use a templating system for your values files also which basically means you are also generating your value files on the go. I'd suggest taking a look at helmfile. This lets you share values file across multiple charts and application environments.

Best way to DRY up deployments that all depend on a very similar init-container

I have 10 applications to deploy to Kubernetes. Each of the deployments depends on an init container that is basically identical except for a single parameter (and it doesn't make conceptual sense for me to decouple this init container from the application). So far I've been copy-pasting this init container into each deployment.yaml file, but I feel like that's got to be a better way of doing this!
I haven't seen a great solution from my research, though the only thing I can think of so far is to use something like Helm to package up the init container and deploy it as part of some dependency-based way (Argo?).
Has anyone else with this issue found a solution they were satisfied with?
A Helm template can contain an arbitrary amount of text, just so long as when all of the macros are expanded it produces a valid YAML Kubernetes manifest. ("Valid YAML" is trickier than it sounds because the indentation matters.)
The simplest way to do this would be to write a shared Helm template that included the definition for the init container:
_init_container.tpl:
{{- define "common.myinit" -}}
name: myinit
image: myname/myinit:{{ .Values.initTag }}
# Other things from a container spec
{{ end -}}
Then in your deployment, include this:
deployment.yaml:
apiVersion: v1
kind: Deployment
spec:
template:
spec:
initContainers:
- {{ include "common.myinit" . | indent 10 | strip }}
Then you can copy the _init_container.tpl file into each of your individual services.
If you want to avoid the copy-and-paste (reasonable enough) you can create a Helm chart that contains only templates and no actual Kubernetes resources. You need to set up some sort of repository to hold this chart. Put the _init_container.tpl into that shared chart, declare it as a dependency is the chart metadata, and reference the template in your deployment YAML in the same way (Go template names are shared across all included charts).