Helm - Templating variables in values.yaml - kubernetes

I'm trying to template variables from a map inside the values.yaml into my final Kubernetes ConfigMap YAML.
I've read through https://github.com/helm/helm/issues/2492 and https://helm.sh/docs/chart_template_guide/ but can't seem to find an answer.
For some context, this is roughly what I'm trying to do:
values.yaml
config:
key1: value
key2: value-{{ .Release.Name }}
configmap.yaml
kind: ConfigMap
data:
config-file: |
{{- range $key, $value := .Values.config }}
{{ $key }} = {{ $value }}
{{- end }}
Where the desired output with would be:
helm template --name v1 mychart/
kind: ConfigMap
data:
config-file: |
key1 = value
key2 = value-v1
I've tried a few variations using template functions and pipelining, but to no avail:
{{ $key }} = {{ tpl $value . }}
{{ $key }} = {{ $value | tpl . }}
{{ $key }} = {{ tpl $value $ }}

The above would also have worked in this way
values.yaml
config:
key1: "value"
key2: "value-{{ .Release.Name }}"
configmap.yaml
kind: ConfigMap
data:
config-file: |
{{- range $key, $value := .Values.config }}
{{ $key }} = {{ tpl $value $ }}
{{- end }}
What I changed was : I put value in quotes in value.yaml and used template tpl in the config map.

I'll refer to the question's title regarding templating variables in helm and suggest another option to use on values.yaml which is YAML Anchors.
Docs reference
As written in here:
The YAML spec provides a way to store a reference to a value, and
later refer to that value by reference. YAML refers to this as
"anchoring":
coffee: "yes, please"
favorite: &favoriteCoffee "Cappucino"
coffees:
- Latte
- *favoriteCoffee
- Espresso
In the above, &favoriteCoffee sets a reference to Cappuccino.
Later, that reference is used as *favoriteCoffee.
So coffees becomes Latte, Cappuccino, Espresso.
A more practical example
Referring to a common image setup (Registry and PullPolicy) in all values.yaml.
Notice how the default values are being set at Global.Image next to the reference definition which starts with &:
Global:
Image:
Registry: &global-docker-registry "12345678910.dkr.ecr.us-west-2.amazonaws.com" # <--- Default value
PullPolicy: &global-pull-policy "IfNotPresent" # <--- Default value
Nginx:
Image:
Registry: *global-docker-registry
PullPolicy: *global-pull-policy
Version: 1.21.4
Port: 80
MySql:
Image:
Registry: *global-docker-registry
PullPolicy: *global-pull-policy
Name: mysql
Version: 8.0.27
Port: 3306

Managed to solve this using the following syntax:
configmap.yaml
kind: ConfigMap
data:
config-file: |
{{- range $key, $value := .Values.config }}
{{ $key }} = {{ tpl ($value | toString) $ }}
{{- end }}

there is fight in this PR here about this topic.
I know that it's possible now, but this require maintenance of the chart to be in-house (e.g. answer of Amrut ).
Let's summarize :
To have templating in values.yaml , these are the available options:
helm may support that in future ( watch this thread about this topic.)
use tpl function inside the chart
use another tool on top of helm : terraform or helmfile.

Related

Is there a clean way to provide env var overrides using defaults, configmap, secrets, and command-line options?

We have some services that can be installed in multiple locations with differing configurations. We've been asked to support multi-level configuration options using environment variables set with defaults, configmaps, secrets, and command-line options passed in via helm install --set. The following works, but is very cumbersome as the number of parameters for some of the services are numerous and the Values dot-notation goes a few levels deeper.
env:
# Set default values
- name: MY_VAR
value: default-value
- name: OTHER_VAR
value: default-other-value
# Allow configmap to override
- name: MY_VAR
valueFrom:
configMapKeyRef:
name: env-configmap
key: MY_VAR
optional: true
- name: OTHER_VAR
valueFrom:
configMapKeyRef:
name: env-configmap
key: OTHER_VAR
optional: true
# Allow secrets to override
- name: MY_VAR
valueFrom:
secretsKeyRef:
name: env-secrets
key: MY_VAR
optional: true
- name: OTHER_VAR
valueFrom:
secretsKeyRef:
name: env-secrets
key: OTHER_VAR
optional: true
# Allow 'helm install --set' to override
{{- if .Values.env }}
{{- if .Values.env.my }}
{{- if .Values.env.my.var }}
- name: MY_VAR
value: {{ .Values.env.my.var }}
{{- end }}
{{- end }}
{{- if .Values.env.other }}
{{- if .Values.env.other.var }}
- name: OTHER_VAR
value: {{ .Values.env.other.var }}
{{- end }}
{{- end }}
{{- end }}
Using envFrom for the ConfigMap and Secrets would be nice, but tests and docs show this would not allow the command-line override, since env: and envFrom: doesn't mix in the way that's needed. As the v1.9 and v2.1 Kubernetes API states:
envFrom: List of sources to populate environment variables in the
container. The keys defined within a source must be a C_IDENTIFIER.
All invalid keys will be reported as an event when the container is
starting. When a key exists in multiple sources, the value associated
with the last source will take precedence. Values defined by an Env
with a duplicate key will take precedence. Cannot be updated.
Is there a better way to provide this default->configmap->secrets->cmd-line override precedence?
I found a solution that I mostly like. My issue was caused by giving too much weight to the "Values defined by an Env with a duplicate key will take precedence" comment in the docs, and thinking I needed to exclusively use Env. The defined precedence is exactly what I needed.
Here's the helm chart files for my current solution.
configmap/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
data:
{{- if .Values.env }}
{{- toYaml $.Values.env | nindent 2 }}
{{- end }}
{{- if .Values.applicationYaml }}
application.yml: |
{{- toYaml $.Values.applicationYaml | nindent 4 }}
{{- end }}
secrets/templates/secrets.yaml
apiVersion: v1
kind: Secret
metadata:
name: my-secrets
type: Opaque
data:
{{- range $key, $val := .Values.env }}
{{ $key }}: {{ $val | b64enc }}
{{- end }}
stringData:
{{- if .Values.applicationYaml }}
application.yml: |
{{- toYaml $.Values.applicationYaml | nindent 4 }}
{{- end }}
deployment.yaml
apiVersion: apps/v1
kind: Deployment
...
spec:
...
containers:
- name: my-deployment
{{- if .Values.env }}
env:
{{- range $key, $val := .Values.env }}
- name: {{ $key }}
value: {{ $val }}
{{- end }}
{{- end }}
envFrom:
- configMapRef:
name: my-configmap
- secretRef:
name: my-secrets
volumeMounts:
- name: configmap-application-config
mountPath: /application/config/configmap/
- name: secrets-application-config
mountPath: /application/config/secrets/
volumes:
- name: configmap-application-config
configMap:
name: my-configmap
optional: true
- name: secrets-application-config
secret:
secretName: my-secrets
optional: true
Since this is a Spring Boot app, I used volumeMounts to allow the application.yml default values to be overridden in the ConfigMap and Secrets. The order of precedence from lowest to highest is:
the application's application.yml (v1 in following examples)
the configmap's applicationYaml (v2)
the secret's applicationYaml (v3)
the configmap env (v4)
the secret env (v5)
the helm install/uninstall --set (v6)
To complete the example, here's test values yaml files and the command-line.
app/src/main/resources/application.yml
applicationYaml:
test:
v1: set-from-this-value
v2: overridden
v3: overridden
v4: overridden
v5: overridden
v6: overridden
configmap/values.yaml
applicationYaml:
test:
v2: set-from-this-value
v3: overridden
v4: overridden
v5: overridden
v6: overridden
env:
TEST_V4: set-from-this-value
TEST_V5: overridden
TEST_V6: overridden
secrets/values.yaml
applicationYaml:
test:
v3: set-from-this-value
v4: overridden
v5: overridden
v6: overridden
env:
TEST_V5: set-from-this-value
TEST_V6: overridden
command-line
helm install --set env.TEST_V6=set-from-this-value ...
Ideally, I'd like to be able to use dot-notation instead of TEST_V6 in the env and --set fields, but I'm not finding a way in helm to operate only on the leaves of yaml. In other words, I'd like something like range $key, $val, but where the key is equal to "test.v6". If that was possible, the key could be internally converted to an environment variable name with {{ $key | upper | replace "-" "_" | replace "." "_" }}.

Transform a helm dict into a list

I'm using a Helm chart to control what environment variables are set for a certain container in a deployment.
In my Values.yaml, I have an entry called env which is a dictionary:
image:
repository: xxxx.yyyyy.com/myimage
pullPolicy: IfNotPresent
# Enviroment variables that will be passed to the container.
env: {}
Now, I'll pass variables to the env dict using --set:
helm upgrade mydeployment chart --set env.VARIABLE=test
However, this must be transformed into a list to adhere to Kubernetes yaml:
spec:
template:
spec:
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
# This should come from that dict
env:
- name: VARIABLE
value: "test"
I don't know how to use the template language from Helm (sprig / go) to achieve that. Is it even possible?
To iterate through the map, the core Go text/template language provides a range keyword that can iterate through maps or arrays.
{{ range $key, $value := .Values.env }}
...
{{ end }}
Inside of this you can put arbitrary text. Helm doesn't require this to be any particular kind of YAML construct, just so long as the final result is valid YAML. For this setup a typical loop would look like
env:
{{- range $key, $value := .Values.env }}
- name: {{ quote $key }}
value: {{ quote $value }}
{{- end }}
You do need to be careful with indentation here. As a rule of thumb it often will work to include a - "swallow whitespace" indicator inside the open {{ and to not include one inside the close }}. The - name: must be at least as indented as the env: above it (ignoring the range line), and value: must be aligned with name:. I might put all of the template-language lines (the range and end) starting at the first column, even if they're embedded in a structure that's nested more.
spec:
template:
spec:
containers:
- name: {{ template "chart.fullname" . }}
env:
{{- range $key, $value := .Values.env }}
- name: {{ quote $key }}
value: {{ quote $value }}
{{- end }}
image: {{ .Values.registry }}/{{ .Values.image }}:{{ .Values.tag }}

Why helm template function is not resolveing $labels var?

I am defining a PrometheusRule as follow:
prometheusRule:
rules:
- alert: SSLCertExpiringSoon
expr: probe_ssl_earliest_cert_expiry - time() < 86400 * 10
for: 0m
labels:
severity: warning
annotations:
summary: Blackbox SSL certificate will expire soon (instance {{ $labels.instance }})
description: "SSL certificate expires in 30 days\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
And the template yml from helm chart:
{{- if .Values.prometheusRule.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: {{ template "prometheus-blackbox-exporter.fullname" . }}
{{- with .Values.prometheusRule.namespace }}
namespace: {{ . }}
{{- end }}
labels:
{{- include "prometheus-blackbox-exporter.labels" . | nindent 4 }}
{{- with .Values.prometheusRule.additionalLabels -}}
{{- toYaml . | nindent 4 -}}
{{- end }}
spec:
{{- with .Values.prometheusRule.rules }}
groups:
- name: {{ template "prometheus-blackbox-exporter.name" $ }}
rules: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- end }}
when I run helm template, tpl func it is not resolving the $labels and $values vars. When I remove annotations then the helm template is not complaining anymore. Where do fail?
error:
Error: template: prometheus-blackbox-exporter/templates/prometheusrule.yaml:18:16: executing "prometheus-blackbox-exporter/templates/prometheusrule.yaml" at <tpl (toYaml .) $>: error calling tpl: error during tpl function execution for "- alert: SSLCertExpiringSoon\n annotations:\n summary: Blackbox SSL certificate will expire soon (instance {{ $labels.instance\n }})\n expr: probe_ssl_earliest_cert_expiry - time() < 86400 * 10\n for: 0m\n labels:\n release: prometheus\n severity: warning\n- alert: SSLCertExpiringSoon\n annotations: null\n expr: probe_ssl_earliest_cert_expiry - time() < 86400 * 3\n for: 0m\n labels:\n severity: critical": parse error at (prometheus-blackbox-exporter/templates/prometheusrule.yaml:3): undefined variable "$labels"
Prometheus's alerting rules also use {{ ... $variable ... }} syntax, similar to Helm but with a different variant on the Go text/template syntax. When you pass this file through tpl, Helm tries to evaluate the embedded {{ ... }} template and evaluate any blocks there. Since $labels and $value aren't local variables defined at the Helm level, you get this error.
If you just want Prometheus to see this file as-is, and you don't need to replace anything at the Helm level (the file doesn't include references to .Values) then you don't need tpl
rules: {{ toYaml . | nindent 8 }}
If you do need tpl, then inside the included file you need to cause {{ to be emitted as a string and not processed as a template. One syntactic approach to it is to create a template block that prints out {{:
description: "VALUE = {{ "{{" }} $value }}"
# ^^^^^^^^^^ a {{ ... }} block that prints "{{"
The working version of syntax is as follow:
{{ `{{` }} $value }}

How to conditionally render helm templates based on existence of nested values

I have a template that renders a secret containing the credentials to log in to my server. There is a deployment configuration in which this is done insecurely, in which case I don't want to render the secret. Normally my values.yaml contains auth.myapp.username, and auth.myapp.password, however when the deployment lacks credentials, the whole auth section no longer exists.
My secret template looks like this:
{{ if .Values.auth.myapp.username }}
apiVersion: v1
kind: Secret
metadata:
name: myapp-credentials-secret
type: Opaque
data:
USERNAME: {{ .Values.auth.myapp.username | b64enc }}
PASSWORD: {{ .Values.auth.myapp.password | b64enc }}
{{ end }}
However I get an error when running helm install on this because the 'auth' value doesn't exist, it fails to lookup myapp from it.
I know I can get around this by not using the --strict flag, but I don't have control over that, is there a proper way of predicating on nested values?
Use haskey function:
{{ if haskey .Values "auth" }}
apiVersion: v1
kind: Secret
metadata:
name: myapp-credentials-secret
type: Opaque
data:
USERNAME: {{ .Values.auth.myapp.username | b64enc }}
PASSWORD: {{ .Values.auth.myapp.password | b64enc }}
{{ end }}
The hasKey function returns true if the given dict contains the given key.
hasKey $myDict "name1"
If the key is not found, this returns false.
You can use the standard default function to fill in layers of the hierarchy that don't exist, combined with the Sprig dict function to create an empty dictionary:
{{- $auth := .Values.auth | default dict -}}
{{/* $auth is the "auth" object from the top-level values, or an empty
dictionary if there was no such value */}}
That will let you step through this structure one level at a time:
{{- $auth := .Values.auth | default dict -}}
{{- $myapp := $auth.myapp | default dict -}}
{{- if and $myapp.username $myapp.password -}}
apiVersion: v1
kind: Secret
...
data:
USERNAME: {{ $myapp.username | b64enc }}
PASSWORD: {{ $myapp.password | b64enc }}
{{ end }}

How to pass dynamic arguments to a helm chart that runs a job

I'd like to allow our developers to pass dynamic arguments to a helm template (Kubernetes job). Currently my arguments in the helm template are somewhat static (apart from certain values) and look like this
Args:
--arg1
value1
--arg2
value2
--sql-cmd
select * from db
If I were run a task using the docker container without Kubernetes, I would pass parameters like so:
docker run my-image --arg1 value1 --arg2 value2 --sql-cmd "select * from db"
Is there any way to templatize arguments in a helm chart in such way that any number of arguments could be passed to a template.
For example.
cat values.yaml
...
arguments: --arg1 value1 --arg2 value2 --sql-cmd "select * from db"
...
or
cat values.yaml
...
arguments: --arg3 value3
...
I've tried a few approaches but was not successful. Here is one example:
Args:
{{ range .Values.arguments }}
{{ . }}
{{ end }}
Yes. In values.yaml you need to give it an array instead of a space delimited string.
cat values.yaml
...
arguments: ['--arg3', 'value3', '--arg2', 'value2']
...
or
cat values.yaml
...
arguments:
- --arg3
- value3
- --arg2
- value2
...
and then you like you mentioned in the template should do it:
args:
{{ range .Values.arguments }}
- {{ . }}
{{ end }}
If you want to override the arguments on the command line you can pass an array with --set like this:
--set arguments={--arg1, value1, --arg2, value2, --arg3, value3, ....}
In your values file define arguments as:
extraArgs:
argument1: value1
argument2: value2
booleanArg1:
In your template do:
args:
{{- range $key, $value := .Values.extraArgs }}
{{- if $value }}
- --{{ $key }}={{ $value }}
{{- else }}
- --{{ $key }}
{{- end }}
{{- end }}
Rico's answer needed to be improved.
Using the previous example I've received errors like:
templates/deployment.yaml: error converting YAML to JSON: yaml or
failed to get versionedObject: unable to convert unstructured object to apps/v1beta2, Kind=Deployment: cannot restore slice from string
This is my working setup with coma in elements:
( the vertical format for the list is more readable )
cat values.yaml
...
arguments: [
"--arg3,",
"value3,",
"--arg2,",
"value2,",
]
...
in the template should do it:
args: [
{{ range .Values.arguments }}
{{ . }}
{{ end }}
]
because of some limitations, I had to work with split and to use a delimiter, so in my case:
deployment.yaml :
{{- if .Values.deployment.args }}
args:
{{- range (split " " .Values.deployment.args) }}
- {{ . }}
{{- end }}
{{- end }}
when use --set:
helm install --set deployment.args="--inspect server.js" ...
results with:
- args:
- --inspect
- server.js
The arguments format needs to be kept consistent in such cases.
Here is my case and it works fine.
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.app.name }}
labels:
app: {{ .Values.app.name }}
instance: test
spec:
replicas: {{ .Values.master.replicaCount }}
selector:
matchLabels:
app: {{ .Values.app.name }}
instance: test
template:
metadata:
labels:
app: {{ .Values.app.name }}
instance: test
spec:
imagePullSecrets:
- name: gcr-pull-secret
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.app.image }}
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
args:
[
"--users={{int .Values.cmd.users}}",
"--spawn-rate={{int .Values.cmd.rate}}",
"--host={{.Values.cmd.host}}",
"--logfile={{.Values.cmd.logfile}}",
"--{{.Values.cmd.role}}"]
ports:
- containerPort: {{ .Values.container.port }}
resources:
requests:
memory: {{ .Values.container.requests.memory }}
cpu: {{ .Values.container.requests.cpu }}
limits:
memory: {{ .Values.container.limits.memory }}
cpu: {{ .Values.container.limits.cpu }}
Unfortunately following mixed args format does not work within container construct -
mycommand -ArgA valA --ArgB valB --ArgBool1 -ArgBool2 --ArgC=valC
The correct format of above command expected is -
mycommand --ArgA=valA --ArgB=valB --ArgC=valC --ArgBool1 --ArgBool2
This can be achieved by following constructs -
#Dockerfile last line
ENTRYPOINT [mycommand]
#deployment.yaml
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.app.image }}
args: [
"--ArgA={{ .Values.cmd.ArgA }}",
"--ArgB={{ .Values.cmd.ArgB }}",
"--ArgC={{ .Values.cmd.ArgC }}",
"--{{ .Values.cmd.ArgBool1 }}",
"--{{ .Values.cmd.ArgBool2 }}" ]
#values.yaml
cmd:
ArgA: valA
ArgB: valB
ArgC: valC
ArgBool1: "ArgBool1"
ArgBool2: "ArgBool2"
helm install --name "airflow" stable/airflow --set secrets.database=mydatabase,secrets.password=mypassword
So this is the helm chart in question: https://github.com/helm/charts/tree/master/stable/airflow
Now I want to overwrite the default values in the helm chart
secrets.database and
secrets.password so I use --set argument and then it is key=value pairs separated by commas.
helm install --name "<name for your chart>" <chart> --set key0=value0,key1=value1,key2=value2,key3=value3
Did you try this?
{{ range .Values.arguments }}
{{ . | quote }}
{{ end }}
Acid R's key/value solution was the only thing that worked for me.
I ended up with this:
values.yaml
arguments:
url1: 'http://something1.example.com'
url2: 'http://something2.example.com'
url3: 'http://something3.example.com'
url4: 'http://something3.example.com'
And in my template:
args:
{{- range $key, $value := .Values.arguments }}
- --url={{ $value }}
{{- end }}