Multiple Env Variables in Helm Charts - kubernetes

I have created common helm charts. In values.yml file, I have set of env variables that need to be set as part of deployment.yaml file.
Snippet of values file.
env:
name: ABC
value: 123
name: XYZ
value: 567
name: PQRS
value: 345
In deployment.yaml, when the values are referred, only the last name/value are set, other values are overwritten. How to read/set all the names/values in the deployment file?

I've gone through a few iterations of how to handle setting sensitive environment variables. Something like the following is the simplest solution I've come up with so far:
template:
{{- if or $.Values.env $.Values.envSecrets }}
env:
{{- range $key, $value := $.Values.env }}
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
{{- range $key, $secret := $.Values.envSecrets }}
- name: {{ $key }}
valueFrom:
secretKeyRef:
name: {{ $secret }}
key: {{ $key | quote }}
{{- end }}
{{- end }}
values:
env:
ENV_VAR: value
envSecrets:
SECRET_VAR: k8s-secret-name
Pros:
syntax is pretty straightforward
keys are easily mergeable. This came in useful when creating CronJobs with shared secrets. I was able to easily override "global" values using the following:
{{- range $key, $secret := merge (default dict .envSecrets) $.Values.globalEnvSecrets }}
Cons:
This only works for secret keys that exactly match the name of the environment variable, but it seems like that is the typical use case.

This is how I solved it in a common helm-chart I developed previously:
env:
{{- if .Values.env }}
{{- toYaml .Values.env | indent 12 }}
{{- end }}
In the values.yaml:
env:
- name: ENV_VAR
value: value
# or
- name: ENV_VAR
valueFrom:
secretKeyRef:
name: secret_name
key: secret_key
An important thing to note here is the indention. Incorrect indentation might lead to a valid helm-chart (yaml file), but the kubernetes API will give an error.

It looks like you've made a typo and forgot your dashes. Without the dashes yaml will evaluate env into a single object instead of a list and overwrite values in unexpected ways.
Your env should like more like this:
env:
- name: ABC
value: 123
- name: XYZ
value: 567
- name: PQRS
value: 345
- name: SECRET
valueFrom:
secretKeyRef:
name: name
key: key
https://www.convertjson.com/yaml-to-json.htm can help visualize how the yaml is being interpreted and investigate syntax issues.

You could let the chart user decide if he wants to take environment variables from a secret, provide the value, or take it from the downward API in the values.yaml
env:
FOO:
value: foo
BAR:
valueFrom:
secretKeyRef:
name: bar
key: barKey
POD_NAME:
valueFrom:
fieldRef:
fieldPath: metadata.name
and render it in the deployment.yaml
spec:
# ...
template:
# ...
spec:
# ...
containers:
- name: {{ .Chart.Name }}
env:
{{- range $name, $item := .Values.env }}
- name: {{ $name }}
{{- $item | toYaml | nindent 14 }}
{{- end }}
# ...
This is relatively simple and flexible.
It has the shortcoming of not keeping the order of the environment variables. This can break dependent environment variables.
I have written a bit longer story on how to support corrrect ordering as well: An Advanced API for Environment Variables in Helm Charts.

Related

Helm - Check if value not exists OR part of list

Assuming I have this values.yaml under my helm chart -
tasks:
- name: test-production-dev
env:
- production
- dev
- name: test-dev
env:
- dev
- name: test-all
environment_variables:
STAGE: dev
I would like to run my cronjob based on these values -
if .env doesn't exist - run any time.
if .env exists - run only if environment_variables.STAGE is in the .env list.
This is what I've done so far ( with no luck ) -
{{- range $.Values.tasks}}
# check if $value.env not exists OR contains stage
{{if or .env (hasKey .env "$.Values.environment_variables.STAGE") }}
apiVersion: batch/v1
kind: CronJob
...
{{- end}}
---
{{- end}}
values.yaml
tasks:
- name: test-production-dev
env:
- production
- dev
- name: test-dev
env:
- dev
- name: test-all
- name: test-production
env:
- production
environment_variables:
STAGE: dev
template/xxx.yaml
plan a
...
{{- range $.Values.tasks }}
{{- $flag := false }}
{{- if .env }}
{{- range .env }}
{{- if eq . $.Values.environment_variables.STAGE }}
{{- $flag = true }}
{{- end }}
{{- end }}
{{- else }}
{{- $flag = true }}
{{- end }}
{{- if $flag }}
apiVersion: batch/v1
kind: CronJob
meta:
name: {{ .name }}
{{- end }}
{{- end }}
...
plan b
...
{{- range $.Values.tasks }}
{{- if or (not .env) (has $.Values.environment_variables.STAGE .env) }}
apiVersion: batch/v1
kind: CronJob
meta:
name: {{ .name }}
{{- end }}
{{- end }}
...
output
...
apiVersion: batch/v1
kind: CronJob
meta:
name: test-production-dev
apiVersion: batch/v1
kind: CronJob
meta:
name: test-dev
apiVersion: batch/v1
kind: CronJob
meta:
name: test-all
...

helm chart getting secrets and configmap values using envFrom

I m trying to inject env vars in my helm chart deployment file. my values file looks like this.
values.yaml
envFrom:
- configMapRef:
name: my-config
- secretRef:
name: my-secret
I want to iterate through secrets and configmaps values . This is what I did in deployment.yaml file
envFrom:
{{- range $item := .Values.envFrom }}
{{- $item | toYaml | nindent 14 }}
{{- end }}
But i didn t get the desired result
You can directly use the defined value like:
...
envFrom:
{{- toYaml .Values.envFrom | nindent 6 }}
...
Or Instead of use range, you can use with.
Here is an example:
values.yaml:
envFrom:
- configMapRef:
name: my-config
- secretRef:
name: my-secret
pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: dapi-test-pod
namespace: test
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "env" ]
# {{- with .Values.envFrom }} can be here if you dont
# want to define envFrom in this container if envFrom
# is not defined in values.yaml.
# If you want to do that, remove the one below.
envFrom:
{{- with .Values.envFrom }}
{{- toYaml . | nindent 8 }}
{{- end }}
restartPolicy: Never
The output is:
c[_] > helm template test .
---
# Source: test/templates/test.yaml
apiVersion: v1
kind: Pod
metadata:
name: dapi-test-pod
namespace: test
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "env" ]
envFrom:
- configMapRef:
name: my-config
- secretRef:
name: my-secret
restartPolicy: Never

helm chart always not going into the if condition

yaml file and in that below values are defined including one specific value called "environment"
image:
repository: my_repo_url
tag: my_tag
pullPolicy: IfNotPresent
releaseName: cron_script
schedule: "0 10 * * *"
namespace: deploy_cron
rav_admin_password: asdf
environment: testing
testing_forwarder_ip: 10.2.71.21
prod_us_forwarder_ip: 10.2.71.15
Now in my helm chart based on this environment value i need to assign a value to new variable and for that I have written code like below, but always it is not entering into the if else block itself
{{- $fwip := .Values.prod_us_forwarder_ip }}
{{- if contains .Values.environment "testing" }}
{{- $fwip := .Values.testing_forwarder_ip }}
{{- end }}
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: "{{ .Values.releaseName }}"
namespace: "{{ .Values.namespace }}"
labels:
....................................
....................................
....................................
spec:
restartPolicy: Never
containers:
- name: "{{ .Values.releaseName }}"
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: IfNotPresent
args:
- python3
- test.py
- --data
- 100
- {{ $fwip }}
In the above code always i get $fwip value as 10.2.71.21 what ever environment value is either testing or production for both i am getting same value
And if i don't declare the variable $fwip before the if else statement then it says $fwip variable is not defined error, So i am not sure why exactly if else statement is not getting used at all, How to debug further ?
This is a syntax problem of variables and local variables.
The fwip in if should use = instead of :=
{{- $fwip := .Values.prod_us_forwarder_ip }}
{{- if contains .Values.environment "testing" }}
{{- $fwip = .Values.testing_forwarder_ip }}
{{- end }}
I translated it into go code to make it easier for you to understand.
(In the go language, := means definition and assignment, = means assignment)
// :=
env := "testing"
test := "10.2.71.21"
prod := "10.2.71.15"
fwip := prod
if strings.Contains(env,"testing"){
fwip := test
fmt.Println(fwip) // 10.2.71.21
}
fmt.Println(fwip) // 10.2.71.15
// =
env := "testing"
test := "10.2.71.21"
prod := "10.2.71.15"
fwip := prod
if strings.Contains(env,"testing"){
fwip = test
fmt.Println(fwip) // 10.2.71.21
}
fmt.Println(fwip) // 10.2.71.21

Helm template - how to use "if exists at least one of" in array?

I'm trying to make a list of env vars from a values.yaml become a single secret.yaml file containing the list of envs of the type "secret". The idea is to only create this secret file if at least one of the types equals "secret".
Eg:
values.yaml
env:
- name: PLAIN_TEXT_ENV_VAR1
type: plain
value: text value
- name: PLAIN_TEXT_ENV_VAR2
type: plain
value: text value
- name: TOP_SECRET_ENV_VAR_1
type: secret
- name: TOP_SECRET_ENV_VAR_2
type: secret
- name: TOP_SECRET_ENV_VAR_3
type: secret
resulting secret.yaml
kind: Secret
metadata:
name: test
data:
TOP_SECRET_ENV_VAR_1: change_me
TOP_SECRET_ENV_VAR_2: change_me
TOP_SECRET_ENV_VAR_3: change_me
I've already tried to create some flow control using range to iterate, boolean variables and if statements, but go template seems to ignore my ifs after I change it to another value.
Now my secret template is like the one below.
{{ $flowcontrol := true -}}
{{ if $flowcontrol -}}
{{ range $env := $.Values.env -}}
{{ if eq $env.type "secret" -}}
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
{{- range $env := $.Values.env }}
{{- if eq $env.type "secret" }}
{{ $env.name }}: "change_me"
{{- end }}
{{- end }}
{{ $flowcontrol := false }}
{{ end -}}
{{ end -}}
{{ end -}}
It results in three replicated secret.yaml files with 3 variables:
$ helm template .
---
# Source: teste/templates/secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
TOP_SECRET_ENV_VAR_1: "change_me"
TOP_SECRET_ENV_VAR_2: "change_me"
TOP_SECRET_ENV_VAR_3: "change_me"
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
TOP_SECRET_ENV_VAR_1: "change_me"
TOP_SECRET_ENV_VAR_2: "change_me"
TOP_SECRET_ENV_VAR_3: "change_me"
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
TOP_SECRET_ENV_VAR_1: "change_me"
TOP_SECRET_ENV_VAR_2: "change_me"
TOP_SECRET_ENV_VAR_3: "change_me"
How can one control the flow as: if the first one item of a list satisfies the condition, iterate through the rest of the same list only in another section after?
I've managed to create this feature! :D
Basicaly i've created a "template function" using define which iterates the env list from values.yaml and writes a string containing only the envs which the type property matches the word "secret".
Then I call that function using include and assign it's output to a variable.
If the variable length is greater than 0 (meaning it's not an empty string), the secret file is created and then I use the same string to fill the data property.
Here's the code containing the function and the secret template:
{{- define "get-secrets-from-env-list" -}}
{{- $allenv := index . 0 -}}
{{- range $i, $scrts := $allenv -}}
{{- if eq $scrts.type "secret" -}}
{{- nindent 0 $scrts.name -}}: {{ "change_me" | b64enc -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{ $secrets := include "get-secrets-from-env-list" (list .Values.env ) }}
{{- if gt (len $secrets) 0 -}}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: {{ include "awesome-chart.fullname" $ }}
labels:
app.kubernetes.io/name: {{ include "awesome-chart.name" $ }}
helm.sh/chart: {{ include "awesome-chart.chart" $ }}
app.kubernetes.io/instance: {{ $.Release.Name }}
app.kubernetes.io/managed-by: {{ $.Release.Service }}
data:
{{- nindent 2 $secrets -}}
{{- end -}}%
Here's a sample output:
$ helm template .
---
# Source: awesome-chart/templates/secret.yaml
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: my-app
labels:
app.kubernetes.io/name: awesome-chart
helm.sh/chart: awesome-chart
app.kubernetes.io/instance: release-name
app.kubernetes.io/managed-by: Tiller
data:
TOP_SECRET_ENV_VAR_1: Y2hhbmdlX21l
TOP_SECRET_ENV_VAR_2: Y2hhbmdlX21l
TOP_SECRET_ENV_VAR_3: Y2hhbmdlX21l
Works like a charm! ;)

How not to overwrite randomly generated secrets in Helm templates

I want to generate a password in a Helm template, this is easy to do using the randAlphaNum function. However the password will be changed when the release is upgraded. Is there a way to check if a password was previously generated and then use the existing value? Something like this:
apiVersion: v1
kind: Secret
metadata:
name: db-details
data:
{{ if .Secrets.db-details.db-password }}
db-password: {{ .Secrets.db-details.db-password | b64enc }}
{{ else }}
db-password: {{ randAlphaNum 20 | b64enc }}
{{ end }}
You can build on shaunc's idea to use the lookup function to fix the original poster's code like this:
apiVersion: v1
kind: Secret
metadata:
name: db-details
data:
{{- if .Release.IsInstall }}
db-password: {{ randAlphaNum 20 | b64enc }}
{{ else }}
# `index` function is necessary because the property name contains a dash.
# Otherwise (...).data.db_password would have worked too.
db-password: {{ index (lookup "v1" "Secret" .Release.Namespace "db-details").data "db-password" }}
{{ end }}
Only creating the Secret when it doesn't yet exist won't work because Helm will delete objects that are no longer defined during the upgrade.
Using an annotation to keep the object around has the disadvantage that it will not be deleted when you delete the release with helm delete ....
I've got a lot of trouble with the answers from Jan Dubois and shaunc. So I built a combined solution.
The downside of Jan's answer: It leads to errors, when it is used with --dry-run.
The downside of shaunc's answer: It won't work, because the resources will be deleted on helm upgrade.
Here is my code:
# store the secret-name as var
# in my case, the name was very long and containing a lot of fields
# so it helps me a lot
{{- $secret_name := "your-secret-name" -}}
apiVersion: v1
kind: Secret
metadata:
name: {{ $secret_name }}
data:
# try to get the old secret
# keep in mind, that a dry-run only returns an empty map
{{- $old_sec := lookup "v1" "Secret" .Release.Namespace $secret_name }}
# check, if a secret is already set
{{- if or (not $old_sec) (not $old_sec.data) }}
# if not set, then generate a new password
db-password: {{ randAlphaNum 20 | b64enc }}
{{ else }}
# if set, then use the old value
db-password: {{ index $old_sec.data "db-password" }}
{{ end }}
It's still one of the biggest issues of Helm. As far as I understand no good solution is available yet (see https://github.com/helm/charts/issues/5167).
One dirty workaround is to create secret as pre-install hook. Obvious downside of this approach is that secret will not be deleted on helm delete.
apiVersion: v1
kind: Secret
metadata:
name: {{ template "helm-random-secret.fullname" . }}
annotations:
"helm.sh/hook": "pre-install"
"helm.sh/hook-delete-policy": "before-hook-creation"
labels:
app: {{ template "helm-random-secret.name" . }}
chart: {{ template "helm-random-secret.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
data:
some-password: {{ default (randAlphaNum 10) .Values.somePassword | b64enc | quote }}
Building on shaunc's idea to use the lookup function, I've created the following template:
{{/*
Returns a secret if it already in Kubernetes, otherwise it creates
it randomly.
*/}}
{{- define "getOrGeneratePass" }}
{{- $len := (default 16 .Length) | int -}}
{{- $obj := (lookup "v1" .Kind .Namespace .Name).data -}}
{{- if $obj }}
{{- index $obj .Key -}}
{{- else if (eq (lower .Kind) "secret") -}}
{{- randAlphaNum $len | b64enc -}}
{{- else -}}
{{- randAlphaNum $len -}}
{{- end -}}
{{- end }}
Then you can simply configure secrets like:
---
apiVersion: v1
kind: Secret
metadata:
name: my-secret
type: Opaque
data:
PASSWORD: "{{ include "getOrGeneratePass" (dict "Namespace" .Release.Namespace "Kind" "Secret" "Name" "my-secret" "Key" "PASSWORD") }}"
You can use the lookup function and skip generation if secret already exists:
{{- if not (lookup "v1" "secret" .Release.Namespace "db-details") -}}
<create secret here>
{{- end -}}
The actual tools are all here. My workaround is just another combination of suggested tools
{{- if not (lookup "v1" "Secret" .Release.Namespace "mysecret") }}
apiVersion: v1
kind: Secret
metadata:
name: mysecret
annotations:
"helm.sh/resource-policy": "keep"
type: Opaque
stringData:
password: {{ randAlphaNum 24 }}
{{- end }}
So if there is no such secret, it will be created. If the secret is present, it will be removed from the chart, but not from the cluster, the "helm.sh/resource-policy": "keep" will prevent it.
You may ask (as someone already did above) why lookup, not .Release.IsUpdate. Imagine the situation: your secret is a password to a database. You keep the data in the persistent volume, the claim for which is also annotated by "helm.sh/resource-policy": "keep", so if you even uninstall and reinstall the chart, the data would persist. If you do so with .Release.IsUpdate as condition, then you password will be recreated, the old password will be lost and you will loose the access to your data. If you query for the secret existence, it won't happen.
I've rewritten kubernetes replicator and added some annotations to deal with this kind of problems: https://github.com/olli-ai/k8s-replicator#use-random-password-generated-by-an-helm-chart
Now can generate a random password with helm, and replicate it only once to another secret thus it won't be change by helm in the future.
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: admin-password-source
annotations:
k8s-replicator/replicate-to: "admin-password"
k8s-replicator/replicate-once: "true"
stringData:
password: {{ randAlphaNum 64 | quote }}
Hope it will help people.
You can leverage definitions in the _helpers.tpl
_helpers.tpl
{{/*
Create the secret name
*/}}
{{- define "mssql-server.secretName" -}}
{{- include "mssql-server.name" . }}-mssql-secret
{{- end }}
{{/*
Get sa password value
*/}}
{{- define "mssql-server.sapassword" -}}
{{- if .Release.IsInstall -}}
{{ .Values.sa_password | default (randAlphaNum 20) | b64enc | quote }}
{{- else -}}
{{ index (lookup "v1" "Secret" .Release.Namespace (include "mssql-server.secretName" .)).data "sa_password" }}
{{- end }}
{{- end }}
secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: {{ include "mssql-server.secretName" . }}
labels:
{{- include "mssql-server.labels" . | nindent 4 }}
type: Opaque
data:
sa_password: {{ include "mssql-server.sapassword" . }}
A bit late here, and most people may just catch it in the documentation:
helm does this for you with the annotation "helm.sh/resource-policy": keep
see:
https://helm.sh/docs/howto/charts_tips_and_tricks/#tell-helm-not-to-uninstall-a-resource