Helm - Configmap - Replace only the contents of file and not File Names - kubernetes

I have defined application.properties files in a config dir. Config dir is on the below structure.
config
application.properties
application-test.properties
application-dev.properties
application-prod.properties
I have created a helm chart to create the configmap. Helm chart is defined as below
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configmap
data:
{{- (.Files.Glob "config/*").AsConfig | nindent 2 }}
I see that the configmap is created.
We are consuming the ConfigMap via environment variables in a running container using the envFrom property. (This in my deployment yaml file)
spec:
containers:
- envFrom:
- configMapRef:
name: nginx-configmap
I see that the values are stored as environment variables. However the variables are in lower cases.
server.port=8080
server.name=localhost
Since these are env variables, they have to be uppercase and . should be replaced with _. So, I have modifed my chart as below
data:
{{- (.Files.Glob "config/*").AsConfig | nindent 2 | upper | replace "." "_" }}
The generated configmap is as below
APPLICATION_PROPERTIES: |
SERVER_PORT = 8080
SERVER_NAME = LOCALHOST
Below is the env variables inside container
APPLICATION_PROPERTIES=SERVER_PORT = 8080
SERVER_NAME = LOCALHOST
My requirement is that only the contents of the file should be upper case and . should be replaced with _. The filename should not be converted. The filename should be as is.
Can this be achieved?

Try this:
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configmap
data:
{{ $root := . }}
{{ range $path, $bytes := .Files.Glob "config/*" }}
{{ base $path }}: '{{ $root.Files.Get $path | nindent 2 | upper | replace "." "_" }}'
{{ end }}

Related

helm chart getting secrets and configmap values using envFrom

I m trying to inject env vars in my helm chart deployment file. my values file looks like this.
values.yaml
envFrom:
- configMapRef:
name: my-config
- secretRef:
name: my-secret
I want to iterate through secrets and configmaps values . This is what I did in deployment.yaml file
envFrom:
{{- range $item := .Values.envFrom }}
{{- $item | toYaml | nindent 14 }}
{{- end }}
But i didn t get the desired result
You can directly use the defined value like:
...
envFrom:
{{- toYaml .Values.envFrom | nindent 6 }}
...
Or Instead of use range, you can use with.
Here is an example:
values.yaml:
envFrom:
- configMapRef:
name: my-config
- secretRef:
name: my-secret
pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: dapi-test-pod
namespace: test
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "env" ]
# {{- with .Values.envFrom }} can be here if you dont
# want to define envFrom in this container if envFrom
# is not defined in values.yaml.
# If you want to do that, remove the one below.
envFrom:
{{- with .Values.envFrom }}
{{- toYaml . | nindent 8 }}
{{- end }}
restartPolicy: Never
The output is:
c[_] > helm template test .
---
# Source: test/templates/test.yaml
apiVersion: v1
kind: Pod
metadata:
name: dapi-test-pod
namespace: test
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "env" ]
envFrom:
- configMapRef:
name: my-config
- secretRef:
name: my-secret
restartPolicy: Never

helm chart: Include multiple lines from values.yaml into configmap

I want to create a helm chart that results in a config map that looks like this:
apiVersion: v1
kind: ConfigMap
metadata:
name: myconfigmap
data:
myconfigfile1.properties: |
property11 = value11
property12 = value12
myconfigfile1.properties: |
property21 = value21
property22 = value22
whereas this part shall be configurable in the values.yaml:
myconfig:
myconfigfile1.properties: |
property11 = value11
property12 = value12
myconfigfile1.properties: |
property21 = value21
property22 = value22
Now I want to iterate over all the children of myconfig in the values.yaml and add them to my helm template. My attempts so far with this template:
apiVersion: v1
kind: ConfigMap
metadata:
name: myconfigmap
data:
# {{- range $key, $val := .Values.myconfig}}
# {{ $key }}: |
# {{ $val }}
# {{- end }}
resulted in this error message:
$ helm install --dry-run --debug ./mychart/ --generate-name
install.go:159: [debug] Original chart version: ""
install.go:176: [debug] CHART PATH: /home/my/helmcharts/mychart
Error: YAML parse error on mychart/templates/myconfig.yaml: error converting YAML to JSON: yaml: line 11: could not find expected ':'
helm.go:84: [debug] error converting YAML to JSON: yaml: line 11: could not find expected ':'
YAML parse error on mychart/templates/myconfig.yaml
I can avoid the error by removing the | after myconfigfile1.properties: in my values.yaml, however then I lose the line breaks and the result is not what I want.
Many thanks for your help in advance.
Kind regards,
Martin
A few minutes after writing this question I stubled upon Question #62432632 convert-a-yaml-to-string-in-helm which does not exactly answer my question but with its help I could find the correct syntax.
values.yaml:
myconfig:
myconfigfile1.properties: |-
property11 = value11
property12 = value12
myconfigfile2.properties: |-
property21 = value21
property22 = value22
template:
apiVersion: v1
kind: ConfigMap
metadata:
name: myconfigmap
data:
{{- range $name, $config := .Values.myconfig }}
{{ $name }}: |-
{{ tpl $config $ | indent 4 }}
{{- end }}

Helm template doesnt work when binded to configmap

I have a file which will be binded to a configmap. Once I put a tpl function in it, it stops working when other lines are included in the file.
I use this helper tpl:
{{- define "call-nested" }}
{{- $dot := index . 0 }}
{{- $subchart := index . 1 | splitList "." }}
{{- $template := index . 2 }}
{{- $values := $dot.Values }}
{{- range $subchart }}
{{- $values = index $values . }}
{{- end }}
{{- include $template (dict "Chart" (dict "Name" (last $subchart)) "Values" $values "Release" $dot.Release "Capabilities" $dot.Capabilities) }}
{{- end }}
testing in some.yaml:
psqlhost: {{include "call-nested" (list . "postgresql" "postgresql.fullname")}}
newlinekey: value
It works well if some.yaml is a standalone file. But once I bind it a configmap, it gives this error:
executing "mytestchart/templates/my-configmap.yaml" at <tpl (.Files.Glob "config/*").AsConfig .>: error calling tpl: Error during tpl function execution for "some.yaml:<br>\"name: {{include \\\"call-nested\\\" (list . \\\"postgresql\\\" \\\"postgresql.fullname\\\")}}\\r\\nnewlinekey:\n value\"\n": parse error in "mytestchart/templates/my-configmap.yaml": template: mytestchart/templates/my-configmap.yam:1: unexpected "\\" in operand
Once I remove the new line it will also work well.
edit: Configmap:
apiVersion: v1
kind: ConfigMap
metadata:
name: somename
data:
{{ tpl (.Files.Glob "config/*").AsConfig .| indent 2 }}
Your template is not rendering correctly. In order to make it work you need to change the following:
Add a .tmpl suffix to the file you'd like to put to your ConfigMap, for example: some.yaml.tmpl.
Set your ConfigMap's data: to: {{- tpl ((.Files.Glob "config/*.tmpl").AsConfig) . | indent 2 }}. Add a indentation to that line also.
Your ConfigMap would than look something like this:
apiVersion: v1
kind: ConfigMap
metadata:
name: somename
data:
{{- tpl ((.Files.Glob "config/*.tmpl").AsConfig) . | indent 2 }}
You can find more info with some examples here.
Please let me know if that helped.

Helm template - how to use "if exists at least one of" in array?

I'm trying to make a list of env vars from a values.yaml become a single secret.yaml file containing the list of envs of the type "secret". The idea is to only create this secret file if at least one of the types equals "secret".
Eg:
values.yaml
env:
- name: PLAIN_TEXT_ENV_VAR1
type: plain
value: text value
- name: PLAIN_TEXT_ENV_VAR2
type: plain
value: text value
- name: TOP_SECRET_ENV_VAR_1
type: secret
- name: TOP_SECRET_ENV_VAR_2
type: secret
- name: TOP_SECRET_ENV_VAR_3
type: secret
resulting secret.yaml
kind: Secret
metadata:
name: test
data:
TOP_SECRET_ENV_VAR_1: change_me
TOP_SECRET_ENV_VAR_2: change_me
TOP_SECRET_ENV_VAR_3: change_me
I've already tried to create some flow control using range to iterate, boolean variables and if statements, but go template seems to ignore my ifs after I change it to another value.
Now my secret template is like the one below.
{{ $flowcontrol := true -}}
{{ if $flowcontrol -}}
{{ range $env := $.Values.env -}}
{{ if eq $env.type "secret" -}}
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
{{- range $env := $.Values.env }}
{{- if eq $env.type "secret" }}
{{ $env.name }}: "change_me"
{{- end }}
{{- end }}
{{ $flowcontrol := false }}
{{ end -}}
{{ end -}}
{{ end -}}
It results in three replicated secret.yaml files with 3 variables:
$ helm template .
---
# Source: teste/templates/secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
TOP_SECRET_ENV_VAR_1: "change_me"
TOP_SECRET_ENV_VAR_2: "change_me"
TOP_SECRET_ENV_VAR_3: "change_me"
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
TOP_SECRET_ENV_VAR_1: "change_me"
TOP_SECRET_ENV_VAR_2: "change_me"
TOP_SECRET_ENV_VAR_3: "change_me"
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
TOP_SECRET_ENV_VAR_1: "change_me"
TOP_SECRET_ENV_VAR_2: "change_me"
TOP_SECRET_ENV_VAR_3: "change_me"
How can one control the flow as: if the first one item of a list satisfies the condition, iterate through the rest of the same list only in another section after?
I've managed to create this feature! :D
Basicaly i've created a "template function" using define which iterates the env list from values.yaml and writes a string containing only the envs which the type property matches the word "secret".
Then I call that function using include and assign it's output to a variable.
If the variable length is greater than 0 (meaning it's not an empty string), the secret file is created and then I use the same string to fill the data property.
Here's the code containing the function and the secret template:
{{- define "get-secrets-from-env-list" -}}
{{- $allenv := index . 0 -}}
{{- range $i, $scrts := $allenv -}}
{{- if eq $scrts.type "secret" -}}
{{- nindent 0 $scrts.name -}}: {{ "change_me" | b64enc -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{ $secrets := include "get-secrets-from-env-list" (list .Values.env ) }}
{{- if gt (len $secrets) 0 -}}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: {{ include "awesome-chart.fullname" $ }}
labels:
app.kubernetes.io/name: {{ include "awesome-chart.name" $ }}
helm.sh/chart: {{ include "awesome-chart.chart" $ }}
app.kubernetes.io/instance: {{ $.Release.Name }}
app.kubernetes.io/managed-by: {{ $.Release.Service }}
data:
{{- nindent 2 $secrets -}}
{{- end -}}%
Here's a sample output:
$ helm template .
---
# Source: awesome-chart/templates/secret.yaml
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: my-app
labels:
app.kubernetes.io/name: awesome-chart
helm.sh/chart: awesome-chart
app.kubernetes.io/instance: release-name
app.kubernetes.io/managed-by: Tiller
data:
TOP_SECRET_ENV_VAR_1: Y2hhbmdlX21l
TOP_SECRET_ENV_VAR_2: Y2hhbmdlX21l
TOP_SECRET_ENV_VAR_3: Y2hhbmdlX21l
Works like a charm! ;)

How not to overwrite randomly generated secrets in Helm templates

I want to generate a password in a Helm template, this is easy to do using the randAlphaNum function. However the password will be changed when the release is upgraded. Is there a way to check if a password was previously generated and then use the existing value? Something like this:
apiVersion: v1
kind: Secret
metadata:
name: db-details
data:
{{ if .Secrets.db-details.db-password }}
db-password: {{ .Secrets.db-details.db-password | b64enc }}
{{ else }}
db-password: {{ randAlphaNum 20 | b64enc }}
{{ end }}
You can build on shaunc's idea to use the lookup function to fix the original poster's code like this:
apiVersion: v1
kind: Secret
metadata:
name: db-details
data:
{{- if .Release.IsInstall }}
db-password: {{ randAlphaNum 20 | b64enc }}
{{ else }}
# `index` function is necessary because the property name contains a dash.
# Otherwise (...).data.db_password would have worked too.
db-password: {{ index (lookup "v1" "Secret" .Release.Namespace "db-details").data "db-password" }}
{{ end }}
Only creating the Secret when it doesn't yet exist won't work because Helm will delete objects that are no longer defined during the upgrade.
Using an annotation to keep the object around has the disadvantage that it will not be deleted when you delete the release with helm delete ....
I've got a lot of trouble with the answers from Jan Dubois and shaunc. So I built a combined solution.
The downside of Jan's answer: It leads to errors, when it is used with --dry-run.
The downside of shaunc's answer: It won't work, because the resources will be deleted on helm upgrade.
Here is my code:
# store the secret-name as var
# in my case, the name was very long and containing a lot of fields
# so it helps me a lot
{{- $secret_name := "your-secret-name" -}}
apiVersion: v1
kind: Secret
metadata:
name: {{ $secret_name }}
data:
# try to get the old secret
# keep in mind, that a dry-run only returns an empty map
{{- $old_sec := lookup "v1" "Secret" .Release.Namespace $secret_name }}
# check, if a secret is already set
{{- if or (not $old_sec) (not $old_sec.data) }}
# if not set, then generate a new password
db-password: {{ randAlphaNum 20 | b64enc }}
{{ else }}
# if set, then use the old value
db-password: {{ index $old_sec.data "db-password" }}
{{ end }}
It's still one of the biggest issues of Helm. As far as I understand no good solution is available yet (see https://github.com/helm/charts/issues/5167).
One dirty workaround is to create secret as pre-install hook. Obvious downside of this approach is that secret will not be deleted on helm delete.
apiVersion: v1
kind: Secret
metadata:
name: {{ template "helm-random-secret.fullname" . }}
annotations:
"helm.sh/hook": "pre-install"
"helm.sh/hook-delete-policy": "before-hook-creation"
labels:
app: {{ template "helm-random-secret.name" . }}
chart: {{ template "helm-random-secret.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
data:
some-password: {{ default (randAlphaNum 10) .Values.somePassword | b64enc | quote }}
Building on shaunc's idea to use the lookup function, I've created the following template:
{{/*
Returns a secret if it already in Kubernetes, otherwise it creates
it randomly.
*/}}
{{- define "getOrGeneratePass" }}
{{- $len := (default 16 .Length) | int -}}
{{- $obj := (lookup "v1" .Kind .Namespace .Name).data -}}
{{- if $obj }}
{{- index $obj .Key -}}
{{- else if (eq (lower .Kind) "secret") -}}
{{- randAlphaNum $len | b64enc -}}
{{- else -}}
{{- randAlphaNum $len -}}
{{- end -}}
{{- end }}
Then you can simply configure secrets like:
---
apiVersion: v1
kind: Secret
metadata:
name: my-secret
type: Opaque
data:
PASSWORD: "{{ include "getOrGeneratePass" (dict "Namespace" .Release.Namespace "Kind" "Secret" "Name" "my-secret" "Key" "PASSWORD") }}"
You can use the lookup function and skip generation if secret already exists:
{{- if not (lookup "v1" "secret" .Release.Namespace "db-details") -}}
<create secret here>
{{- end -}}
The actual tools are all here. My workaround is just another combination of suggested tools
{{- if not (lookup "v1" "Secret" .Release.Namespace "mysecret") }}
apiVersion: v1
kind: Secret
metadata:
name: mysecret
annotations:
"helm.sh/resource-policy": "keep"
type: Opaque
stringData:
password: {{ randAlphaNum 24 }}
{{- end }}
So if there is no such secret, it will be created. If the secret is present, it will be removed from the chart, but not from the cluster, the "helm.sh/resource-policy": "keep" will prevent it.
You may ask (as someone already did above) why lookup, not .Release.IsUpdate. Imagine the situation: your secret is a password to a database. You keep the data in the persistent volume, the claim for which is also annotated by "helm.sh/resource-policy": "keep", so if you even uninstall and reinstall the chart, the data would persist. If you do so with .Release.IsUpdate as condition, then you password will be recreated, the old password will be lost and you will loose the access to your data. If you query for the secret existence, it won't happen.
I've rewritten kubernetes replicator and added some annotations to deal with this kind of problems: https://github.com/olli-ai/k8s-replicator#use-random-password-generated-by-an-helm-chart
Now can generate a random password with helm, and replicate it only once to another secret thus it won't be change by helm in the future.
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: admin-password-source
annotations:
k8s-replicator/replicate-to: "admin-password"
k8s-replicator/replicate-once: "true"
stringData:
password: {{ randAlphaNum 64 | quote }}
Hope it will help people.
You can leverage definitions in the _helpers.tpl
_helpers.tpl
{{/*
Create the secret name
*/}}
{{- define "mssql-server.secretName" -}}
{{- include "mssql-server.name" . }}-mssql-secret
{{- end }}
{{/*
Get sa password value
*/}}
{{- define "mssql-server.sapassword" -}}
{{- if .Release.IsInstall -}}
{{ .Values.sa_password | default (randAlphaNum 20) | b64enc | quote }}
{{- else -}}
{{ index (lookup "v1" "Secret" .Release.Namespace (include "mssql-server.secretName" .)).data "sa_password" }}
{{- end }}
{{- end }}
secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: {{ include "mssql-server.secretName" . }}
labels:
{{- include "mssql-server.labels" . | nindent 4 }}
type: Opaque
data:
sa_password: {{ include "mssql-server.sapassword" . }}
A bit late here, and most people may just catch it in the documentation:
helm does this for you with the annotation "helm.sh/resource-policy": keep
see:
https://helm.sh/docs/howto/charts_tips_and_tricks/#tell-helm-not-to-uninstall-a-resource