helm chart: Include multiple lines from values.yaml into configmap - kubernetes

I want to create a helm chart that results in a config map that looks like this:
apiVersion: v1
kind: ConfigMap
metadata:
name: myconfigmap
data:
myconfigfile1.properties: |
property11 = value11
property12 = value12
myconfigfile1.properties: |
property21 = value21
property22 = value22
whereas this part shall be configurable in the values.yaml:
myconfig:
myconfigfile1.properties: |
property11 = value11
property12 = value12
myconfigfile1.properties: |
property21 = value21
property22 = value22
Now I want to iterate over all the children of myconfig in the values.yaml and add them to my helm template. My attempts so far with this template:
apiVersion: v1
kind: ConfigMap
metadata:
name: myconfigmap
data:
# {{- range $key, $val := .Values.myconfig}}
# {{ $key }}: |
# {{ $val }}
# {{- end }}
resulted in this error message:
$ helm install --dry-run --debug ./mychart/ --generate-name
install.go:159: [debug] Original chart version: ""
install.go:176: [debug] CHART PATH: /home/my/helmcharts/mychart
Error: YAML parse error on mychart/templates/myconfig.yaml: error converting YAML to JSON: yaml: line 11: could not find expected ':'
helm.go:84: [debug] error converting YAML to JSON: yaml: line 11: could not find expected ':'
YAML parse error on mychart/templates/myconfig.yaml
I can avoid the error by removing the | after myconfigfile1.properties: in my values.yaml, however then I lose the line breaks and the result is not what I want.
Many thanks for your help in advance.
Kind regards,
Martin

A few minutes after writing this question I stubled upon Question #62432632 convert-a-yaml-to-string-in-helm which does not exactly answer my question but with its help I could find the correct syntax.
values.yaml:
myconfig:
myconfigfile1.properties: |-
property11 = value11
property12 = value12
myconfigfile2.properties: |-
property21 = value21
property22 = value22
template:
apiVersion: v1
kind: ConfigMap
metadata:
name: myconfigmap
data:
{{- range $name, $config := .Values.myconfig }}
{{ $name }}: |-
{{ tpl $config $ | indent 4 }}
{{- end }}

Related

helm to create single configmap with different labels

can anyone help me creating configmaps in helm with different labels.
I have 6 configmaps each have their own labels which will be used by pods. I would like to have the same labels in helm as well. Is it possible in Helm?
These are my values for configmap. I want to render these values and create separate configmap.
values.yaml/
configmaptestingforhelm:
configmapref1:
confname: serviceaccount-config
key: serviceaccount.test.path
value: /root/test_key.json
configmapref2:
confname: host-credentials-config
key: secret.credentials.host.path
value: /root/host_key.json
I am trying to create separate configmap for each configmappref* as below
configmap1
apiVersion: v1
kind: ConfigMap
{{- range $value := .Values.configmap.configmapref1 }}
metadata:
name: {{ $value.confname }}
data:
{{ $value.key }} : {{ $value.value }}
{{- end}}
configmap2
apiVersion: v1
kind: ConfigMap
{{- range $value := .Values.configmap.configmapref2 }}
metadata:
name: {{ $value.confname }}
data:
{{ $value.key }} : {{ $value.value }}
{{- end}}
My expected output is as below ,but I am not getting the desired output
apiVersion: v1
data:
secret.credentials.host.path: /root/host_key.json
kind: ConfigMap
metadata:
name: host-credentials-config
apiVersion: v1
data:
serviceaccount.test.path: /root/test_key.json
kind: ConfigMap
metadata:
name: serviceaccount-config
values.yaml
testLables:
cm0:
a: b
c: d
cm1:
e: f
g: k
templates/configmap.yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "test.fullname" . }}-0
labels:
{{- include "test.labels" . | nindent 4 }}
{{- range $k, $v := .Values.testLables.cm0 }}
{{ $k }}: {{ $v }}
{{- end }}
data:
data: |-
k: v
---
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "test.fullname" . }}-1
labels:
{{- include "test.labels" . | nindent 4 }}
{{- range $k, $v := .Values.testLables.cm1 }}
{{ $k }}: {{ $v }}
{{- end }}
data:
data: |-
k: v
cmd:
helm template --debug test .
output
---
# Source: test/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test-0
labels:
helm.sh/chart: test-0.1.0
app.kubernetes.io/name: test
app.kubernetes.io/instance: test
app.kubernetes.io/version: "1.18.0"
app.kubernetes.io/managed-by: Helm
a: b
c: d
data:
data: |-
k: v
---
# Source: test/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test-1
labels:
helm.sh/chart: test-0.1.0
app.kubernetes.io/name: test
app.kubernetes.io/instance: test
app.kubernetes.io/version: "1.18.0"
app.kubernetes.io/managed-by: Helm
e: f
g: k
data:
data: |-
k: v
update:
values.yaml
configmaptestingforhelm:
configmapref1:
confname: serviceaccount-config
key: serviceaccount.test.path
value: /root/test_key.json
configmapref2:
confname: host-credentials-config
key: secret.credentials.host.path
value: /root/host_key.json
templates/configmap.yaml
{{- range $k, $cm := .Values.configmaptestingforhelm }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ $cm.confname }}
data:
{{ $cm.key }}: {{ $cm.value }}
{{- end }}
cmd
helm template --debug test .
output:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: serviceaccount-config
data:
serviceaccount.test.path: /root/test_key.json
---
apiVersion: v1
kind: ConfigMap
metadata:
name: host-credentials-config
data:
secret.credentials.host.path: /root/host_key.json

Helm named template explodes with 'unsupported value: encountered a cycle via map[string]interface' if called more than once

I am trying to refactor a Helm chart for a large enterprise application with numerous deployments, services, ingresses etc and am trying to reduce copy-and-paste. Since I've not found any generally accepted DRY design patterns for Helm charts that have enterprise applications in mind, I am simply working from this guide as starting point: https://faun.pub/dry-helm-charts-for-micro-services-db3a1d6ecb80
I would like to define a named template for each high level resource, such as a deployment and call it as needed for each component in the application that I'm deploying. Using a simple ConfigMap as an example, I have come up with the following example that works perfectly as expected if I invoke the named template only once:
{{- define "mergeproblem.configmap" -}}
{{- $ := index . 0 -}}
{{- $name := index . 2 -}}
{{- with index . 1 -}}
{{- $this := dict "Values" (get .Values $name) -}}
{{- $defaultRoot := dict "Values" (omit .Values $name) -}}
{{- $noValues := omit . "Values" -}}
{{- with merge $noValues $this $defaultRoot -}}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "mergeproblem.fullname" . }}-{{ $name }}
labels:
{{- include "mergeproblem.labels" . | nindent 4 }}
data:
MergeProblemExample: |
Values: {{ .Values | toYaml | nindent 6 }}
Template: {{ .Template | toYaml | nindent 6 }}
Chart: {{ .Chart | toYaml | nindent 6 }}
Release: {{ .Release | toYaml | nindent 6 }}
{{- end -}}
{{- end -}}
{{- end -}}
The idea here is that I can call this named template, giving it the name of the resource I want to create. It will pull the defaults from the top-level .Values but merge in the .Values.$name over the top with a higher precedence so that the template contents can be relatively simple without having to perform individual merges or ternaries on every variable interpolation -- instead the main scoped input will already have the merging done on .Values.
An example of the working output if I make only a single call: https://pastebin.com/ZCJ4LTpV
I am invoking it with the following:
{{- template "mergeproblem.configmap" (list $ . "myfoo") -}}
However if I invoke it more than once like so:
{{- template "mergeproblem.configmap" (list $ . "myfoo") -}}
{{- template "mergeproblem.configmap" (list $ . "yourbar") -}}
I receive the following error from Helm:
$ helm upgrade --install --namespace mergeproblem --debug mergeproblem ./mergeproblem/
history.go:56: [debug] getting history for release mergeproblem
upgrade.go:123: [debug] preparing upgrade for mergeproblem
upgrade.go:131: [debug] performing update for mergeproblem
upgrade.go:303: [debug] creating upgraded release for mergeproblem
Error: UPGRADE FAILED: create: failed to encode release "mergeproblem": json: unsupported value: encountered a cycle via map[string]interface {}
helm.go:81: [debug] json: unsupported value: encountered a cycle via map[string]interface {}
create: failed to encode release "mergeproblem"
helm.sh/helm/v3/pkg/storage/driver.(*Secrets).Create
/private/tmp/helm-20210414-93729-197z3ms/pkg/storage/driver/secrets.go:156
helm.sh/helm/v3/pkg/storage.(*Storage).Create
/private/tmp/helm-20210414-93729-197z3ms/pkg/storage/storage.go:69
helm.sh/helm/v3/pkg/action.(*Upgrade).performUpgrade
/private/tmp/helm-20210414-93729-197z3ms/pkg/action/upgrade.go:304
helm.sh/helm/v3/pkg/action.(*Upgrade).Run
/private/tmp/helm-20210414-93729-197z3ms/pkg/action/upgrade.go:132
main.newUpgradeCmd.func2
/private/tmp/helm-20210414-93729-197z3ms/cmd/helm/upgrade.go:155
github.com/spf13/cobra.(*Command).execute
/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra#v1.1.1/command.go:850
github.com/spf13/cobra.(*Command).ExecuteC
/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra#v1.1.1/command.go:958
github.com/spf13/cobra.(*Command).Execute
/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra#v1.1.1/command.go:895
main.main
/private/tmp/helm-20210414-93729-197z3ms/cmd/helm/helm.go:80
runtime.main
/usr/local/Cellar/go/1.16.3/libexec/src/runtime/proc.go:225
runtime.goexit
/usr/local/Cellar/go/1.16.3/libexec/src/runtime/asm_amd64.s:1371
UPGRADE FAILED
main.newUpgradeCmd.func2
/private/tmp/helm-20210414-93729-197z3ms/cmd/helm/upgrade.go:157
github.com/spf13/cobra.(*Command).execute
/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra#v1.1.1/command.go:850
github.com/spf13/cobra.(*Command).ExecuteC
/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra#v1.1.1/command.go:958
github.com/spf13/cobra.(*Command).Execute
/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra#v1.1.1/command.go:895
main.main
/private/tmp/helm-20210414-93729-197z3ms/cmd/helm/helm.go:80
runtime.main
/usr/local/Cellar/go/1.16.3/libexec/src/runtime/proc.go:225
runtime.goexit
/usr/local/Cellar/go/1.16.3/libexec/src/runtime/asm_amd64.s:1371
My values.yaml looks like this:
########
#
# Component specifics
#
myfoo:
image:
repository: REPOSITORY-FROM-MYFOO-BLOCK
tag: TAG-FROM-MYFOO-BLOCK
autoscaling:
minReplicas: 33
yourbar:
image:
repository: Repository-From-Yourbar-Block
tag: Tag-From-Yourbar-Block
########
#
# Example defaults
#
replicaCount: 1
image:
repository: default-repository-from-top-level-image-block
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: "default-tag-from-top-level-image-block"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80

Helm template doesnt work when binded to configmap

I have a file which will be binded to a configmap. Once I put a tpl function in it, it stops working when other lines are included in the file.
I use this helper tpl:
{{- define "call-nested" }}
{{- $dot := index . 0 }}
{{- $subchart := index . 1 | splitList "." }}
{{- $template := index . 2 }}
{{- $values := $dot.Values }}
{{- range $subchart }}
{{- $values = index $values . }}
{{- end }}
{{- include $template (dict "Chart" (dict "Name" (last $subchart)) "Values" $values "Release" $dot.Release "Capabilities" $dot.Capabilities) }}
{{- end }}
testing in some.yaml:
psqlhost: {{include "call-nested" (list . "postgresql" "postgresql.fullname")}}
newlinekey: value
It works well if some.yaml is a standalone file. But once I bind it a configmap, it gives this error:
executing "mytestchart/templates/my-configmap.yaml" at <tpl (.Files.Glob "config/*").AsConfig .>: error calling tpl: Error during tpl function execution for "some.yaml:<br>\"name: {{include \\\"call-nested\\\" (list . \\\"postgresql\\\" \\\"postgresql.fullname\\\")}}\\r\\nnewlinekey:\n value\"\n": parse error in "mytestchart/templates/my-configmap.yaml": template: mytestchart/templates/my-configmap.yam:1: unexpected "\\" in operand
Once I remove the new line it will also work well.
edit: Configmap:
apiVersion: v1
kind: ConfigMap
metadata:
name: somename
data:
{{ tpl (.Files.Glob "config/*").AsConfig .| indent 2 }}
Your template is not rendering correctly. In order to make it work you need to change the following:
Add a .tmpl suffix to the file you'd like to put to your ConfigMap, for example: some.yaml.tmpl.
Set your ConfigMap's data: to: {{- tpl ((.Files.Glob "config/*.tmpl").AsConfig) . | indent 2 }}. Add a indentation to that line also.
Your ConfigMap would than look something like this:
apiVersion: v1
kind: ConfigMap
metadata:
name: somename
data:
{{- tpl ((.Files.Glob "config/*.tmpl").AsConfig) . | indent 2 }}
You can find more info with some examples here.
Please let me know if that helped.

Helm template - how to use "if exists at least one of" in array?

I'm trying to make a list of env vars from a values.yaml become a single secret.yaml file containing the list of envs of the type "secret". The idea is to only create this secret file if at least one of the types equals "secret".
Eg:
values.yaml
env:
- name: PLAIN_TEXT_ENV_VAR1
type: plain
value: text value
- name: PLAIN_TEXT_ENV_VAR2
type: plain
value: text value
- name: TOP_SECRET_ENV_VAR_1
type: secret
- name: TOP_SECRET_ENV_VAR_2
type: secret
- name: TOP_SECRET_ENV_VAR_3
type: secret
resulting secret.yaml
kind: Secret
metadata:
name: test
data:
TOP_SECRET_ENV_VAR_1: change_me
TOP_SECRET_ENV_VAR_2: change_me
TOP_SECRET_ENV_VAR_3: change_me
I've already tried to create some flow control using range to iterate, boolean variables and if statements, but go template seems to ignore my ifs after I change it to another value.
Now my secret template is like the one below.
{{ $flowcontrol := true -}}
{{ if $flowcontrol -}}
{{ range $env := $.Values.env -}}
{{ if eq $env.type "secret" -}}
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
{{- range $env := $.Values.env }}
{{- if eq $env.type "secret" }}
{{ $env.name }}: "change_me"
{{- end }}
{{- end }}
{{ $flowcontrol := false }}
{{ end -}}
{{ end -}}
{{ end -}}
It results in three replicated secret.yaml files with 3 variables:
$ helm template .
---
# Source: teste/templates/secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
TOP_SECRET_ENV_VAR_1: "change_me"
TOP_SECRET_ENV_VAR_2: "change_me"
TOP_SECRET_ENV_VAR_3: "change_me"
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
TOP_SECRET_ENV_VAR_1: "change_me"
TOP_SECRET_ENV_VAR_2: "change_me"
TOP_SECRET_ENV_VAR_3: "change_me"
apiVersion: v1
kind: Secret
metadata:
name: testsecret
data:
TOP_SECRET_ENV_VAR_1: "change_me"
TOP_SECRET_ENV_VAR_2: "change_me"
TOP_SECRET_ENV_VAR_3: "change_me"
How can one control the flow as: if the first one item of a list satisfies the condition, iterate through the rest of the same list only in another section after?
I've managed to create this feature! :D
Basicaly i've created a "template function" using define which iterates the env list from values.yaml and writes a string containing only the envs which the type property matches the word "secret".
Then I call that function using include and assign it's output to a variable.
If the variable length is greater than 0 (meaning it's not an empty string), the secret file is created and then I use the same string to fill the data property.
Here's the code containing the function and the secret template:
{{- define "get-secrets-from-env-list" -}}
{{- $allenv := index . 0 -}}
{{- range $i, $scrts := $allenv -}}
{{- if eq $scrts.type "secret" -}}
{{- nindent 0 $scrts.name -}}: {{ "change_me" | b64enc -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{ $secrets := include "get-secrets-from-env-list" (list .Values.env ) }}
{{- if gt (len $secrets) 0 -}}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: {{ include "awesome-chart.fullname" $ }}
labels:
app.kubernetes.io/name: {{ include "awesome-chart.name" $ }}
helm.sh/chart: {{ include "awesome-chart.chart" $ }}
app.kubernetes.io/instance: {{ $.Release.Name }}
app.kubernetes.io/managed-by: {{ $.Release.Service }}
data:
{{- nindent 2 $secrets -}}
{{- end -}}%
Here's a sample output:
$ helm template .
---
# Source: awesome-chart/templates/secret.yaml
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: my-app
labels:
app.kubernetes.io/name: awesome-chart
helm.sh/chart: awesome-chart
app.kubernetes.io/instance: release-name
app.kubernetes.io/managed-by: Tiller
data:
TOP_SECRET_ENV_VAR_1: Y2hhbmdlX21l
TOP_SECRET_ENV_VAR_2: Y2hhbmdlX21l
TOP_SECRET_ENV_VAR_3: Y2hhbmdlX21l
Works like a charm! ;)

Helm - Configmap - Replace only the contents of file and not File Names

I have defined application.properties files in a config dir. Config dir is on the below structure.
config
application.properties
application-test.properties
application-dev.properties
application-prod.properties
I have created a helm chart to create the configmap. Helm chart is defined as below
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configmap
data:
{{- (.Files.Glob "config/*").AsConfig | nindent 2 }}
I see that the configmap is created.
We are consuming the ConfigMap via environment variables in a running container using the envFrom property. (This in my deployment yaml file)
spec:
containers:
- envFrom:
- configMapRef:
name: nginx-configmap
I see that the values are stored as environment variables. However the variables are in lower cases.
server.port=8080
server.name=localhost
Since these are env variables, they have to be uppercase and . should be replaced with _. So, I have modifed my chart as below
data:
{{- (.Files.Glob "config/*").AsConfig | nindent 2 | upper | replace "." "_" }}
The generated configmap is as below
APPLICATION_PROPERTIES: |
SERVER_PORT = 8080
SERVER_NAME = LOCALHOST
Below is the env variables inside container
APPLICATION_PROPERTIES=SERVER_PORT = 8080
SERVER_NAME = LOCALHOST
My requirement is that only the contents of the file should be upper case and . should be replaced with _. The filename should not be converted. The filename should be as is.
Can this be achieved?
Try this:
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configmap
data:
{{ $root := . }}
{{ range $path, $bytes := .Files.Glob "config/*" }}
{{ base $path }}: '{{ $root.Files.Get $path | nindent 2 | upper | replace "." "_" }}'
{{ end }}