Helm's v3 Example Doesn't Show Multi-line Properties. Get YAML to JSON parse error - kubernetes

In Helm's v3 documentation: Accessing Files Inside Templates, the author gives an example of 3 properties (toml) files; where each file has only one key/value pair.
The configmap.yaml looks like this. I'm only adding one config.toml for simplicity.
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-config
data:
{{- $files := .Files }}
{{- range tuple "config.toml" }}
{{ . }}: |-
{{ $files.Get . }}
{{- end }}
This works fine, until I add a second line to the config.toml file.
config.toml
replicaCount=1
foo=bar
Then I get an Error: INSTALLATION FAILED: YAML parse error on deploy/templates/configmap.yaml: error converting YAML to JSON: yaml: line 9: could not find expected ':'
Any thoughts will be appreciated.
Thanks

Helm will read in that file, but it is (for good or bad) a text templating engine. It does not understand that you are trying to compose a YAML file and thus it will not help you. That's actually why you will see so many, many templates in the wild with {{ .thing | indent 8 }} or {{ .otherThing | toYaml }} -- because you need to help Helm know in what context it is emitting the text
Thus, in your specific case, you'll want the indent filter with a value of 4 because your current template has two spaces for the key indent level, and two more spaces for the value block scalar
data:
{{- $files := .Files }}
{{- range tuple "config.toml" }}
{{ . }}: |-
{{ $files.Get . | indent 4 }}
{{/* notice this ^^^ template expression is flush left,
because the 'indent' is handling whitespace, not the golang template itself */}}
{{- end }}
Also, while this is the specific answer to your question, don't overlook the .AsConfig section on that page which seems much more likely to be what you really want to happen, and requires less indent math

Related

helm --set to select environment

Struggling with some helm templating...
I'm trying to pass a separate yaml file with springboot parameters to helm, and have them split by environment... then I want to pass the environment to helm using --set env=staging
Feels like I've tried everything but clearly I'm lacking a fundamental understanding...
My _helpers.tpl contains these:
{{- define "env" }}
{{- printf "%s" .Values.env }}
{{- end }}
{{ define "configmap.metadata" }}
name: {{ .Values.name }}-config
{{ end }}
{{ define "configmap.properties" }}
{{ index .Values.environment (include "env" .) "properties" | indent 4 }}
{{ end }}
The template for the config map:
apiVersion: v1
kind: ConfigMap
metadata:
{{ include "configmap.metadata" . }}
data:
app.properties: |-
{{ include "configmap.properties" .}}
And the yaml file containing the properties looks like this:
environment:
staging:
properties:
spring:
datasource:
url: something
username: something
password: something
app1:
key: something
secret: something
baseUri: something
app2:
bootstrap_server: something
bootstrap_port: something
registry_schema: something
production:
properties:
spring:
etc, etc
And then I want to select the environment using set. I'm testing with:
helm template test . -f values.yaml -f properties.yaml --set env=staging
I think I've just tried so many things that I just can't see the wood for the trees! The error I'm seeing is:
Error: template: microservice/templates/configmap.yaml:7:7: executing "microservice/templates/configmap.yaml" at <include "configmap.properties" .>: error calling include: template: microservice/templates/_helpers.tpl:56:76: executing "configmap.properties" at <4>: wrong type for value; expected string; got map[string]interface {}
EDIT:
After tweaking, I'm still getting an error, but I'm seeing something in the configmap.. but I wonder if the error is due to the 8 spaces on the first line..
data:
app.properties: |-
app2:
bootstrap_port: something
bootstrap_server: something
registry_schema: something
app1:
baseUri: something
key: something
secret: something
spring:
datasource:
password: something
url: something
username: something
I think your actual error message is around the way you're using the .Values.environment.production.properties value. It's a YAML map, but the indent function expects it to be a string. You should be able to see some odd indentation and maybe an odd [map spring [map datasource ...]] string if you use the helm template --debug option.
When you go to render the ConfigMap, you need to make sure to do two things. Since the data you have is structured properties, you need to use the lightly-documented toYaml function to convert it back to YAML. This will begin at the first column, so you need to apply the indent function to it, and then you need to make sure the markup that invokes it is also at the first column (indent should be the only thing that supplies indentation).
data:
app.properties: |-
{{ include "configmap.properties" . | indent 4}}
{{/*- starts at column 1, but includes the `indent` function */}}
{{ define "configmap.properties" }}
{{- index .Values.environment (.Values.env) "properties" | toYaml }}
{{/*- starts at first column, includes `toYaml`, does not include `indent` */}}
{{- end }}

How to filter out file names from a .File.Glob in a Helm Chart

I have the following snippet in a helm library which is supposed to add in all the files in a folder into a ConfigMap except for those ending in .tz.
{{- /*
add the contents of every file in the config folder of this IOC helm chart
into the config map - this must include start.sh the startup script.
The files must be text only. Any files ending in .tz are explicitly ommitted
*/ -}}
{{ (.Files.Glob "config/*[!.tz]").AsConfig | indent 2 }}
version.txt: |
IOC {{ .Release.Name }} version {{ .Chart.AppVersion }}
This does not quite work in that it filters out all files ending in . or t or z.
I cant see how to do this with Go globbing. I also cannot work out the syntax for using 'without' for the list created by .Files.Glob.
Can anyone enlighten me? Thanks!
Thanks #David Maze. This almost worked but the reference to .Files.Get failed and I think that is because the . context becomes the range counter within the range loop.
Adding $ to .Files within the loop got it working.
Also to fully replicate AsConfig I needed to extract the basename from
the path which I did by adding regexReplaceAll.
data:
{{- range $path, $_ := .Files.Glob "config/*" }}
{{- if not (hasSuffix ".tz" $path) }}
{{ regexReplaceAll "(.*)/" $path "" }}: |
{{ $.Files.Get $path | indent 4 }}
{{- end }}
{{- end }}
The Helm documentation references the Go "github.com/gobwas/glob" package for the supported glob syntax; that doesn't support any sort of "except" cases, except for single characters.
What you can do instead is iterate over all of the files, and then use a normal conditional to exclude the ones you don't want. You have to construct the ConfigMap structure yourself rather than relying on the AsConfig helper.
data:
{{- range $path, $_ := .Files.Glob "config/*" }}
{{- if not (hasSuffix ".tz" $path) }}
{{ $path }}: |
{{ .Files.Get $path | indent 4 }}
{{- end }}
{{- end }}

How to use if/else loop in Helm

I am trying to use if/else-if/else loop in helm chart. Basically, I want to add ENV configs in configfile based on the if/else condition. Below is the logic:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.applicationName }}-configmap
labels:
projectName: {{ .Values.applicationName }}
environment: {{ .Values.environment }}
type: configmap
data:
{{- if eq .Values.environment "production" }}
{{ .Files.Get "config-prod.yaml" | nindent 2}}
{{- else if eq .Values.environment "development" }}
{{ .Files.Get "config-dev.yaml" | nindent 2}}
{{- else }}
{{ .Files.Get "config-stage.yaml" | nindent 2}}
{{- end }}
But I am not getting the desired output and facing some issue. Can anybody help me out with this?
Edit1: I have added my modified configmap.yaml as per the suggestions, helm install/template command gives Error: YAML parse error on demo2/templates/configmap.yaml: error converting YAML to JSON: yaml: line 14: did not find expected key error.
also my config-prod and config-stage is being rendered (as per the condition if I give environment: production then config-prod.yaml is being added and if I give environment: stage/null then config-stage.yaml is being added.
Your question would benefit from more specifics.
Please consider adding the following to your question:
How are you trying this? What commands exactly did you run?
How are you "not getting the desired output"? What output did you get?
Please also include:
the relevant entries from your values.yaml
the config-dev.yaml and config-stage.yaml files
Have you run helm template to generate the templates that Helm would apply to your cluster? This would be a good way to diagnose the issue.
I wonder whether you're chomping too much whitespace.
And you should just chomp left, i.e. {{- .... }} rather than left+right {{- ... -}}.
Sorry guys, it was my mistake, my dev-config.yaml has envs and it was defined like key=value, instead of key: value.

Is it possile to create yaml object from helm from file

I want to iterate over the Yaml list which is defined in the file, and use it in Job. For example i have
test.yaml
list:
- first element
- second element
In _helpers.tpl i can define
something like
{{- define "mychart.list" -}}
{{ .Files.Get "test.yaml"| toYaml }}
{{- end }}
And then in Job i want do something like
{{- $lists := include "mychart.list" . }}
{{- range $element := $lists.list}}
apiVersion: batch/v1
kind: Job
metadata:
and then use the $element.
But when i am trying to do the dry-run with --debug it complains about
at <$lists.list>: can't evaluate field list in type string.
Looks like whole value is coming as string rather than Yaml, does it needs explicit call to Yaml parser ? If yes, is there a way to do that ?
BTW i have also tried various combinations of
{{- $lists := include "mychart.list" . | toYaml }}
or loading the file inline, but none of them helps.
I can put the list in Values.yaml, but don't want to do that purposely.
Any help is really appreciated.
Just putting for future reference, if someone comes and have look.
There's fromYaml function, which is undocumented. I found it hard way, but that solved the isssue.
{{- $lists := include "mychart.list" . | fromYaml}}

Helm include templates

Please! Is it possible to squash multiple helm templates into one and then refer to it as a one-liner in a deployment file?
EG:
{{- define "foo.deploy" -}}
value:
{{- include "foo.1" . | nindent 6 }}
{{- include "foo.2" . | nindent 6 }}
{{- include "foo.3" . | nindent 6 }}
And then do an {{- include "foo.deploy" . }} in a separate deployment file.
Which should then contain foo.1, foo.2 and foo.3, and their respective definitions.
As opposed to literally writing out all three different 'includes' especially if you've got loads.
Much appreciated,
Thanks,
A named template (sometimes called a partial or a subtemplate) is simply a template defined inside of a file, and given a name. We’ll see two ways to create them, and a few different ways to use them.
Template names are global. As a result of this, if two templates are declared with the same name the last occurrence will be the one that is used. Since templates in subcharts are compiled together with top-level templates, it is best to name your templates with chart specific names. A popular naming convention is to prefix each defined template with the name of the chart: {{ define "mychart.labels" }}.
More information about named templates you can find here: named-template.
Proper configuration file should look like:
{{/* Generate basic labels */}}
{{- define "mychart.labels" }}
labels:
generator: helm
date: {{ now | htmlDate }}
{{- end }}
In your case part of file should looks like:
{{- define "foo.deploy" -}}
{{- include "foo.1" . | nindent 6 }}
{{- include "foo.2" . | nindent 6 }}
{{- include "foo.3" . | nindent 6 }}
{{ end }}