What is the difference between helm syntax {{ something }} and {{- something }}? - kubernetes-helm

I'm trying to understand helm templates and found syntax like this:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
So I thought every thing had to start with {{- but then I found other syntax that did not have that syntax:
- name: {{ .Chart.Name }}
So my question is what is the difference between those two syntaxs? What does the dash do? When is it needed?

The Helm template syntax is based on the Go programming language's text/template package.
The braces {{ and }} are the opening and closing brackets to enter and exit template logic.
The Helm documentation at https://helm.sh/docs/chart_template_guide/control_structures/
discusses why this syntax is needed in an example.
YAML ascribes meaning to whitespace, so managing the whitespace becomes pretty important. [...] the curly brace syntax {{ of template declarations can be modified with special characters to tell the template engine to chomp whitespace. {{- (with the dash and space added) indicates that whitespace should be chomped left, while -}} means whitespace to the right should be consumed. Be careful! Newlines are whitespace!
So the answer is this. The difference between the {{ syntax and the {{- syntax is that the {{- something }} will result in space on the left being removed. Without this any extra space would be included which could result in incorrectly formatted YAML.
Refer to the Helm documentation which goes into great length about how this syntax works and removes extra spaces.
You'll frequently see the dash showing up in control structures because without this extra space would be added to your YAML file which could result in invalid syntax being created. So, for example,
{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1beta1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
Causes the property apiVersion to be output (in the YAML file) without adding blank lines before and after the property.
Simple example
The Go templating documentation says
when executing the template whose source is
"{{23 -}} < {{- 45}}"
the generated output would be
"23<45"
This shows that the dash syntax causes white space to be removed.
Learning to experiment with Helm syntax
Below I'll explain how you can begin experimenting with helm syntax
using a simple throw away project.
The commands below I create a temp directory testhelm and go into it, and then run create helm mytest to create a helm application.
Next, I create a sample helm YAML file. This is the file you want to put what you want to test inside of. Below I used the file mytest/templates/my.yaml but any file can be created.
Helm apparently takes all of the files in the templates directory and parses/processes them to create the YAML output (which is used to create a Kubernetes YAML file to configure a K8S application).
In our case, we just leverage the helm command create a test bed for us to play around with.
If you are on a UNIX-based system you should be able to copy and paste the entire code sample below to create the testbed to begin experimenting.
mkdir testhelm
cd testhelm
helm create mytest
cat <<EOF > mytest/templates/my.yaml
expression1: "{{ 23 }} < {{ 45 }}"
expression2: "{{ 23 -}} < {{- 45 }}"
aTest0: ArgWithNoSpace
aTest1: Arg with spaces on left and right
aTest2: " spaces-on-left-and-right "
aTest3: {{ " spaces-on-left-and-right " }}
aTest4: {{ " spaces-on-left-and-right " | trim | quote }}
aTest5: Some
{{- "Thing Funky is" -}} goingOn
{{- " here"}}
drink2: {{ .Values.drink2 | default "coffee" | quote }}
aTest6: Some {{ "Thing Funky is" }}goingOn {{ " here"}}
aTest7: Some {{ "Thing Funky is" }}goingOn {{ " here"}}
EOF
Then run run the helm template command as shown below, and study the output that you get.
helm template myproj ./mychart | less
. . . output trimmed . . .
# Source: mychart/templates/my.yaml
expression1: "23 < 45"
expression2: "23<45"
aTest0: ArgWithNoSpace
aTest1: Arg with spaces on left and right
aTest2: " spaces-on-left-and-right "
aTest3: spaces-on-left-and-right
aTest4: "spaces-on-left-and-right"
aTest5: SomeThing Funky isgoingOn here
drink2: "coffee"
aTest6: Some Thing Funky isgoingOn here
aTest7: Some Thing Funky isgoingOn here
The first two name/value pairs expression1 and expression2 show the difference with and without dash syntax being used.
Notice also the syntax for aTest5 which resulted in several lines being merged into a single line of output.
Notice also aTest6 and aTest7 look different in the source but produce the same output; spaces within the {{ }} are not output unless they within quotes.
Using this approach digest the Helm syntax in bite sized chunks and so when you need to fix something you can understand what you are seeing.

Related

How to know, when I have to use {{- instead of {{ when producing yaml?

Completely new to helm, I cannot understand how this whitespace removal work. I've read that {{- should just remove whitespaces/newlines. I'd expect it to remove from expression within brackets, but I would put my had into mildly warm water that I saw removing white spaces before brackets. What does it actually do?
Also, using helm producing yaml(where whitespaces, indentation and newlines are significant characters :| ), single missing/extra dash will produce error.
Is there some recommendation which pattern of dashes to use to avoid issues? Since I really didn't find reliable definition of what is removed I constantly have to test all permutations to find winning combination...
EDIT: or how to debug it to know what is wrong? Unable to parse yaml to json is not enough to find out what is the actual problem.
First of all, you may need to know that helm essentially uses golang for template replacement, so the underlying implementation can refer to go template
According helm doc
Notice that we received a few empty lines in our YAML. Why? When the
template engine runs, it removes the contents inside of {{ and }}, but
it leaves the remaining whitespace exactly as is.
YAML ascribes meaning to whitespace, so managing the whitespace
becomes pretty important. Fortunately, Helm templates have a few tools
to help.
First, the curly brace syntax of template declarations can be modified
with special characters to tell the template engine to chomp
whitespace. {{- (with the dash and space added) indicates that
whitespace should be chomped left, while -}} means whitespace to the
right should be consumed. Be careful! Newlines are whitespace!
Make sure there is a space between the - and the rest of your
directive. {{- 3 }} means "trim left whitespace and print 3" while
{{-3 }} means "print -3".
e.g.
values.yaml
atest: test
templates/cm.yaml
```yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
cfg1: |-
{{ if eq $.Values.atest "test" }}
ok1
{{ end }}
cfg2: |-
{{- if eq $.Values.atest "test" }}
ok2
{{- end }}
cfg3: |-
ok
{{- if eq $.Values.atest "test" -}}
3
{{- end -}}
xxxxx
cmd
helm template --debug test .
output
---
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
cfg1: |-
ok1
cfg2: |-
ok2
cfg3: |-
ok3xxxxx
In detail:
For cfg1, there is no - in {{ or }}, after rendering, the carriage returns at the beginning and end of lines {{ if }} and {{ end }} are preserved, and the spaces before ok1 are also preserved
For cfg2, there is a - in {{, after rendering, the carriage returns at the end of lines {{ if }} and {{ end }} are retained, but the enter or spaces at the beginning are deleted
For cfg3, there is - in {{ and }}, after rendering, Therefore, after rendering, all enter and spaces before and after lines {{ if }} and {{ end }}, including those before ok 3, all will be deleted
Last, For debug.
helm Debugging Templates
helm template --debug test .
It's a great way to have the server render your templates, then return the resulting manifest file.
In the Go text/template language, a hyphen - directly inside curly braces {{ ... }} causes whitespace outside the curly braces to be removed. It's any whitespace (space, tab, carriage return, newline) on the same side as the hyphen.
plain text: two words
has two spaces between words: two {{ printf "" }} words
has only one space: two {{- printf "" }} words
has only one space: two {{ printf "" -}} words
has no spaces: two {{- printf "" -}} words
# kept vv vvvv dropped
has exactly two spaces: two {{ printf "" -}} words
has exactly four spaces: two {{- printf "" }} words
# dropped ^^ ^^^^ kept
on one line since newlines are whitespace: two
{{- printf "" }} words
Other punctuation, including hyphens outside of curly braces, is always kept as-is.
As a rule of thumb it's often right to start template expressions at the first column and include a hyphen inside the open brace but not the closing brace. That preserves a newline and indentation for the following line.
metadata:
labels:
{{- include "common.labels" . | indent 4 }}
another: label
Running helm template on your chart will render it to YAML, and also try to parse it. helm template --debug will print out the output even on a parse failure. In this situation I tend to visually scan through the generated YAML to look for indentation errors or especially missing newlines (that is, a line like value: foovolumeMounts: that's two lines concatenated together).

How does "template" and "include" differ in helm

How does "template" and "include" keywords differ in helm. Both seem to be used to render template parameters
template is part of the core Go text/template language. It always renders its results to the template output; it does not produce a value and its result cannot be captured in a variable or included in a pipeline.
include is a Helm extension. It captures the template output and returns it as a string. Its result can be used the same way as any other support function call. include is not a "keyword" or "action" or "special form", from the point of view of the templating language it is an ordinary extension function.
If you're unsure, in the context of a Helm chart, include is usually not wrong.
The most important place where this difference matters is where you have a block that produces a YAML fragment, and you need to indent it. Helm includes an indent function that can do this, but it needs a string to do this, so you need to use include and not template here.
{{- define "foo.labels" -}}
foo: bar
{{ end -}}
metadata:
labels:
{{ include "foo.labels" . | indent 4 }}
spec:
template:
metadata:
{{ include "foo.labels" . | indent 8 }}
For a more direct example, consider a template that just quotes its parameter. If you call this with template, the template pipeline syntax applies to the template parameter. If you call it with include, it applies to the result of the template call. Combining this with indent, there's a visible difference whether you see the indentation inside or outside the quotes.
{{ define "quote" }}{{ quote . }}{{ end }}
{{/* "hello" is indented, then " hello" is quoted */}}
Template: {{ template "quote" "hello" | indent 2 }}
{{/* "hello" is quoted, then '"hello"' is indented */}}
Include: {{ include "quote" "hello" | indent 2 }}
Template: " hello"
Include: "hello"

can't access helm .Values from a named template with non global context passed in

I'm trying to use a Helm named template that I plan to include with several different contexts, and the template has many values that are the same for all contexts.
Whenever I pass a context to template or include to invoke the named template, the references to .Values do not work, which is understandable because I'm explicitly setting a lower context.
In the Helm documentation for with, it claims there is a "global" variable $ that will allow reference to the global .Values, e.g., {{ $.Values... }}. This does not work (the example below shows the error).
I've also tried defining variables (using :=) and "enclosing" the include inside that variable definition (via indentation - I don't know if it matters) to make that variable available within the named template, but this doesn't work either.
I've also tried putting these in "globals" as described here which is more of a subchart thing and this doesn't work either.
So, I'm out of Helm tricks to make this work and will sadly have to re-define these many same variable many times - which makes the entire named template solution a bit less elegant - or just go back to having largely duplicate partially-parameterized templates.
What am I missing?
$ helm version
Client: &version.Version{SemVer:"v2.9+unreleased", GitCommit:"", GitTreeState:"clean"}
Values.yaml:
---
commonSetting1: "common1"
commonSetting2: "common2"
context1:
setting1: "c1s1"
setting2: "c1s2"
context2:
setting1: "c2s1"
setting2: "c2s2"
deployment.yaml:
---
{{- define "myNamedTemplate" }}
- name: {{ .setting1 }}
image: {{ $.Values.commonSetting1 }}
{{- include "myNamedTemplate" .Values.context1 }}
{{- include "myNamedTemplate" .Values.context2 }}
$ helm template test-0.1.0.tgz
Error: render error in "test/templates/deployment.yaml": template: test/templates/deployment.yaml:7:4: executing "test/templates/deployment.yaml" at <include "myNamedTemp...>: error calling include: template: test/templates/deployment.yaml:4:19: executing "myNamedTemplate" at <$.Values.commonSetti...>: can't evaluate field commonSetting1 in type interface {}
When I do this, I tend to explicitly pass in the top-level context object as a parameter. This gets a little tricky because the Go text/template templates only take a single parameter, so you need to use the (Helm/Sprig) list function to package multiple parameters together, and then the (standard text/template) index function to unpack them.
The template definition would look like:
{{- define "myNamedTemplate" }}
{{- $top := index . 0 }}
{{- $context := index . 1 }}
- name: {{ $context.setting1 }}
image: {{ $top.Values.commonSetting1 }}
{{ end }}
When you invoke it, you would then need to explicitly pass the current context as a parameter:
{{ include "myNamedTemplate" (list . .Values.context1) }}

How can we specify custom path to .Files.Get when creating ConfigMap with Helm

I am creating a config map as below
kubectl create configmap testconfigmap --from-file=testkey=/var/opt/testfile.txt
As I am using helm charts, I would like to create the config map using YAML file instead of running kubectl.
I went through Kubernetes - How to define ConfigMap built using a file in a yaml? and we can use .Files.Get to access the files.
But then testfile.txt needs to be a part of helm. I would like to have something like
kind: ConfigMap
metadata:
name: testconfigmap
data:
fromfile: |-
{{ .Files.Get "/var/opt/testfile.txt" | indent 4 }}
It works when "testfile.txt" is under the main helm directory. So, {{ .Files.Get "testfile.txt" | indent 4 }} works but {{ .Files.Get "/var/opt/testfile.txt" | indent 4 }} doesn't. With custom path, the value for the ConfigMap is empty.
Is is possible to place the file at a custom path outside the helm folder, so I can define my path in Values.yaml and read it in my ConfigMap yaml ?
This is a Community Wiki answer so feel free to edit it and add any additional details you consider important.
As mdaniel has already stated in his comment:
Is is possible to place the file at a custom path outside the helm
folder no, because helm considers that a security risk – mdaniel 2
days ago
You can also compare it with this feature request on GitHub where you can find very similar requirement described in short e.g. in this comment:
I have this exact need. My chart publishes a secret read from file at
/keybase. This file is deliberately not in the chart.
I believe files for .Files.Get should not be assumed to be inside the
chart ...
One interesting comment:
lenalebt commented on Dec 23, 2017 I am quite sure .Files.Get not
being able to access the file system arbitrarily is a security
feature, so I don't think the current behaviour is wrong - it just
does not fulfill all use cases.
This issue was created quite long time ago (Dec 19, 2017) but has been recently reopened. There are even some specific proposals on how it could be handled:
titou10titou10 commented on Apr 2 #misberner can you confirm that
using--include-dir =will allow us to use
.Files.Glob().AsConfig(), and so create a ConfigMap with one
entry in the CM per file in?
#misberner misberner commented on Apr 2 Yeah that's the idea. An open
question from my point of view is whether an --include-dir with a
specified introduces an overlay, or shadows everything under
/ from previous args and from the bundle itself. I'm not super
opinionated on that one but would prefer the former.
The most recent comments give some hope that this feature might become available in future releases of helm.
As mdaniel and mario already mentioned, for now this is not possible, as it's considered a security risk.
But actually there is a workaround.
You can use Helm templating to parse your property file and load it into a ConfigMap.
# create the following ConfigMap in your Chart
# this is just a simple prototype
# it requires strict key=value syntax in your property file (no empty strings etc.)
# but it shows the idea - improve the syntax, if needed
apiVersion: v1
kind: ConfigMap
metadata:
name: example
data:
{{- if .Values.example.map }}
{{- range $line := splitList "\n" .Values.example.map }}
{{- $words := splitList "=" $line }}
{{- $key := index $words 0 | trim }}
{{- $value := rest $words | join "=" | trim }}
{{ $key }}: "{{ $value }}"
{{- end }}
{{- end }}
{{- end }}
And after that you may load your properties file into this ConfigMap.
helm install mychart --set-file example.map="/test/my.properties"
Of course it is safe to use ONLY if you fully control the input, i. e. how each and every line of your property file is populated.

helm template: how do i assign the result of a template to a variable

I'm trying to do something like:
{{- $cassandrafullname := template "cassandra.fullname" . -}}
but I'm getting this error on a dry run:
Error: UPGRADE FAILED: parse error in "cassandra/templates/service.yaml": template: cassandra/templates/service.yaml:1: unexpected <template> in command
The reason why I have this issue is because I am unable to use the template cassandra.fullname within a range, so I'm trying to put the value into a variable and use it in the range instead. So if there's a solution for that, it would also be accepted!
Helm defines an include function which is identical to the standard template, except that it returns the rendered output instead of outputting it. You should be able to write
{{- $cassandrafullname := include "cassandra.fullname" . -}}
Unfortunately this does not work with fromYaml, so you can't read yaml structs into pipeline operations as usual. A rather big shortcoming. Lots of times I need to filter a list into another list, but this seems impossible with helm:
{{- define "sometpl" -}}
- bla: dibla
- oki: doki
{{- end -}}
---
{{- $v := include "sometpl" . | fromYaml }}
some: {{- $v | toYaml | nindent 2 }}
Will give
some:
Error: 'error unmarshaling JSON: while decoding JSON: json: cannot unmarshal array
into Go value of type map[string]interface {}'