helm check for required variables in a single file - kubernetes-helm

I'd like to have a file that checks for all "required" variables. So far I came up with this, but it doesn't look like this is "best" practices
values.yaml
# this is required name of the hosted zone
domainZone: ""
someOtherRequireVar: ""
/templates/required.yaml
# the block below makes sure that our required variables are
# declared in the values file and if not that will throw the error
# so even the block below is commeted, it's still evaluated by the HELM and error
# will be generated if the variable's value is not set
# {{ .Values.domainZone | required "Domain Zone is required and should be non-empty string" }}
# {{ .Values.someOtherRequireVar | required "The someOtherRequireVar is also required and should be non-empty string" }}
Are there better ways to achieve this?

I'd recommend using named templates for validation, which you can execute in your installation notes instead of regular template files. Furthermore, by using failsafe validation functions (unlike require) for individual values, you can validate multiple values before failing execution of your helm command.
Bitnami charts use this strategy.
Setup
In a partials file, e.g. templates/_helpers.tpl, define named templates for validating the supplied values:
{{/* Compile all validation warnings into a single message and call fail. */}}
{{- define "mychart.validateValues" -}}
{{- $messages := list -}}
{{- $messages = append $messages (include "mychart.validateValues.foo" .) -}}
{{- $messages = append $messages (include "mychart.validateValues.bar" .) -}}
{{- $messages = without $messages "" -}}
{{- $message := join "\n" $messages -}}
{{- if $message -}}
{{- printf "\nVALUES VALIDATION:\n%s" $message | fail -}}
{{- end -}}
{{- end -}}
{{/* Validate value of foo */}}
{{- define "mychart.validateValues.foo" -}}
{{- if not (and .Values.foo (kindIs "string" .Values.foo)) -}}
mychart: foo
`foo` is required and should be a non-empty string
{{- end -}}
{{- end -}}
{{/* Validate the value of bar */}}
{{- define "mychart.validateValues.bar" -}}
{{- if not (and .Values.bar (kindIs "string" .Values.bar)) -}}
mychart: bar
`bar` is required and should be a non-empty string
{{- end -}}
{{- end -}}
In this case, the named template mychart.validateValues will run multiple validations (i.e. mychart.validateValues.foo, mychart.validateValues.bar). If one or more validations produce a validation warning, it will call fail with a summary of the warnings.
In templates/NOTES.txt, evaluate the named template for validation:
{{- include "mychart.validateValues" . }}
Test
With the following values.yaml file:
bar: 24
Templating, installing, or upgrading the chart will fail with the following message:
Error: execution error at (test/templates/NOTES.txt:1:4):
VALUES VALIDATION:
mychart: foo
`foo` is required and should be a non-empty string
mychart: bar
`bar` is required and should be a non-empty string

Related

helm using argument in template to check if value exists in Values.yaml

The problem is following: I want to check if field in Values.yaml exists based on argument given to the template in _helpers.tpl:
{{- define "example-name" -}}
{{- $objectRef := index . 0 -}}
{{- if .Values.custom -}}
{{- if .Values.custom.$objectRef -}}
{{- if .Values.custom.$objectRef.annotations -}}
{{- include "some-library" (tuple .Values.custom.$objectRef.annotations) | indent 4 }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
Then in my deployment for example:
{{- template "example-name" "someField" }}
I want the result to be following:
{{- define "example-name" -}}
{{- $objectRef := index . 0 -}}
{{- if .Values.custom -}}
{{- if .Values.custom.someField-}}
{{- if .Values.custom.someField.annotations -}}
{{- include "some-library" (tuple .Values.custom.someField.annotations) | indent 4 }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
But the only thing I get is following error: bad character U+0024 '$'
I want to use template multiple times with various arguments. I haven't seen anyone dealing with the same problem before.
Any ideas?
The standard template function you're looking for is index. In its simplest form, index $map $key does a dynamic lookup of the $key (can be any expression) in the provided $map (can be any expression). It can also do integer-index lookups in arrays (Go slices) and nested lookups if you need to.
The other problem that you'll run into is that $map.undefinedKey (or index $map "undefinedKey"), assuming undefinedKey is not present in $map, is valid but evaluates to Go nil. So you can't do further lookups in that. The workaround to this I typically use is to use the Helm (Sprig) default function to use an empty dict if a value is not present.
That would give you a template like:
{{- define "example-name" -}}
{{- $top := index . 0 -}}
{{- $objectRef := index . 1 -}}
{{- $custom := $top.Values.custom | default dict -}}
{{- $object := index $custom $objectRef | default dict -}}
{{- with $object.annotations -}}
{{- include "some-library" (list $top .) | indent 4 }}
{{- end }}
{{- end }}
This is called with a list of two values, the top-level Helm object and a reference to a key in .Values.custom
{{- include "example-name" (list . "someField") -}}
The template extracts the two values from the list parameter. It then traverses the values structure one level at a time, at each level defaulting to an empty dictionary. So for example if there is no .Values.custom then $custom is set to an empty dictionary, which allows index $custom $objectRef to execute successfully (and return nil, but not abort). At the bottom level we use the with template function to check to see if come value is truthy, and if so, temporarily bind . to its value. When we make the inner call, we already have the top-level Helm object in a variable, and we can assemble a list of $top and the non-empty annotation structure . as the single template parameter.

Passing Dictionary as an argument in Helm Template

I wonder if there is anyway for us to pass dictionary as an argument in Helm Templates. In Python, I could simply create some scripts like this but I am not sure how to translate this script into Helm.
>>> def display(d):
... for key in d:
... print("key:", key, "Value:", d[key])
...
>>> def func():
... D = {'a':1, 'b':2, 'c':3}
... display(D)
...
>>> func()
key: a Value: 1
key: b Value: 2
key: c Value: 3
I tried to implement some script like this but they seem does not solve out.
_helpers.tpl
{{- define "display"}}
{{- $d := .d -}}
{{- range $key, $value := $d -}}
{{- printf "%s-%s" $key $value-}}
{{- end -}}
_test.tpl
{{- define "func"}}
{{- $D = dict 'a' 1 'b' 2 'c' 3} -}}
{{- $res := include "display" (dict "d" $D) -}}
{{- $res -}}
{{- end -}}
Would you give me some working examples of how a dictionary is passed as an arguments in helm?
Thank you in advance for your help! I am much appreciate it!
A Go text/template template takes only one parameter. It has the special name . inside the template. That parameter can be any type, though, and in a Helm context there are extension functions like list and dict that can assemble the containers.
A template can only return a string, and if you need to capture that return value then you need to use the Helm include extension function rather than the standard template template directive. Anything the template outputs is part of that "return value" and there is not a specific "return" command.
So, for example, your display function is pretty easy to write: its one parameter is a dictionary and you use a range loop to iterate over its contents. (I'm dumping it out as a YAML list, which would be a more typical Kubernetes/Helm output.)
{{- define "display" -}}
{{- range $k, $v := . -}}
- key: {{ $k }}
value: {{ $v }}
{{ end -}}
{{ end -}}
And then to call it, you'd construct a dictionary with dict (or extract one from .Values) and pass that dictionary to the function.
{{- $d := dict "a" 1 "b" 2 "c" 3 }}
{{- include "display" $d -}}

How to concatenate variables inside a ternary statement?

I'm trying to do this:
name: {{ $value.enable | ternary $value.prefix $.Release.Namespace $value.suffix $value.override }}
But that syntax is wrong. I can't find any examples for how I would concatenate these vars together: $value.prefix $.Release.Namespace $value.suffix
Edit
I thought I could use print like this:
name: {{ true | ternary (print $value.prefix $.Release.Namespace $value.suffix) $value.fullnameOverride }}
But if you don't specify one of the fields it prints <nil> instead of not printing anything which is what I want.
Helm includes the Sprig template library which includes many useful composable parts.
For the "true" half, you have the three components; you'd like them joined together; but you'd like nil parts to be removed. The Sprig list function constructs a list from arbitrary items; compact takes a list and returns a new list without empty items (either empty string or nil, anything that's a Go zero value); and then join combines the list together into a single string. You can assign that to a temporary variable, and use it in the ternary call:
{{- $qualifiedName := list $value.prefix $.Release.Namespace $value.suffix | compact | join "" }}
name: {{ $value.enable | ternary $qualifiedName $value.fullnameOverride }}
I find the ternary syntax a little hard to read (even if it matches the C-style expr ? t : f syntax) and in this context it's not necessary. A helper template that spelled this out could be easier to understand later.
{{-/* Generate the name of the thing. Call with a list containing
the top-level Helm object and an item from the values file.
(Indented for readability, the template engine removes all of
the whitespace.) */-}}
{{- define "name" -}}
{{- $top := index . 0 -}}
{{- $value := index . 1 -}}
{{- if $value.enable -}}
{{- with $value.prefix -}}{{- . -}}{{- end -}}
{{- with $top.Release.Namespace -}}{{- . -}}{{- end -}}
{{- with $value.suffix -}}{{- . -}}{{- end -}}
{{- else -}}
{{- $value.fullnameOverride -}}
{{- end -}}
{{- end -}}
name: {{ include "name" (list $ .) }}

Adding news lines when defining collection

I am trying to define a collection (dict), and I would like to add a new line on each definition (for readability), Eg:
{{ $deployment := dict
"Release" .Release
"Chart" .Chart
"Values" .Values }}
But when I do this, helm respond a parse error :
Error: parse error in "XXX": template: XXX:2: unclosed action
Error: UPGRADE FAILED: parse error in "XXX": template: XXX:2: unclosed action
Is there a way in HELM to do this?
I achieved this by defining the dict first and then setting one key per line.
{{- $myDict := dict "" "" -}}
{{- $_ := set $myDict "myKey1" "myValue1" -}}
{{- $_ := set $myDict "myKey2" "myValue2" -}}
{{- $_ := set $myDict "myKey3" "myValue3" -}}
{{- $_ := set $myDict "myKey4" "myValue4" -}}
Bonus Tip: Since dict get function is available seemingly in only helm3 and later, you can use this hack to get a value from a dict to a string.
{{/* Hack needed until helm 3 which has 'get' for 'dict' */}}
{{- $myValue3Var := pluck "myKey3" $myDict | first -}}
TLDR;
It's impossible to declare dict in multiline way, like with Perl fat comma operator.
Please check the reference of "Sprig: Template functions for Go templates."
Instead you could use this sort of hacky way to achieve similar result:
Keep each key value pair in separate line, in Global Values file for readability:
# values.yaml
--
global:
someMap:
coffee: robusta
origin: Angola
crema: yes
Define helper template in _helpers.tpl:
{{- define "mychart.labels.standard"}}
{{- $global := default (dict) .Values.global.someMap -}}
Release: {{ .Release.Name | quote }}
Chart: {{ .Chart.Name }}
Values:
{{- $global := default (dict) .Values.global.someMap -}}
{{- range $key, $value := $global }}
{{ $key }}: {{ $value }}
{{- end }}
{{- end -}}
Include it in another template:
helm_data:
{{- $global := default (dict) .Values.global -}}
{{- range $key, $value := $global }}
{{ $key }}: {{ $value }}
{{- end }}
{{ include "mychart.labels.standard" . | nindent 0 -}}
Render it to verify the result (helm template --name dict-chart .)
---
# Source: mychart/templates/data_type.yaml
helm_data:
someMap: map[crema:true origin:Angola coffee:robusta]
Release: "dict-chart"
Chart: mychart
Values:
coffee: robusta
crema: true
origin: Angol
It seems it's impossible to do so. The Helm templating system is basically the Go templating system. As stated in the Go templating docs:
Except for raw strings, actions may not span newlines, although comments can.
For people coming across this question, this functionality works in recent versions of HELM. For me, OPs example works as-is (Helm v3.8.2).
(I came across this question myself due to a mismatched ) in my template.)

How best to say a value is required in a helm chart?

I am doing this now:
value: {{ required "A valid .Values.foo entry required!" .Values.foo }}
But to give this same message for all required values in the templates is cumbersome and clutters the templates in my opinion.
Is there a better way where we could define it outside the template \ or a cleaner way to do it within the template itself?
You could do something by taking advantage of range and the fact that null will fail the required check. So in your values.yaml you could have this section for required env vars:
reqEnv:
- name: "VAR1"
value: null
- name: "VAR2"
value: null
And in the env section of the Deployment you then have:
{{- range .Values.reqEnv }}
{{ .name }}: {{ required "A value must be entered for all reqEnv entries" .value }}
{{- end }}
Then the user gets an error unless they set all required values of the reqEnv section in their values file or as paramters. Unfortunately what you lose by doing this is the detail of which var is missing. This could be why the official helm charts seem to prefer using required in the way that you already are.
Define the required values on top of your manifest as variables utilizing the required function.
E.g. deployment.yaml:
{{- $name := .Values.name | required ".Values.name is required." -}}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ $name }}
....
You can use helm lint with --strict flag to check undefined values
$ helm lint --strict .
==> Linting .
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: render error in "mychart/templates/service.yaml": template: mychart/templates/service.yaml:10:19: executing "mychart/templates/service.yaml" at <.Values.foo>: map has no entry for key "foo"
Error: 1 chart(s) linted, 1 chart(s) failed
To expose name of missing item to required text you can do something like this:
{{- range $field, $my_key := $data }}
{{- if hasKey $dic1 $my_key }}
{{ $field }}: {{ index $dic1 $my_key | b64enc}}
{{- else if hasKey $dic2 $my_key }}
{{ $field }}: {{ index $dic2 $my_key | b64enc}}
{{- else }}
{{ $field }}: {{ required (printf "key %s is missing" $my_key) nil }}
{{- end }}
{{- end }}
I've come up with an alternative hack trying to solve this problem. It's questionable whether this is actually any better that the built in solution, but I thought it might be worth noting down here as an option.
Add a function to your _helpers.tpl (or wherever):
{{/*
Require and include a value
*/}}
{{- define "require" -}}
{{- $scope := index . 0 -}}
{{- $name := index . 1 -}}
{{required (print "Missing required value: " $name) (index $scope "Values" $name)}}
{{- end}}
Then in your template call it with:
value: {{ include "require" (list . "foo") }}
If a value is missing, it errors with message:
Missing required value: foo
Otherwise, it inserts the value.
Edit: To make this work for nested values, you need a slightly more complex helper:
{{/*
Index a nested component
*/}}
{{- define "indexNested" -}}
{{- $message := index . 0 -}}
{{- $object := index . 1 -}}
{{- $path := (mustRegexSplit "\\." (index . 2) -1) -}}
{{- range $path -}}
{{- if not $object -}}
{{ fail $message }}
{{- end -}}
{{- $object = index $object . -}}
{{- end -}}
{{ required $message $object }}
{{- end}}
{{/*
Require and include a value
*/}}
{{- define "require" -}}
{{- $scope := index . 0 -}}
{{- $name := index . 1 -}}
{{ include "indexNested" (list (print "Missing required value: " $name) $scope.Values $name) }}
{{- end}}
Now you can access the foo.bar value with:
{{ include "require" (list . "foo.bar") }}