I'm using sub-charts. Here's my directory structure
/path/microservice-base-chart
/path/myApp
I have this values.yaml for my "base" (generic) chart
# Default region and repository
aws_region: us-east-1
repository: 012234567890.dkr.ecr.us-east-1.amazonaws.com
repositories:
us-east-1: 01234567890.dkr.ecr.us-east-1.amazonaws.com
eu-north-1: 98765432109.dkr.ecr.eu-north-1.amazonaws.com
image:
name: ""
version: ""
...and this in the base chart's templates/_helpers.yaml file
{{/*
Get the repository from the AWS region
*/}}
{{- define "microservice-base-chart.reponame" -}}
{{- $repo := index .Values.repositories .Values.aws_region | default .Values.repository }}
{{- printf "%s" $repo }}
{{- end }}
...and this in the base chart's templates/deployment.yaml file
apiVersion: apps/v1
kind: Deployment
...
spec:
...
template:
...
spec:
...
containers:
- name: {{ .Values.image.name }}
image: {{ include "microservice-base-chart.reponame" . }}/{{ .Values.image.name }}:{{ .Values.image.version }}
I have this in the Chart.yaml of a sub chart that uses the base chart.
dependencies:
- alias: microservice-0
name: microservice-base-chart
version: "0.1.0"
repository: file://../microservice-base-chart
...and this in the values.yaml of a sub chart
microservice-0:
image:
name: myApp
version: 1.2.3
However, when I run this, where I set aws_region
$ helm install marcom-stats-svc microservice-chart/ \
--set image.aws_region=eu-north-1 \
--set microservice-0.image.version=2.0.0 \
--dry-run --debug
I get this for the image name of the above deployment.yaml template
image: 01234567890.dkr.ecr.us-east-1.amazonaws.com/myApp:2.0.0
instead of the expected
image: 98765432109.dkr.ecr.eu-north-1.amazonaws.com/myApp:2.0.0
What am I missing? TIA
structure of folders:
files
alertmanager
rules
- alertmanager.rules
- nodes.rules
...
templates
- _helpers.tpl
- prometheus.yaml
files/alertmanager/rules/alertmanager.rules
- name: Alertmanager
rules:
- alert: PrometheusAlertmanagerConfigurationReloadFailure
expr: |
# Without max_over_time, failed scrapes could create false negatives, see
# https://www.robustperception.io/alerting-on-gauges-in-prometheus-2-0 for details.
max_over_time(alertmanager_config_last_reload_successful{app_kubernetes_io_name="alertmanager"}[5m]) == 0
for: 10m
labels:
severity: critical
{{ include _default_rule_labels . }}
annotations:
type: Alertmanager
summary: Prometheus AlertManager configuration reload failure
description: |
The error could be caused by recent changes and could be caused by an incorrect configuration of alertmanager template (defined in templates/prometheus.yaml).
Or it can be caused by incorrect route(s) configuration (typically in argocd/apps/values.yaml)
templates/_helpers.tpl
{{/*
Collect alertmanager rules from files
*/}}
{{- define "alertmanager.rules" -}}
{{- range $path, $_ := .Files.Glob "files/alertmanager/rules/**.rules" }}
{{ tpl ($.Files.Get $path) $ }}
{{- end }}
{{- end }}
{{/*
Set default alert rule labels
*/}}
{{- define "_default_rule_labels" -}}
environment: {{ .Values.environment }}
client: {{ .Values.client }}
cluster: {{ .Values.eks_cluster }}
sla: {{ .Values.sla }}
{{- end }}
templates/prometheus.yaml
# ? PROMETHEUS ALERT RULES
serverFiles:
alerting_rules.yml:
groups:
{{- include "alertmanager.rules" . | nindent 14 }}
I'm getting the following error when trying to render the prometheus.yaml template:
✗ helm template . -s templates/prometheus.yaml
parse error at (root/templates/prometheus.yaml:15): function "_default_rule_labels" not defined
How to approach?
I can render {{ .Values.something }} inside files/alertmanager/rules/alertmanager.rules, but inclusion of named templates throws an error.
I am defining a PrometheusRule as follow:
prometheusRule:
rules:
- alert: SSLCertExpiringSoon
expr: probe_ssl_earliest_cert_expiry - time() < 86400 * 10
for: 0m
labels:
severity: warning
annotations:
summary: Blackbox SSL certificate will expire soon (instance {{ $labels.instance }})
description: "SSL certificate expires in 30 days\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
And the template yml from helm chart:
{{- if .Values.prometheusRule.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: {{ template "prometheus-blackbox-exporter.fullname" . }}
{{- with .Values.prometheusRule.namespace }}
namespace: {{ . }}
{{- end }}
labels:
{{- include "prometheus-blackbox-exporter.labels" . | nindent 4 }}
{{- with .Values.prometheusRule.additionalLabels -}}
{{- toYaml . | nindent 4 -}}
{{- end }}
spec:
{{- with .Values.prometheusRule.rules }}
groups:
- name: {{ template "prometheus-blackbox-exporter.name" $ }}
rules: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- end }}
when I run helm template, tpl func it is not resolving the $labels and $values vars. When I remove annotations then the helm template is not complaining anymore. Where do fail?
error:
Error: template: prometheus-blackbox-exporter/templates/prometheusrule.yaml:18:16: executing "prometheus-blackbox-exporter/templates/prometheusrule.yaml" at <tpl (toYaml .) $>: error calling tpl: error during tpl function execution for "- alert: SSLCertExpiringSoon\n annotations:\n summary: Blackbox SSL certificate will expire soon (instance {{ $labels.instance\n }})\n expr: probe_ssl_earliest_cert_expiry - time() < 86400 * 10\n for: 0m\n labels:\n release: prometheus\n severity: warning\n- alert: SSLCertExpiringSoon\n annotations: null\n expr: probe_ssl_earliest_cert_expiry - time() < 86400 * 3\n for: 0m\n labels:\n severity: critical": parse error at (prometheus-blackbox-exporter/templates/prometheusrule.yaml:3): undefined variable "$labels"
Prometheus's alerting rules also use {{ ... $variable ... }} syntax, similar to Helm but with a different variant on the Go text/template syntax. When you pass this file through tpl, Helm tries to evaluate the embedded {{ ... }} template and evaluate any blocks there. Since $labels and $value aren't local variables defined at the Helm level, you get this error.
If you just want Prometheus to see this file as-is, and you don't need to replace anything at the Helm level (the file doesn't include references to .Values) then you don't need tpl
rules: {{ toYaml . | nindent 8 }}
If you do need tpl, then inside the included file you need to cause {{ to be emitted as a string and not processed as a template. One syntactic approach to it is to create a template block that prints out {{:
description: "VALUE = {{ "{{" }} $value }}"
# ^^^^^^^^^^ a {{ ... }} block that prints "{{"
The working version of syntax is as follow:
{{ `{{` }} $value }}
I have a template that get rendered several times with a range iteration and I can access variables external variables such as $.Release.Name without a problem. However, when I include templates I can't get it to work:
{{ range $key, $val := $.Values.resources }}
...
annotations:
checksum/config: {{ include (print $.Template.BasePath "/secrets.yaml") . | sha256sum }}
{{ end }}
And in secrets.yaml:
apiVersion: "v1"
kind: "Secret"
metadata:
name: {{ $.Release.Name }}-secrets
I got this error:
Error: render error in "botfront-project/templates/deployment.yaml": template: [filename] :19:28: executing [filename] at <include (print $.Template.BasePath "/secrets.yaml") .>: error calling include: template: .../secrets.yaml:4:19: executing ".../secrets.yaml" at <$.Release.Name>: nil pointer evaluating interface {}.Name
How do I access variables inside an included template?
TL;DR;
just replace . with $ to use the global scope instead of the local one you created .
Example:
{{- include "my-chart.labels" $ | nindent 4 }}
Explanations
According to the docs, https://helm.sh/docs/chart_template_guide/control_structures/#modifying-scope-using-with:
we can use $ for accessing the object Release.Name from the parent
scope.
$ is mapped to the root scope when template execution begins
and it does not change during template execution
With range we change the scope inside the loop. Indeed, {{- include "my-chart.labels" . | nindent 4 }} would invoke the current scope ..
So if you dig into this "scope" thing in helm doc, you eventually find this part: https://helm.sh/docs/chart_template_guide/variables/
With this example:
{{- range .Values.tlsSecrets }}
apiVersion: v1
kind: Secret
metadata:
name: {{ .name }}
labels:
# Many helm templates would use `.` below, but that will not work,
# however `$` will work here
app.kubernetes.io/name: {{ template "fullname" $ }}
# I cannot reference .Chart.Name, but I can do $.Chart.Name
helm.sh/chart: "{{ $.Chart.Name }}-{{ $.Chart.Version }}"
app.kubernetes.io/instance: "{{ $.Release.Name }}"
# Value from appVersion in Chart.yaml
app.kubernetes.io/version: "{{ $.Chart.AppVersion }}"
app.kubernetes.io/managed-by: "{{ $.Release.Service }}"
type: kubernetes.io/tls
data:
tls.crt: {{ .certificate }}
tls.key: {{ .key }}
---
{{- end }}
I'd like to allow our developers to pass dynamic arguments to a helm template (Kubernetes job). Currently my arguments in the helm template are somewhat static (apart from certain values) and look like this
Args:
--arg1
value1
--arg2
value2
--sql-cmd
select * from db
If I were run a task using the docker container without Kubernetes, I would pass parameters like so:
docker run my-image --arg1 value1 --arg2 value2 --sql-cmd "select * from db"
Is there any way to templatize arguments in a helm chart in such way that any number of arguments could be passed to a template.
For example.
cat values.yaml
...
arguments: --arg1 value1 --arg2 value2 --sql-cmd "select * from db"
...
or
cat values.yaml
...
arguments: --arg3 value3
...
I've tried a few approaches but was not successful. Here is one example:
Args:
{{ range .Values.arguments }}
{{ . }}
{{ end }}
Yes. In values.yaml you need to give it an array instead of a space delimited string.
cat values.yaml
...
arguments: ['--arg3', 'value3', '--arg2', 'value2']
...
or
cat values.yaml
...
arguments:
- --arg3
- value3
- --arg2
- value2
...
and then you like you mentioned in the template should do it:
args:
{{ range .Values.arguments }}
- {{ . }}
{{ end }}
If you want to override the arguments on the command line you can pass an array with --set like this:
--set arguments={--arg1, value1, --arg2, value2, --arg3, value3, ....}
In your values file define arguments as:
extraArgs:
argument1: value1
argument2: value2
booleanArg1:
In your template do:
args:
{{- range $key, $value := .Values.extraArgs }}
{{- if $value }}
- --{{ $key }}={{ $value }}
{{- else }}
- --{{ $key }}
{{- end }}
{{- end }}
Rico's answer needed to be improved.
Using the previous example I've received errors like:
templates/deployment.yaml: error converting YAML to JSON: yaml or
failed to get versionedObject: unable to convert unstructured object to apps/v1beta2, Kind=Deployment: cannot restore slice from string
This is my working setup with coma in elements:
( the vertical format for the list is more readable )
cat values.yaml
...
arguments: [
"--arg3,",
"value3,",
"--arg2,",
"value2,",
]
...
in the template should do it:
args: [
{{ range .Values.arguments }}
{{ . }}
{{ end }}
]
because of some limitations, I had to work with split and to use a delimiter, so in my case:
deployment.yaml :
{{- if .Values.deployment.args }}
args:
{{- range (split " " .Values.deployment.args) }}
- {{ . }}
{{- end }}
{{- end }}
when use --set:
helm install --set deployment.args="--inspect server.js" ...
results with:
- args:
- --inspect
- server.js
The arguments format needs to be kept consistent in such cases.
Here is my case and it works fine.
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.app.name }}
labels:
app: {{ .Values.app.name }}
instance: test
spec:
replicas: {{ .Values.master.replicaCount }}
selector:
matchLabels:
app: {{ .Values.app.name }}
instance: test
template:
metadata:
labels:
app: {{ .Values.app.name }}
instance: test
spec:
imagePullSecrets:
- name: gcr-pull-secret
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.app.image }}
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
args:
[
"--users={{int .Values.cmd.users}}",
"--spawn-rate={{int .Values.cmd.rate}}",
"--host={{.Values.cmd.host}}",
"--logfile={{.Values.cmd.logfile}}",
"--{{.Values.cmd.role}}"]
ports:
- containerPort: {{ .Values.container.port }}
resources:
requests:
memory: {{ .Values.container.requests.memory }}
cpu: {{ .Values.container.requests.cpu }}
limits:
memory: {{ .Values.container.limits.memory }}
cpu: {{ .Values.container.limits.cpu }}
Unfortunately following mixed args format does not work within container construct -
mycommand -ArgA valA --ArgB valB --ArgBool1 -ArgBool2 --ArgC=valC
The correct format of above command expected is -
mycommand --ArgA=valA --ArgB=valB --ArgC=valC --ArgBool1 --ArgBool2
This can be achieved by following constructs -
#Dockerfile last line
ENTRYPOINT [mycommand]
#deployment.yaml
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.app.image }}
args: [
"--ArgA={{ .Values.cmd.ArgA }}",
"--ArgB={{ .Values.cmd.ArgB }}",
"--ArgC={{ .Values.cmd.ArgC }}",
"--{{ .Values.cmd.ArgBool1 }}",
"--{{ .Values.cmd.ArgBool2 }}" ]
#values.yaml
cmd:
ArgA: valA
ArgB: valB
ArgC: valC
ArgBool1: "ArgBool1"
ArgBool2: "ArgBool2"
helm install --name "airflow" stable/airflow --set secrets.database=mydatabase,secrets.password=mypassword
So this is the helm chart in question: https://github.com/helm/charts/tree/master/stable/airflow
Now I want to overwrite the default values in the helm chart
secrets.database and
secrets.password so I use --set argument and then it is key=value pairs separated by commas.
helm install --name "<name for your chart>" <chart> --set key0=value0,key1=value1,key2=value2,key3=value3
Did you try this?
{{ range .Values.arguments }}
{{ . | quote }}
{{ end }}
Acid R's key/value solution was the only thing that worked for me.
I ended up with this:
values.yaml
arguments:
url1: 'http://something1.example.com'
url2: 'http://something2.example.com'
url3: 'http://something3.example.com'
url4: 'http://something3.example.com'
And in my template:
args:
{{- range $key, $value := .Values.arguments }}
- --url={{ $value }}
{{- end }}