I am trying to override default values from -f .YAML, and it seems that there are some logic error or it is unsupported.
values.yaml
hive:
siteXml:
properties:
- name: xyz.catalog.credentials.provider.factory.class
# value: com.xyz.dataplatform.credentialproviders.xyz1CredentialsProvider
value: com.xyz.auth.profile.ProfileCredentialsProvider
overiding from .YAML values (failed to override).
helm template someinstall chartname -f myvalues.yaml
hive:
siteXml:
properties:
- name: xyz.catalog.credentials.provider.factory.class
value: com.xyz.someplatform.credentialproviders.xyz1CredentialsProvider
Configmap.yaml logic
{{- range $k, $v := .Values.hive.siteXml.properties }}
{{- if eq .name "xyz.catalog.credentials.provider.factory.class" }}
spark.hadoop.fs.s3a.xyz.credentials.provider={{ $v.value }}
{{- end }}
{{- end }}
Result:
spark.hadoop.fs.s3a.xyz.credentials.provider=com.xyz.auth.profile.ProfileCredentialsProvider
expected:
spark.hadoop.fs.s3a.xyz.credentials.provider=com.xyz.someplatform.credentialproviders.xyz1CredentialsProvider
The above code is working fine if Dict values located on 1 single file, if values are concatenated then helm grab the values from value.yaml.
hive:
siteXml:
properties:
- name: xyz.catalog.credentials.provider.factory.class
value: com.xyz.someplatform.credentialproviders.xyz1CredentialsProvider
- name: a
value: b
- name: c
value: d
Related
We have some services that can be installed in multiple locations with differing configurations. We've been asked to support multi-level configuration options using environment variables set with defaults, configmaps, secrets, and command-line options passed in via helm install --set. The following works, but is very cumbersome as the number of parameters for some of the services are numerous and the Values dot-notation goes a few levels deeper.
env:
# Set default values
- name: MY_VAR
value: default-value
- name: OTHER_VAR
value: default-other-value
# Allow configmap to override
- name: MY_VAR
valueFrom:
configMapKeyRef:
name: env-configmap
key: MY_VAR
optional: true
- name: OTHER_VAR
valueFrom:
configMapKeyRef:
name: env-configmap
key: OTHER_VAR
optional: true
# Allow secrets to override
- name: MY_VAR
valueFrom:
secretsKeyRef:
name: env-secrets
key: MY_VAR
optional: true
- name: OTHER_VAR
valueFrom:
secretsKeyRef:
name: env-secrets
key: OTHER_VAR
optional: true
# Allow 'helm install --set' to override
{{- if .Values.env }}
{{- if .Values.env.my }}
{{- if .Values.env.my.var }}
- name: MY_VAR
value: {{ .Values.env.my.var }}
{{- end }}
{{- end }}
{{- if .Values.env.other }}
{{- if .Values.env.other.var }}
- name: OTHER_VAR
value: {{ .Values.env.other.var }}
{{- end }}
{{- end }}
{{- end }}
Using envFrom for the ConfigMap and Secrets would be nice, but tests and docs show this would not allow the command-line override, since env: and envFrom: doesn't mix in the way that's needed. As the v1.9 and v2.1 Kubernetes API states:
envFrom: List of sources to populate environment variables in the
container. The keys defined within a source must be a C_IDENTIFIER.
All invalid keys will be reported as an event when the container is
starting. When a key exists in multiple sources, the value associated
with the last source will take precedence. Values defined by an Env
with a duplicate key will take precedence. Cannot be updated.
Is there a better way to provide this default->configmap->secrets->cmd-line override precedence?
I found a solution that I mostly like. My issue was caused by giving too much weight to the "Values defined by an Env with a duplicate key will take precedence" comment in the docs, and thinking I needed to exclusively use Env. The defined precedence is exactly what I needed.
Here's the helm chart files for my current solution.
configmap/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
data:
{{- if .Values.env }}
{{- toYaml $.Values.env | nindent 2 }}
{{- end }}
{{- if .Values.applicationYaml }}
application.yml: |
{{- toYaml $.Values.applicationYaml | nindent 4 }}
{{- end }}
secrets/templates/secrets.yaml
apiVersion: v1
kind: Secret
metadata:
name: my-secrets
type: Opaque
data:
{{- range $key, $val := .Values.env }}
{{ $key }}: {{ $val | b64enc }}
{{- end }}
stringData:
{{- if .Values.applicationYaml }}
application.yml: |
{{- toYaml $.Values.applicationYaml | nindent 4 }}
{{- end }}
deployment.yaml
apiVersion: apps/v1
kind: Deployment
...
spec:
...
containers:
- name: my-deployment
{{- if .Values.env }}
env:
{{- range $key, $val := .Values.env }}
- name: {{ $key }}
value: {{ $val }}
{{- end }}
{{- end }}
envFrom:
- configMapRef:
name: my-configmap
- secretRef:
name: my-secrets
volumeMounts:
- name: configmap-application-config
mountPath: /application/config/configmap/
- name: secrets-application-config
mountPath: /application/config/secrets/
volumes:
- name: configmap-application-config
configMap:
name: my-configmap
optional: true
- name: secrets-application-config
secret:
secretName: my-secrets
optional: true
Since this is a Spring Boot app, I used volumeMounts to allow the application.yml default values to be overridden in the ConfigMap and Secrets. The order of precedence from lowest to highest is:
the application's application.yml (v1 in following examples)
the configmap's applicationYaml (v2)
the secret's applicationYaml (v3)
the configmap env (v4)
the secret env (v5)
the helm install/uninstall --set (v6)
To complete the example, here's test values yaml files and the command-line.
app/src/main/resources/application.yml
applicationYaml:
test:
v1: set-from-this-value
v2: overridden
v3: overridden
v4: overridden
v5: overridden
v6: overridden
configmap/values.yaml
applicationYaml:
test:
v2: set-from-this-value
v3: overridden
v4: overridden
v5: overridden
v6: overridden
env:
TEST_V4: set-from-this-value
TEST_V5: overridden
TEST_V6: overridden
secrets/values.yaml
applicationYaml:
test:
v3: set-from-this-value
v4: overridden
v5: overridden
v6: overridden
env:
TEST_V5: set-from-this-value
TEST_V6: overridden
command-line
helm install --set env.TEST_V6=set-from-this-value ...
Ideally, I'd like to be able to use dot-notation instead of TEST_V6 in the env and --set fields, but I'm not finding a way in helm to operate only on the leaves of yaml. In other words, I'd like something like range $key, $val, but where the key is equal to "test.v6". If that was possible, the key could be internally converted to an environment variable name with {{ $key | upper | replace "-" "_" | replace "." "_" }}.
I have the following values file:
MYVAR: 12123
MYVAR2: 214123
I want to iterate over them and use them as env variables in my deployment template:
env:
{{- range .Values.examplemap }}
- name: {{ .name }}
value: {{ .value }}
{{- end }}
I tried this
For iterate over a map in helm you can try put this in the values.yaml
extraEnvs:
- name: ENV_NAME_1
value: value123
- name: ENV_NAME_2
value: value123
So in your template you must iterate the extraEnvs like this:
extraEnvs:
{{- range .Values.image.extraEnvs }}
- name: {{ .name | quote }}
value: {{ .value | quote }}
{{- end }}
In the core Go text/template language, the range operator can iterate over either a list or a map. There's specific syntax to assign the key-value pairs in a map to local variables:
env:
{{- $k, $v := range .Values.examplemap }}
- name: {{ $k }}
value: {{ $v }}
{{- end }}
I'm trying to iterate over list of secrets in my values files and mount it as env variables on the pods but having hard time with it.
The helm template command which spits the sample template lists the keys & value as I expect it to be but when i deploy only the "name" key gets mounted on the pods but nothing after that like the valuefrom and secretkeyref. Really appreciate any help on this :)
**DEPLOYMENT YAML**
{{- if $root.Values.secrets }}
{{- range $secrets := $root.Values.secrets }}
{{- range $data := $secrets.data }}
- name: {{ $data.name }}
valuefrom:
secretKeyRef:
name: {{ $secrets.name }}
key: {{ $data.name }}
{{ end }}
{{ end }}
{{ end }}
secrets:
- name: aws-secrets
data:
- key: "secret1"
name: DATABASE_HOST
- key: "secret2"
name: DATABASE_NAME
I fat fingered should be "valueFrom". 🤦
I'm trying to template variables from a map inside the values.yaml into my final Kubernetes ConfigMap YAML.
I've read through https://github.com/helm/helm/issues/2492 and https://helm.sh/docs/chart_template_guide/ but can't seem to find an answer.
For some context, this is roughly what I'm trying to do:
values.yaml
config:
key1: value
key2: value-{{ .Release.Name }}
configmap.yaml
kind: ConfigMap
data:
config-file: |
{{- range $key, $value := .Values.config }}
{{ $key }} = {{ $value }}
{{- end }}
Where the desired output with would be:
helm template --name v1 mychart/
kind: ConfigMap
data:
config-file: |
key1 = value
key2 = value-v1
I've tried a few variations using template functions and pipelining, but to no avail:
{{ $key }} = {{ tpl $value . }}
{{ $key }} = {{ $value | tpl . }}
{{ $key }} = {{ tpl $value $ }}
The above would also have worked in this way
values.yaml
config:
key1: "value"
key2: "value-{{ .Release.Name }}"
configmap.yaml
kind: ConfigMap
data:
config-file: |
{{- range $key, $value := .Values.config }}
{{ $key }} = {{ tpl $value $ }}
{{- end }}
What I changed was : I put value in quotes in value.yaml and used template tpl in the config map.
I'll refer to the question's title regarding templating variables in helm and suggest another option to use on values.yaml which is YAML Anchors.
Docs reference
As written in here:
The YAML spec provides a way to store a reference to a value, and
later refer to that value by reference. YAML refers to this as
"anchoring":
coffee: "yes, please"
favorite: &favoriteCoffee "Cappucino"
coffees:
- Latte
- *favoriteCoffee
- Espresso
In the above, &favoriteCoffee sets a reference to Cappuccino.
Later, that reference is used as *favoriteCoffee.
So coffees becomes Latte, Cappuccino, Espresso.
A more practical example
Referring to a common image setup (Registry and PullPolicy) in all values.yaml.
Notice how the default values are being set at Global.Image next to the reference definition which starts with &:
Global:
Image:
Registry: &global-docker-registry "12345678910.dkr.ecr.us-west-2.amazonaws.com" # <--- Default value
PullPolicy: &global-pull-policy "IfNotPresent" # <--- Default value
Nginx:
Image:
Registry: *global-docker-registry
PullPolicy: *global-pull-policy
Version: 1.21.4
Port: 80
MySql:
Image:
Registry: *global-docker-registry
PullPolicy: *global-pull-policy
Name: mysql
Version: 8.0.27
Port: 3306
Managed to solve this using the following syntax:
configmap.yaml
kind: ConfigMap
data:
config-file: |
{{- range $key, $value := .Values.config }}
{{ $key }} = {{ tpl ($value | toString) $ }}
{{- end }}
there is fight in this PR here about this topic.
I know that it's possible now, but this require maintenance of the chart to be in-house (e.g. answer of Amrut ).
Let's summarize :
To have templating in values.yaml , these are the available options:
helm may support that in future ( watch this thread about this topic.)
use tpl function inside the chart
use another tool on top of helm : terraform or helmfile.
I'd like to allow our developers to pass dynamic arguments to a helm template (Kubernetes job). Currently my arguments in the helm template are somewhat static (apart from certain values) and look like this
Args:
--arg1
value1
--arg2
value2
--sql-cmd
select * from db
If I were run a task using the docker container without Kubernetes, I would pass parameters like so:
docker run my-image --arg1 value1 --arg2 value2 --sql-cmd "select * from db"
Is there any way to templatize arguments in a helm chart in such way that any number of arguments could be passed to a template.
For example.
cat values.yaml
...
arguments: --arg1 value1 --arg2 value2 --sql-cmd "select * from db"
...
or
cat values.yaml
...
arguments: --arg3 value3
...
I've tried a few approaches but was not successful. Here is one example:
Args:
{{ range .Values.arguments }}
{{ . }}
{{ end }}
Yes. In values.yaml you need to give it an array instead of a space delimited string.
cat values.yaml
...
arguments: ['--arg3', 'value3', '--arg2', 'value2']
...
or
cat values.yaml
...
arguments:
- --arg3
- value3
- --arg2
- value2
...
and then you like you mentioned in the template should do it:
args:
{{ range .Values.arguments }}
- {{ . }}
{{ end }}
If you want to override the arguments on the command line you can pass an array with --set like this:
--set arguments={--arg1, value1, --arg2, value2, --arg3, value3, ....}
In your values file define arguments as:
extraArgs:
argument1: value1
argument2: value2
booleanArg1:
In your template do:
args:
{{- range $key, $value := .Values.extraArgs }}
{{- if $value }}
- --{{ $key }}={{ $value }}
{{- else }}
- --{{ $key }}
{{- end }}
{{- end }}
Rico's answer needed to be improved.
Using the previous example I've received errors like:
templates/deployment.yaml: error converting YAML to JSON: yaml or
failed to get versionedObject: unable to convert unstructured object to apps/v1beta2, Kind=Deployment: cannot restore slice from string
This is my working setup with coma in elements:
( the vertical format for the list is more readable )
cat values.yaml
...
arguments: [
"--arg3,",
"value3,",
"--arg2,",
"value2,",
]
...
in the template should do it:
args: [
{{ range .Values.arguments }}
{{ . }}
{{ end }}
]
because of some limitations, I had to work with split and to use a delimiter, so in my case:
deployment.yaml :
{{- if .Values.deployment.args }}
args:
{{- range (split " " .Values.deployment.args) }}
- {{ . }}
{{- end }}
{{- end }}
when use --set:
helm install --set deployment.args="--inspect server.js" ...
results with:
- args:
- --inspect
- server.js
The arguments format needs to be kept consistent in such cases.
Here is my case and it works fine.
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.app.name }}
labels:
app: {{ .Values.app.name }}
instance: test
spec:
replicas: {{ .Values.master.replicaCount }}
selector:
matchLabels:
app: {{ .Values.app.name }}
instance: test
template:
metadata:
labels:
app: {{ .Values.app.name }}
instance: test
spec:
imagePullSecrets:
- name: gcr-pull-secret
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.app.image }}
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
args:
[
"--users={{int .Values.cmd.users}}",
"--spawn-rate={{int .Values.cmd.rate}}",
"--host={{.Values.cmd.host}}",
"--logfile={{.Values.cmd.logfile}}",
"--{{.Values.cmd.role}}"]
ports:
- containerPort: {{ .Values.container.port }}
resources:
requests:
memory: {{ .Values.container.requests.memory }}
cpu: {{ .Values.container.requests.cpu }}
limits:
memory: {{ .Values.container.limits.memory }}
cpu: {{ .Values.container.limits.cpu }}
Unfortunately following mixed args format does not work within container construct -
mycommand -ArgA valA --ArgB valB --ArgBool1 -ArgBool2 --ArgC=valC
The correct format of above command expected is -
mycommand --ArgA=valA --ArgB=valB --ArgC=valC --ArgBool1 --ArgBool2
This can be achieved by following constructs -
#Dockerfile last line
ENTRYPOINT [mycommand]
#deployment.yaml
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.app.image }}
args: [
"--ArgA={{ .Values.cmd.ArgA }}",
"--ArgB={{ .Values.cmd.ArgB }}",
"--ArgC={{ .Values.cmd.ArgC }}",
"--{{ .Values.cmd.ArgBool1 }}",
"--{{ .Values.cmd.ArgBool2 }}" ]
#values.yaml
cmd:
ArgA: valA
ArgB: valB
ArgC: valC
ArgBool1: "ArgBool1"
ArgBool2: "ArgBool2"
helm install --name "airflow" stable/airflow --set secrets.database=mydatabase,secrets.password=mypassword
So this is the helm chart in question: https://github.com/helm/charts/tree/master/stable/airflow
Now I want to overwrite the default values in the helm chart
secrets.database and
secrets.password so I use --set argument and then it is key=value pairs separated by commas.
helm install --name "<name for your chart>" <chart> --set key0=value0,key1=value1,key2=value2,key3=value3
Did you try this?
{{ range .Values.arguments }}
{{ . | quote }}
{{ end }}
Acid R's key/value solution was the only thing that worked for me.
I ended up with this:
values.yaml
arguments:
url1: 'http://something1.example.com'
url2: 'http://something2.example.com'
url3: 'http://something3.example.com'
url4: 'http://something3.example.com'
And in my template:
args:
{{- range $key, $value := .Values.arguments }}
- --url={{ $value }}
{{- end }}