I am trying to override default values from -f .YAML, and it seems that there are some logic error or it is unsupported.
values.yaml
hive:
siteXml:
properties:
- name: xyz.catalog.credentials.provider.factory.class
# value: com.xyz.dataplatform.credentialproviders.xyz1CredentialsProvider
value: com.xyz.auth.profile.ProfileCredentialsProvider
overiding from .YAML values (failed to override).
helm template someinstall chartname -f myvalues.yaml
hive:
siteXml:
properties:
- name: xyz.catalog.credentials.provider.factory.class
value: com.xyz.someplatform.credentialproviders.xyz1CredentialsProvider
Configmap.yaml logic
{{- range $k, $v := .Values.hive.siteXml.properties }}
{{- if eq .name "xyz.catalog.credentials.provider.factory.class" }}
spark.hadoop.fs.s3a.xyz.credentials.provider={{ $v.value }}
{{- end }}
{{- end }}
Result:
spark.hadoop.fs.s3a.xyz.credentials.provider=com.xyz.auth.profile.ProfileCredentialsProvider
expected:
spark.hadoop.fs.s3a.xyz.credentials.provider=com.xyz.someplatform.credentialproviders.xyz1CredentialsProvider
The above code is working fine if Dict values located on 1 single file, if values are concatenated then helm grab the values from value.yaml.
hive:
siteXml:
properties:
- name: xyz.catalog.credentials.provider.factory.class
value: com.xyz.someplatform.credentialproviders.xyz1CredentialsProvider
- name: a
value: b
- name: c
value: d
We have some services that can be installed in multiple locations with differing configurations. We've been asked to support multi-level configuration options using environment variables set with defaults, configmaps, secrets, and command-line options passed in via helm install --set. The following works, but is very cumbersome as the number of parameters for some of the services are numerous and the Values dot-notation goes a few levels deeper.
env:
# Set default values
- name: MY_VAR
value: default-value
- name: OTHER_VAR
value: default-other-value
# Allow configmap to override
- name: MY_VAR
valueFrom:
configMapKeyRef:
name: env-configmap
key: MY_VAR
optional: true
- name: OTHER_VAR
valueFrom:
configMapKeyRef:
name: env-configmap
key: OTHER_VAR
optional: true
# Allow secrets to override
- name: MY_VAR
valueFrom:
secretsKeyRef:
name: env-secrets
key: MY_VAR
optional: true
- name: OTHER_VAR
valueFrom:
secretsKeyRef:
name: env-secrets
key: OTHER_VAR
optional: true
# Allow 'helm install --set' to override
{{- if .Values.env }}
{{- if .Values.env.my }}
{{- if .Values.env.my.var }}
- name: MY_VAR
value: {{ .Values.env.my.var }}
{{- end }}
{{- end }}
{{- if .Values.env.other }}
{{- if .Values.env.other.var }}
- name: OTHER_VAR
value: {{ .Values.env.other.var }}
{{- end }}
{{- end }}
{{- end }}
Using envFrom for the ConfigMap and Secrets would be nice, but tests and docs show this would not allow the command-line override, since env: and envFrom: doesn't mix in the way that's needed. As the v1.9 and v2.1 Kubernetes API states:
envFrom: List of sources to populate environment variables in the
container. The keys defined within a source must be a C_IDENTIFIER.
All invalid keys will be reported as an event when the container is
starting. When a key exists in multiple sources, the value associated
with the last source will take precedence. Values defined by an Env
with a duplicate key will take precedence. Cannot be updated.
Is there a better way to provide this default->configmap->secrets->cmd-line override precedence?
I found a solution that I mostly like. My issue was caused by giving too much weight to the "Values defined by an Env with a duplicate key will take precedence" comment in the docs, and thinking I needed to exclusively use Env. The defined precedence is exactly what I needed.
Here's the helm chart files for my current solution.
configmap/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
data:
{{- if .Values.env }}
{{- toYaml $.Values.env | nindent 2 }}
{{- end }}
{{- if .Values.applicationYaml }}
application.yml: |
{{- toYaml $.Values.applicationYaml | nindent 4 }}
{{- end }}
secrets/templates/secrets.yaml
apiVersion: v1
kind: Secret
metadata:
name: my-secrets
type: Opaque
data:
{{- range $key, $val := .Values.env }}
{{ $key }}: {{ $val | b64enc }}
{{- end }}
stringData:
{{- if .Values.applicationYaml }}
application.yml: |
{{- toYaml $.Values.applicationYaml | nindent 4 }}
{{- end }}
deployment.yaml
apiVersion: apps/v1
kind: Deployment
...
spec:
...
containers:
- name: my-deployment
{{- if .Values.env }}
env:
{{- range $key, $val := .Values.env }}
- name: {{ $key }}
value: {{ $val }}
{{- end }}
{{- end }}
envFrom:
- configMapRef:
name: my-configmap
- secretRef:
name: my-secrets
volumeMounts:
- name: configmap-application-config
mountPath: /application/config/configmap/
- name: secrets-application-config
mountPath: /application/config/secrets/
volumes:
- name: configmap-application-config
configMap:
name: my-configmap
optional: true
- name: secrets-application-config
secret:
secretName: my-secrets
optional: true
Since this is a Spring Boot app, I used volumeMounts to allow the application.yml default values to be overridden in the ConfigMap and Secrets. The order of precedence from lowest to highest is:
the application's application.yml (v1 in following examples)
the configmap's applicationYaml (v2)
the secret's applicationYaml (v3)
the configmap env (v4)
the secret env (v5)
the helm install/uninstall --set (v6)
To complete the example, here's test values yaml files and the command-line.
app/src/main/resources/application.yml
applicationYaml:
test:
v1: set-from-this-value
v2: overridden
v3: overridden
v4: overridden
v5: overridden
v6: overridden
configmap/values.yaml
applicationYaml:
test:
v2: set-from-this-value
v3: overridden
v4: overridden
v5: overridden
v6: overridden
env:
TEST_V4: set-from-this-value
TEST_V5: overridden
TEST_V6: overridden
secrets/values.yaml
applicationYaml:
test:
v3: set-from-this-value
v4: overridden
v5: overridden
v6: overridden
env:
TEST_V5: set-from-this-value
TEST_V6: overridden
command-line
helm install --set env.TEST_V6=set-from-this-value ...
Ideally, I'd like to be able to use dot-notation instead of TEST_V6 in the env and --set fields, but I'm not finding a way in helm to operate only on the leaves of yaml. In other words, I'd like something like range $key, $val, but where the key is equal to "test.v6". If that was possible, the key could be internally converted to an environment variable name with {{ $key | upper | replace "-" "_" | replace "." "_" }}.
I would like to use variables (vars.yaml) to replace the content in a config file (config.yaml) and automatise the process in the playbook (playbook.yaml). But I can't found a type of process that automaticly search for vars inside an external file through the playbook.yaml.
My files look like this :
vars.yaml
---
var1: content1
var2: content2
var3: content3
config.yaml:
apiVersion: 1
datasources:
- name: {{ var1 }}
type: {{ var2 }}
access: proxy
url: {{ var3 }}
playbook.yaml:
---
- name: Deploy
hosts: localhost
tasks:
- include_vars: ./vars.yaml
- name: Update config file
# Something to replace all vars in the config file using the content of vars.yaml
Can you help me and tell me if there is something possible instead of doing a regex for each variable ...?
The file with the unresolved variables is actually a template. Let's name it this way
shell> cat config.yaml.j2
apiVersion: 1
datasources:
- name: {{ var1 }}
type: {{ var2 }}
access: proxy
url: {{ var3 }}
Use the template module to update config.yaml, e.g.
- include_vars: vars.yaml
- name: Update config file
template:
src: config.yaml.j2
dest: config.yaml
gives
shell> cat config.yaml
apiVersion: 1
datasources:
- name: content1
type: content2
access: proxy
url: content3
In the helm-template I'm trying to retrieve a value of the map by key.
I've tried to use the index from the go-templates, as suggested here:
Access a map value using a variable key in a Go template
However it doesn't work for me (see later test). Any idea for the alternative solution?
Chart.yaml:
apiVersion: v1
appVersion: "1.0"
description: A Helm chart for Kubernetes
name: foochart
version: 0.1.0
values.yaml:
label:
- name: foo
value: foo1
- name: bar
value: bar2
templates/test.txt
label: {{ .Values.label }}
Works OK for helm template .:
---
# Source: foochart/templates/test.txt
label: [map[value:foo1 name:foo] map[name:bar value:bar2]]
However once trying to use the index:
templates/test.txt
label: {{ .Values.label }}
foolabel: {{ index .Values.label "foo" }}
It won't work - helm template .:
Error: render error in "foochart/templates/test.txt": template: foochart/templates/test.txt:2:13: executing "foochart/templates/test.txt" at <index .Values.label ...>: error calling index: cannot index slice/array with type string
label is an array, so the index function will only work with integers, this is a working example:
foolabel: {{ index .Values.label 0 }}
The 0 selects the first element of the array.
A better option is to avoid using an array and replace it with a map:
label:
foo:
name: foo
value: foo1
bar:
name: bar
value: bar2
And you dont even need the index function:
foolabel: {{ .Values.label.foo }}
values.yaml
coins:
ether:
host: 10.11.0.50
port: 123
btc:
host: 10.11.0.10
port: 321
template.yaml
{{- range $key, $val := .Values.coins }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ $key }}
- env:
- name: SSH_HOSTNAME
value: {{ $val.host | quote }}
- name: SSH_TUNNEL_HOST
value: {{ $val.port | quote }}
---
{{- end }}
run $ helm template ./helm
---
# Source: test/templates/ether.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: btc
- env:
- name: SSH_HOSTNAME
value: "10.11.0.10"
- name: SSH_TUNNEL_HOST
value: "321"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ether
- env:
- name: SSH_HOSTNAME
value: "10.11.0.50"
- name: SSH_TUNNEL_HOST
value: "123"
---
I'd like to allow our developers to pass dynamic arguments to a helm template (Kubernetes job). Currently my arguments in the helm template are somewhat static (apart from certain values) and look like this
Args:
--arg1
value1
--arg2
value2
--sql-cmd
select * from db
If I were run a task using the docker container without Kubernetes, I would pass parameters like so:
docker run my-image --arg1 value1 --arg2 value2 --sql-cmd "select * from db"
Is there any way to templatize arguments in a helm chart in such way that any number of arguments could be passed to a template.
For example.
cat values.yaml
...
arguments: --arg1 value1 --arg2 value2 --sql-cmd "select * from db"
...
or
cat values.yaml
...
arguments: --arg3 value3
...
I've tried a few approaches but was not successful. Here is one example:
Args:
{{ range .Values.arguments }}
{{ . }}
{{ end }}
Yes. In values.yaml you need to give it an array instead of a space delimited string.
cat values.yaml
...
arguments: ['--arg3', 'value3', '--arg2', 'value2']
...
or
cat values.yaml
...
arguments:
- --arg3
- value3
- --arg2
- value2
...
and then you like you mentioned in the template should do it:
args:
{{ range .Values.arguments }}
- {{ . }}
{{ end }}
If you want to override the arguments on the command line you can pass an array with --set like this:
--set arguments={--arg1, value1, --arg2, value2, --arg3, value3, ....}
In your values file define arguments as:
extraArgs:
argument1: value1
argument2: value2
booleanArg1:
In your template do:
args:
{{- range $key, $value := .Values.extraArgs }}
{{- if $value }}
- --{{ $key }}={{ $value }}
{{- else }}
- --{{ $key }}
{{- end }}
{{- end }}
Rico's answer needed to be improved.
Using the previous example I've received errors like:
templates/deployment.yaml: error converting YAML to JSON: yaml or
failed to get versionedObject: unable to convert unstructured object to apps/v1beta2, Kind=Deployment: cannot restore slice from string
This is my working setup with coma in elements:
( the vertical format for the list is more readable )
cat values.yaml
...
arguments: [
"--arg3,",
"value3,",
"--arg2,",
"value2,",
]
...
in the template should do it:
args: [
{{ range .Values.arguments }}
{{ . }}
{{ end }}
]
because of some limitations, I had to work with split and to use a delimiter, so in my case:
deployment.yaml :
{{- if .Values.deployment.args }}
args:
{{- range (split " " .Values.deployment.args) }}
- {{ . }}
{{- end }}
{{- end }}
when use --set:
helm install --set deployment.args="--inspect server.js" ...
results with:
- args:
- --inspect
- server.js
The arguments format needs to be kept consistent in such cases.
Here is my case and it works fine.
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.app.name }}
labels:
app: {{ .Values.app.name }}
instance: test
spec:
replicas: {{ .Values.master.replicaCount }}
selector:
matchLabels:
app: {{ .Values.app.name }}
instance: test
template:
metadata:
labels:
app: {{ .Values.app.name }}
instance: test
spec:
imagePullSecrets:
- name: gcr-pull-secret
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.app.image }}
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
args:
[
"--users={{int .Values.cmd.users}}",
"--spawn-rate={{int .Values.cmd.rate}}",
"--host={{.Values.cmd.host}}",
"--logfile={{.Values.cmd.logfile}}",
"--{{.Values.cmd.role}}"]
ports:
- containerPort: {{ .Values.container.port }}
resources:
requests:
memory: {{ .Values.container.requests.memory }}
cpu: {{ .Values.container.requests.cpu }}
limits:
memory: {{ .Values.container.limits.memory }}
cpu: {{ .Values.container.limits.cpu }}
Unfortunately following mixed args format does not work within container construct -
mycommand -ArgA valA --ArgB valB --ArgBool1 -ArgBool2 --ArgC=valC
The correct format of above command expected is -
mycommand --ArgA=valA --ArgB=valB --ArgC=valC --ArgBool1 --ArgBool2
This can be achieved by following constructs -
#Dockerfile last line
ENTRYPOINT [mycommand]
#deployment.yaml
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.app.image }}
args: [
"--ArgA={{ .Values.cmd.ArgA }}",
"--ArgB={{ .Values.cmd.ArgB }}",
"--ArgC={{ .Values.cmd.ArgC }}",
"--{{ .Values.cmd.ArgBool1 }}",
"--{{ .Values.cmd.ArgBool2 }}" ]
#values.yaml
cmd:
ArgA: valA
ArgB: valB
ArgC: valC
ArgBool1: "ArgBool1"
ArgBool2: "ArgBool2"
helm install --name "airflow" stable/airflow --set secrets.database=mydatabase,secrets.password=mypassword
So this is the helm chart in question: https://github.com/helm/charts/tree/master/stable/airflow
Now I want to overwrite the default values in the helm chart
secrets.database and
secrets.password so I use --set argument and then it is key=value pairs separated by commas.
helm install --name "<name for your chart>" <chart> --set key0=value0,key1=value1,key2=value2,key3=value3
Did you try this?
{{ range .Values.arguments }}
{{ . | quote }}
{{ end }}
Acid R's key/value solution was the only thing that worked for me.
I ended up with this:
values.yaml
arguments:
url1: 'http://something1.example.com'
url2: 'http://something2.example.com'
url3: 'http://something3.example.com'
url4: 'http://something3.example.com'
And in my template:
args:
{{- range $key, $value := .Values.arguments }}
- --url={{ $value }}
{{- end }}