Helm variables used at custom places - kubernetes

I am not able to reference variable inside a nested variable in Helm. Also, I am not able to do this as nested reference. I want to retrieve all client variables under client name using the value of the client_name variable. How can I do that?
Values.yaml
clients:
client1:
incomigBucket: databucket
outgoingBucket: tempbucket
webURL: http://example.com
client2:
incomingBucket: databucket
outgoingBucket: tempbucket
webURL: http://example.com
I want to store client variables values in one variable and want to use it at different places in my Json file. if I use range function then it create section twice(as I have mentioned 2 clients), is there any thing in Helm I can use which can store these variables dynamically and use it in custom places in json file?
Sample File Section:
"FileConfig": {
"Client1": {
"incomingLocationPath": "s3://{{ .Values.clients.client1.incomingBucket }}/dir1/dir2",
"outgoingLocationPath": "s3://{{ .Values.clients.client1.outgoingBucket }}/dir1/dir2",
},
"Client2": {
"incomingLocationPath": "s3://{{ .Values.clients.client2.incomingBucket }}/dir1/dir2",
"outgoingLocationPath": "s3://{{ .Values.clients.client2.outgoingBucket }}/dir1/dir2",
}
}

I am not entirely sure where you intend to use this snippet. I assume it is inside a configMap. At least, I am not aware of any resource that had FileConfig section. I wouldn't know why it should be JSON either.
The basic pattern could look like this.
fileConfig:
{{- range $client, $config := .Values.clients }}
{{ $client }}:
incomingLocationPath: "s3://{{ $config.incomingBucket }}/dir1/dir2"
outgoingLocationPath: "s3://{{ $config.outgoingBucket }}/dir1/dir2"
{{- end }}
In something like a configMap to create JSON it could look like this.
kind: ConfigMap
apiVersion: v1
metadata:
name: file-config-json
data:
config.json: |
"fileConfig": {
{{- range $client, $config := .Values.clients }}
"{{ $client }}": {
"incomingLocationPath": "s3://{{ $config.incomingBucket }}/dir1/dir2",
"outgoingLocationPath": "s3://{{ $config.outgoingBucket }}/dir1/dir2"
}
{{- end }}
}

Related

When including a template in another template, from an ansible role, what is the path?

I am attempting to use templates to seed configurations for apps in an ansible role. The role will configure some related services.
Template 1 - logging_config.json.j2
{
"logger_type": "foo_bar",
"logger_host": "https://fo_host",
"logger_token": "{{ secret_vault_logger_token }}"
}
Template 2 - app1_config.json.j2
{
"api_url_foo": "{{ app1_foo_api_url }}",
"logger": {% include "logging_config.json.j2" %}
}
Template 3 - xapp2_config.json.j2x
{
"port": {{ app2_port }},
"logger": {% include "logging_config.json.j2" %}
}
These are in a role - say roles/my_apps/templates. The role names templates app2_config.json.j2 or app1_config.json.j2 using the k8s module.
I always see this:
fatal: [k8s-non-devops]: FAILED! => {
"changed": false,
"msg": "TemplateNotFound: logging_config.json.j2"
}
In that path I have tried: templates/logging_config.json.j2, myapps/templates/logging_config.json.j2, roles/myapps/templates/logging_config.json.j2.
The variables template_path and template_fullpath are undefined.
Where is it?
In this instance, the j2 {% include %} syntax won't work. The paths don't appear to make sense. However, the ansible lookup plugin will do the job:
{
"port": {{ app2_port }},
"logger": {{ lookup('template', 'logging_config.json.j2' ) }}
}
This however come with a slight cost, ansible seems to recognise and alter the json, turning double quotes into single quotes and making it invalid. So if the config is json, you need to add | to_json:
{
"port": {{ app2_port }},
"logger": {{ lookup('template', 'logging_config.json.j2' ) | to_json }}
}

How to add spaces in helm set switch in kubernetes

How can I add space in --set switch with helm command.
I need spaces between "mongodb://datadog:'${DB_PASSWORD}'#%%host%%:%%port%%", and "replica_check": true, and also between "replica_check": true, and "additional_metrics": ["metrics.commands","tcmalloc","top","collection"]}]'
It gives me error as :
that set cannot end with ,
helm upgrade myservice helm/charts/myservice \
-f helm/charts/chat/values.yaml \
--set "mongodb-replicaset.podAnnotations.ad\.datadoghq\.com/mongodb-replicaset\.instances"='[{"server": "mongodb://datadog:'${DB_PASSWORD}'#%%host%%:%%port%%", "replica_check": true, "additional_metrics": ["metrics.commands","tcmalloc","top","collection"]}]' \
--wait --install```
If you're trying to pass something really complicated like that, it might be easier to package it up in a YAML file and pass it with the helm upgrade -f option.
mongodb-replicaset.podAnnotations.ad.datadoghq.com/mongodb-replicaset.instances: >-
[
{
"server": "mongodb://datadog:passw0rd#%%host%%:%%port%%",
"replica_check": true,
"additional_metrics": ["metrics.commands","tcmalloc","top","collection"]
}
]
(The >- marker is YAML syntax causing the following indented block to be processed as a string, folding newlines into spaces, and trimming leading and trailing newlines.)
Since valid JSON is valid YAML, if you're trying to launch this from a script, you could use your language-native JSON serializer.
import json, os
db_password = os.environ.get('DB_PASSWORD')
instances = [{
"server": "mongodb://datadog:" + db_password + "#%%host:%%port",
"replica_check": True,
"additional_metrics": ["metrics.commands", "tcmalloc", "top", "collection"]
}]
annotations = {
"mongodb-replicaset.podAnnotations.ad.datadoghq.com/mongodb-replicaset.instances": json.dumps(instances)
}
print(json.dumps(annotations))
Once you've done this you can pass the generated file to the helm upgrade -f option. (You don't need to include the chart's own values.yaml file, Helm includes this by default and your values override it.)
helm upgrade myservice helm/charts/myservice \
-f mongo-dd.yaml \
--wait --install
Yet another option is to generate this inside the chart itself. Helm includes an undocumented toJson template function; you can construct objects using the Sprig dict function.
{{- $server := printf "mongodb://datadog:%s#%%%%host%%%%:%%%%port%%%%" .Values.mongodbPassword -}}
{{- $metrics := list "metrics.commands" "tcmalloc" "top" "collection" -}}
{{- $instance := dict "server" $server "replica_check" true "additional_metrics" $metrics -}}
{{- $instances := list $instance -}}
mongodb-replicaset...instances: {{ toJson $instances }}
Your error message suggests your problem isn't the spaces per se, but that helm --set treats commas as separators between multiple key/value pairs. You can backslash-escape the commas
... --set 'mongodb-replicaset..."replica_check": true\, "additional_metrics": ["metrics.commands\,"tcmalloc"\,...]}]'

Writing to Vault secret with consul-template

Im struggling to understand from the doc on how to write to Vault KV using consul-template.
I enabled KV with vault secrets enable -path=secret -version=2 kv
I have the following template,
vault {
ssl {
ca_cert = "tls/ca.pem"
}
retry {
backoff = "1s"
}
}
template {
contents = <<EOH
---
{{ secret "secret/data/test/admin" "value=test" }}
EOH
}
However I keep getting,
2019/08/08 22:28:43.201250 [WARN] (view) vault.write(secret/data/test/admin/password -> 2b955093): vault.write(secret/data/test/admin/password -> 2b955093): Error making API request.
URL: PUT http://<vault address>/v1/secret/data/test/admin
Code: 400. Errors:
* no data provided (retry attempt 2 after "500ms")
I have figured out how to read from the store as its well documented but not writing to it. Any help is appreciated
This has been raised as a bug in #1252.
Experiencing the same issue with secret updates.
The cert update works as expected.
{{ with secret "pki/issue/exampledotcom" "common_name=example.com"}}
{{ .Data.certificate }}
{{ end }}

Get array of strings from helm config

Ultimately i'm trying to get an array of strings e.g. ['foo', 'bar'] in my js app from my helm config.
./vars/dev/organizations.yaml
...
organizations:
- 'foo'
- 'bar'
...
./templates/configmap.yaml
...
data:
organizations.yaml: |
organizations: "{{ toYaml .Values.organizations | indent 4 }}"
...
./templates/deployment.yaml
...
containers:
args:
- "--organizations-config"
- "/etc/app/cfg/organizations.yaml"
...
index.js
...
const DEFAULT_ORGANIZATIONS_PATH = './vars/local/organizations.yaml'
const program = require('commander')
program
.option(
'--organizations-config <file path>',
'The path to the organizations config file.', DEFAULT_ORGANIZATIONS_PATH)
.parse(process.argv)
function readConfigs () {
return Promise.all(configs.map(path => {
return new Promise((resolve, reject) => {
fs.readFile(path, (err, data) => {
err ? reject(err) : resolve(yaml.safeLoad(data))
})
})
}))
}
readConfigs()
.then(configs => {
let organizationsConfig = configs[3]
console.log('organizationsConfig = ', organizationsConfig)
console.log('organizationsConfig.organizations = ', organizationsConfig.organizations)
...
The output from above is:
organizationsConfig = { organizations: ' - foo - bar' }
organizationsConfig.organizations = - foo - bar
How can I modify my helm config so that organizationsConfig.organizations will be ['foo', 'bar']
One way to get the output you're looking for is to change:
...
organizations:
- 'foo'
- 'bar'
...
To:
organizations: |
[ 'foo', 'bar']
So helm treats it as a single string. We happen to know that it contains array content but helm just thinks it's a string. Then we can set that string directly in the configmap:
organizations: {{ .Values.organizations | indent 4 }}
What this does is what the grafana chart does in that it forces the user to specify the list in the desired format in the first place. Perhaps you'd prefer to take an array from the helm values and convert it to your desired format, which appears to me to be json format. To do that you could follow the example of the vault chart. So the configmap line becomes:
organizations: {{ .Values.organizations | toJson | indent 4 }}
Then the yaml that the user puts in can be as you originally had it i.e. a true yaml array. I tried this and it works but I notice that it gives double-quoted content like ["foo","bar"]
The other way you can do it is with:
organizations:
{{- range .Values.organizations }}
- {{ . }}
{{- end }}
Sometimes the root cause is that you forget to surround each item by quotes:
organizations:
{{- range .Values.organizations }}
- {{ . | quote }} # <-- SEE Here
{{- end }}

Variable value as yaml key in helm chart

I want to choose config section from values.yaml by setting a variable in helm command line.
example part of values.yaml:
aaa:
x1: "az1"
x2: "az2"
bbb:
x1: "bz1"
x2: "bz2"
example part of configmap.yaml
data:
{{ .Values.outsideVal.x1 }}
Expected result should looks like this
data:
az1
Test helm output
helm template --set outsideVal=aaa mychart
And got this error
Error: render error in "./templates/configmap.yaml": template: ./templates/configmap.yaml:21:12: executing "./templates/configmap.yaml" at <.Values.outsideVal.x...>: can't evaluate field x1 in type interface {}
So the question is how get the result as expected?
I suspect you're looking for the text/template index function, which can look up a value in a map by a variable key.
{{ (index .Values .Values.outsideVal).x1 }}