accessing dynamic file in helm - kubernetes

I am new to kubernetes helm chart. There is a yaml file named configmap. This file containing all the configuration which are related to the application. Since this file containing a lot of data, I was trying to put some data to the new file and access using FILE object. So created two different file with the name:
data1.yaml and data2.yaml
data1.yaml file have only static data. On the other hand data2.yaml file contains dynamic data (some variables also like $.Values.appUrl).
I am able to read static file (data1.yaml) to the configmap.yaml file using FILE object. I am also able to read data2.yaml file but since this file containing some variable also, the variable value is not replacing by actual value. It printing the same variable in configmap file. So my question is,
Is there any way to access dynamic file (which contains variable type data) ?
Below is the example data shown.
configmap.yaml file is->
kind: ConfigMap
apiVersion: v1
metadata:
name: example-configmap
namespace: default
data1: {{ .File.Get "data1.yaml" | indent 2 }}
data2: {{ .File.Get "data2.yaml" | indent 2 }}
data1.yaml file is ->
data1:
ui.type:2
ui.color1:red
ui.color2:green
data2.yaml file is ->
system.data.name: "app-name"
system.data.url: {{ $.Values.appUrl }} # variable
system.data.type_one: "app-type-xxx"
system.data.value: "3"
system.interface.properties: |
Values.yaml file is ->
appUrl: "https://app-name.com"
Output:
kind: ConfigMap
apiVersion: v1
metadata:
name: example-configmap
namespace: default
data1:
ui.type:2
ui.color1:red
ui.color2:green
data2:
system.data.name: "app-name"
system.data.url: {{ $.Values.appUrl }} # here should be "https://app-name.com"
system.data.type_one: "app-type-xxx"
system.data.value: "3"
system.interface.properties: |

{{ (tpl (.Files.Glob "data2.yaml").AsConfig . ) | indent 2 }}
Using above syntax it's picking the actual value of variables. But it also printing the file name like below:
data2.yaml: |-
So I resolve the issue by using below syntax:
{{ (tpl (.Files.Get "data2.yaml") . ) | indent 2 }}

You can use the tpl function to process the templated file like this:
configmap.yaml:
apiVersion: v1
kind: ConfigMap
metadata:
name: example-configmap
data:
{{ (tpl (.Files.Get "data2.yaml") . ) | indent 2 }}
Note that configmap has data key by default. I think you should capture your data under this key, instead of creating data1 and data2 but not sure.
values.yaml:
appUrl: "https://app-name.com"
data2.yaml: (indentations corrected, otherwise it fails)
system.data.name: "app-name"
system.data.url: {{ .Values.appUrl }} # variable
system.data.type_one: "app-type-xxx"
system.data.value: "3"
system.interface.properties: |

Related

Using toJson or toRawJson without having it automatically adding quotation marks?

So I have a values.yaml file with an string variable representing a database connection string with no quotes looking like this (don't worry, not the real password):
ActionLogsConnectionString: Database=ActionLogs;Server=SQL_DEV;User Id=sa;Password=Y19|yx\dySh53&h
My goal is to print it inside a ConfigMap resource so that it can then be injected in my pod as a .json configuration file for a dotnet app. I also want to append the application name in the connection string:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "asp.fullname" . }}
labels:
{{- include "asp.labels" . | nindent 4 }}
data:
appsettings.k8s.json: |-
{
"ConnectionStrings": {
"ActionLogsConnectionString": "{{ .Values.ActionLogsConnectionString }};Application Name=Asp;"
}
}
This produce this result:
"ActionLogsConnectionString": "Database=ActionLogs;Server=SQL_DEV;User Id=sa;Password=Y19|yx\dySh53&h;Application Name=Asp;"
Look great! And at this point I don't have a quote problem.
Now problem, the slash isn't escaped for the json file format. Good thing, helm provide a toJson function. Unfortunately, it also transform the "&" to unicode value. I then found toRawJson and it gives the expected results.
My problem is that, when using either toJson or toRawJson, it adds extra quotes and break my result:
so this yalm file:
"ActionLogsConnectionString": "{{ .Values.ActionLogsConnectionString | toRawJson }};Application Name=Asp;"
results in this json file (note the extra quotes):
"ActionLogsConnectionString": ""Database=ActionLogs;Server=SQL_DEV;User Id=sa;Password=Y19|yx\\dySh53&h";Application Name=Asp;"
I see there's a function called | quote, but this only add some. No way to use toRawJson without adding any?
Using toJson or toRawJson is the wrong solution here, because the JSON representation of a string by definition includes the double quotes. "foo" is a JSON string, foo isn't valid JSON.
But you're only working with a scalar value, so there's not much point in marshaling it to JSON in the first place. I think the following gets you what you want:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "asp.fullname" . }}
labels:
{{- include "asp.labels" . | nindent 4 }}
data:
appsettings.k8s.json: |-
{
"ConnectionStrings": {
"ActionLogsConnectionString": {{ printf "%s;Application Name=asp" .Values.ActionLogsConnectionString | quote }}
}
}
Here, we're using the printf to produce the desired string (and then passing it to the quote function for proper quoting).
This produces:
---
# Source: example/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: my-example-fullname
labels:
# This is a test
data:
appsettings.k8s.json: |-
{
"ConnectionStrings": {
"ActionLogsConnectionString": "Database=ActionLogs;Server=SQL_DEV;User Id=sa;Password=Y19|yx\\dySh53&h;Application Name=asp"
}
}

Read an array in helm template from value file

I am trying to set and read array value like described below , parsing of yaml template file to json is failing , please suggest resolution.
template file :
spec:
arguments: {{ toYaml .Values.sparkapplication.spec.arguments | indent 6 }}
values file :
sparkapplication :
spec:
arguments:
- val: "/spark_app_prop/config.properties"
- val: "/spark_app_prop/log4j.properties"
expected parsed result in template file should be :
spec:
arguments: [ "/tmp/spark-streaming-poc-app-1.0.0.1/infa_conf/config.properties", "/tmp/spark-streaming-poc-app-1.0.0.1/infa_conf/log4j.properties" ]
values.yaml
sparkapplication :
spec:
arguments:
- val: "/spark_app_prop/config.properties"
- val: "/spark_app_prop/log4j.properties"
template/cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
{{- $list := list }}
{{- range .Values.sparkapplication.spec.arguments }}
{{- $list = append $list .val }}
{{- end }}
arguments: {{ toJson $list }}
output
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
arguments: ["/spark_app_prop/config.properties","/spark_app_prop/log4j.properties"]

How do I pass a map as a Helm tpl string?

I'm trying to populate the annotations on a service which is templated like this:
apiVersion: v1
kind: Service
metadata:
name: example
namespace: {{ .Release.Namespace }}
annotations:
{{- if .Values.service.annotations }}
{{ tpl .Values.service.annotations . | nindent 4 | trim }}
{{- end }}
(rest omitted for brevity.)
I tried to populate the service.annotations like this in my values.yaml:
service:
annotations:
foo: my-foo
bar: 6
baz: false
... but Helm errored out with [ERROR] templates/: template: my-chart/templates/service.yaml:20:18: executing "my-chart/templates/service.yaml" at <.Values.service.annotations>: wrong type for value; expected string; got map[string]interface {}
I tried reading the docs for the tpl function but they just showed examples for how to interpolate a string from a different variable, or how to pass values from an external file.
How do I set the annotations on this service? I don't have any control over the template.
update
You need to understand the essence of the problem, my previous answer is the style required for the final rendering, based on this, you just need to understand the usage of tpl and try to render the template as written before.
helm tpl
values.yaml
service:
annotations: "{{ toYaml .Values.service.fields | trim }}"
fields:
foo: my-foo
bar: 6
baz: false
template
apiVersion: v1
kind: Service
metadata:
name: example
namespace: {{ .Release.Namespace }}
annotations:
{{- if .Values.service.annotations }}
{{ tpl .Values.service.annotations . | nindent 4 | trim }}
{{- end }}
...
cmd
helm template --debug test .
output
apiVersion: v1
kind: Service
metadata:
name: test
namespace: default
annotations:
bar: 6
baz: false
foo: my-foo
...
As David Maze said.
In service.yaml, annotations must be a YAML string.
You may try
tempaltes/service.yaml
apiVersion: v1
kind: Service
metadata:
name: example
namespace: {{ .Release.Namespace }}
{{- if .Values.service.annotations }}
annotations:
{{ toYaml .Values.service.annotations | nindent 4 | trim }}
{{- end }}

Helm: "Template" keyword

Can someone explain to me what the role of the keyword "template" is in this code :
apiVersion: v1
kind: Secret
metadata:
name: {{ template "identity-openidconnect" . }}
namespace: {{ .Release.Namespace }}
labels:
app: {{ template "microService.name" . }}
release: "{{ .Release.Name }}"
xxxx
xxxxxxxxxxxx
The keyword "template" means, that Helm will find the previously created template and complete the yaml file according to the template in the template. It has to be created in advance. This type of construction allows you to refer to the same scheme many times.
For example, we can define a template to encapsulate a Kubernetes block of labels:
{{- define "mychart.labels" }}
labels:
generator: helm
date: {{ now | htmlDate }}
{{- end }}
Now we can embed this template inside of our existing ConfigMap, and then include it with the template action:
{{- define "mychart.labels" }}
labels:
generator: helm
date: {{ now | htmlDate }}
{{- end }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-configmap
{{- template "mychart.labels" }}
data:
myvalue: "Hello World"
{{- range $key, $val := .Values.favorite }}
{{ $key }}: {{ $val | quote }}
{{- end }}
When the template engine reads this file, it will store away the reference to mychart.labels until template "mychart.labels" is called. Then it will render that template inline. So the result will look like this:
# Source: mychart/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: running-panda-configmap
labels:
generator: helm
date: 2016-11-02
data:
myvalue: "Hello World"
drink: "coffee"
food: "pizza"
Note: a define does not produce output unless it is called with a template, as in this example.
For more info about templates you can read this page.

Using include inside range in Go templates (helm)

I have a template that get rendered several times with a range iteration and I can access variables external variables such as $.Release.Name without a problem. However, when I include templates I can't get it to work:
{{ range $key, $val := $.Values.resources }}
...
annotations:
checksum/config: {{ include (print $.Template.BasePath "/secrets.yaml") . | sha256sum }}
{{ end }}
And in secrets.yaml:
apiVersion: "v1"
kind: "Secret"
metadata:
name: {{ $.Release.Name }}-secrets
I got this error:
Error: render error in "botfront-project/templates/deployment.yaml": template: [filename] :19:28: executing [filename] at <include (print $.Template.BasePath "/secrets.yaml") .>: error calling include: template: .../secrets.yaml:4:19: executing ".../secrets.yaml" at <$.Release.Name>: nil pointer evaluating interface {}.Name
How do I access variables inside an included template?
TL;DR;
just replace . with $ to use the global scope instead of the local one you created .
Example:
{{- include "my-chart.labels" $ | nindent 4 }}
Explanations
According to the docs, https://helm.sh/docs/chart_template_guide/control_structures/#modifying-scope-using-with:
we can use $ for accessing the object Release.Name from the parent
scope.
$ is mapped to the root scope when template execution begins
and it does not change during template execution
With range we change the scope inside the loop. Indeed, {{- include "my-chart.labels" . | nindent 4 }} would invoke the current scope ..
So if you dig into this "scope" thing in helm doc, you eventually find this part: https://helm.sh/docs/chart_template_guide/variables/
With this example:
{{- range .Values.tlsSecrets }}
apiVersion: v1
kind: Secret
metadata:
name: {{ .name }}
labels:
# Many helm templates would use `.` below, but that will not work,
# however `$` will work here
app.kubernetes.io/name: {{ template "fullname" $ }}
# I cannot reference .Chart.Name, but I can do $.Chart.Name
helm.sh/chart: "{{ $.Chart.Name }}-{{ $.Chart.Version }}"
app.kubernetes.io/instance: "{{ $.Release.Name }}"
# Value from appVersion in Chart.yaml
app.kubernetes.io/version: "{{ $.Chart.AppVersion }}"
app.kubernetes.io/managed-by: "{{ $.Release.Service }}"
type: kubernetes.io/tls
data:
tls.crt: {{ .certificate }}
tls.key: {{ .key }}
---
{{- end }}