Helm Construct dynamic configmap from multiple configuration YAML file - kubernetes-helm

I have 2 files as follows:
_config-dev.yaml
frontend:
NODE_ENV: dev
REACT_APP_API_URL: 'https://my-dev-apiurl/'
database:
DB_USER: admin-dev
DB_PASSWORD: password-dev
_config-stag.yaml
frontend:
NODE_ENV: stag
REACT_APP_API_URL: 'https://my-stag-api-url/'
database:
DB_USER: admin-stag
DB_PASSWORD: password-stag
myConfigMap.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: {{ .Release.Name }}-frontend
namespace: {{ .Values.global.namespace }}
data:
# Here I want to insert only frontend data from _config-dev.yaml file if my {{ eq .Values.global.environment "dev" }} like below
NODE_ENV: dev
REACT_APP_API_URL: 'https://my-dev-apiurl/'
# if my {{ eq .Values.global.environment "stag" }} i want to get frontend values from _config-dev.yaml like below
NODE_ENV: stag
REACT_APP_API_URL: 'https://my-stag-api-url/'
Can anyone figure out how to insert the data as per above scenario mentioned in myConfigMap.yaml file as a comment under data:.

my test project
test
├── Chart.yaml
├── cfg
│   ├── _config-dev.yaml
│   └── _config-stag.yaml
├── templates
│   └── configmap.yaml
└── values.yaml
values.yaml
global:
environment: dev
test/cfg/_config-dev.yaml
frontend:
NODE_ENV: dev
REACT_APP_API_URL: 'https://my-dev-apiurl/'
database:
DB_USER: admin-dev
DB_PASSWORD: password-dev
test/cfg/_config-stag.yaml
frontend:
NODE_ENV: stag
REACT_APP_API_URL: 'https://my-stag-api-url/'
database:
DB_USER: admin-stag
DB_PASSWORD: password-stag
test/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "test.fullname" . }}
data:
{{- $data := .Files.Get "cfg/_config-stag.yaml" }}
{{- if eq .Values.global.environment "dev" }}
{{- $data = .Files.Get "cfg/_config-dev.yaml" }}
{{- end }}
{{- $cfg := fromYaml $data }}
{{- range $k, $v := $cfg.frontend }}
{{ $k }}: {{ $v }}
{{- end }}
output
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
NODE_ENV: dev
REACT_APP_API_URL: https://my-dev-api-url/

Related

Helm not using values override?

I'm using sub-charts. Here's my directory structure
/path/microservice-base-chart
/path/myApp
I have this values.yaml for my "base" (generic) chart
# Default region and repository
aws_region: us-east-1
repository: 012234567890.dkr.ecr.us-east-1.amazonaws.com
repositories:
us-east-1: 01234567890.dkr.ecr.us-east-1.amazonaws.com
eu-north-1: 98765432109.dkr.ecr.eu-north-1.amazonaws.com
image:
name: ""
version: ""
...and this in the base chart's templates/_helpers.yaml file
{{/*
Get the repository from the AWS region
*/}}
{{- define "microservice-base-chart.reponame" -}}
{{- $repo := index .Values.repositories .Values.aws_region | default .Values.repository }}
{{- printf "%s" $repo }}
{{- end }}
...and this in the base chart's templates/deployment.yaml file
apiVersion: apps/v1
kind: Deployment
...
spec:
...
template:
...
spec:
...
containers:
- name: {{ .Values.image.name }}
image: {{ include "microservice-base-chart.reponame" . }}/{{ .Values.image.name }}:{{ .Values.image.version }}
I have this in the Chart.yaml of a sub chart that uses the base chart.
dependencies:
- alias: microservice-0
name: microservice-base-chart
version: "0.1.0"
repository: file://../microservice-base-chart
...and this in the values.yaml of a sub chart
microservice-0:
image:
name: myApp
version: 1.2.3
However, when I run this, where I set aws_region
$ helm install marcom-stats-svc microservice-chart/ \
--set image.aws_region=eu-north-1 \
--set microservice-0.image.version=2.0.0 \
--dry-run --debug
I get this for the image name of the above deployment.yaml template
image: 01234567890.dkr.ecr.us-east-1.amazonaws.com/myApp:2.0.0
instead of the expected
image: 98765432109.dkr.ecr.eu-north-1.amazonaws.com/myApp:2.0.0
What am I missing? TIA

Create kubernetes docker-registry secret from yaml file for each lookup namespaces?

I am trying to dynamic lookup available namespaces and able to create secrets in the namespaces using below helm chart.
templates/secrets.yaml
{{ range $index, $namespace := (lookup "v1" "Namespace" "" "").items }}
apiVersion: v1
kind: Secret
metadata:
name: myregcred
namespace: {{ $namespace.metadata.name }}
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: {{ template "imagePullSecret" . }}
{{- end}}
values.yaml
imageCredentials:
registry: quay.io
username: someone
password: sillyness
email: someone#host.com
_helpers.tpl
{{- define "imagePullSecret" }}
{{- with .Values.imageCredentials }}
{{- printf "{\"auths\":{\"%s\":{\"username\":\"%s\",\"password\":\"%s\",\"email\":\"%s\",\"auth\":\"%s\"}}}" .registry .username .password .email (printf "%s:%s" .username .password | b64enc) | b64enc }}
{{- end }}
{{- end }}
When i run this helm chart, i get below error
Error: INSTALLATION FAILED: template: secrets/templates/_helpers.tpl:2:16: executing "imagePullSecret" at <.Values.imageCredentials>: nil pointer evaluating interface {}.imageCredentials
I dont know what I am doing wrong here.
When you reference the named template "imagePullSecret" inside the range, the context "." you are providing refers to the body of the loop, which does not have the "Values" attribute.
Try providing the root context instead:
{{ range $index, $namespace := (lookup "v1" "Namespace" "" "").items }}
apiVersion: v1
kind: Secret
metadata:
name: myregcred
namespace: {{ $namespace.metadata.name }}
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: {{ template "imagePullSecret" $ }}
---
{{- end}}

How to create a k8s configmap by looping over nested helm values

The helm values look like in the following example. Here appdata can scale to any number but will contain the same set of keys.
data:
appdata:
config0:
url: 'https://example1.com'
port: '80'
config1:
url: 'https://example2.com'
port: '8673'
someotherconfig:
url: 'https://example3.com'
port: '9887'
...
...
This is what I have so far. This keeps updating the last config's data from someotherconfig key and also I want to have the config map name to contain the config name for each iteration, like {{ template "app" $ }}-config0, {{ template "app" $ }}-config1 and so on based on iteration.
{{- range $Mydata := $.Values.data.appdata }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "app" $ }}
labels:
data:
url: {{ $Mydata.url }}
port: {{ $Mydata.port }}
{{- end }}
You're almost there. You need to use the key/value notation to get the key name. Try the following.
{{- range $configuration, $Mydata := $.Values.data.appdata }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "app" $ }}-{{ $configuration }}
data:
url: {{ $Mydata.url }}
port: {{ $Mydata.port }}
{{- end }}

Helm3: Create .properties files recursively in Configmap

Below are files that I have:
users-values.yaml file :
users:
- foo
- baz
other-values.yaml file:
foo_engine=postgres
foo_url=some_url
foo_username=foofoo
baz_engine=postgres
baz_url=some_url
baz_username=bazbaz
config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test-catalog
data:
{{- range $user := .Values.users }}
{{ . }}: |
engine.name={{ printf ".Values.%s_engine" ($user) }}
url={{ printf ".Values.%s_url" ($user) }}
username={{ printf".Values.%s_username" ($user) }}
{{- end }}
deployment-coordinator.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: "{{ .Release.Name }}-coordinator"
labels:
app.kubernetes.io/name: "{{ .Release.Name }}-coordinator"
spec:
replicas: 1
...
template:
metadata:
labels:
app.kubernetes.io/name: "{{ .Release.Name }}-coordinator"
spec:
volumes:
- name: config
configMap:
name: test-catalog
...
volumeMounts:
- name: config
mountPath: "/etc/config"
Then, I do a helm install test mychart.
When I exec into the pod, and cd to /etc/config, I expect to see foo.properties and baz.properties files in there, and each file looks like:
foo.properties: |
engine.name=postgres
url=some_url
username=foofoo
baz.properties: |
engine.name=postgres
url=some_url
username=bazbaz
The answer from Pawel below solved the error I got previously
unexpected bad character U+0022 '"' in command
But, the files are still not created in the /etc/config directory.
So, I was wondering if it's even possible to create the .properties files using helm range as I mentioned in my config.yaml file above.
The reason I wanted to do it the below way is because I have more than 10 users to create .properties files on, not just foo and baz. Just thought it'll be easier if I can do a for loop on it if possible.
data:
{{- range $user := .Values.users }}
{{ . }}: |
engine.name={{ printf ".Values.%s_engine" ($user) }}
url={{ printf ".Values.%s_url" ($user) }}
username={{ printf".Values.%s_username" ($user) }}
{{- end }}

Helm - Templating variables in values.yaml

I'm trying to template variables from a map inside the values.yaml into my final Kubernetes ConfigMap YAML.
I've read through https://github.com/helm/helm/issues/2492 and https://helm.sh/docs/chart_template_guide/ but can't seem to find an answer.
For some context, this is roughly what I'm trying to do:
values.yaml
config:
key1: value
key2: value-{{ .Release.Name }}
configmap.yaml
kind: ConfigMap
data:
config-file: |
{{- range $key, $value := .Values.config }}
{{ $key }} = {{ $value }}
{{- end }}
Where the desired output with would be:
helm template --name v1 mychart/
kind: ConfigMap
data:
config-file: |
key1 = value
key2 = value-v1
I've tried a few variations using template functions and pipelining, but to no avail:
{{ $key }} = {{ tpl $value . }}
{{ $key }} = {{ $value | tpl . }}
{{ $key }} = {{ tpl $value $ }}
The above would also have worked in this way
values.yaml
config:
key1: "value"
key2: "value-{{ .Release.Name }}"
configmap.yaml
kind: ConfigMap
data:
config-file: |
{{- range $key, $value := .Values.config }}
{{ $key }} = {{ tpl $value $ }}
{{- end }}
What I changed was : I put value in quotes in value.yaml and used template tpl in the config map.
I'll refer to the question's title regarding templating variables in helm and suggest another option to use on values.yaml which is YAML Anchors.
Docs reference
As written in here:
The YAML spec provides a way to store a reference to a value, and
later refer to that value by reference. YAML refers to this as
"anchoring":
coffee: "yes, please"
favorite: &favoriteCoffee "Cappucino"
coffees:
- Latte
- *favoriteCoffee
- Espresso
In the above, &favoriteCoffee sets a reference to Cappuccino.
Later, that reference is used as *favoriteCoffee.
So coffees becomes Latte, Cappuccino, Espresso.
A more practical example
Referring to a common image setup (Registry and PullPolicy) in all values.yaml.
Notice how the default values are being set at Global.Image next to the reference definition which starts with &:
Global:
Image:
Registry: &global-docker-registry "12345678910.dkr.ecr.us-west-2.amazonaws.com" # <--- Default value
PullPolicy: &global-pull-policy "IfNotPresent" # <--- Default value
Nginx:
Image:
Registry: *global-docker-registry
PullPolicy: *global-pull-policy
Version: 1.21.4
Port: 80
MySql:
Image:
Registry: *global-docker-registry
PullPolicy: *global-pull-policy
Name: mysql
Version: 8.0.27
Port: 3306
Managed to solve this using the following syntax:
configmap.yaml
kind: ConfigMap
data:
config-file: |
{{- range $key, $value := .Values.config }}
{{ $key }} = {{ tpl ($value | toString) $ }}
{{- end }}
there is fight in this PR here about this topic.
I know that it's possible now, but this require maintenance of the chart to be in-house (e.g. answer of Amrut ).
Let's summarize :
To have templating in values.yaml , these are the available options:
helm may support that in future ( watch this thread about this topic.)
use tpl function inside the chart
use another tool on top of helm : terraform or helmfile.