How to parse a JSON secret to YAML in Vault? - kubernetes

I have a secret in Vault which is really a "complex" structure of JSON, meaning it's not just a key/value but there are several keys at different sublevels.
I need to somehow get this secret and convert it to the YAML representation of that JSON. If it was a simple structure (like several k/v at the same level), I could use something as simple as
{{- with secret "secret/foo" -}}
{{ range $k, $v := .Data.data }}
{{ $k }}: {{ $v }}
{{- end }}
{{- end }}
however, as this is not the case, and the structure of the JSON is complex, trying to come up with a template is rather impossible.
However, I found that Vault uses Consul templates, and Consul has a parseYAML function, so my question is, how can I template this so that I get all the content of .Data.data and translate it into YAML?
I have tried several approaches similar to this one below:
{{- with secret "secret/foo" -}}
{{ .Data.data| parseYAML }}
{{- end }}
but I'm always getting the same error wrong type for value; expected string; got map[string]interface {}"
UPDATE
Sample yaml:
apiVersion: v1
kind: ServiceAccount
metadata:
name: app
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: app
spec:
selector:
matchLabels:
app: app
template:
metadata:
labels:
app: app
annotations:
vault.hashicorp.com/agent-inject: 'true'
vault.hashicorp.com/agent-inject-secret-foo: 'secret/foo'
vault.hashicorp.com/agent-inject-template-secret-foo: |
{{- with secret "secret/foo" -}}
{{ .Data.data| parseYAML }}
{{- end }}
vault.hashicorp.com/role: 'app'
spec:
containers:
- name: app
image: 'app:1.0.0'
serviceAccountName: app
where secret/foo is a long JSON with no clear structure. A random example (actual JSON is about 300 lines I think).
{
"a": {
"a": "a",
"b": "b",
"c": {
"a": "a",
"b": {
"c": "c"
},
"d": "a"
},
"e": {
"a": {
"b": {
"c": {
"a": "a",
"b": "b"
}
}
}
}
}
}

Ok, just figure this out.
{{- with secret "secret/foo" -}}
{{ .Data.data| toYAML }}
{{- end }}
this is correct, but instead of parseYAML the right function is toYAML.

Related

Create kubernetes docker-registry secret from yaml file for each lookup namespaces?

I am trying to dynamic lookup available namespaces and able to create secrets in the namespaces using below helm chart.
templates/secrets.yaml
{{ range $index, $namespace := (lookup "v1" "Namespace" "" "").items }}
apiVersion: v1
kind: Secret
metadata:
name: myregcred
namespace: {{ $namespace.metadata.name }}
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: {{ template "imagePullSecret" . }}
{{- end}}
values.yaml
imageCredentials:
registry: quay.io
username: someone
password: sillyness
email: someone#host.com
_helpers.tpl
{{- define "imagePullSecret" }}
{{- with .Values.imageCredentials }}
{{- printf "{\"auths\":{\"%s\":{\"username\":\"%s\",\"password\":\"%s\",\"email\":\"%s\",\"auth\":\"%s\"}}}" .registry .username .password .email (printf "%s:%s" .username .password | b64enc) | b64enc }}
{{- end }}
{{- end }}
When i run this helm chart, i get below error
Error: INSTALLATION FAILED: template: secrets/templates/_helpers.tpl:2:16: executing "imagePullSecret" at <.Values.imageCredentials>: nil pointer evaluating interface {}.imageCredentials
I dont know what I am doing wrong here.
When you reference the named template "imagePullSecret" inside the range, the context "." you are providing refers to the body of the loop, which does not have the "Values" attribute.
Try providing the root context instead:
{{ range $index, $namespace := (lookup "v1" "Namespace" "" "").items }}
apiVersion: v1
kind: Secret
metadata:
name: myregcred
namespace: {{ $namespace.metadata.name }}
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: {{ template "imagePullSecret" $ }}
---
{{- end}}

Read an array in helm template from value file

I am trying to set and read array value like described below , parsing of yaml template file to json is failing , please suggest resolution.
template file :
spec:
arguments: {{ toYaml .Values.sparkapplication.spec.arguments | indent 6 }}
values file :
sparkapplication :
spec:
arguments:
- val: "/spark_app_prop/config.properties"
- val: "/spark_app_prop/log4j.properties"
expected parsed result in template file should be :
spec:
arguments: [ "/tmp/spark-streaming-poc-app-1.0.0.1/infa_conf/config.properties", "/tmp/spark-streaming-poc-app-1.0.0.1/infa_conf/log4j.properties" ]
values.yaml
sparkapplication :
spec:
arguments:
- val: "/spark_app_prop/config.properties"
- val: "/spark_app_prop/log4j.properties"
template/cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
{{- $list := list }}
{{- range .Values.sparkapplication.spec.arguments }}
{{- $list = append $list .val }}
{{- end }}
arguments: {{ toJson $list }}
output
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
arguments: ["/spark_app_prop/config.properties","/spark_app_prop/log4j.properties"]

How to create a k8s configmap by looping over nested helm values

The helm values look like in the following example. Here appdata can scale to any number but will contain the same set of keys.
data:
appdata:
config0:
url: 'https://example1.com'
port: '80'
config1:
url: 'https://example2.com'
port: '8673'
someotherconfig:
url: 'https://example3.com'
port: '9887'
...
...
This is what I have so far. This keeps updating the last config's data from someotherconfig key and also I want to have the config map name to contain the config name for each iteration, like {{ template "app" $ }}-config0, {{ template "app" $ }}-config1 and so on based on iteration.
{{- range $Mydata := $.Values.data.appdata }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "app" $ }}
labels:
data:
url: {{ $Mydata.url }}
port: {{ $Mydata.port }}
{{- end }}
You're almost there. You need to use the key/value notation to get the key name. Try the following.
{{- range $configuration, $Mydata := $.Values.data.appdata }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "app" $ }}-{{ $configuration }}
data:
url: {{ $Mydata.url }}
port: {{ $Mydata.port }}
{{- end }}

lookup configmap value in helm template

Say I have a configmap like this, and I want to get the value(12301) of version 123v1 in map_a, what is the correct way?
apiVersion: v1
kind: ConfigMap
metadata:
name: myconfig
namespace: default
data:
test.yml: |
map_a:
"123v1":
"Version": 12301
"Type": "abc"
......
Here is my attempts:
apiVersion: v1
kind: ConfigMap
metadata:
name: test-config
data:
{{- $configmap := (lookup "v1" "ConfigMap" "default" "myconfig") }}
{{- if $configmap }}
{{- $models := get $configmap.data "test.yml" }}
version: {{ $models.map_a.123v1.Version }}
{{- end }}
$ helm template .
install.go:173: [debug] Original chart version: ""
install.go:190: [debug] CHART PATH: /root/test-lookup
Error: parse error at (test-lookup/templates/config.yaml:9): ".123v"
helm.go:88: [debug] parse error at (test-lookup/templates/config.yaml:9): ".123v"
The pipe symbol at the end of a line in YAML signifies that any indented text that follows should be interpreted as a multi-line scalar value. See the YAML spec. This is also mentioned in the helm doc |-
{{ $models }} is a value behind a pipe symbol of myconfig>data>test.yml.
So, the value of the expression {{ $models }} is a string instead of a map. In other words, {{ $models. map_a }} is an illegal operation.
The real value of {{ $models }} is a multi-line string, the string content is:
map_a:
"123v1":
"Version": 12301
"Type": "abc"
This can work here, of course the result is not what you want:
apiVersion: v1
kind: ConfigMap
metadata:
name: test-config
data:
{{- $configmap := (lookup "v1" "ConfigMap" "default" "myconfig") }}
{{- if $configmap }}
{{- $models := get $configmap.data "test.yml" }}
version: {{ $models | quote }}
{{- end }}
output:
apiVersion: v1
data:
version: |
map_a:
"123v1":
"Version": 12301
"Type": "abc"
kind: ConfigMap
metadata:
annotations:
...
So in order to achieve your purpose, you have to operate {{ $models }} in accordance with the method of operating strings.
config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test-config
data:
{{- $configmap := (lookup "v1" "ConfigMap" "default" "myconfig") }}
{{- if $configmap }}
{{- $models := get $configmap.data "test.yml" }}
{{- range ( split "\n" $models) }}
{{- if ( contains "Version" .) }}
version: {{ (split ":" .)._1 | trim | quote }}
{{- end }}
{{- end }}
{{- end }}
output:
apiVersion: v1
data:
version: "12301"
kind: ConfigMap
metadata:
annotations:
...

Helm: "Template" keyword

Can someone explain to me what the role of the keyword "template" is in this code :
apiVersion: v1
kind: Secret
metadata:
name: {{ template "identity-openidconnect" . }}
namespace: {{ .Release.Namespace }}
labels:
app: {{ template "microService.name" . }}
release: "{{ .Release.Name }}"
xxxx
xxxxxxxxxxxx
The keyword "template" means, that Helm will find the previously created template and complete the yaml file according to the template in the template. It has to be created in advance. This type of construction allows you to refer to the same scheme many times.
For example, we can define a template to encapsulate a Kubernetes block of labels:
{{- define "mychart.labels" }}
labels:
generator: helm
date: {{ now | htmlDate }}
{{- end }}
Now we can embed this template inside of our existing ConfigMap, and then include it with the template action:
{{- define "mychart.labels" }}
labels:
generator: helm
date: {{ now | htmlDate }}
{{- end }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-configmap
{{- template "mychart.labels" }}
data:
myvalue: "Hello World"
{{- range $key, $val := .Values.favorite }}
{{ $key }}: {{ $val | quote }}
{{- end }}
When the template engine reads this file, it will store away the reference to mychart.labels until template "mychart.labels" is called. Then it will render that template inline. So the result will look like this:
# Source: mychart/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: running-panda-configmap
labels:
generator: helm
date: 2016-11-02
data:
myvalue: "Hello World"
drink: "coffee"
food: "pizza"
Note: a define does not produce output unless it is called with a template, as in this example.
For more info about templates you can read this page.