How to mount Vault secret as a file in Kubernetes? - kubernetes

Question

I'm using Hashicorp Vault in Kubernetes. I'm trying to mount secret file into main folder where my application resides. It would look like that: /usr/share/nginx/html/.env while application files are in /usr/share/nginx/html. But the container is not starting because of that. I suspect that that /usr/share/nginx/html was overwritten by Vault (annotation: vault.hashicorp.com/secret-volume-path). How can I mount only file /usr/share/nginx/html/.env?
My annotations:
vault.hashicorp.com/agent-init-first: "true"
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-inject-secret-.env: configs/data/app/dev
vault.hashicorp.com/agent-inject-template-.env: |
{{- with secret (print "configs/data/app/dev") -}}{{- range $k, $v := .Data.data -}}
{{ $k }}={{ $v }}
{{ end }}{{- end -}}
vault.hashicorp.com/role: app
vault.hashicorp.com/secret-volume-path: /usr/share/nginx/html

Answer 1

I tried to replicate the use case, but I got an error
2022/10/21 06:42:12 [error] 29#29: *9 directory index of "/usr/share/nginx/html/" is forbidden, client: 20.1.48.169, server: localhost, request: "GET / HTTP/1.1", host: "20.1.55.62:80"
so it seems like vault changed the directory permission as well, as it create .env in the path, here is the config
vault.hashicorp.com/agent-init-first: "true"
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-inject-secret-.env: kv/develop/us-west-2/app1-secrets
vault.hashicorp.com/agent-inject-template-.env: |
"{{ with secret "kv/develop/us-west-2/app1-secrets" }}
{{ range $k, $v := .Data.data }}
{{ $k }} = "{{ $v }}"
{{ end }}
{{ end }} "
vault.hashicorp.com/agent-limits-ephemeral: ""
vault.hashicorp.com/secret-volume-path: /usr/share/nginx/html/
vault.hashicorp.com/agent-inject-file-.env: .env
vault.hashicorp.com/auth-path: auth/kubernetes/develop/us-west-2
vault.hashicorp.com/role: rolename
The work around was to overide the command of the desired container, for this use case, i used nginx
command: ["bash", "-c", "cat /vault/secret/.env > /usr/share/nginx/html/.env && nginx -g 'daemon off;' "]
Here is the compelete example with dummy value of my-app
apiVersion: apps/v1
kind: Deployment
metadata:
name: debug-app
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
annotations:
vault.hashicorp.com/agent-init-first: "true"
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-inject-secret-.env: kv/my-app/develop/us-west-2/develop-my-app
vault.hashicorp.com/agent-inject-template-.env: |
"{{ with secret "kv/my-app/develop/us-west-2/develop-my-app" }}
{{ range $k, $v := .Data.data }}
{{ $k }} = "{{ $v }}"
{{ end }}
{{ end }} "
vault.hashicorp.com/agent-limits-ephemeral: ""
vault.hashicorp.com/secret-volume-path: /vault/secret/
vault.hashicorp.com/agent-inject-file-.env: .env
vault.hashicorp.com/auth-path: auth/kubernetes/develop/us-west-2
vault.hashicorp.com/role: my-app-develop-my-app
spec:
serviceAccountName: develop-my-app
containers:
- name: debug
image: nginx
command: ["bash", "-c", "cat /vault/secret/.env > /usr/share/nginx/html/.env && nginx -g 'daemon off;' "]
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http