How do I load a configMap into an environment variable?
Things I've done
Kubernetes documentation describes just this scenario, and following it:
I've actually set up my configMap through Terraform with this:
resource "kubernetes_config_map" "production_database_host" {
metadata {
name = "production-database-host"
}
data {
connection_name = "${google_sql_database_instance.master.connection_name}"
}
}
But via Kubernetes, it would look like this:
apiVersion: v1
data:
connection_name: this_string_is_redacted
kind: ConfigMap
metadata:
creationTimestamp: 2018-10-12T05:49:49Z
name: production-database-host
namespace: default
resourceVersion: "316273"
selfLink: /api/v1/namespaces/default/configmaps/production-database-host
uid: a1c06423-cde2-11e8-b615-42010a800235
(Fetched by running kubectl get configmap production-database-host -o yaml)
Now, I also have a working container, in a deployment, where I added an environment variable like so:
env:
- name: INSTANCE_CONNECTION_NAME
valueFrom:
configMapKeyRef:
name: production-database-host
key: connection_name
However, applying this config gives me:
$ kubectl apply -f .
error: error converting YAML to JSON: yaml: line 39: did not find expected key
What am I doing wrong here? Why won't this simply load this_string_is_redacted into the INSTANCE_CONNECTION_NAME environment variable?
Edit: All the source for my infrastructure is in this repo. The Terraform files are applied first, they create the Kubnernetes cluster and add the configMap. Then I apply the Kubernetes config.
It was a formatting issue, unfortunately the block:
env:
- name: INSTANCE_CONNECTION_NAME
valueFrom:
configMapKeyRef:
name: production-database-host
key: connection_name
Was indented one space more than I should have been. Everything else works fine.
Related
I have a pod with the following specs
kind: Pod
metadata:
name: busybox
namespace: default
spec:
containers:
- image: busybox
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
name: busybox
env:
- name: WATCH_NAMESPACE
valueFrom:
configMapKeyRef:
name: watch-namespace-config
key: WATCH_NAMESPACE
restartPolicy: Always
I also created a ConfigMap
kubectl create configmap watch-namespace-config \
--from-literal=WATCH_NAMESPACE=dev
The pod looks for values in the watch-namespace-config configmap.
When I manually change the configmap values, I want the pod to restart automatically to reflect this change. Checking if that is possible in any way.
This is currently a feature in progress https://github.com/kubernetes/kubernetes/issues/22368
For now, use Reloader - https://github.com/stakater/Reloader
It watches if some change happens in ConfigMap and/or Secret; then performs a rolling upgrade on relevant DeploymentConfig, Deployment, Daemonset, Statefulset and Rollout
How to use it - https://github.com/stakater/Reloader#how-to-use-reloader
As you mentioned correctly once you update a ConfigMap or Secret the Deployment/Pod/Stateful set is not updated.
An optional solution for this scenario is to use Kustomization.
Kustomization generates a unique name every time you update the ConfigMap/Secret with a generated hash, for example: ConfigMap-xxxxxx.
If you will will use:
kubectl kustomize . | kubectl apply -f -
kubectl will "update" the changes with the new config map values.
Working Example(s) using Kustomization:
https://github.com/nirgeier/KubernetesLabs/tree/master/Labs/08-Kustomization
I am trying to map kubernetes secret value to a environment variable . My secret is as shown below
apiVersion: v1
kind: Secret
metadata:
name: test-secret
type: opaque
data:
tls.crt: {{ required "A valid value is required for tls.crt" .Values.tlscrt }}
Mapped the key to environment variable in the deployment yaml
env:
- name: TEST_VALUE
valueFrom:
secretKeyRef:
name: test-secret
key: tls.crt
The value gets mapped when i do helm install. However when i do helm upgrade , the changed value is not reflected in the environment variable , it still has the old value. Can anyone please help here ?
Changes to secret or configMap data are not reflected in existing pods. You have to delete and recreate the pod in order to see changes. There are ways to automate the process (see this Q/A for example: Helm chart restart pods when configmap changes) and they all have one thing in common: you need to modify something in pod definition to trigger a restart. It does not happen when you update a linked secret or a configMap because the link remains the same.
Is there a way to set the values of some keys in a Knative service.yaml file using environment variables?
More detail
I am trying to deploy a Knative service to a Kubernetes cluster using GitLab CI. Some of the variables in my service.yaml file depend on the project and environment of the GitLab CI pipeline. Is there a way I can seamlessly plug those values into my service.yaml file without resorting to hacks like sed -i ...?
For example, given the following script, I want the $(KUBE_NAMESPACE), $(CI_ENVIRONMENT_SLUG), and $(CI_PROJECT_PATH_SLUG) values to be replaced by accordingly-named environment variables.
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: design
namespace: "$(KUBE_NAMESPACE)"
spec:
template:
metadata:
name: design-v1
annotations:
app.gitlab.com/env: "$(CI_ENVIRONMENT_SLUG)"
app.gitlab.com/app: "$(CI_PROJECT_PATH_SLUG)"
spec:
containers:
- name: user-container
image: ...
timeoutSeconds: 600
containerConcurrency: 8
I don't think there is a great way to expand environment variables inside of an existing yaml, but if you don't want to use sed, you might be able to use envsubst:
envsubst < original.yaml > modified.yaml
You would just run this command before you use the yaml to expand the environment variables contained within it.
Also I think you'll need your variables to use curly braces, instead of parentheses, like this: ${KUBE_NAMESPACE}.
EDIT: You might also be able to use this inline like this: kubectl apply -f <(envsubst < service.yaml)
More than a Knative issue this is more of Kubernetes limitation. Kubernetes allows some expansion but not in annotations or namespace definitions. For example, you can do it in container env definitions:
containers:
- env:
- name: PODID
valueFrom: ...
- name: LOG_PATH
value: /var/log/$(PODID)
If this is a CI/CD system like Gitlab the environment variables should be in a shell environment, so a simple shell expansion will do. For example.
#!/bin/bash
echo -e "
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: design
namespace: "${KUBE_NAMESPACE}"
spec:
template:
metadata:
name: design-v1
annotations:
app.gitlab.com/env: "${CI_ENVIRONMENT_SLUG}"
app.gitlab.com/app: "${CI_PROJECT_PATH_SLUG}"
spec:
containers:
- name: user-container
image: ...
timeoutSeconds: 600
containerConcurrency: 8
" | kubectl apply -f -
You can also use envsubst as a helper like mentioned in the other answer.
In order to a service work, it needs an environment variable called DSN which prints to something like postgres://user:password#postgres.svc.cluster.local:5432/database. This value I built with a ConfigMap resource:
apiVersion: v1
kind: ConfigMap
metadata:
name: idp-config
namespace: diary
data:
DSN: postgres://user:password#postgres.svc.cluster.local:5432/database
This ConfigMap mounts as environment variable in my service Pod. Since the values are different from user and password and these PostgreSQL credentials are in another k8s resource (a Secret and a ConfigMap), how can I properly build this DSN environment in a k8s resource yaml so my service can connect to the database?
Digging into Kubernetes Docs I was able to find. According to Define Environment Variables for a Container :
Environment variables that you define in a Pod’s configuration can be used elsewhere in the configuration, for example in commands and arguments that you set for the Pod’s containers. In the example configuration below, the GREETING, HONORIFIC, and NAME environment variables are set to Warm greetings to, The Most Honorable, and Kubernetes, respectively. Those environment variables are then used in the CLI arguments passed to the env-print-demo container.
apiVersion: v1
kind: Pod
metadata:
name: print-greeting
spec:
containers:
- name: env-print-demo
image: bash
env:
- name: GREETING
value: "Warm greetings to"
- name: HONORIFIC
value: "The Most Honorable"
- name: NAME
value: "Kubernetes"
command: ["echo"]
args: ["$(GREETING) $(HONORIFIC) $(NAME)"]
In Kubernetes you have the ability to dynamically grab the name of a pod and reference it in a yaml file (Pod Field) like so:
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
and reference it later in the yaml file like so:
- name: FOO
value: $(POD_NAME)-bar
Where in the case of a StatefulSet the value of foo may be something like "app_thing-0-bar, app_thing-1-bar ... etc". However this doesn't seem to work in dynamically setting the name of a configmap. For example, the following configmap:
apiVersion: v1
kind: ConfigMap
metadata:
name: app_thing-0-config
data:
FOO: BAR
and this in the StatefulSet deployment yaml:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: app_thing
.
.
.
.
.
envFrom:
- configMapRef:
name: $(POD_NAME)-config
will not reference the configmap correctly as it doesn't seem to like the $() syntax. Is there any way to do this without resorting to init containers and entrypoint scripting?
If I understand you correctly there is a tool that can make it work. It's called RELOADER:
Problem: We would like to watch if some change happens in ConfigMap and/or Secret; then perform a rolling upgrade on relevant
DeploymentConfig, Deployment, Daemonset and Statefulset
Solution: Reloader can watch changes in ConfigMap and Secret and do rolling upgrades on Pods with their associated DeploymentConfigs,
Deployments, Daemonsets and Statefulsets.
You can find all the necessary info in the link above.
Also if you'd need more details than you can check the documentation.
Please let me know if that helped.