i am running a statefulset where i use volumeClaimTemplates. everything's fine there.
i also have a configmap where i would like to essentially replace some entries with the name of the pod for each pod that this config file is projected onto; eg, if the configmap data is:
ThisHost=<hostname -s>
OtherConfig1=1
OtherConfig1=2
...
then for the statefulset pod named mypod-0, the config file should contain ThisHost=mypod-0 and ThisHost=mypod-1 for mypod-1.
how could i do this?
The hostnames are contained in environment variables within the pod by default called HOSTNAME.
It is possible to modify the configmap itself if you first:
mount the configmap and set it to ThisHost=hostname -s (this will create a file in the pod's filesystem with that text)
pass a substitution command to the pod when starting (something like $ sed 's/hostname/$HOSTNAME/g' -i /path/to/configmapfile)
Basically, you mount the configmap and then replace it with the environment variable information that is available within the pod. It's just a substitution operation.
Look at the example below:
apiVersion: v1
kind: Pod
metadata:
name: command-demo
labels:
purpose: demonstrate-command
spec:
containers:
- name: command-demo-container
image: debian
command: ["sed"]
args: ["'s/hostname/$HOSTNAME'", "-i", "/path/to/config/map/mount/point"]
restartPolicy: OnFailure
The args' syntax might need some adjustments but you get the idea.
Please let me know if that helped.
Related
I have a configmap.yaml file as below :
apiVersion: v1
kind: ConfigMap
metadata:
name: abc
namespace: monitoring
labels:
app: abc
version: 0.17.0
data:
application.yml: |-
myjava:
security:
enabled: true
abc:
server:
access-log:
enabled: ${myvar}. ## this is not working
"myvar" value is available in pod as shell environment variable from secretkeyref field in deployment file.
Now I want to replace myvar shell environment variable in configmap above i.e before application.yml file is available in pod it should have replaced myvar value. which is not working i tried ${myvar} and $(myvar) and "#{ENV['myvar']}"
Is that possible in kubernetes configmap to reference with in data section pod's environment variable if yes how or should i need to write a script to replace with sed -i application.yml etc.
Is that possible in kubernetes configmap to reference with in data section pod's environment variable
That's not possible. A ConfigMap is not associated with a particular pod, so there's no way to perform the sort of variable substitution you're asking about. You would need to implement this logic inside your containers (fetch the ConfigMap, perform variable substitution yourself, then consume the data).
Spinnaker mounts volumes like this:
apiVersion: v1
kind: Pod
metadata:
...
spec:
containers:
...
volumes:
- configMap:
defaultMode: 420
items:
- key: config
path: config
name: kubectl-k8s-integration
name: "1551221025832"
- ...
I need the config file to be writeble by everyone so that I can use kubectl config use-context in the container, ie I need defaultMode to be 666 instead of 420. There doesn't seem to be place in the Spinnaker GUI to set this when defining volumes. What am I missing?
Based on https://github.com/spinnaker/spinnaker/issues/2118, it is not possible.
My workaround: I configure spinnaker to mount the configmap volume to a different filename, and added code to the container that automatically copies to the expected folder; the copy has write-access.
E.g. say my configmap has a key named "config", and I was previously mounting the configmap as /home/user/.kube, so .kube/config was the file with permission 420 instead of 666. Well I now mount it as /home/user/root.kube, and the container does the equivalent of cp -r /home/user/root.kube /home/user/.kube when it starts. Now /home/user/.kube/config is writable by user.
I'm new at kubernetes, and Im wondering the best way to inject values to ConfigMap.
for now, I defined Deployment object which takes the relevant values from ConfigMap file. I wish to use the same .yml file for my production and staging environments. so only the values in the configMap will be changed, while the file itself will be the same.
Is there any way to do it built-in in kubernetes, without using configuration management tools (like Ansible, puppet, etc.)?
You can find the links to the quoted text in the end of the answer.
A good practice when writing applications is to separate application code from configuration. We want to enable application authors to easily employ this pattern within Kubernetes. While the Secrets API allows separating information like credentials and keys from an application, no object existed in the past for ordinary, non-secret configuration. In Kubernetes 1.2, we’ve added a new API resource called ConfigMap to handle this type of configuration data.
Besides, Secrets data will be stored in a base64 encoded form, which is also suitable for binary data such as keys, whereas ConfigMaps data will be stored in plain text format, which is fine for text files.
The ConfigMap API is simple conceptually. From a data perspective, the ConfigMap type is just a set of key-value pairs.
There are several ways you can create config maps:
Using list of values in the command line
$ kubectl create configmap special-config --from-literal=special.how=very --from-literal=special.type=charm
Using a file on the disk as a source of data
$ kubectl create configmap game-config-2 --from-file=docs/user-guide/configmap/kubectl/game.properties --from-file=docs/user-guide/configmap/kubectl/ui.properties
$ kubectl create configmap game-config-3 --from-file=game-special-key=docs/user-guide/configmap/kubectl/game.properties
Using directory with files as a source of data
$ kubectl create configmap game-config --from-file=configure-pod-container/configmap/kubectl/
Combining all three previously mentioned methods
There are several ways to consume a ConfigMap data in Pods
Use values in ConfigMap as environment variables
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "echo $(SPECIAL_LEVEL_KEY)" ]
env:
- name: SPECIAL_LEVEL_KEY
valueFrom:
configMapKeyRef:
name: special-config
key: SPECIAL_LEVEL
Use data in ConfigMap as files on the volume
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "ls /etc/config/" ]
volumeMounts:
- name: config-volume
mountPath: /etc/config
volumes:
- name: config-volume
configMap:
# ConfigMap containing the files
name: special-config
Only changes in ConfigMaps that are consumed in a volume will be visible inside the running pod. Kubelet is checking whether the mounted ConfigMap is fresh on every periodic sync. However, it is using its local ttl-based cache for getting the current value of the ConfigMap. As a result, the total delay from the moment when the ConfigMap is updated to the moment when new keys are projected to the pod can be as long as kubelet sync period + ttl of ConfigMaps cache in kubelet.
Pod that contains in specification any references to non-existent ConfigMap or Secrets won't start.
Consider to read official documentation and other good articles for even more details:
Configuration management with Containers
Configure a Pod to Use a ConfigMap
Using ConfigMap
Kubernetes ConfigMaps and Secrets
Managing Pod configuration using ConfigMaps and Secrets in Kubernetes
You also create configmap
kubectl create configmap special-config \
--from-env-file=configure-pod-container/configmap/kubectl/game-env-file.properties
and access it in the container
apiVersion: v1
kind: Pod
metadata:
name: dapi-test-pod
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "env" ]
envFrom:
- configMapRef:
name: special-config
restartPolicy: Never
If you're thinking of ansible then I suspect you'll want to look at helm for this. I don't think it is a concern that kubernetes itself would address but helm is a kubernetes project.
If I understand correctly you've got a configmap yaml file and you want to deploy it with one set of values for staging and one for production.
A natural way to do this would be to keep two copies of the file with '-staging' and '-prod' appended on the name and have your CI choose the one for the environment it is deploying to. Or you could have a shell script in your CI that does a sed/replace on the particular values you want to switch for the environment.
Using helm you could pass in command-line parameters at deploy time or via a parameter-file (the values.yaml).
In K8S, what is the best way to execute scripts in container (POD) once at deployment, which reads from confuguration files which are part of the deployment and seed ex mongodb once?
my project consist of k8s manifest files + configuration files
I would like to be able to update the config files locally and then redeploy via kubectl or helm
In docker-compose i could create a volume ponting at the directory where the config files resides and then in the command part execute bash -c cmds reading from the config files in the volume. How is this best done in K8S? I don't want to include the configuration files in a image via dockerfile, forcing me to rebuild the image before redeploying again via kubectl or helm
How is this best done in K8S?
There are several ways to skin a cat, but my suggestion would be to do the following:
Keep configuration in configMap and mount it as separate volume. Such a map is kept as k8s manifest, making all changes to it separate from docker build image - no need to rebuild or keep sensitive data within image. You can also use instead (or together with) secret in the same manner as configMap.
Use initContainers to do the initialization before main container is to be brought online, covering for your 'once on deployment' automatically. Alternatively (if init operation is not repeatable) you can use Jobs instead and start it when necessary.
Here is excerpt of example we are using on gitlab runner:
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: ss-my-project
spec:
...
template:
....
spec:
...
volumes:
- name: volume-from-config-map-config-files
configMap:
name: cm-my-config-files
- name: volume-from-config-map-script
projected:
sources:
- configMap:
name: cm-my-scripts
items:
- key: run.sh
path: run.sh
mode: 0755
# if you need to run as non-root here is how it is done:
securityContext:
runAsNonRoot: true
runAsUser: 999
supplementalGroups: [999]
containers:
- image: ...
name: ...
command:
- /scripts/run.sh
...
volumeMounts:
- name: volume-from-config-map-script
mountPath: "/scripts"
readOnly: true
- mountPath: /usr/share/my-app-config/config.file
name: volume-from-config-map-config-files
subPath: config.file
...
You can, ofc, mount several volumes from config maps or combine them in one single, depending on frequency of your changes and affected parts. This is example with two separately mounted configMaps just to illustrate the principle (and mark script executable), but you can use only one for all required files, put several files into one or put single file into each - as per your need.
Example of such configMap is like so:
apiVersion: v1
kind: ConfigMap
metadata:
name: cm-my-scripts
data:
run.sh: |
#!/bin/bash
echo "Doing some work here..."
And example of configMap covering config file is like so:
kind: ConfigMap
apiVersion: v1
metadata:
name: cm-my-config-files
data:
config.file: |
---
# Some config.file (example name) required in project
# in whatever format config file actually is (just example)
... (here is actual content like server.host: "0" or EFG=True or whatever)
Playing with single or multiple files in configMaps can yield result you want, and depending on your need you can have as many or as few as you want.
In docker-compose i could create a volume ponting at the directory where the config files resides and then in the command part execute bash -c cmds reading from the config files in the volume.
In k8s equivalent of this would be hostPath but then you would seriously hamper k8s ability to schedule pods to different nodes. This might be ok if you have single node cluster (or while developing) to ease change of config files, but for actual deployment above approach is advised.
Creating a new ConfigMap from a file:
kubernetes create configmap foo --from-file=foo
This is how the ConfigMap looks internally:
kubernetes get configmaps foo -o yaml
apiVersion: v1
data:
foo: |
VAR1=value1
VAR2=value2
Then, I use this ConfigMap to create a set of environment variables in the container:
apiVersion: v1
kind: Pod
metadata:
labels:
name: app
name: app
spec:
containers:
- name: app-server
image: app:latest
ports:
- containerPort: 3000
envFrom:
- configMapRef:
name: foo
command: ["/bin/bash", "-c", "printenv"]
When the container command runs, I see the following output for printenv:
foo=VAR1=value1
VAR2=value2
So, an echo $foo command in the pod returns:
VAR1=value1 VAR2=value2
According to the documentation for ConfigMap with --from-file, this is expected behaviour.
What would be a creative way (and the proper place) to somehow get the values of this file available to the pod as individual env variables VAR1, VAR2, VAR3, etc. ?
This is not possible with the current version (1.6.x) of Kubernetes. As written in the offical documentation for kubectl create configmap:
--from-file: Key file can be specified using its file path, in which case file basename will be used as configmap key, or optionally with a key and file path, in which case the given key will be used. Specifying a directory will iterate each named file in the directory whose basename is a valid configmap key.
When you want to create a configmap which is used like this, as input for the envFrom container configuration you could create it with the --from-literal option like this:
kubectl create configmap foo --from-literal=var1=value1 --from-literal=var2=value2
To still keep the file, you could transform your file into somethings which then runs this command like this:
eval "kubectl create configmap foo $(cat foo.txt | sed -e 's/^/--from-literal=/' | tr "\n" ' ')"
Along with that maybe checking the outstanding proposals like the --flatten flag proposal on Github are worth your time.
Also keep an eye on the variable naming. iirc VAR1 and VAR2 are not valid property names - they have to be lower case which might cause some issues when passing them on.