Automatic subdirectories in Kubernetes configmaps? - kubernetes

Question

(A very similar question was asked about 2 years ago, though it was specifically about secrets, I doubt the story is any different for configmaps... but at the least, I can present the use case and why the existing workarounds aren't viable for us.)
Given a simple, cut-down deployment.yaml:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: example
spec:
template:
spec:
containers:
- name: example
volumeMounts:
- name: vol
mountPath: /app/Configuration
volumes:
- name: vol
configMap:
name: configs
and the matching configmap.yaml:
apiVersion: v1
kind: ConfigMap
metadata:
name: configs
labels:
k8s-app: example
data:
example1.json: |-
{
"key1": "value1"
}
example2.json: |-
{
"key2": "value2"
}
the keys in configmap.yaml, whatever they may be, are simply created as files, without deployment.yaml needing to be modified or have any specifics other than the mountPath.
The problem is that the actual structure has subfolders to handle region-specific values that override the root ones:
Configuration \ example1.json
Configuration \ example2.json
Configuration \ us \ example1.json
Configuration \ us \ ca \ example2.json
The number and nature of these could obviously vary, for as many different countries and regions imaginable and for each separately configured module. The intent was to provide a tool to the end user that would allow them to set up and manage these configurations, which would behind the scenes automatically generate the configmap.yaml and update it in kubernetes.
However, unless there's a trick I haven't found yet, this seems to be outside of kubernetes's abilities, in a couple ways.
First of all, there is no syntax that allows one to specify configmap keys that are directories, nor include a subdirectory path in a key:
data:
# one possible approach (currently complains that it doesn't validate '[-._a-zA-Z0-9]+')
/us/example1.json: |-
{
"key1": "value1"
}
# another idea; this obviously results in 'invalid type for io.k8s.api.core.v1.ConfigMap.data: got "map", expected "string"'
us:
example2.json: |-
{
"key2": "value2"
}
So what are our options to accomplish this?
Wellll, we could map the keys to specific locations using the items: -key: path: approach in the deployment.yaml's volumes: -configMap: node,
and/or generate several nodes in the deployment.yaml's volumeMounts: node,
using either subPath: (which is basically the same as using items: -key: -path: in the volumes: configMap:),
or individual separate configmaps for each subdirectory, and mounting them all as different volumes in the deployment.yaml.
All of these methods would require massive and incredibly verbose changes to the deployment.yaml, leaking out knowledge it shouldn't have any reason to know about, making it mutable and continually re-generated rather than static, complicating rolling out settings updates to deployed pods, etc. etc. etc. It's just Not Good. And all of that just to have mapped one directory, just because it contains subdirectories...
Surely this CAN'T be the way it's SUPPOSED to work? What am I missing? How should I proceed?

Answer 1

From a "container-native" perspective, having a large file system tree of configuration files that the application processes at startup to arrive at its canonical configuration is an anti-pattern. Better to have a workflow that produces a single file, which can be stored in a ConfigMap and easily inspected in its final form. See, for instance, nginx ingress.
But obviously not everyone is rewriting their apps to better align with the kubernetes approach. The simplest way then to get a full directory tree of configuration files into a container at deploy time is to use initContainers and emptyDir mounts.
Package the config file tree into a container (sometimes called a "data-only" container), and have the container start script just copy the config tree into the emptyDir mount. The application can then consume the tree as it expects to.

Answer 2

Depending on the scale of your config tree another viable option might be to simulate a subtree with, say, underscores instead of slashes in the file "paths" inside the configmap. This will make you loose general filesystem performance (which should never be a problem if you are just having configs to read) and force you to rewrite a little bit of your applications code (file pattern traversing instead of directory traversing when accessing configs), but should solve your use case at a fairly cheap price.

Answer 3

A few workarounds:
have a data only container built with the data on it...
FROM scratch
... # copy data here
then add it as a sidecar mounting the volume on the other container...
create a tar ball from the config, convert it to a configmap, mount in a container and change the container command to untar the config before start...
rename the files with some special char instead of /, like us#example.json and use a script to mv them as in the beginning.
All of this is very hacky... The best scenario is to refactor them to be used in flat folder and create them with something like kustomize:
kustomize edit add configmap my-configmap --from-file='./*.json'