Cron Jobs in Kubernetes - connect to existing Pod, execute script - kubernetes

I'm certain I'm missing something obvious. I have looked through the documentation for ScheduledJobs / CronJobs on Kubernetes, but I cannot find a way to do the following on a schedule:
Connect to an existing Pod
Execute a script
Disconnect
I have alternative methods of doing this, but they don't feel right.
Schedule a cron task for: kubectl exec -it $(kubectl get pods --selector=some-selector | head -1) /path/to/script
Create one deployment that has a "Cron Pod" which also houses the application, and many "Non Cron Pods" which are just the application. The Cron Pod would use a different image (one with cron tasks scheduled).
I would prefer to use the Kubernetes ScheduledJobs if possible to prevent the same Job running multiple times at once and also because it strikes me as the more appropriate way of doing it.
Is there a way to do this by ScheduledJobs / CronJobs?
http://kubernetes.io/docs/user-guide/cron-jobs/

As far as I'm aware there is no "official" way to do this the way you want, and that is I believe by design. Pods are supposed to be ephemeral and horizontally scalable, and Jobs are designed to exit. Having a cron job "attach" to an existing pod doesn't fit that module. The Scheduler would have no idea if the job completed.
Instead, a Job can to bring up an instance of your application specifically for running the Job and then take it down once the Job is complete. To do this you can use the same Image for the Job as for your Deployment but use a different "Entrypoint" by setting command:.
If they job needs access to data created by your application then that data will need to be persisted outside the application/Pod, you could so this a few ways but the obvious ways would be a database or a persistent volume.
For example useing a database would look something like this:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: APP
spec:
template:
metadata:
labels:
name: THIS
app: THAT
spec:
containers:
- image: APP:IMAGE
name: APP
command:
- app-start
env:
- name: DB_HOST
value: "127.0.0.1"
- name: DB_DATABASE
value: "app_db"
And a job that connects to the same database, but with a different "Entrypoint" :
apiVersion: batch/v1
kind: Job
metadata:
name: APP-JOB
spec:
template:
metadata:
name: APP-JOB
labels:
app: THAT
spec:
containers:
- image: APP:IMAGE
name: APP-JOB
command:
- app-job
env:
- name: DB_HOST
value: "127.0.0.1"
- name: DB_DATABASE
value: "app_db"
Or the persistent volume approach would look something like this:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: APP
spec:
template:
metadata:
labels:
name: THIS
app: THAT
spec:
containers:
- image: APP:IMAGE
name: APP
command:
- app-start
volumeMounts:
- mountPath: "/var/www/html"
name: APP-VOLUME
volumes:
- name: APP-VOLUME
persistentVolumeClaim:
claimName: APP-CLAIM
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: APP-VOLUME
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
nfs:
path: /app
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: APP-CLAIM
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
selector:
matchLabels:
service: app
With a job like this, attaching to the same volume:
apiVersion: batch/v1
kind: Job
metadata:
name: APP-JOB
spec:
template:
metadata:
name: APP-JOB
labels:
app: THAT
spec:
containers:
- image: APP:IMAGE
name: APP-JOB
command:
- app-job
volumeMounts:
- mountPath: "/var/www/html"
name: APP-VOLUME
volumes:
- name: APP-VOLUME
persistentVolumeClaim:
claimName: APP-CLAIM

Create a scheduled pod that uses the Kubernetes API to run the command you want on the target pods, via the exec function. The pod image should contain the client libraries to access the API -- many of these are available or you can build your own.
For example, here is a solution using the Python client that execs to each ZooKeeper pod and runs a database maintenance command:
import time
from kubernetes import config
from kubernetes.client import Configuration
from kubernetes.client.apis import core_v1_api
from kubernetes.client.rest import ApiException
from kubernetes.stream import stream
import urllib3
config.load_incluster_config()
configuration = Configuration()
configuration.verify_ssl = False
configuration.assert_hostname = False
urllib3.disable_warnings()
Configuration.set_default(configuration)
api = core_v1_api.CoreV1Api()
label_selector = 'app=zk,tier=backend'
namespace = 'default'
resp = api.list_namespaced_pod(namespace=namespace,
label_selector=label_selector)
for x in resp.items:
name = x.spec.hostname
resp = api.read_namespaced_pod(name=name,
namespace=namespace)
exec_command = [
'/bin/sh',
'-c',
'opt/zookeeper/bin/zkCleanup.sh -n 10'
]
resp = stream(api.connect_get_namespaced_pod_exec, name, namespace,
command=exec_command,
stderr=True, stdin=False,
stdout=True, tty=False)
print("============================ Cleanup %s: ============================\n%s\n" % (name, resp if resp else "<no output>"))
and the associated Dockerfile:
FROM ubuntu:18.04
ADD ./cleanupZk.py /
RUN apt-get update \
&& apt-get install -y python-pip \
&& pip install kubernetes \
&& chmod +x /cleanupZk.py
CMD /cleanupZk.py
Note that if you have an RBAC-enabled cluster, you may need to create a service account and appropriate roles to make this API call possible. A role such as the following is sufficient to list pods and to run exec, such as the example script above requires:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: pod-list-exec
namespace: default
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: [""] # "" indicates the core API group
resources: ["pods/exec"]
verbs: ["create", "get"]
An example of the associated cron job:
apiVersion: v1
kind: ServiceAccount
metadata:
name: zk-maint
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: zk-maint-pod-list-exec
namespace: default
subjects:
- kind: ServiceAccount
name: zk-maint
namespace: default
roleRef:
kind: Role
name: pod-list-exec
apiGroup: rbac.authorization.k8s.io
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: zk-maint
namespace: default
labels:
app: zk-maint
tier: jobs
spec:
schedule: "45 3 * * *"
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
concurrencyPolicy: Forbid
jobTemplate:
spec:
template:
spec:
containers:
- name: zk-maint
image: myorg/zkmaint:latest
serviceAccountName: zk-maint
restartPolicy: OnFailure
imagePullSecrets:
- name: azure-container-registry

This seems like an anti-pattern. Why can't you just run your worker pod as a job pod?
Regardless you seem pretty convinced you need to do this. Here is what I would do.
Take your worker pod and wrap your shell execution in a simple webservice, it's 10 minutes of work with just about any language. Expose the port and put a service in front of that worker/workers. Then your job pods can simply curl ..svc.cluster.local:/ (unless you've futzed with dns).

It sounds as though you might want to run scheduled work within the pod itself rather than doing this at the Kubernetes level. I would approach this as a cronjob within the container, using traditional Linux crontab. Consider:
kind: Pod
apiVersion: v1
metadata:
name: shell
spec:
init-containers:
- name: shell
image: "nicolaka/netshoot"
command:
- /bin/sh
- -c
- |
echo "0 */5 * * * /opt/whatever/bin/do-the-thing" | crontab -
sleep infinity
If you want to track logs from those processes, that will require a fluentd type of mechanism to track those log files.

I managed to do this by creating a custom image with doctl (DigitalOcean's command line interface) and kubectl. The CronJob object would use these two commands to download the cluster configuration and run a command against a container.
Here is a sample CronJob:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: drupal-cron
spec:
schedule: "*/5 * * * *"
concurrencyPolicy: Forbid
jobTemplate:
spec:
template:
spec:
containers:
- name: drupal-cron
image: juampynr/digital-ocean-cronjob:latest
env:
- name: DIGITALOCEAN_ACCESS_TOKEN
valueFrom:
secretKeyRef:
name: api
key: key
command: ["/bin/bash","-c"]
args:
- doctl kubernetes cluster kubeconfig save drupster;
POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='{.items[0].metadata.name}');
kubectl exec $POD_NAME -c drupal -- vendor/bin/drush core:cron;
restartPolicy: OnFailure
Here is the Docker image that the CronJob uses: https://hub.docker.com/repository/docker/juampynr/digital-ocean-cronjob
If you are not using DigitalOcean, figure out how to download the cluster configuration so kubectl can use it. For example, with Google Cloud, you would have to download gcloud.
Here is the project repository where I implemented this https://github.com/juampynr/drupal8-do.

This one should help .
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: hello
spec:
schedule: "*/30 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: hello
image: busybox
args:
- /bin/sh
- -c
kubectl exec -it <podname> "sh script.sh ";
restartPolicy: OnFailure

Related

Populating a Containers environment values with mounted configMap in Kubernetes

I'm currently learning Kubernetes and recently learnt about using ConfigMaps for a Containers environment variables.
Let's say I have the following simple ConfigMap:
apiVersion: v1
data:
MYSQL_ROOT_PASSWORD: password
kind: ConfigMap
metadata:
creationTimestamp: null
name: mycm
I know that a container of some deployment can consume this environment variable via:
kubectl set env deployment mydb --from=configmap/mycm
or by specifying it manually in the manifest like so:
containers:
- env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
configMapKeyRef:
key: MYSQL_ROOT_PASSWORD
name: mycm
However, this isn't what I am after, since I'd to manually change the environment variables each time the ConfigMap changes.
I am aware that mounting a ConfigMap to the Pod's volume allows for the auto-updating of ConfigMap values. I'm currently trying to find a way to set a Container's environment variables to those stored in the mounted config map.
So far I have the following YAML manifest:
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: mydb
name: mydb
spec:
replicas: 1
selector:
matchLabels:
app: mydb
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: mydb
spec:
containers:
- image: mariadb
name: mariadb
resources: {}
args: ["export MYSQL_ROOT_PASSWORD=$(cat /etc/config/MYSQL_ROOT_PASSWORD)"]
volumeMounts:
- name: config-volume
mountPath: /etc/config
env:
- name: MYSQL_ROOT_PASSWORD
value: temp
volumes:
- name: config-volume
configMap:
name: mycm
status: {}
I'm attempting to set the MYSQL_ROOT_PASSWORD to some temporary value, and then update it to mounted value as soon as the container starts via args: ["export MYSQL_ROOT_PASSWORD=$(cat /etc/config/MYSQL_ROOT_PASSWORD)"]
As I somewhat expected, this didn't work, resulting in the following error:
/usr/local/bin/docker-entrypoint.sh: line 539: /export MYSQL_ROOT_PASSWORD=$(cat /etc/config/MYSQL_ROOT_PASSWORD): No such file or directory
I assume this is because the volume is mounted after the entrypoint. I tried adding a readiness probe to wait for the mount but this didn't work either:
readinessProbe:
exec:
command: ["sh", "-c", "test -f /etc/config/MYSQL_ROOT_PASSWORD"]
initialDelaySeconds: 5
periodSeconds: 5
Is there any easy way to achieve what I'm trying to do, or is it impossible?
So I managed to find a solution, with a lot of inspiration from this answer.
Essentially, what I did was create a sidecar container based on the alpine K8s image that mounts the configmap and constantly watches for any changes, since the K8s API automatically updates the mounted configmap when the configmap is changed. This required the following script, watch_passwd.sh, which makes use of inotifywait to watch for changes and then uses the K8s API to rollout the changes accordingly:
update_passwd() {
kubectl delete secret mysql-root-passwd > /dev/null 2>&1
kubectl create secret generic mysql-root-passwd --from-file=/etc/config/MYSQL_ROOT_PASSWORD
}
update_passwd
while true
do
inotifywait -e modify "/etc/config/MYSQL_ROOT_PASSWORD"
update_passwd
kubectl rollout restart deployment $1
done
The Dockerfile is then:
FROM docker.io/alpine/k8s:1.25.6
RUN apk update && apk add inotify-tools
COPY watch_passwd.sh .
After building the image (locally in this case) as mysidecar, I create the ServiceAccount, Role, and RoleBinding outlined here, adding rules for deployments so that they can be restarted by the sidecar.
After this, I piece it all together to create the following YAML Manifest (note that imagePullPolicy is set to Never, since I created the image locally):
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: mydb
name: mydb
spec:
replicas: 3
selector:
matchLabels:
app: mydb
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: mydb
spec:
serviceAccountName: secretmaker
containers:
- image: mysidecar
name: mysidecar
imagePullPolicy: Never
command:
- /bin/sh
- -c
- |
./watch_passwd.sh $(DEPLOYMENT_NAME)
env:
- name: DEPLOYMENT_NAME
valueFrom:
fieldRef:
fieldPath: metadata.labels['app']
volumeMounts:
- name: config-volume
mountPath: /etc/config
- image: mariadb
name: mariadb
resources: {}
envFrom:
- secretRef:
name: mysql-root-passwd
volumes:
- name: config-volume
configMap:
name: mycm
status: {}
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: secretmaker
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app: mydb
name: secretmaker
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create", "get", "delete", "list"]
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app: mydb
name: secretmaker
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: secretmaker
subjects:
- kind: ServiceAccount
name: secretmaker
namespace: default
---
It all works as expected! Hopefully this is able to help someone out in the future. Also, if anybody comes across this and has a better solution please feel free to let me know :)

Kubernetes: set environment variables from file?

My Kubernetes deployment has an initContainer which fetches a token from a URL. My app container (3rd party) then needs that token as an environment variable.
A possible approach would be: the initContainer creates a Kubernetes Secret with the token value; the app container uses the secret as an environment variable via env[].valueFrom.secretKeyRef.
Creating the Secret from the initContainer requires accessing the Kubernetes API from a Pod though, which tends to be a tad cumbersome. For example, directly accessing the REST API requires granting proper permissions to the pod's service account; otherwise, creating the secret will fail with
secrets is forbidden: User \"system:serviceaccount:default:default\"
cannot create resource \"secrets\" in API group \"\" in the namespace \"default\"
So I was wondering, isn't there any way to just write the token to a file on an emptyDir volume...something like this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-deployment
labels:
app: my-app
spec:
selector:
matchLabels:
app: my-app
template:
metadata:
labels:
app: my-app
spec:
initContainers:
- name: fetch-auth-token
image: curlimages/curl
command:
- /bin/sh
args:
- -c
- |
echo "Fetching token..."
url=https://gist.githubusercontent.com/MaxHorstmann/a99823d5aff66fe2ad4e7a4e2a2ee96b/raw/662c19aa96695e52384337bdbd761056bb324e72/token
curl $url > /auth-token/token
volumeMounts:
- mountPath: /auth-token
name: auth-token
...
volumes:
- name: auth-token
emptyDir: {}
... and then somehow use that file to populate an environment variable in the app container, similar to env[].valueFrom.secretKeyRef, along the lines of:
containers:
- name: my-actual-app
image: thirdpartyappimage
env:
- name: token
valueFrom:
fileRef:
path: /auth-token/token
# ^^^^ this does not exist
volumeMounts:
- mountPath: /auth-token
name: auth-token
Unfortunately, there's no env[].valueFrom.fileRef.
I considered overwriting the app container's command with a shell script which loads the environment variable from the file before launching the main command; however, the container image doesn't even contain a shell.
Is there any way to set the environment variable in the app container from a file?
Creating the Secret from the initContainer requires accessing the Kubernetes API from a Pod though, which tends to be a tad cumbersome...
It's not actually all that bad; you only need to add a ServiceAccount, Role, and RoleBinding to your deployment manifests.
The ServiceAccount manifest is minimal, and you only need it if you don't want to grant permissions to the default service account in your namespace:
apiVersion: v1
kind: ServiceAccount
metadata:
name: secretmaker
Then your Role grants access to secrets (we need create and delete permissions, and having get and list is handy for debugging):
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app: env-example
name: secretmaker
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- create
- get
- delete
- list
A RoleBinding connects the ServiceAccount to the Role:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app: env-example
name: secretmaker
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: secretmaker
subjects:
- kind: ServiceAccount
name: secretmaker
namespace: default
And with those permissions in place, the Deployment is relatively simple:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: env-example
name: env-example
namespace: env-example
spec:
selector:
matchLabels:
app: env-example
template:
metadata:
labels:
app: env-example
spec:
serviceAccountName: secretmaker
initContainers:
- command:
- /bin/sh
- -c
- |
echo "Fetching token..."
url=https://gist.githubusercontent.com/MaxHorstmann/a99823d5aff66fe2ad4e7a4e2a2ee96b/raw/662c19aa96695e52384337bdbd761056bb324e72/token
curl $url -o /tmp/authtoken
kubectl delete secret authtoken > /dev/null 2>&1
kubectl create secret generic authtoken --from-file=AUTH_TOKEN=/tmp/authtoken
image: docker.io/alpine/k8s:1.25.6
name: create-auth-token
containers:
- name: my-actual-app
image: docker.io/alpine/k8s:1.25.6
command:
- sleep
- inf
envFrom:
- secretRef:
name: authtoken
The application container here is a no-op that runs sleep inf; that gives you the opportunity to inspect the environment by running:
kubectl exec -it deployment/env-example -- env
Look for the AUTH_TOKEN variable created by our initContainer.
All the manifests mentioned here can be found in this repository.

Executing a Script using a Cronjob Kubernetes Cluster

I have a 3 node K8 v1.21 cluster in AWS and looking for SOLID config to run a script using a cronjob. I have seen many documents on here and Google using cronjob and hostPath to Persistent Volumes/Claims to using ConfigMaps, the list goes one.
I keep getting "Back-off restarting failed container/CrashLoopBackOff" errors.
Any help is much appreciated.
cronjob.yaml
The script I am trying to run is basic for testing only
#! /bin/<br/>
kubectl create deployment nginx --image=nginx
Still getting the same error.
kubectl describe pod/xxxx
This hostPath in AWS cluster created using eksctl works.
apiVersion: v1
kind: Pod
metadata:
name: redis-hostpath
spec:
containers:
- image: redis
name: redis-container
volumeMounts:
- mountPath: /test-mnt
name: test-vol
volumes:
- name: test-vol
hostPath:
path: /test-vol
UPDATE
Tried running your config in GCP on a fresh cluster. Only thing I changed was the /home/script.sh to /home/admin/script.sh
Did you test this on your cluster?
Warning FailedPostStartHook 5m27s kubelet Exec lifecycle hook ([/home/mchung/script.sh]) for Container "busybox" in Pod "dumb-job-1635012900-qphqr_default(305c4ed4-08d1-4585-83e0-37a2bc008487)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to create exec "0f9f72ccc6279542f18ebe77f497e8c2a8fd52f8dfad118c723a1ba025b05771": cannot exec in a deleted state: unknown, message: ""
Normal Killing 5m27s kubelet FailedPostStartHook
Assuming you're running it in a remote multi-node cluster (since you mentioned AWS in your question), hostPath is NOT an option there for volume mount. Your best choice would be to use a ConfigMap and use it as volume mount.
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-script
data:
script.sh: |
# write down your script here
And then:
apiVersion: batch/v1
kind: CronJob
metadata:
name: redis-job
spec:
schedule: '*/5 * * * *'
jobTemplate:
spec:
template:
spec:
containers:
- name: redis-container
image: redis
args:
- /bin/sh
- -c
- /home/user/script.sh
volumeMounts:
- name: redis-data
mountPath: /home/user/script.sh
subPath: script.sh
volumes:
- name: redis-data
configMap:
name: redis-script
Hope this helps. Let me know if you face any difficulties.
Update:
I think you're doing something wrong. kubectl isn't something you should run from another container / pod. Because it requires the necessary binary to be existed into that container and an appropriate context set. I'm putting a working manifest below for you to understand the whole concept of running a script as a part of cron job:
apiVersion: v1
kind: ConfigMap
metadata:
name: script-config
data:
script.sh: |-
name=StackOverflow
echo "I love $name <3"
---
apiVersion: batch/v1
kind: CronJob
metadata:
name: dumb-job
spec:
schedule: '*/1 * * * *' # every minute
jobTemplate:
spec:
template:
spec:
containers:
- name: busybox
image: busybox:stable
lifecycle:
postStart:
exec:
command:
- /home/script.sh
volumeMounts:
- name: some-volume
mountPath: /home/script.sh
volumes:
- name: some-volume
configMap:
name: script-config
restartPolicy: OnFailure
What it'll do is it'll print some texts in the STDOUT in every minute. Please note that I have put only the commands that container is capable to execute, and kubectl is certainly not one of them which exists in that container out-of-the-box. I hope that is enough to answer your question.

How to delete Kubernetes job automatically after job completion

I am running a kubernetes job on GKE and want to delete the job automatically after the job is completed.
Here is my configuration file for the job.
I set ttlSecondsAfterFinished: 0 but the job was not deleted automatically.
Am I missing something?
cluster / node version: 1.12.8-gke.10
apiVersion: batch/v1
kind: Job
metadata:
name: myjob
spec:
# automatically clean up finished job
ttlSecondsAfterFinished: 0
template:
metadata:
name: myjob
spec:
containers:
- name: myjob
image: gcr.io/GCP_PROJECT/myimage:COMMIT_SHA
command: ["bash"]
args: ["deploy.sh"]
# Do not restart containers after they exit
restartPolicy: Never
Looks like this feature is still not available on GKE now.
https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/
https://cloud.google.com/kubernetes-engine/docs/concepts/alpha-clusters#about_feature_stages
To ensure stability and production quality, normal GKE clusters only enable features that
are beta or higher. Alpha features are not enabled on normal clusters because they are not
production-ready or upgradeable.
It depends how did you create job.
If you are using CronJob you can use spec.successfulJobsHistoryLimit and spec.failedJobsHistoryLimit and set values to 0. It will say K8s to not sotre any previously finished jobs.
If you are creating pods using YAMLs you have to deleted it manually. However you can also set CronJob to execute command each 5 minutes.
kubectl delete job $(kubectl get job -o=jsonpath='{.items[?(#.status.succeeded==1)].metadata.name}')
It will delete all jobs with status succeded.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: jp-runner
rules:
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["get", "list", "delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: jp-runner
subjects:
- kind: ServiceAccount
name: sa-jp-runner
roleRef:
kind: Role
name: jp-runner
apiGroup: ""
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: sa-jp-runner
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: clean-jobs
spec:
concurrencyPolicy: Forbid
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
serviceAccountName: sa-jp-runner
containers:
- name: clean-jobs
image: bitnami/kubectl:latest
command:
- /bin/sh
- -c
- kubectl delete jobs $(kubectl get jobs -o=jsonpath='{.items[?(#.status.succeeded==1)].metadata.name}')
restartPolicy: Never
backoffLimit: 0

Kubernetes puzzle: Populate environment variable from file (mounted volume)

I have a Pod or Job yaml spec file (I can edit it) and I want to launch it from my local machine (e.g. using kubectl create -f my_spec.yaml)
The spec declares a volume mount. There would be a file in that volume that I want to use as value for an environment variable.
I want to make it so that the volume file contents ends up in the environment variable (without me jumping through hoops by somehow "downloading" the file to my local machine and inserting it in the spec).
P.S. It's obvious how to do that if you have control over the command of the container. But in case of launching arbitrary image, I have no control over the command attribute as I do not know it.
apiVersion: batch/v1
kind: Job
metadata:
generateName: puzzle
spec:
template:
spec:
containers:
- name: main
image: arbitrary-image
env:
- name: my_var
valueFrom: <Contents of /mnt/my_var_value.txt>
volumeMounts:
- name: my-vol
path: /mnt
volumes:
- name: my-vol
persistentVolumeClaim:
claimName: my-pvc
You can create deployment with kubectl endless loop which will constantly poll volume and update configmap from it. After that you can mount created configmap into your pod. It's a little bit hacky but will work and update your configmap automatically. The only requirement is that PV must be ReadWriteMany or ReadOnlyMany (but in that case you can mount it in read-only mode to all pods).
apiVersion: v1
kind: ServiceAccount
metadata:
name: cm-creator
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: cm-creator
rules:
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["create", "update", "get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cm-creator
namespace: default
subjects:
- kind: User
name: system:serviceaccount:default:cm-creator
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: cm-creator
apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: cm-creator
namespace: default
labels:
app: cm-creator
spec:
replicas: 1
serviceAccountName: cm-creator
selector:
matchLabels:
app: cm-creator
template:
metadata:
labels:
app: cm-creator
spec:
containers:
- name: cm-creator
image: bitnami/kubectl
command:
- /bin/bash
- -c
args:
- while true;
kubectl create cm myconfig --from-file=my_var=/mnt/my_var_value.txt --dry-run -o yaml | kubectl apply -f-;
sleep 60;
done
volumeMounts:
- name: my-vol
path: /mnt
readOnly: true
volumes:
- name: my-vol
persistentVolumeClaim:
claimName: my-pvc