Automated way to create multiple kubernetes Job manifests - kubernetes

Question

Cron template
kind: CronJob
metadata:
name: some-example
namespace: some-example
spec:
schedule: "* 12 * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: some-example
image: gcr.io/some-example/some-example
imagePullPolicy: Always
env:
- name: REPO_URL
value: https://example.com/12/some-example
I need to create multiple Job files with different URLs of REPO_URL over 100s save in a file. I am looking for a solution where I can set Job template and get the required key:value from another file.
so far I've tried https://kustomize.io/, https://ballerina.io/, and https://github.com/mikefarah/yq. But I am not able to find a great example to fit the scenario.

Answer 1

That would be pretty trivial with yq and a shell script. Assuming
your template is in cronjob.yml, we can write something like this:
let count=0
while read url; do
yq -y '
.metadata.name = "some-example-'"$count"'"|
.spec.jobTemplate.spec.template.spec.containers[0].env[0].value = "'"$url"'"
' cronjob.yml
echo '---'
let count++
done < list_of_urls.txt | kubectl apply -f-
E.g., if my list_of_urls.txt contains:
https://google.com
https://stackoverflow.com
The above script will produce:
[...]
metadata:
name: some-example-0
namespace: some-example
spec:
[...]
env:
- name: REPO_URL
value: https://google.com
---
[...]
metadata:
name: some-example-1
namespace: some-example
spec:
[...]
env:
- name: REPO_URL
value: https://stackoverflow.com
You can drop the | kubectl apply -f- if you just want to see the
output instead of actually creating resources.
Or for more structured approach, we could use Ansible's k8s
module:
- hosts: localhost
gather_facts: false
tasks:
- k8s:
state: present
definition:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: "some-example-{{ count }}"
namespace: some-example
spec:
schedule: "* 12 * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: some-example
image: gcr.io/some-example/some-example
imagePullPolicy: Always
env:
- name: REPO_URL
value: "{{ item }}"
loop:
- https://google.com
- https://stackoverflow.com
loop_control:
index_var: count
Assuming that the above is stored in playbook.yml, running this with
ansible-playbook playbook.yml would create the same resources as the
earlier shell script.