I'm trying to apply the same job history limits to a number of CronJobs using a patch like the following, named kubeJobHistoryLimit.yml:
apiVersion: batch/v1beta1
kind: CronJob
spec:
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 1
My kustomization.yml looks like:
bases:
- ../base
configMapGenerator:
- name: inductions-config
env: config.properties
patches:
- path: kubeJobHistoryLimit.yml
target:
kind: CronJob
patchesStrategicMerge:
- job_specific_patch_1.yml
- job_specific_patch_2.yml
...
resources:
- secrets-uat.yml
And at some point in my CI pipeline I have:
kubectl --kubeconfig $kubeconfig apply --force -k ./
The kubectl version is 1.21.9.
The issue is that the job history limit values don't seem to be getting picked up. Is there something wrong w/ the configuration or the version of K8s I'm using?
With kustomize 4.5.2, your patch as written doesn't apply; it fails with:
Error: trouble configuring builtin PatchTransformer with config: `
path: kubeJobHistoryLimit.yml
target:
kind: CronJob
`: unable to parse SM or JSON patch from [apiVersion: batch/v1
kind: CronJob
spec:
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 1
]
This is because it's missing metadata.name, which is required, even if it's ignored when patching multiple objects. If I modify the patch to look like this:
apiVersion: batch/v1
kind: CronJob
metadata:
name: ignored
spec:
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 1
It seems to work.
If I have base/cronjob1.yaml that looks like:
apiVersion: batch/v1
kind: CronJob
metadata:
name: cronjob1
spec:
failedJobsHistoryLimit: 2
successfulJobsHistoryLimit: 5
jobTemplate:
spec:
template:
spec:
containers:
- command:
- sleep
- 60
image: docker.io/alpine:latest
name: example
schedule: 30 3 * * *
Then using the above patch and a overlay/kustomization.yaml like this:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../base
patches:
- path: kubeJobHistoryLimit.yml
target:
kind: CronJob
I see the following output from kustomize build overlay:
apiVersion: batch/v1
kind: CronJob
metadata:
name: cronjob2
spec:
failedJobsHistoryLimit: 1
jobTemplate:
spec:
template:
spec:
containers:
- command:
- sleep
- 60
image: docker.io/alpine:latest
name: example
schedule: 30 3 * * *
successfulJobsHistoryLimit: 1
You can see the two attributes have been updated correctly.
Related
when i update my imagestream and then trigger a run of a cronjob, the cronjob will still use the previous imagestream instead of pulling latest. despite the fact i have configured the cronjob to always pull.
the way ive been testing this is to:
push a new image to the stream, and verify the imagestream is updated
check the cronjob obj and verify the image associated to the container still has the old image stream hash
trigger a new run of the cronjob, which I would think would pull a new image since the pull policy is always. -- but it does not, the cronjob starts with a container using the old image stream.
heres the yaml:
apiVersion: template.openshift.io/v1
kind: Template
metadata:
name: cool-cron-job-template
parameters:
- name: ENVIRONMENT
displayName: Environment
required: true
objects:
- apiVersion: v1
kind: ImageStream
metadata:
name: cool-cron-job
namespace: cool-namespace
labels:
app: cool-cron-job
owner: cool-owner
spec:
lookupPolicy:
local: true
- apiVersion: batch/v1
kind: CronJob
metadata:
name: cool-cron-job-cron-job
namespace: cool-namespace
labels:
app: cool-cron-job
owner: cool-owner
spec:
schedule: "10 0 1 * *"
concurrencyPolicy: "Forbid"
startingDeadlineSeconds: 200
suspend: false
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 1
jobTemplate:
spec:
template:
metadata:
labels:
app: cool-cron-job
cronjob: "true"
annotations:
alpha.image.policy.openshift.io/resolve-names: '*'
spec:
dnsPolicy: ClusterFirst
restartPolicy: OnFailure
securityContext: { }
terminationGracePeriodSeconds: 0
containers:
- command: [ "python", "-m", "cool_cron_job.handler" ]
imagePullPolicy: Always
name: cool-cron-job-container
image: cool-cron-job:latest
This is my yaml file that i am trying to use for cronJob creation. I am getting error like unknown field "container" in io.k8s.api.core.v1.PodSpec,
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: abc-service-cron-job
spec:
schedule: "* * * * *"
jobTemplate:
spec:
template:
spec:
container:
- name: abc-service-cron-job
image: docker.repo1.xyz.com/hui-services/abc-application/REPLACE_ME
imagePullPolicy: Always
command:
- /bin/sh
- -c
- date; echo Hello from the Kubernetes cluster
restartPolicy: OnFailure
apiVersion: batch/v1beta1
kind: CronJob
metadata:
...
spec:
...
jobTemplate:
spec:
template:
spec:
containers: # <-- you have spelling error here, should be "containers"
...
I want to periodically restart the deployment using k8s cronjob.
Please check what is the problem with the yaml file.
When I execute the command from the local command line, the deployment restarts normally, but it seems that the restart is not possible with cronjob.
e.g $ kubectl rollout restart deployment my-ingress -n my-app
my cronjob yaml file
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: deployment-restart
namespace: my-app
spec:
schedule: '0 8 */60 * *'
jobTemplate:
spec:
backoffLimit: 2
activeDeadlineSeconds: 600
template:
spec:
serviceAccountName: deployment-restart
restartPolicy: Never
containers:
- name: kubectl
image: bitnami/kubectl:latest
command:
- 'kubectl'
- 'rollout'
- 'restart'
- 'deployment/my-ingress -n my-app'
as David suggested run cron of kubectl is like by executing the command
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: hello
spec:
schedule: "*/5 * * * *"
jobTemplate:
spec:
template:
spec:
serviceAccountName: sa-jp-runner
containers:
- name: hello
image: bitnami/kubectl:latest
command:
- /bin/sh
- -c
- kubectl rollout restart deployment my-ingress -n my-app
restartPolicy: OnFailure
i would also suggest you to check the role and service account permissions
example for ref :
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: kubectl-cron
rules:
- apiGroups:
- extensions
- apps
resources:
- deployments
verbs:
- 'patch'
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: kubectl-cron
namespace: default
subjects:
- kind: ServiceAccount
name: sa-kubectl-cron
namespace: default
roleRef:
kind: Role
name: kubectl-cron
apiGroup: ""
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: sa-kubectl-cron
namespace: default
---
I have a HorizontalPodAutoscalar to scale my pods based on CPU. The minReplicas here is set to 5:
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-web
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp-web
minReplicas: 5
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 50
I've then added Cron jobs to scale up/down my horizontal pod autoscaler based on time of day:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: production
name: cron-runner
rules:
- apiGroups: ["autoscaling"]
resources: ["horizontalpodautoscalers"]
verbs: ["patch", "get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: cron-runner
namespace: production
subjects:
- kind: ServiceAccount
name: sa-cron-runner
namespace: production
roleRef:
kind: Role
name: cron-runner
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: sa-cron-runner
namespace: production
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: django-scale-up-job
namespace: production
spec:
schedule: "56 11 * * 1-6"
successfulJobsHistoryLimit: 0 # Remove after successful completion
failedJobsHistoryLimit: 1 # Retain failed so that we see it
concurrencyPolicy: Forbid
jobTemplate:
spec:
template:
spec:
serviceAccountName: sa-cron-runner
containers:
- name: django-scale-up-job
image: bitnami/kubectl:latest
command:
- /bin/sh
- -c
- kubectl patch hpa myapp-web --patch '{"spec":{"minReplicas":8}}'
restartPolicy: OnFailure
----
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: django-scale-down-job
namespace: production
spec:
schedule: "30 20 * * 1-6"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 0 # Remove after successful completion
failedJobsHistoryLimit: 1 # Retain failed so that we see it
jobTemplate:
spec:
template:
spec:
serviceAccountName: sa-cron-runner
containers:
- name: django-scale-down-job
image: bitnami/kubectl:latest
command:
- /bin/sh
- -c
- kubectl patch hpa myapp-web --patch '{"spec":{"minReplicas":5}}'
restartPolicy: OnFailure
This works really well, except that now when I deploy it overwrites this minReplicas value with the minReplicas in the HorizontalPodAutoscaler spec (in my case, this is set to 5)
I'm deploying my HPA using kubectl apply -f ~/autoscale.yaml
Is there a way of handling this situation? Do I need to create some kind of shared logic so that my deployment scripts can work out what the minReplicas value should be? Or is there a simpler way of handling this?
I think you could also consider the following two options:
Use helm to manage the life-cycle of your application with lookup function:
The main idea behind this solution is to query the state of specific cluster resource (here HPA) before trying to create/recreate it with helm install/upgrade commands.
Helm.sh: Docs: Chart template guide: Functions and pipelines: Using the lookup function
I mean to check the current minReplicas value each time before you upgrade your application stack.
Manage the HPA resource separately to application manifest files
Here you can handover this task to a dedicated HPA operator, which can coexist with your CronJobs that adjust minReplicas according specific schedule:
Banzaicloud.com: Blog: K8S HPA Operator
I am running a kubernetes job on GKE and want to delete the job automatically after the job is completed.
Here is my configuration file for the job.
I set ttlSecondsAfterFinished: 0 but the job was not deleted automatically.
Am I missing something?
cluster / node version: 1.12.8-gke.10
apiVersion: batch/v1
kind: Job
metadata:
name: myjob
spec:
# automatically clean up finished job
ttlSecondsAfterFinished: 0
template:
metadata:
name: myjob
spec:
containers:
- name: myjob
image: gcr.io/GCP_PROJECT/myimage:COMMIT_SHA
command: ["bash"]
args: ["deploy.sh"]
# Do not restart containers after they exit
restartPolicy: Never
Looks like this feature is still not available on GKE now.
https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/
https://cloud.google.com/kubernetes-engine/docs/concepts/alpha-clusters#about_feature_stages
To ensure stability and production quality, normal GKE clusters only enable features that
are beta or higher. Alpha features are not enabled on normal clusters because they are not
production-ready or upgradeable.
It depends how did you create job.
If you are using CronJob you can use spec.successfulJobsHistoryLimit and spec.failedJobsHistoryLimit and set values to 0. It will say K8s to not sotre any previously finished jobs.
If you are creating pods using YAMLs you have to deleted it manually. However you can also set CronJob to execute command each 5 minutes.
kubectl delete job $(kubectl get job -o=jsonpath='{.items[?(#.status.succeeded==1)].metadata.name}')
It will delete all jobs with status succeded.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: jp-runner
rules:
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["get", "list", "delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: jp-runner
subjects:
- kind: ServiceAccount
name: sa-jp-runner
roleRef:
kind: Role
name: jp-runner
apiGroup: ""
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: sa-jp-runner
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: clean-jobs
spec:
concurrencyPolicy: Forbid
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
serviceAccountName: sa-jp-runner
containers:
- name: clean-jobs
image: bitnami/kubectl:latest
command:
- /bin/sh
- -c
- kubectl delete jobs $(kubectl get jobs -o=jsonpath='{.items[?(#.status.succeeded==1)].metadata.name}')
restartPolicy: Never
backoffLimit: 0