I have a pod that is meant to run a code excerpt and exit afterwards. I do not want this pod to restart after exiting, but apparently it is not possible to set a restart policy in Kubernetes (see here and here).
Therefore my question is: how can I implement a pod that runs only once?
Thank you
You need to deploy a job. A deployment is meant to keep the containers running all the time. Give a check on:
https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/
Related
I am using Spark, which has a predefined script to create a pod in my kubernetes cluster.
After the pod is created and running, I want to check if it's still alive. I could do this by using a livenessProbe, however this is configured in the configuration file for the Pod, which I do not have control over, as my pod is created by Spark and I cannot change its config file.
So my question is, after the pod has been already created and running, how can I change the configuration for it so that is uses livenessProbe?
Or is there any other way to check the liveness of the pod?
I am a beginner to Kubernetes, sorry for this question!
After a Pod is created you can't change the livenessProe definition.
You could use a second Pod to report on the status of your workload, if that works for your use case.
The other option is to use a Mutating Admission Controller to modify the Pod definition from your Spark script, though I would consider this not exactly beginner friendly.
https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook
https://www.trion.de/news/2019/04/25/beispiel-kubernetes-mutating-admission-controller.html
https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/
I have a problem statement where in there is a Kubernetes cluster and I have some pods running on it.
Now, I want some functions/processes to run once per deployment, independent of number of replicas.
These processes use the same image like the image in deployment yaml.
I cannot use initcontainers and sidecars, because they will run along with main container on pod for each replica.
I tried to create a new image and then a pod out of it. But this pod keeps on running, which is not good for cluster resource, as it should be destroyed after it has done its job. Also, the main container depends on the completion on this process, in order to run the "command" part of K8 spec.
Looking for suggestions on how to tackle this?
Theoretically, You could write an admission controller webhook for intercepting create/update deployments and triggering your functions as you want. If your functions need to be checked, use ValidatingWebhookConfiguration for validating the process and then deny or accept commands.
I am trying to deploy updates to pods. However I want the current pods to terminate only when all the containers inside the pod have terminated and their process is complete.
The new pods can keep waiting to start untill all container in the old pods have completed. We have a mechanism to stop old pods from picking up new tasks and therefore they should eventually terminate.
It's okay if twice the pods exist at some instance of time. I tried finding solution for this in kubernetes docs but wan't successful. Pointers on how / if this is possible would be helpful.
well I guess then you may have to create a duplicate kind of deployment with new image as required and change the selector in service to new deployment, which will prevent external traffic from entering pre-existing pods and new calls can go to new pods. Then later you can check for something like -
Kubectl top pods -c containers
and if the load appears to be static and low, then preferrably you can delete the old pods related deployment later.
But for this thing everytime the service selectors have to be updated and likely for keeping track of things you can append the git commit hash to the service selector to keep it unique everytime.
But rollback to previous versions if required from inside Kubernetes cluster will be difficult, so preferably you can trigger the wanted build again.
I hope this makes some sense !!
Assume deployment like that:
Deployment contains two types of pods Config and App
Each App pod to start needs to have access to Config pod
There is always only one Config pod
Already launched App pods can work without access to Config pod service
Situation I would like to manage:
Node containing some of App pods and Config pod going down for any reason
On another Node first starts Config pod
After Config pod is successfully started App pods are launched
Already read about:
InitContainers - couldn't find an information if Config pod would be of type Init if in above situation it would rerun - I think not
StatueFullSet - I cannot find a way how this could help me in that situation
From my perspective I was thinking about a loop for App pods before running target application, that would wait for Config pod to come up and in case of unavailability after timeout force them to fail. But I'm not sure if that is best practice, would like better to handle this with Kubernetes configuration rather that with such script.
You would use either code in your app or an initContainer to block until a config pod is available. Combine this with a readinessProbe that checks if the app is up. Doing the block-and-retry loop in your own code is a bit more work but recommended since you can more carefully control the behavior. This means that app pods can launch whenever, but they won't be marked as ready for traffic until the initialize.
I have a cronjob that runs and does things regularly. I want to send a slack message with the technosophos/slack-notify container when that cronjob fails.
Is it possible to have a container run when a pod fails?
There is nothing built in for this that i am aware of. You could use a web hook to get notified when a pod changes and look for state stuff in there. But you would have to build the plumbing yourself or look for an existing third party tool.
Pods and Jobs are different things. If you want to wait for a job that has failed and send an email after it has, you can do something like this in bash:
while true
do
kubectl wait --for=condition=failed job/myjob
kubectl run --image=technosophos/slack-notify --env="EMAIL=failure#yourdomain.com"
done
To the question: Is it possible to have a container run when a pod fails?
Yes , although there is nothing out of the box right now , but you can define a health check.
Then you can write a cron job , or a Jenkins job , or a custom kubernetes cluster service/controller that checks/probes that health check regularly and if the health check fails then you can run a container based on that.