Kubernetes pod Rollback and Restart - kubernetes

Deployment resource object is still not supported in our cluster and not enabled.
We are using Pod resource object Yaml file. something like below:
apiVersion: v1
kind: Pod
metadata:
name: sample-test
namespace: default
spec:
automountServiceAccountToken: false
containers:
I have explored patch and Put rest api for Pod(Kubectl patch and replace) - it will update to new image version and pod restarts.
I need help in below:
When the image version is same, it will not update and pod will not restart.
How can i acheive Pod restart, is there any API for this or any alternate
approach for this. Because My pod also refers configmap and secret. After i
make changes to secret, i want to restart pod so that it can take updated
value.
Suppose when patch applied with new container image and it fails status is failed, I want to rollback to previous version, How can i acheive this with standalone pod without using deployment. Is there any alternate approach.

Achieving solutions for your scenario, can be handled like this:
When the image version is same, it will not update and pod will not restart. How can i acheive Pod restart, is there any API for this or any alternate approach for this. Because My pod also refers configmap and secret. After i make changes to secret, i want to restart pod so that it can take updated value
Create a new secret/configmap each time and update the pod yaml to use the new configmap/secret rather than the old name.
Suppose when patch applied with new container image and it fails status is failed, I want to rollback to previous version, How can i acheive this with standalone pod without using deployment. Is there any alternate approach
Before you do a Pod update, get the current Pod yaml using kubectl like this,
kubectl get pod <pod-name> -o yaml -n <namespace>
After getting the yaml, generate the new pod yaml and apply it. In case of failure, clean up the new resources created(configmaps & secrets) and apply the older version of pod to achieve rollback

Related

How to automatically roll a CronJob on ConfigMap change

When my ConfigMap changes, I'd like my CronJob to restart.
There's a trick in Helm to make this happen for Deployments using an annotation that contains the SHA of the ConfigMap (see here or here).
This works for my Deployment, doing kubectl get deployments show the AGE is just a couple minutes.
But not my CronJob, doing kubectl get cronjobs, the "AGE" is still hours old implying they haven't restarted.
So, it turns out CronJobs don't need to be restarted when the ConfigMap changes anyway. Everytime a Job is created, the Pod that spins up to execute the Job seems to use the latest ConfigMap values anyway.

Is kubectl apply safe enough to update all pods no matter how they were created?

A pod can be created by Deployment or ReplicaSet or DaemonSet, if I am updating a pod's container specs, is it OK for me to simply modify the yaml file that created the pod? Would it be erroneous once I have done that?
Brief Question:
Is kubectl apply -f xxx.yml the silver bullet for all pod update?
...if I am updating a pod's container specs, is it OK for me to simply modify the yaml file that created the pod?
The fact that the pod spec is part of the controller spec (eg. deployment, daemonset), to update the container spec you naturally start with the controller spec. Also, a running pod is largely immutable, there isn't much you can change directly unless you do a replace - which is what the controller already doing.
you should not make changes to the pods directly, but update the spec.template.spec section of the deployment used to create the pods.
reason for this is that the deployment is the controller that manages the replicasets and therefore the pods that are created for your application. that means if you apply changes to the pods manifest directly, and something like a pod rescheduling/restart happens, the changes made to the pod will be lost because the replicaset will recreate the pod according to its own specification and not the specification of the last running pod.
you are safe to use kubectl apply to apply changes to existing resources but if you are unsure, you can always extract the current state of the deployment from kubernetes and pipe that output into a yaml file to create a backup:
kubectl get deploy/<name> --namespace <namespace> -o yaml > deploy.yaml
another option is to use the internal rollback mechanism of kubernetes to restore a previous revision of your deployment. see https://learnk8s.io/kubernetes-rollbacks for more infos on that.

Difference between kubernetes recreate update strategy vs simply uninstall, install

When deciding on update strategy for a kubernetes application, there is an option to use Recreate strategy.
How would this be different from just uninstalling and installing the app?
I assume that by "just uninstalling and installing the app" you mean complete deletion of your deployment e.g.:
kubectl delete deployment nginx-deployment
and creating it again:
kubectl apply -f nginx-deployment.yaml
Note that when using Recreate strategy there is no complete deletion of the deployment so there is fundamental difference here. By choosing this strategy you only inform kubernetes that all the pods managed by your deployment should be deleted and recreated when you update them (e.g. you update the image version of the container) rather than deleting and recreating their new versions one at a time what takes place when using RollingUpdate strategy. This way you make sure that certain number of pods serving an old version of the application are still available when the update occurs and pods with a new version of the image appear.
When you delete your deployment and create a new one, your new deployment has nothing to do with the old one. In other words, completely new Deployment resource is created and no history of the changes you made is preserved.
I believe the best way of explaining things is always an example. So let's move on to the following one.
Let's say you've created a new nginx deployment based on your yaml manifest:
kubectl apply -f nginx-deployment.yaml
and then you decided to update the image version, either by editing nginx-deployment.yaml manifest and re-applying it or this way:
kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.161 --record=true
In either case you will be able to check rollout history by running:
kubectl rollout history deployment nginx-deployment
and you should see something like this:
$ kubectl rollout history deployment nginx-deployment
deployment.apps/nginx-deployment
REVISION CHANGE-CAUSE
1 kubectl apply --filename=nginx-deployment.yaml --record=true
2 kubectl set image deployment nginx-deployment nginx=nginx:1.16.1 --record=true
When you have rollout history you're able to undo your latest change and go back to the previous revision:
kubectl rollout undo deployment.v1.apps/nginx-deployment
Now your rollout history for this deployment may look like this:
$ kubectl rollout history deployment nginx-deployment
deployment.apps/nginx-deployment
REVISION CHANGE-CAUSE
2 kubectl set image deployment nginx-deployment nginx=nginx:1.16.1 --record=true
3 kubectl apply --filename=nginx-deployment.yaml --record=true
When you simply delete your deployment and recreate it again you will have nothing in rollout history for newly created deployment and you won't be able to roll it back to some older revision in such an easy way.
Recreate strategy will delete your Pods and then add new Pods - you will get short downtime, but on the other side you will not use much extra resources during upgrade.
You typically want RollingUpgrade since that takes a few Pods at a time and you can deploy stateless applications without downtime.

Reapply updated configuration to a statefulset, using Helm

I have a rather peculiar use case. Specifically, prior to the deployment of my statefulset I am deploying a ConfigMap which contains an environment variable setting (namely RECREATE_DATADIR) which instructs the pod's container to create a new data structure on the file system.
However, during the typical lifetime of the container the data structure should NOT be recreated. Hence, right after the pod is successfully running, I am changing the ConfigMap and then reapply it. Hence - if the pod ever fails, it won't recreate the data directory structure when it respawns.
How can I achieve this same result using Helm charts?
You can create a job as part of your helm chart, with the post-install helm hook which will have configmap edit permissions, will use a kubectl image (bitnami/kubectl for example), and it will patch the configmap to false using kubectl commands.

Kubernetes kubectl patch pod vs delete/create pod

I have a Pod - I want to update to latest version of container image, if it fails then want to revert it back to earlier version of container image.
Which approach of below will be good to update to latest or earlier container image.
1> using kubectl patch is good option?
2> deleting the pod and again creating the pod is good option?
what are benefits/disadvantages of both approach.
kubectl patch pod test-pod -p '{"spec":{"containers":[{"name":"test","image":"1.0"}]}}'
kubectl delete pod test-pod and kubectl apply -f testpod.yaml
From the docs here one of the use cases of deployment
Rollback to an earlier Deployment revision if the current state of the
Deployment is not stable. Each rollback updates the revision of the
Deployment.
So use deployment to perform rolling update and rollback when necessary
kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.161 --record=true
Check rollout history
kubectl rollout history deployment.v1.apps/nginx-deployment
Rollback to a previous version
kubectl rollout undo deployment.v1.apps/nginx-deployment
based on the scenario put up by you, you can choose to look for doing canary deployments by using the rolling update strategy, if the new deployment's containers fail to serve the load then the old ones will not be destroyed and load will continuously be served by pre-existing containers.
The other way can be to keep deployment history, you can define the number of manifests you want to be saved by kubernetes and then you can manually rollback to your desired version
And if you are not aware about what version carries which image then you can go for patch command, as portrayed by you in the question