The Kubernetes StatefulSet RollingUpdate strategy deletes and recreates each Pod in order. I am interested in updating a StatefulSet by recreating a pod and then deleting the old Pod (note the reversal), one-by-one.
This is interesting to me because:
There is no reduction in the number of Ready Pods. I understand this is how a normal Deployment update works too (i.e. a Pod is only deleted after the new Pod replacing it is Ready).
More importantly, it allows me to perform application-specific live migration during my StatefulSet upgrade. I would like to "migrate" data from (old) pod-i to (new) pod-i before (old) pod-i is terminated (I would implement this in (new) pod-i readiness logic).
Is such an update strategy possible?
This is inherently possible with Deployments, but not StatefulSets. StatefulSets are used when you care strongly about an exact number of replicas with well known names. Deployments are used for more elastic workloads.
You may be able to accomplish your goal by using multiple StatefulSets- e.g. instead of a StatefulSet of 3 replicas, use 3 StatefulSets of 1 replica each. Then deploy an additional StatefulSet for your data migration before removing one of the previous ones.
Alternatively, this may be a use case for an Operator to manage the application.
No, because pods have specific names based on their ordinal (-0, -1, etc) and there can only be one pod at a time with a given name. Deployments and DaemonSets can burst for updates because their names are randomized so it doesn't matter what order you do things in.
Related
This question already has answers here:
k8s - Why we need ReplicaSet when we have Deployments
(2 answers)
Kubernetes: what's the difference between Deployment and Replica set?
(4 answers)
Closed last month.
I know that deployment uses replicaset underneath it, has revision control, creates another replicaset during rolling upgrade/downgrade.
I want to know what is the scenario in which only replicaset can be used and deployment can't be used.
ReplicaSet's purpose is to maintain a stable set of replica Pods running at any given time and it checks how many pods need to maintain bases on which it creates or deletes the pods. ReplicaSet then fulfills its purpose by creating and deleting Pods as needed to reach the desired number. ReplicaSets can be used independently. With ReplicaSet you define the number of replicas you want to run for a particular service. You would have those many replicas running.
Whereas Deployment is the advancement of replica sets. When you use Deployments you don't have to worry about managing the ReplicaSets that they create. Deployments own and manage their ReplicaSets. As such, it is recommended to use Deployments when you want ReplicaSets. As a replica looks only on creating and deleting the pods. Deployment is recommended for application services and
With deployment you should be able to do rolling upgrades or rollbacks. You can update images from v1 to v2.
Refer to this SO1 , SO2 and official documentation of Replicasets and Deployment.
what is the scenario in which only replicaset can be used and deployment can't be used.
There is no such common scenario. ReplicaSets are a lower level abstraction for maintaining stateless pods of the same image / config version. You typically creating new ReplicaSets when you want to change image or pod configuration, it is recommended to use Deployment for such changes.
By its own, it is not very useful to use ReplicaSet directly, it is more a lower level abstraction to maintain the number of replicas with the same configuration.
Does Kubernetes have any strategy of picking which older version pods to be deleted during rolling update? I didn't find any existing documents describing this.
Pods are supposed to be ephemeral and disposable, and thus if you do care about their ordering, you'd want to use a StatefulSet with podManagementPolicy: which allows you to control the order they are created and then destroyed.
When i scale the application by creating deployment .Let's say i am running nginx service on 3 cluster.
Nginx is running in containers in multiple pods .
If i change nginx configuration in one of the pod ,does it propagate to all the nodes and pods because it is running in cluster and scaled.
does it propagate to all the nodes and pods because it is running in
cluster and scaled.
No. Only when you change the deployment yaml. Then it re-creates pods 1 by 1 with the new configuration.
I would like to add a few more things to what was already said. First of all you are even not supposed to do any changes to Pods which are managed let's say by ReplicaSet, ReplicationController or Deployment. This are objects which provide additional abstraction layer and it is their responsibility to ensure that there are given number of Pods of a certain kind running in your kubernetes cluster.
It doesn't matter how many nodes your cluster consists of as mentioned controllers span across all nodes in the cluster.
Changes made in a single Pod will not only not propagate to other Pods but may be easily lost if such newly created Pod with changed configuration crashes.
Remember that one of the tasks of the Deployment is to make sure that certain number of Pods of a given type ( specified in a Pod template section of the Deployment ) are always up and running. When your manually reconfigured Pod goes down then your Deployment (actually ReplicaSet created by the Deployment) acts behind the scenes and recreates such Pod. But how does it recreate it ? Does it take into consideration changes introduced by you to such Pod ? Of course not, it will recreate it based on the template it is given in the Deployment.
If you want to make changes in your Pods one by one kubernetes allows you to do so by providing so called rolling update mechanism.
Here you can read about old-fashioned approach using ReplicationController which is not used any more as it is replaced by Deployments and ReplicaSets but I think it's still worth reading just to grasp the concept.
Currently Deployment is the way to go. About updating a Deployment you can read here. Note that the default update strategy is RollingUpdate which ensures that changes are not applied to all Pods at once but one by one.
I had a quick question about rolling deploys. I'm trying to make sure that the app pods creation is staggered. I looked at maxSurge and maxUnavailable which seems to be the only settings for controlling rolling deploys. Both these settings talk about pod creation in terms of old replicaset. I want to make sure that pod creation is staggered even when there is no deployment currently running.
example: If I set maxSurge to 1 and I have the replication set to 5 then in the presence of old deployment, the rolling update strategy will do the right thing and get one pod up at a time, but if there is no old deployment, all the 5 pods will come up together on a new deployment which is something I am trying to avoid.
What you have explaied is the ecpected behaviour in case there are no existing deployments.
So you want to do a ordered deployment - one pod after the other.
Try deploying the application as a statefulset.
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
Also note the differences b/w a deployment and a statefulset, for example no rollback in case of statefulset
https://blog.thecodeteam.com/2017/08/16/technical-dive-statefulsets-deployments-kubernetes/
You could try leveraging a HorizontalPodAutoscaler with maybe a custom metric that could be set to whatever value will result in your desired number of replicas. Then just configure your HPA so it only scales up so many at a time.
"How to permanently delete pods inside "default" namespace? As when I delete pods, they are coming back because of "replication controller".
As this is in a Default namespace, I am sure that we can delete it permanently. Any idea how to do it ?
I'd like to add some update to what was already said in previous answer.
Basically in kubernetes you have several abstraction layers. As you can read in the documentation:
A Pod is the basic execution unit of a Kubernetes application–the
smallest and simplest unit in the Kubernetes object model that you
create or deploy. A Pod represents processes running on your Cluster .
It is rarely deployed as separate entity. In most cases it is a part of higher level object such as Deployment or ReplicationController. I would advise you to familiarize with general concept of controllers, especially Deployments, as they are currently the recommended way of setting up replication [source]:
Note: A Deployment that configures a ReplicaSet is now the recommended
way to set up replication.
As you can read further:
A ReplicationController ensures that a specified number of pod
replicas are running at any one time. In other words, a
ReplicationController makes sure that a pod or a homogeneous set of
pods is always up and available.
It applies also to situation when certain pods are deleted by user. Replication controller doesn't care why the pods were deleted. Its role is just to make sure they are always up and running. Its very simple concept. When you don't want certain pods to exist any more, you must get rid of the higher level object that manages them and ensures they are always available.
Read about Replication Controllers, then delete the ReplicationController.
It can't "ensure that a specified number of pod replicas are running" when it's dead.
kubectl delete replicationcontroller <name>