I have been trying different things around k8s these days. I am wondering about the field nodeSelector in the POD specification.
As I understand we have to assign some labels to the nodes and these labels can further be used in the nodeSelector field part of the POD specification.
Assignment of the node to pods based on nodeSelector works fine. But, after I create the pod, now I want to update/overwrite the nodeSelector field which would deploy my pod to new node based on new nodeSelector label updated.
I am thinking this in the same way it is done for the normal labels using kubectl label command.
Are there any hacks to achieve such a case ?
If this is not possible for the current latest versions of the kubernetes, why should not we consider it ?
Thanks.
While editing deployment manually as cookiedough suggested is one of the options, I believe using kubctl patch would be a better solution.
You can either patch by using yaml file or JSON string, which makes it easier to integrate thing into scripts. Here is a complete reference.
Example
Here's a simple deployment of nginx I used, which will be created on node-1:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
nodeSelector:
kubernetes.io/hostname: node-1
JSON patch
You can patch the deployment to change the desired node as follows:kubectl patch deployments nginx-deployment -p '{"spec": {"template": {"spec": {"nodeSelector": {"kubernetes.io/hostname": "node-2"}}}}}'
YAML patch
By running kubectl patch deployment nginx-deployment --patch "$(cat patch.yaml)", where patch.yaml is prepared as follows:
spec:
template:
spec:
nodeSelector:
kubernetes.io/hostname: node-2
Both will result in scheduler scheduling new pod on requested node, and terminating the old one as soon as the new one is ready.
Remove the nodeSelector line & save it
Related
I'm working through Kubernetes in Action (copyright 2018), and at least one of the examples is out-of-date with respect to current versions of kubectl.
Currently I'm stuck in section 2.3 on just trying to demo a simple web-server docker container ("kubia"):
kubectl run kubia --image=Dave/kubia --port=8080 --generator=run/v1
the --generator option has been removed from current versions of kubectl. What command(s) achieve the same end in the current version of kubectl?
Note: I'm literally just 2 chapters into learning about Kubernetes, so I don't really know what a deployment or anything else (so the official kubernetes docuementation doesn't help), I just need the simplest way to verify that that I can, in fact, run this container in my minikube "cluster".
in short , you can use following commands to create pods and deployments (imperative way) using following commands which are similar to the commands mentioned in that book :
To create a pod named kubia with image Dave/kubia
kubectl run kubia --image=Dave/kubia --port=8080
To create a deployment named kubia with image Dave/kubia
kubectl create deployment kubia --image=Dave/kubia --port=8080
You can just instantiated the pod, since --generator has been deprecated.
apiVersion: v1
kind: Pod
metadata:
name: kubia
spec:
containers:
- name: kubia
image: Dave/kubia
ports:
- containerPort: 8080
Alternatively, you can use a deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: kubia-deployment
labels:
app: kubia
spec:
replicas: 1
selector:
matchLabels:
app: kubia
template:
metadata:
labels:
app: kubia
spec:
containers:
- name: kubia
image: Dave/kubia
ports:
- containerPort: 8080
Save either one to a something.yaml file and run
kubectl create -f something.yaml
And to clean up
kubectl delete -f something.yaml
✌️
If someone who read same book (Kubernetes in Action, copyright 2018) have same issue in the future, just run pod instead of the replication controller and expose pod instead of rc in following chapter.
Both replica set and deployment have the attribute replica: 3, what's the difference between deployment and replica set? Does deployment work via replica set under the hood?
configuration of deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-deployment
labels:
my-label: my-value
spec:
replicas: 3
selector:
matchLabels:
my-label: my-value
template:
metadata:
labels:
my-label: my-value
spec:
containers:
- name: app-container
image: my-image:latest
configuration of replica set
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: my-replicaset
labels:
my-label: my-value
spec:
replicas: 3
selector:
matchLabels:
my-label: my-value
template:
metadata:
labels:
my-label: my-value
spec:
containers:
- name: app-container
image: my-image:latest
Kubernetes Documentation
When to use a ReplicaSet
A ReplicaSet ensures that a specified number of pod replicas are running at any given time. However, Deployment is a higher-level concept that manages ReplicaSets and provides declarative updates to Pods along with a lot of other useful features. Therefore, we recommend using Deployments instead of directly using ReplicaSets, unless you require custom update orchestration or don't require updates at all.
This actually means that you may never need to manipulate ReplicaSet objects: use a Deployment instead, and define your application in the spec section.
Deployment resource makes it easier for updating your pods to a newer version.
Lets say you use ReplicaSet-A for controlling your pods, then You wish to update your pods to a newer version, now you should create Replicaset-B, scale down ReplicaSet-A and scale up ReplicaSet-B by one step repeatedly (This process is known as rolling update). Although this does the job, but it's not a good practice and it's better to let K8S do the job.
A Deployment resource does this automatically without any human interaction and increases the abstraction by one level.
Note: Deployment doesn't interact with pods directly, it just does rolling update using ReplicaSets.
A ReplicaSet ensures that a number of Pods is created in a cluster. The pods are called replicas and are the mechanism of availability in Kubernetes.
But changing the ReplicaSet will not take effect on existing Pods, so it is not possible to easily change, for example, the image version.
A deployment is a higher abstraction that manages one or more ReplicaSets to provide a controlled rollout of a new version. When the image version is changed in the Deployment, a new ReplicaSet for this version will be created with initially zero replicas. Then it will be scaled to one replica, after that is running, the old ReplicaSet will be scaled down. (The number of newly created pods, the step size so to speak, can be tuned.)
As long as you don't have a rollout in progress, a deployment will result in a single replicaset with the replication factor managed by the deployment.
I would recommend to always use a Deployment and not a bare ReplicaSet.
One of the differences between Deployments and ReplicaSet is changes made to container are not reflected once the ReplicaSet is created
For example:
This is my replicaset.yaml
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: nginx-replicaset
spec:
selector:
matchLabels:
app: nginx
replicas: 5 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.13.2
ports:
- containerPort: 80
I will apply this replicaset using this command
kubectl apply -f replicaset.yaml
kubectl get pods
kubectl describe pod <<name_of_pod>>
So from pod definition, we can observe that nginx is using 1.13.2. Now let's change image version to 1.14.2 in replicaset.yaml
Again apply changes
kubectl apply -f replicaset.yaml
kubectl get pods
kubectl describe pod <<name_of_pod>>
Now we don't see any changes in Pod and they are still using old image.
Now let us repeat the same with a deployment (deployment.yaml)
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 5 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.13.2
ports:
- containerPort: 80
I will apply this deployment using this command
kubectl apply -f deployment.yaml
kubectl get pods
kubectl describe pod <<name_of_pod>>
Change the deployment.yaml file with some other version of nginx image
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 5 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
I will again apply this deployment using this command
kubectl apply -f deployment.yaml
kubectl get pods
kubectl describe pod <<name_of_pod>>
Now we can see these pods and we can see updated image in the description of pod
TLDR:
Deployment manages -> Replicatset manages -> pod(s) abstraction of
-> container (e.g docker container)
A Deployment in Kubernetes is a higher-level abstraction that represents a set of replicas of your application. It ensures that your desired number of replicas of your application are running and available.
A ReplicaSet, on the other hand, is a lower-level resource that is responsible for maintaining a stable set of replicas of your application. ReplicaSet ensures that a specified number of replicas are running and available.
Example:
Suppose you have a web application that you want to run on 3 replicas for high availability.
You would create a Deployment resource in Kubernetes, specifying the desired number of replicas as 3 pods. The deployment would then create and manage the ReplicaSet, which would in turn create and manage 3 replicas of your web application pod.
In summary, Deployment provides higher-level abstractions for scaling, rolling updates and rolling back, while ReplicaSet provides a lower-level mechanism for ensuring that a specified number of replicas of your application are running.
On my deployment I can set replicas: 3 but then it only spins up one pod. Its my understanding that kubernetes will then fire up more pods as needed, up to three.
But for the sake of uptime I want to be able to have a minimum of three pods at all times and for it to maybe create more as needed but to never scale down lower than 3.
Is this possible with a Deployment?
It is exactly as you did, you define 3 replicas in your Deployment.
Have you verified that you have enough resources for your Deployments?
Replicas example:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3 # <------- The number of your desired replicas
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
K8S will spin up a new pod if it can do it meaning, if you have enough resources, the resources are valid (like nodes, selectors, image, volumes, configuration, and more)
If you have defined 3 replicas and you still getting only 1, examine your deployment or your events.
How to view events
# To view all the events (don't specify pod name or namespace)
kubectl get events
# Get event for specific pod
kubectl describe event <pod name> --namespace <namespace>
What is the difference between the following syntax usage:
kubectl get deployments
kubectl get deployment.apps
kubectl get deployment.v1.apps
There are references to deployment.v1.apps and deployment.apps in the documentation specially when talking about rollouts and upgrades.
For example:
To see the Deployment rollout status, run kubectl rollout status deployment.v1.apps/nginx-deployment
For example:
Let's update the nginx Pods to use the nginx:1.16.1 image instead of the nginx:1.14.2 image.
kubectl --record deployment.apps/nginx-deployment set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1
There is no difference. They show you in examples different ways to access the resource.
This is the reference to app/v1 api that you can see in example nginx deployment:
apiVersion: apps/v1
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
You can use shorten way like kubectl get deployments either longer ones you provided in the question.
However, its obvious, you cant use e.g app/v2
kubectl get deployment.v2.apps/nginx-deployment
error: the server doesn't have a resource type "deployment"
I want to deploy a Postgres service with replication on Kubernetes cluster.
I have defined a PetSet and a Service for this. But i am only able to define same resource limits to all pods in a service, due to which Kubernetes assigns these nodes randomly to nodes.
Is there a way, where i can have a service with different pods resource conf ?
My current yaml for reference.
https://github.com/kubernetes/charts/blob/master/incubator/patroni/templates/ps-patroni.yaml
You cannot assign different configuration options (i.e. resource limits) to pods in the same Replica. Inherently they are meant to be identical. You would need to create multiple replicas in order to accomplish this.
for a service you could have multiple deployments with different nodeSelector configurations behind one Service definition.
E.g. you could label your nodes like this:
kubectl label nodes node-1 pool=freshhardware
kubectl label nodes node-2 pool=freshhardware
kubectl label nodes node-3 pool=shakyhardware
And then have two deployments like this:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: my-nginx
spec:
replicas: 4
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
nodeSelector:
pool: freshhardware
... the second one could look the same with just these fields exchanged:
nodeSelector:
pool: shakyhardware
A service definition like this would then take all pods from both deployments into account:
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
ports:
- port: 80
protocol: TCP
selector:
run: my-nginx
The drawback is of course that you'd have to manage two Deployments at a time, but that's kind of build-in to this problem anyways.