Being new to the K8s, I am trying to clean up the whole namespace after running some tests on a Windows 10 machine. In short, I thought it would be as easy as running kubectl.exe delete deployment but the deployments are created back after a second and I don't know how to get rid of them. See the followings for the details of what I did:
1.kubectl get deployments,rs (to see what we already have)
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.extensions/postgresql 1 1 1 1 18m
deployment.extensions/redis 1 1 1 1 16m
NAME DESIRED CURRENT READY AGE
replicaset.extensions/postgresql-c8cb9fff6 1 1 1 18m
replicaset.extensions/redis-5678477b7c 1 1 1 16m
2. kubectl scale deployment redis --replicas=0 (Scale down the deployment)
deployment.extensions "redis" scaled
3. kubectl get deployments,rs (Check again how it looks)
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.extensions/postgresql 1 1 1 1 21m
deployment.extensions/redis 0 0 0 0 19m
NAME DESIRED CURRENT READY AGE
replicaset.extensions/postgresql-c8cb9fff6 1 1 1 21m
replicaset.extensions/redis-5678477b7c 0 0 0 19m
4. kubectl delete deployment.extensions/redis (Delete the deployment)
deployment.extensions "redis" deleted
5. kubectl get deployments,rs (Check again and see that it is back!)
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.extensions/postgresql 1 1 1 1 23m
deployment.extensions/redis 1 1 1 1 27s
NAME DESIRED CURRENT READY AGE
replicaset.extensions/postgresql-c8cb9fff6 1 1 1 23m
replicaset.extensions/redis-5678477b7c 1 1 1 27s
6. kubectl.exe get events (Looking into the events):
Among other things I can see "Scaled down replica set redis-5678477b7c to 0" and then "Scaled up replica set redis-5678477b7c to 1" which looks like it was never actually deleted but immediately scaled up again after the delete command was executed.
Not sure what I am missing but have already checked a couple of other posts like Kubernetes pod gets recreated when deleted and How to delete all resources from Kubernetes one time? but neither one worked for me.
Forgot to say that the K8s cluster is managed by the Docker Desktop.
Use kubectl delete deployment <the name of deployment >
If you need to clean up whole namespace , use kubectl delete namespace <namespace-name>
Then re-create the same namespace by kubectl create ns command , if you need the same namespace.
You can also clean up the namespace by using --all options with objects:
e.g
kubecetl delete deployment --all
kubecetl delete statefulset --all
kubectl delete pvc --all
kubectl delete secrets --all
kubectl delete service --all
and so on.
As pointed out by #David Maze, you're deleting the ReplicaSet instead of the Deployment that's managing it.
From the documentation:
You can define Deployments to create new ReplicaSets
The Deployment will automatically create and manage a ReplicaSet to control the pods. You need to delete the Deployment to erase it's managed resources.
Related
I see a lot of heavy documentation online related to Kubernetes deployment but still can't find the definition of 0/0.
> $ k get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
async-handler-redis-master 1/1 1 1 211d
bbox-inference-4k-pilot-2d-boxes 0/0 0 0 148d
What exactly does it mean to be 0/0? It's deployed but not ready? Why is it not ready? How do I make this deployment READY?
It means replica of your deployment is 0. In other words you don't have any pods under this deployment so 0/0 means 0 out of 0 pod is ready.
You can;
kubectl scale deployment <deployment-name> --replicas=1
I get the server could not find the requested resource error when I try to delete a deployment, event when it is listed.
D:\cmd
λ kubectl -n gdpr-tr get all
NAME READY STATUS RESTARTS AGE
pod/scv-turkey-iys-integration-6dd784689f-z6hqc 0/1 InvalidImageName 0 1m
pod/scv-turkey-iys-integration-79c4d7ffcd-pwmq4 0/1 ErrImagePull 0 6m
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/scv-turkey-iys-integration 1 2 1 0 1h
NAME DESIRED CURRENT READY AGE
replicaset.apps/scv-turkey-iys-integration-6998c5b5f9 0 0 0 1h
replicaset.apps/scv-turkey-iys-integration-6dd784689f 1 1 0 34m
replicaset.apps/scv-turkey-iys-integration-79c4d7ffcd 1 1 0 25m
D:\cmd
λ kubectl -n gdpr-tr delete deployment.apps/scv-turkey-iys-integration
Error from server (NotFound): the server could not find the requested resource
You can use any of below commands to delete the deployment but make sure kubectl client and kubernetes api server version matches
Because in 1.16 version deployments is migrated to apps/v1 from extensions/v1beta1. So if you have a kubectl client which is of older version it will not be able to find the deployment in apps api version.
kubectl -n gdpr-tr delete deployment scv-turkey-iys-integration
kubectl -n gdpr-tr delete deployments/scv-turkey-iys-integration
I'm trying to delete some old deployments / replicasets I have in my cluster but when I run kubectl delete deployment
It'll say the deployment is deleted and the pod from that deployment is Terminating, but then a few seconds later the deployment is magically recreated and the pod comes back.
This is the same result for another replicaset I have.
What could be re-creating these deployments / replicasets and how can I stop it so I can permanently delete these deployments/rs?
Edit: Here's some output. This is on a kubernetes cluster in GKE btw:
kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
quickstart-kb 1/1 1 1 41m
ubuntu 1/1 1 1 66d
kubectl get pods
NAME READY STATUS RESTARTS AGE
ubuntu-677fc9fd77-fgd7k 1/1 Running 0 19d
quickstart-kb-f9b65577f-4fxph 1/1 Running 0 40m
kubectl delete deployment quickstart-kb
deployment.extensions "quickstart-kb" deleted
kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
quickstart-kb 0/1 1 0 7s
ubuntu 1/1 1 1 66d
kubectl get pods
NAME READY STATUS RESTARTS AGE
quickstart-kb-6cb6cf897d-qcjff 0/1 Running 0 11s
ubuntu-677fc9fd77-fgd7k 1/1 Running 0 19d
kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
quickstart-kb 1/1 1 1 4m6s
ubuntu 1/1 1 1 66d
kubectl get pods
NAME READY STATUS RESTARTS AGE
quickstart-kb-6cb6cf897d-qcjff 1/1 Running 0 4m13s
ubuntu-677fc9fd77-fgd7k 1/1 Running 0 19d
I think your deployment object is created with the deployment of some custom resources (CRD).
When you created the CRD, the CRD controller created the deployment object. So, even if you delete the deployment object, the CRD controller re-creates it.
Delete the CRD object itself, to delete the deployment and other objects (if any) that were created with it.
From the name, it seems like Kibana CRD object:
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
Use the following command to delete the Kibana object:
$ kubectl delete Kibana quickstart-kb
I do not want to decrease the number of pods controlled by StatefulSet, and i think that decreasing pods is a dangerous operation in production env.
so... , is there some way ? thx ~
I'm not sure if this is what you are looking for but you can scale a StatefulSet
Use kubectl to scale StatefulSets
First, find the StatefulSet you want to scale.
kubectl get statefulsets <stateful-set-name>
Change the number of replicas of your StatefulSet:
kubectl scale statefulsets <stateful-set-name> --replicas=<new-replicas>
To show you an example, I've deployed a 2 pod StatefulSet called web:
$ kubectl get statefulsets.apps web
NAME READY AGE
web 2/2 60s
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
web-0 1/1 Running 0 63s
web-1 1/1 Running 0 44s
$ kubectl describe statefulsets.apps web
Name: web
Namespace: default
CreationTimestamp: Wed, 23 Oct 2019 13:46:33 +0200
Selector: app=nginx
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"name":"web","namespace":"default"},"spec":{"replicas":2,"select...
Replicas: 824643442664 desired | 2 total
Update Strategy: RollingUpdate
Partition: 824643442984
Pods Status: 2 Running / 0 Waiting / 0 Succeeded / 0 Failed
...
Now if we do scale this StatefulSet up to 5 replicas:
$ kubectl scale statefulset web --replicas=5
statefulset.apps/web scaled
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
web-0 1/1 Running 0 3m41s
web-1 1/1 Running 0 3m22s
web-2 1/1 Running 0 59s
web-3 1/1 Running 0 40s
web-4 1/1 Running 0 27s
$ kubectl get statefulsets.apps web
NAME READY AGE
web 5/5 3m56s
You do not have any downtime in already working pods.
i think that decreasing pods is a dangerous operation in production env.
I agree with you.
As Crou wrote, it is possible to do this operation with kubectl scale statefulsets <stateful-set-name> but this is an imperative operation and it is not recommended to do imperative operations in a production environment.
In a production environment it is better to use a declarative operation, e.g. have the number of replicas in a text file (e.g. stateful-set-name.yaml) and deploy them with kubectl apply -f <stateful-set-name>.yaml with this way of working, it is easy to store the yaml-files in Git so you have full control of all changes and can revert/rollback to a previous configuration. When you store the declarative files in a Git repository you can use a CICD solution e.g. Jenkins or ArgoCD to 1) validate the operation (e.g. not allow decrease) and 2) first deploy to a test-environment and see that it works, before applying the changes to the production environment.
I recommend the book (new edition) Kubernetes Up&Running 2nd ed that describes this procedure in Chapter 18 (new chapter).
I would like to perform rolling back a deployment in my environment.
Command:
kubectl rollout undo deployment/foo
Steps which are perform:
create pods with old configurations
delete old pods
Is there a way to not perform last step - for example - developer would like to check why init command fail and debug.
I didn't find information about that in documentation.
Yes it is possible, before doing rollout, first you need to remove labels (corresponding to replica-set controlling that pod) from unhealthy pod. This way pod won't belong anymore to the deployment and even if you do rollout, it will still be there. Example:
$kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
sleeper 1/1 1 1 47h
$kubectl get pod --show-labels
NAME READY STATUS RESTARTS AGE LABELS
sleeper-d75b55fc9-87k5k 1/1 Running 0 5m46s pod-template-hash=d75b55fc9,run=sleeper
$kubectl label pod sleeper-d75b55fc9-87k5k pod-template-hash- run-
pod/sleeper-d75b55fc9-87k5k labeled
$kubectl get pod --show-labels
NAME READY STATUS RESTARTS AGE LABELS
sleeper-d75b55fc9-87k5k 1/1 Running 0 6m34s <none>
sleeper-d75b55fc9-swkj9 1/1 Running 0 3s pod-template-hash=d75b55fc9,run=sleeper
So what happens here, we have a pod sleeper-d75b55fc9-87k5k which belongs to sleeper deployment, we remove all labels from it, deployment detects that pod "has gone" so it creates a new one sleeper-d75b55fc9-swkj9, but the old one is still there and ready for debugging. Only pod sleeper-d75b55fc9-swkj9 will be affected by rollout.