Imagine you have a k8s cluster set up with Traefik as an Ingress Controller.
An HTTP API application is deployed to the cluster (with an ingress resource) that is able to handle SIGTERM and does not exit until all active requests are handled.
Let's say you deploy the application with 10 replicas, get some traffic to it and scale the deployment down to 5 replicas. Those 5 Pods will be pulled out from the matching Service resource.
For those 5 Pods, the application will receive SIGTERM and start the graceful shutdown.
The question is, what will Traefik do with those active connections to the pulled out 5 Pods?
Will it wait until all the responses are received from the 5 Pods and not send any traffic to them during and after that?
Will it terminate the ongoing connections to those 5 Pods and forget about them?
Traefik will do the first: it will gracefully let those pending, in-flight requests finish but not forward any further requests to the terminating pods.
To add some technical background: once a pod is deemed to terminate from Kubernetes' perspective, the Endpoints controller (also part of the Kubernetes control plane) will remove its IP address from the associated endpoints object. Traefik watches for updates on the endpoints, receives a notification, and updates its forwarding rules accordingly. So technically, it is not able to forward any further traffic while those final requests will continue to be served (by previously established goroutines from Go's http package).
Related
I am running multiple pods with python/gunicorn serving web request. At times, request get really slow (up to 60s) which blocks all workers and makes the livenessProbe fail.
In some instances, all pods are blocked in this state and are restarted at the same time (graceful shutdown takes up to 60s). This means that no pod is available to take new requests.
Is there a way of telling k8s to cover for pods that it is restarting? For example starting a new pod when other pods are unhealthy.
You can have an ingress or a load balancer at L7 layer which can route traffic to kubernetes service which can have multiple backend pods(selected by labels of the pods and label selector of the service) which spread across different deployments running in different nodes. The ingress controller or loadbalancer can do health check on backends and stop routing traffic to unhealthy pods.This topology overall increases the availability and resiliency of the application.
According to some of the tech blogs (e.g. Understanding kubernetes networking: services), k8s service dispatch all the requests through iptable rules.
What if one of the upstream pods crashed when a request happened to be routed on that pods.
Is there a failover mechanism in kubernetes service?
Will the request will be forwarded to next pod automatically?
How does kubernetes solve this through iptable?
Kubernetes offers a simple Endpoints API that is updated whenever the set of Pods in a Service changes. For non-native applications, Kubernetes offers a virtual-IP-based bridge to Services which redirects to the backend Pods
Here is the detail k8s service & endpoints
So your answer is endpoint Object
kubectl get endpoints,services,pods
There are liveness and readiness checks which decides if the pod is able to process the request or not. Kubelet with docker has mechanism to control the life cycle of pods. If the pod is healthy then its the part of the endpoint object.
Is it possible to send requests to all pods behind a service/ingress controller based on the requests?
My requirement is to send requests to all the pods if the request is /send/all.
Thanks
It's not possible because ingress controller can't do this (for sure nginx and GLBC based ingress can't do it, bud due to the way how to http works I assume this is the case for all ingress controllers).
Depending what your exact case is you have few options.
If your case is just monitoring and you can afford using control on number of request sending to your pods you can just set http liveness probe for your pods. Then you will be sure that if pod doesn't return correct response k8s won't send traffic to it.
If you need to trigger some action on all pods you have few options:
Use messaging - for example you can use rabbitmq chart to deploy rabbitmq and write some application that will handle your traffic.
Using DB - create some app that will set some flag in DB abd add some logic to your app to monitor the flag, or create cron job and to monitor the flag and trigger and trigger required actions on pods (in this case you can use service account to give your cron job pod to k8s API to list pods.
A service dispatches to multiple replicas of a deployment (default round-robin).
The backend instances temporarily have to go offline, i.e. they will close their port 80, take some time, and then open the port again.
deployment.yaml is using readinessProbe to regularly check which backend instances are ready to serve requests.
But what happens in the following scenario?
1) readiness check backend A: OK
2) backend A goes offline
3) requests to service is forwarded to backend A
4) readiness check backend A: fail
Will the service send the request again but to another backend instance, or is it lost?
It depends on the type of Service.
If the Service is a ClusterIP or NodePort, it's instantiated as iptables rules. Packets destined for the now offline pod will be undeliverable, causing the request to timeout.
If the Service is a LoadBalancer, the implementation is an application, like nginx or an equivalent. It will watch for timeouts, and generally speaking- though dependent on configuration- will retry, allowing the request to make it to an online pod.
I have a kubernetes setup that contains 4 minions (node1,2,3,4). I created a service that exposes port 80 as node port of 30010. There are 4 nginx pods that accepts the traffic from above service. However distribution of pods among nodes may vary. For example node 1 has 2 pods, node 2 has 1 pod and node 3 has 1 pod. Node 4 doesn't have any pod deployed. My requirement is, whenever I send a request to node1:30010 it should hit only 2 pods on node 1 and it should not hit other pods. Traffic should be routed to other nodes if and only if there is no pod in local node. For example node4 may have to route requests to node4:30010 to other nodes because it has no suitable pod deployed on it. Can I facilitate this requirement by changing configurations of kube-proxy?
As far as I'm aware, no. Hitting node1:30010 will pass traffic to the service, the service will then round robin the response.
Kubernetes is designed as a layer of abstraction above nodes, so you don't have to worry about where traffic is being sent, trying to control which node traffic goes to goes against that idea.
Could you explain your end goal? If your different pods are serving different responses then you may want to create more services, or if you are worried about latency and want to serve traffic from the node closest to the user you may want to look at federating your cluster.