As I can see in below diagram I figure out in kubernetes we have two loadbalancer. One of them loadbalance between nodes and one of them loadbalance between pods.
If I use them both I have two loadbalancer.
Imagine some user want to connect to 10.32.0.5 the kubernetes send its request to node1(10.0.0.1) and after that send the request to pod (10.32.0.5) in nod3(10.0.0.3) but it is unuseful because the best route is to send request nod3(10.0.0.3) directly.
Why the NodePort is insufficient for load-balancing?
Why the NodePort is not LoadBalancer?(it LoadBalance between pods in different node but why we need another load balancer?)
note: I know that if I use NodePort and the node goes down it creates problem but I can say that I can use keepalived for it. The question is
why we need to loadbalance between nodes? keepalived attract all request to one IP.
Why we have two loadbalancer?
Wether you have two load-balancers depends on your setup.
In your example you have 3 nginx pods and 1 nginx service to access the pods. The service builds an abstraction layer, so you don't have to know how many pods there are and what IP addresses they have. You just have to talk to the service and it will loadbalance to one of the pods (docs).
It now depends on your setup how you access the service:
you might want to publish the service via NodePort. Then you can directly access the service on a node.
you might also publish it via LoadBalancer. This gives you another level of abstraction and the caller needs to know less about the actual setup of your cluster.
See docs for details.
Related
Could you please explain use of each "Kind" of OpenShift in a short sentences?
It is okay, that deployment contains data about, image source, pod counts, limits etc.
With the route we can determine the URL for each deployment as well as Ingress, but what is the difference and when should use route and when ingress?
And what is the exact use of service?
Thanks for your help in advance!
Your question cannot be answered simply in short words or one line answers, go through the links and explore more,
Deployment: It is used to change or modify the state of the pod. A pod can be one or more running containers or a group of duplicate pods called ReplicaSets.
Service: Each pod is given an IP address when using a Kubernetes service. The service provides accessibility, connects the appropriate pod automatically, and this address may not be directly identifiable.
Route:Similar to the Kubernetes Ingress resource, OpenShift's Route was developed with a few additional features, including the ability to split traffic between multiple backends.
Ingress: It offers routing rules for controlling who can access the services in a Kubernetes cluster.
Difference between route and ingress?
OpenShift uses HAProxy to get (HTTP) traffic into the cluster. Other Kubernetes distributions use the NGINX Ingress Controller or something similar. You can find more in this doc.
when to use route and ingress: It depends on your requirements. From the image below you can find the feature of the ingress and route and you select according to your requirements.
Exact use of service:
Each pod in a Kubernetes cluster has its own unique IP address. However, the IP addresses of the Pods in a Deployment change as they move around. Therefore, using Pod IP addresses directly is illogical. Even if the IP addresses of the member Pods change, you will always have a consistent IP address with a Service.
A Service also provides load balancing. Clients call a single, dependable IP address, and the Service's Pods distribute their requests evenly.
As per the Headless service definition:
Kubernetes allows clients to discover pod IPs through DNS lookups. Usually, when you perform a DNS lookup for a service, the DNS server returns a single IP which is the service’s cluster IP. But if you don’t need the cluster IP for your service, you can set ClusterIP to None , then the DNS server will return the individual pod IPs instead of the service IP.Then client can connect to any of them
Looks like its similar to creating a clusterIP with one backend pod. If so why should we use clusterIP with one backend pod?
You don't usually control the Pod's name, and you can't guarantee that there will always be exactly one matching Pod.
There are a couple of standard reasons to not directly create Pods, but instead to rely on higher-level constructs. A Deployment can do zero-downtime upgrades; a StatefulSet manages associated PersistentVolumeClaims; a Job can retry on failure. A Pod also can't be edited once created, so you'd need to delete and recreate the Pod on any update. If you're using a Deployment you'll have Pod names like deployment-name-12345678-abcde, but you don't control the ending parts at all. A Service will give you a consistent name.
During an upgrade there won't necessarily be exactly the requested number of replicas:. A Deployment by default will create an additional Pod, wait for it to pass its readiness checks, then tear down the old Pod; an associated Service will route traffic correctly in this case. The alternative is to tear down the old Pod first. In both cases, you could have zero or two matching Pods and not just one.
Having the Service as the standard pattern also helps if you do ever decide to increase the replica count; for example you might choose to run multiple replicas just for additional resiliency in case ons of the Nodes fails. Communicating via a Service works exactly the same way whether you have just one Pod or several.
There is a difference. A DNS lookup for Headless service returns as many IP addresses as there are pods running, while a DNS lookup for ClusterIP service always returns exactly one IP address.
So yes, if you ever have only one Pod then Headless service may be sufficient.
But if you have more than one pod, for Headless service the client must implement some kind of load balancing while for ClusterIP kubernetes will do the load balancing.
One use case for Headless service with multiple pods is when you want to send requests to every pod, e.g. local cache draining.
Basically I have Following Hdfs Cluster setup using docker-compose:
Node 1 with IP: 192.168.1.1 having service deployed as below:
Namenode1:9000
HMaster1: 8300
ZooKeeper1:1291
Node 2 with IP: 192.168.1.2 having service deployed as below:
Namenode2:9000
ZooKeeper2:1291
How does Traefik / Ngnix - (Ingress Controllers) forwards request to two different services having configured with same port number?
There are several great tutorials on how ingress and load balancing works in kubernetes, e.g. this one by Mark Betz. As a general rule, it helps to think in terms of services and workloads instead of specific nodes where your workloads are running on.
A workload deployed in Kubernetes (a so called Pod) has its own internal IP address, called a ClusterIP. That pod can have one or more ports open, just on that pod-owned ip address.
If you now have several pods to distribute the load, e.g. like 5 web server processes or backend logic, it would be hard for a client (inside the cluster) to keep track of all those pod IPs, because they also change when a pod is updated or just restarted due to a crash. This is why Kubernetes has a so called concept of services. Those provide a stable DNS name and IP which then transparently "forwards" to one of the healthy pods. So your client only needs to know the DNS name and not keep track of the specific pod IPs.
If you now want to expose such a service to the public, there are different ways. Either you set your service to type: LoadBalancer which then sets up some load balancer infrastructure on your cloud provider and routes traffic to the nodes and then to the pods - or - you already have an ingress controller in place and just define the routing based on host names and paths. An ingress controller itself is such a loadbalanced service with an attached cloud load balancer and also has some pods (with e.g. a traefik or nginx container) which then route your packets accordingly.
So coming back to your initial question: If you want to expose a service with several pods of the same kind, then you would first create a Service resource that matches your Pods using the selector and then you create one single ingress resource that provides a hostname/path and references this service. The ingress controller will pick up those ingress resources and configure the traefik or nginx accordingly. The ingress controller doesn't really care about the host IPs and port numbers, because it acts on the internal kubernetes ClusterIPs, so you even don't need (and shouldn't) expose such a service directly when you have an ingress in place.
I hope this answers your question regarding exposing two workloads over an ingress controller. For details, check the Kubernetes docs on Ingresses. Based on the services you named (zookeeper, hdfs) load balancing and ingresses might not be what you need for that case. Zookeeper instances should be internal in most cases and need to be adressed individually, so you might want to check out headless services, for this use case. Also check the Kubernetes docs for a way to run zookeeper.
I need for a service in a K8 pod to be able to make HTTP calls to downstream services, load balanced by a NodePort, within the same cluster and namespace.
My constraints are these:
I can do this only through manipulation of deployment and service
entities (no ingress. I don't have that level of access to the
cluster)
I cannot add any K8 plugins
The port that the NodePort exposes must be randomized, not hard coded
This whole thing must be automated. I can't set the deployment with the literal value of
the exposed port. It needs to be set by some sort of variable, or
similar process.
Is this possible, and, if so, how?
It probably can be done but it will not be straight forward and you might have to add some custom automation. A NodePort service is meant to be used by an entity outside your cluster.
For inter-cluster communication, a regular service (with a ClusterIP) will work as designed. Your service can reach another service using DNS service discovery. For example. svc-name.mynamespace.svc.cluster.local would be the DNS entry for a svc-name in the mynamespace namespace.
If you can only do a NodePort which essentially is a port on your K8s nodes, you could create another Deployment or Pod of something like nginx or haproxy. Then have this deployment being serviced by regular K8s service with a ClusterIP. Then have nginx or haproxy point to the NodePort on all your nodes in your Kubernetes cluster. Also, have it configured so that it only forwards to listening NodePorts with some kind of healthcheck.
The above seems like an extra necessary step, but if NodePort from within the cluster is what you need (for some reason), it should do the trick.
I have a question related to Kubernetes networking.
I have a microservice (say numcruncherpod) running in a pod which is serving requests via port 9000, and I have created a corresponding Service of type NodePort (numcrunchersvc) and node port which this service is exposed is 30900.
My cluster has 3 nodes with following IPs:
192.168.201.70,
192.168.201.71
192.168.201.72
I will be routing the traffic to my cluster via reverse proxy (nginx). As I understand in nginx I need to specify IPs of all these cluster nodes to route the traffic to the cluster, is my understanding correct ?
My worry is since nginx won't have knowledge of cluster it might not be a good judge to decide the cluster node to which the traffic should be sent to. So is there a better way to route the traffic to my kubernetes cluster ?
PS: I am not running the cluster on any cloud platform.
This answer is a little late, and a little long, so I ask for forgiveness before I begin. :)
For people not running kubernetes clusters on Cloud Providers there are 4 distinct options for exposing services running inside the cluster to the world outside.
Service of type: NodePort. This is the simplest and default. Kubernetes assigns a random port to your service. Every node in the cluster listens for traffic to this particular port and then forwards that traffic to any one of the pods backing that service. This is usually handled by kube-proxy, which leverages iptables and load balances using a round-robin strategy. Typically since the UX for this setup is not pretty, people often add an external "proxy" server, such as HAProxy, Nginx or httpd to listen to traffic on a single IP and forward it to one of these backends. This is the setup you, OP, described.
A step up from this would be using a Service of type: ExternalIP. This is identical to the NodePort service, except it also gets kubernetes to add an additional rule on all kubernetes nodes that says "All traffic that arrives for destination IP == must also be forwarded to the pods". This basically allows you to specify any arbitrary IP as the "external IP" for the service. As long as traffic destined for that IP reaches one of the nodes in the cluster, it will be routed to the correct pod. Getting that traffic to any of the nodes however, is your responsibility as the cluster administrator. The advantage here is that you no longer have to run an haproxy/nginx setup, if you specify the IP of one of the physical interfaces of one of your nodes (for example one of your master nodes). Additionally you cut down the number of hops by one.
Service of type: LoadBalancer. This service type brings baremetal clusters at parity with cloud providers. A fully functioning loadbalancer provider is able to select IP from a pre-defined pool, automatically assign it to your service and advertise it to the network, assuming it is configured correctly. This is the most "seamless" experience you'll have when it comes to kubernetes networking on baremetal. Most of LoadBalancer provider implementations use BGP to talk and advertise to an upstream L3 router. Metallb and kube-router are the two FOSS projects that fit this niche.
Kubernetes Ingress. If your requirement is limited to L7 applications, such as REST APIs, HTTP microservices etc. You can setup a single Ingress provider (nginx is one such provider) and then configure ingress resources for all your microservices, instead of service resources. You deploy your ingress provider and make sure it has an externally available and routable IP (you can pin it to a master node, and use the physical interface IP for that node for example). The advantage of using ingress over services is that ingress objects understand HTTP mircoservices natively and you can do smarter health checking, routing and management.
Often people combine one of (1), (2), (3) with (4), since the first 3 are L4 (TCP/UDP) and (4) is L7. So things like URL path/Domain based routing, SSL Termination etc is handled by the ingress provider and the IP lifecycle management and routing is taken care of by the service layer.
For your use case, the ideal setup would involve:
A deployment for your microservice, with health endpoints on your pod
An Ingress provider, so that you can tweak/customize your routing/load-balancing as well as use for SSL termination, domain matching etc.
(optional): Use a LoadBalancer provider to front your Ingress provider, so that you don't have to manually configure your Ingress's networking.
Correct. You can route traffic to any or all of the K8 minions. The K8 network layer will forward to the appropriate minion if necessary.
If you are running only a single pod for example, nginx will most likely round-robin the requests. When the requests hit a minion which does not have the pod running on it, the request will be forwarded to the minion that does have the pod running.
If you run 3 pods, one on each minion, the request will be handled by whatever minion gets the request from nginx.
If you run more than one pod on each minion, the requests will be round-robin to each minion, and then round-robin to each pod on that minion.