I have 2 node js backend applications, they depend on each other, but I'm confused about how these 2 backends communicate in Kubernetes AWS EKS
In a default k8s setting, creating a service and registering pods into it will create an internal dns address each pod can use to reach another one.
So, if your services are named service-a and service-b, service-a can reach service-b by sending requests to the service-a host.
The FQDN for each service is service-<x>.<namespace>.svc.cluster.local
More information can be found here
Related
Exposing Service from a BareMetal(Kubeadm) Build Kubernetes Cluster to the outside world. I am trying to access my Nginx as a service outside of the cluster to get NGINX output in the web browser.
For that, I have created a deployment and service for NGINX as shown below,
As per my search, found that we have below to expose to outside world
MetalLb
Ingress NGINX
Some HELM resources
I would like to know all these 3 or any more approaches in such way it help me to learn new things.
GOAL
Exposing Service from a BareMetal(Kubeadm) Built Kubernetes Cluster to the outside world.
How Can I make my service has its own public IP to access from the outside cluster?
You need to set up MetalLB to get an external IP address for the LoadBalancer type services. It will give a local network IP address to the service.
Then you can do port mapping (configuration in the router) of incoming traffic of port 80 and port 443 to your external service IP address.
I have done a similar setup you can check it here in detail:
https://developerdiary.me/lets-build-low-budget-aws-at-home/
You need to deploy an ingress controller in your cluster so that it gives you an entrypoint where your applications can be accessed. Traditionally, in a cloud native environment it would automatically provision a LoadBalancer for you that will read the rules you define inside your Ingress object and route your request to the appropriate service.
One of the most commonly used ingress controller is the Nginx Ingress Controller. There are multiple ways you can use to deploy it (mainfests, helm, operators). In case of bare metal clusters, there are multiple considerations which you can read here.
MetalLB is still in beta stage so its your choice if you want to use. If you don't have a hard requirement to expose the ingress controller as a LoadBalancer, you can expose it as a NodePort Service that will accessible across all your nodes in the cluster. You can then map that NodePort Service in your DNS so that the ingress rules are evaluated.
Basically I have Following Hdfs Cluster setup using docker-compose:
Node 1 with IP: 192.168.1.1 having service deployed as below:
Namenode1:9000
HMaster1: 8300
ZooKeeper1:1291
Node 2 with IP: 192.168.1.2 having service deployed as below:
Namenode2:9000
ZooKeeper2:1291
How does Traefik / Ngnix - (Ingress Controllers) forwards request to two different services having configured with same port number?
There are several great tutorials on how ingress and load balancing works in kubernetes, e.g. this one by Mark Betz. As a general rule, it helps to think in terms of services and workloads instead of specific nodes where your workloads are running on.
A workload deployed in Kubernetes (a so called Pod) has its own internal IP address, called a ClusterIP. That pod can have one or more ports open, just on that pod-owned ip address.
If you now have several pods to distribute the load, e.g. like 5 web server processes or backend logic, it would be hard for a client (inside the cluster) to keep track of all those pod IPs, because they also change when a pod is updated or just restarted due to a crash. This is why Kubernetes has a so called concept of services. Those provide a stable DNS name and IP which then transparently "forwards" to one of the healthy pods. So your client only needs to know the DNS name and not keep track of the specific pod IPs.
If you now want to expose such a service to the public, there are different ways. Either you set your service to type: LoadBalancer which then sets up some load balancer infrastructure on your cloud provider and routes traffic to the nodes and then to the pods - or - you already have an ingress controller in place and just define the routing based on host names and paths. An ingress controller itself is such a loadbalanced service with an attached cloud load balancer and also has some pods (with e.g. a traefik or nginx container) which then route your packets accordingly.
So coming back to your initial question: If you want to expose a service with several pods of the same kind, then you would first create a Service resource that matches your Pods using the selector and then you create one single ingress resource that provides a hostname/path and references this service. The ingress controller will pick up those ingress resources and configure the traefik or nginx accordingly. The ingress controller doesn't really care about the host IPs and port numbers, because it acts on the internal kubernetes ClusterIPs, so you even don't need (and shouldn't) expose such a service directly when you have an ingress in place.
I hope this answers your question regarding exposing two workloads over an ingress controller. For details, check the Kubernetes docs on Ingresses. Based on the services you named (zookeeper, hdfs) load balancing and ingresses might not be what you need for that case. Zookeeper instances should be internal in most cases and need to be adressed individually, so you might want to check out headless services, for this use case. Also check the Kubernetes docs for a way to run zookeeper.
https://github.com/kubernetes-sigs/aws-alb-ingress-controller/blob/master/docs/guide/ingress/annotation.md#target-type
In above link it is mentioned that "instance mode" will route traffic to all ec2 instances within cluster on NodePort opened for your service.
so how does kube-proxy make sure that request is served only once in case multiple replicas of pods are running in different instances and how does it makes sure that requests are evenly served from all pods?
As per documentation:
Amazon Elastic Load Balancing Application Load Balancer (ALB) is a popular AWS service that load balances incoming traffic at the application layer (layer 7) across multiple targets, such as Amazon EC2 instances.
The AWS ALB Ingress controller is a controller that triggers the creation of an ALB and the necessary supporting AWS resources whenever a Kubernetes user declares an Ingress resource on the cluster. The Ingress resource uses the ALB to route HTTP[s] traffic to different endpoints within the cluster.
With instance mode, ingress traffic start from ALB and reach Node Port opened for service. Traffic is routed to the container POD within cluster.
Moreover target-type: "instance mode" is default setting in AWS ALB ingress controller and service must be type of "NodePort" or "LoadBalancer" to use this mode.
Managing ALBs is automatic, and you only need to define your ingress resources as you would typically do. ALB ingress controller POD which is running inside the Kubernetes cluster communicates with Kubernetes API and does all the work. However, this POD is only a control plane, it doesn't do any proxying and stuff like that.
Your Application Load Balancer periodically sends requests to its registered targets to test their status. These tests are called health checks. Alb-ingress-controller is performing "health checks" for targets groups. Different "health check's" on target groups can be controlled using annotations.
You can find more information about ALB ingress and NodePort here and here
Hope this help.
What I have:
I have created one Kubernetes cluster using single node Rancher 2.0 deployment. which has 3 etcd, control nodes & 2 worker nodes attached to cluster.
What I did:
I deployed one API gateway to this cluster & one express mydemoapi service (no db) with 5 pods on 2 nodes on port 5000, which I don't want to expose publicly. So, I just mapped that service endpoint with service name in API gateway http:\\mydemoapi:5000 & it was accessible by gateway public endpoint.
Problem statement:
mydemoapi service is served in random fashion, not in round robin, because default setting of kube-proxy is random as per Rancher documentation Load balancing in Kubernetes
Partial success:
I created one ingress loadbalancer with Keep the existing hostname option in Rancher rules with this URL mydemoapi.<namespace>.153.xx.xx.102.xip.io & attached this service to ingress, it is served in round robin fashion, but having one problem. This service was using xip.io with public ip of my worker node & exposed publicly.
Help needed:
I want to map my internal clusterIP service into gateway with internal access, so that it can be served to gateway internally in round robin fashion and hence to gateway public endpoint. I don't want to expose my service publicly without gateway.
Not sure which cloud you are running on, but if you are running in something like AWS you can set the following annotation to true on your Service definition:
service.beta.kubernetes.io/aws-load-balancer-internal: "true"
Other Cloud providers have similar solutions and some don't even have one. In that case, you will have to use a NodePort service and redirect an external load balancer such as one with haproxy or nginx to forward traffic to that NodePort
Another option is to not use an Ingress at all if you want to do round robin between your services is to change your kube-proxy configs to use either the old namespace proxy mode or the more enhanced ipvs proxy mode.
I created a kubernetes service something like this on my 4 node cluster:
kubectl expose deployment distcc-deploy --name=distccsvc --port=8080
--target-port=3632 --type=LoadBalancer
The problem is how do I expose this service to an external ip. Without an external ip you can not ping or reach this service endpoint from outside network.
I am not sure if i need to change the kubedns or put some kind of changes.
Ideally I would like the service to be exposed on the host ip.
Like http://localhost:32876
hypothetically let's say
i have a 4 node vm on which i am running let's say nginx service. i expose it as a lodabalancer service. how can i access the nginx using this service from the vm ?
let's say the service name is nginxsvc is there a way i can do http://:8080. how will i get this here for my 4 node vm ?
LoadBalancer does different things depending on where you deployed kubernetes. If you deployed on AWS (using kops or some other tool) it'll create an elastic load balancer to expose the service. If you deployed on GCP it'll do something similar - Google terminology escapes me at the moment. These are separate VMs in the cloud routing traffic to your service. If you're playing around in minikube LoadBalancer doesn't really do anything, it does a node port with the assumption that the user understands minikube isn't capable of providing a true load balancer.
LoadBalancer is supposed to expose your service via a brand new IP address. So this is what happens on the cloud providers, they requisition VMs with a separate public IP address (GCP gives a static address and AWS a DNS). NodePort will expose as a port on kubernetes node running the pod. This isn't a workable solution for a general deployment but works ok while developing.