I've just deployed websocket based echo-server on AWS EKS. I see it's running stable and okay but when I was searching for implementation details I was finding only articles that were saying something about nginx ingress controller or AWS application loadbalancer and a lot of troubles with them.
Do I miss anything in my current, vanilla config? Do I need the AWS ALB or nginx ingress controller?
Thank you for all the replies.
All the best.
Do I miss anything in my current, vanilla config?
You probably exposed your echo-server app using service type - ClusterIP or NodePort which is fine if you only need to access your app locally in the cluster (ClusterIP) or using your node IP address (NodePort).
Do I need the AWS ALB or nginx ingress controller?
They both are different things, but they have similar common goal - to make your websocket app available externally and distribute traffic based on defined L7 routing routes. It's good solution if you have multiple deployments. So you need to answer yourself if you need some kind of Ingress Controller. If you are planning to deploy your application into production you should consider using those solutions, but probably it may be fine with service type LoadBalancer.
EDIT:
If you are already using service type LoadBalancer your app is already available externally. Ingress controller provides additional configuration possibilities to configure L7 traffic route to your cluster (Ingress Controllers are often using LoadBalancer under the hood). Check this answer for more details about differences between LoadBalancer and Ingress.
Also check:
Choosing the Right Load Balancer on Amazon: AWS Application Load Balancer vs. NGINX Plus
Configuring Kubernetes Ingress on AWS? Don’t Make These Mistakes
WebSocket - Deploy to Kubernetes
LoadBalancer vs Ingress
Related
Hello Kubernetes Experts,
Trying to get a better understanding here.
I have created a deployment with a regular deployment yaml and service yaml
The service is node port, I then created a ingress and pointed the service
Tried to access the service and it works as expected on the default port of 80 on nginx ingress.
Next created the same deployment and service file. The only exception here was insted of node port is chose ClusterIP. Created a Ingress and pointed the service.
Tried to access the service and it simply fails with the nginx home page and does not do any routing to my application.
I understand that nodeport is what exposes the application to the external world.
But then I'm using Ingress to attain the same functionality.
Do we really need to set the service as node port even if we use Ingress???
Or is something terribly wrong with my yaml files. I tried reading about it and could not get any relevant explanation.
Thank you,
Anish
First, the Service and Ingress resources works a bit different across cloud providers. E.g. on Google Cloud Platform and AWS, you need to use a NodePort service when using Ingress but on e.g. OpenShift ClusterIP is working.
Mostly, the reason is that the Load Balancer is located outside of your cluster (this is not the case on the OpenShift environment where I work).
From Google Cloud documentation, use NodePort for load balancing but ClusterIP if your load balancer is "container native".
In the Service manifest, you must use type: NodePort unless you're using container native load balancing. If using container native load balancing, use the type: ClusterIP.
This is more a design question than an issue. We have deployed in our company our own Kubernetes infrastructure and we are trying to use ingresses and NGINX ingress controller to externally expose our services, but since it is not a cloud environment such as GCP or AWS, we can't use service type "LoadBalancer". Should we just expose our ingress controller through a service type "NodePort"? Is that the normal way to go for production environments (non-cloud)?
From what I've read in another post, one suitable recommendation is to use NodePort, and manually point yet another external load balancer to the port on your Kubernetes nodes.
It just seems that exposing the ingress controller through this mechanism is somehow not very practical or robust (e.g. you don’t know what port your service is going to be allocated, and the port might get re-allocated at some point, etc.)
Is there any other mechanism maybe to expose the ingress controller to the external world?
The Loadbalancer service approach is one way to do it but behind it it's nothing more than a nodeport on the cluster.
Even if you use a service that create a LB on cloud provider, the LB needs to have a target port to communicate with the cluster.
When using a nginx-ingress that will mostly handle web requests, it's common usage to put an ingress in front of a nodeport service.
So with this I think using NodePort services is a good idea to do what you want ;)
This is my opinion, I'm interested if anyone else has another way to do it.
You can specify the port via nodePort in the service. Then it would not be random.
I'm not sure how load balancing works with Ingress.
If I understand correctly, what happens is actually something like this:
I fail to see how the load balancing is performed.
What is wrong in the above scheme that I have drawn?
Can you help me rectify it?
Notes:
- The following answer tells me that the Ingress controller itself is of type 'loadbalancer': Ingress service type
- I use kind ClusterIP because I don't want to expose the loadbalancer to the outside world. The following article does not support this claim, where the load balancer would be provided by the service:
https://medium.com/google-cloud/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0
The ClusterIP services themselves perform load balancing. The naming can be confusing as LoadBalancer services are not the only services that involve load balancing - LoadBalancer actually means something more like 'cloud provider please create an external load balancer and point it at this service'. The kubernetes ClusterIP services also load-balance across Pods in different Nodes using the kube-proxy. If you don't want kubernetes to do load balancing then you have to specifically disable it by creating a headless service.
It seems like the first scheme you drew is correct. But I think you get confused in terminology. Particularly in the difference between ingress and ingress-controller.
Ingress is a type of resources in k8s (like Service, Deployment, ReplicaSet etc). We use ingress if we want to expose some services to an external world with binding to some path and host (i.e. myapp.com/api -> my-api-service).
The job of ingress-controller is to handle creation/update/deletion of ingress resources and implement all the functionality needed for ingress. Under the hood ingress-controller is a simple deployment exposed as LoadBalancer or NodePort service depending on where k8s is deployed. And image-controller forwards received request further to one of pods of service which matches host and path in some of the deployed ingress resources.
I got curious and thought: why would I need an Ingress for load balancing on layer 7 if the only thing it does is forward the traffic to a Service that implements the load balancing on layer 4?
Most Ingress controller implementations I looked up, talk to the Kubernetes API server to keep track of all Pods associated with a Service. Instead of forwarding traffic to the Service, they skip the intermediary and directly forward to the Pods. Since an ingress controller operates on layer 7, it enables a more application-oriented load balancing.
all
I knew well about k8s' nodePort and ClusterIP type in services.
But I am very confused about the Ingress way, because how will a request come into a pod in k8s by this Ingress way?
Suppose K8s master IP is 1.2.3.4, after Ingress setup, and can connect to backend service(e.g, myservice) with a port(e.g, 9000)
Now, How can I visit this myservice:9000 outside? i.e, through 1.2.3.4? As there's no entry port on the 1.2.3.4 machine.
And many docs always said visit this via 'foo.com' configed in the ingress YAML file. But that is really funny, because xxx.com definitely needs DNS, it's not a magic to let you new-invent any xxx.com you like be a real website and can map your xxx.com to your machine!
The key part of the picture is the Ingress Controller. It's an instance of a proxy (could be nginx or haproxy or another ingress type) and runs inside the cluster. It acts as an entrypoint and lets you add more sophisticated routing rules. It reads Ingress Resources that are deployed with apps and which define the routing rules. This allows each app to say what the Ingress Controller needs to do for routing to it.
Because the controller runs inside the cluster, it needs to be exposed to the outside world. You can do this by NodePort but if you're using a cloud provider then it's more common to use LoadBalancer. This gives you an external IP and port that reaches the Ingress controller and you can point DNS entries at that. If you do point DNS at it then you have the option to use routing rules base on DNS (such as using different subdomains for different apps).
The article 'Kubernetes NodePort vs LoadBalancer vs Ingress? When should I use what?' has some good explanations and diagrams - here's the diagram for Ingress:
I am new to Kubernetes and wanted to understand how I can expose a service running in Kubernetes to the outside world. I have exposed it using a NodePort on the cluster.
So, for example: A service exposes port 31234 on the host and I can get to the service from another server through https://kubeserverIP:31234.
What I want to achieve is serve this service through nginx (on a different server, out of Kube control) via a url,say, http://service.example.com. I have tried deploying nginx with an upstream pointing to the service but that is not working and get a bad gateway error.
Is there something which I am missing here? Or is there a neater way of achieving this.
I have a baremetal installation of Kubernetes cluster and have no access to gce load balancer or other vendor LBs.
Thanks
Thanks for pointing in the right direction.
Essential steps broadly were:
Create an app and its service definition.
Create a namespace for ingress.
Create a default backend deployment and service for redirecting all requests not defined in Ingress rules. Create these in the ingress space
Create the nginx ingress controller deployment.
Create RBAC rules.
Finally create the ingress rule for the applications with the paths and the ports.
Found a very useful guide which explained things in details:
https://akomljen.com/kubernetes-nginx-ingress-controller/
You're almost there! Your next step will be to setup a ingress controller. There is an NGINX Ingress controller plugin that you can checkout here.
Edit: Here's an example configuration: https://github.com/nginxinc/kubernetes-ingress/tree/master/examples/complete-example