We have an application and for each customer we provision a new namespace. There are two deployments running inside a single namespace:
front-end Deployment
Back-end Deployment
The front-end should be accessed by the users hence we are using LoadBalancer for each customer (We have a VM Based k8s cluster).
The problem is, as of now we have a few customers and when the business grows, the customers will be increasing and will be having more NameSpaces.
For example: If there are 100 Users, we have to have 100 LoadBalancers. This is not practical and can we have a single LoadBalancer instead and allow all the 100 Users to access through that LoadBalancer?
Can we do this using Ingress?
Yes, ingress is a right way to manage your case.
Generally you've already mentioned why ingress should be used - get a single entry point to the cluster and not having a lot of load balancers which is not convenient and may be expensive in cloud environment.
Main benefits of using ingress are:
TLS termination
hosts/paths based routing
serves itself as a loadbalancer
and many more.
You can choose an ingress which fits better for your use-case. Ingress options
Most common are:
nginx ingress supported by kubernetes community
nginx ingress supported by Nginx inc and community
Please consider getting familiar with general concepts and examples of kubernetes ingress
Related
I've just deployed websocket based echo-server on AWS EKS. I see it's running stable and okay but when I was searching for implementation details I was finding only articles that were saying something about nginx ingress controller or AWS application loadbalancer and a lot of troubles with them.
Do I miss anything in my current, vanilla config? Do I need the AWS ALB or nginx ingress controller?
Thank you for all the replies.
All the best.
Do I miss anything in my current, vanilla config?
You probably exposed your echo-server app using service type - ClusterIP or NodePort which is fine if you only need to access your app locally in the cluster (ClusterIP) or using your node IP address (NodePort).
Do I need the AWS ALB or nginx ingress controller?
They both are different things, but they have similar common goal - to make your websocket app available externally and distribute traffic based on defined L7 routing routes. It's good solution if you have multiple deployments. So you need to answer yourself if you need some kind of Ingress Controller. If you are planning to deploy your application into production you should consider using those solutions, but probably it may be fine with service type LoadBalancer.
EDIT:
If you are already using service type LoadBalancer your app is already available externally. Ingress controller provides additional configuration possibilities to configure L7 traffic route to your cluster (Ingress Controllers are often using LoadBalancer under the hood). Check this answer for more details about differences between LoadBalancer and Ingress.
Also check:
Choosing the Right Load Balancer on Amazon: AWS Application Load Balancer vs. NGINX Plus
Configuring Kubernetes Ingress on AWS? Don’t Make These Mistakes
WebSocket - Deploy to Kubernetes
LoadBalancer vs Ingress
In GKE Ingress documentation
it states that:
When you create an Ingress object, the GKE Ingress controller creates a Google Cloud HTTP(S) Load Balancer and configures it according to the information in the Ingress and its associated Services.
To me it seems that I can not have multiple ingress resources with single GCP ingress controller. Instead, GKE creates a new ingress controller for every ingress resource.
Is this really so, or is it possible to have multiple ingress resources with a single ingress controller in GKE?
I would like to have one GCP LoadBalancer as ingress controller with static IP and DNS configured, and then have multiple applications running in cluster, each application registering its own ingress resource with application specific host and/or path specifications.
Please note that I'm very new to GKE, GCP and Kubernetes in general, so it might be that I have misunderstood something.
I think the question you're actually asking is slightly different than what you have written. You want to know if multiple Ingress resources can be linked to a single GCP Load Balancer, not GKE Ingress controller. Based on the concept of a controller, there is only one GKE Ingress controller in a cluster, which is responsible for fulfilling multiple resources and provisioning multiple load balancers.
So, to answer the question directly (because I've been searching for a straight answer for a long time!):
Combining multiple Ingress resources into a single Google Cloud load
balancer is not supported.
Source: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress
Sad.
However, using the nginx-ingress controller is one way to at least minimize the number of external (GCP) load balancers provisioned (it only provisions a single TCP load balancer), but since the load balancer is for TCP traffic, it cannot terminate SSL, or apply Firewall rules for you (Cloud Armor cannot be used, for instance).
The only way I know of to have a single HTTPS load-balancer in GCP terminate SSL and route traffic to multiple services in GKE is to combine the ingresses into a single resource with all paths and certificates defined in one place.
(If anybody figures out a way to do it with multiple separate ingress resources, I'd love to hear it!)
Yes it is possible to have the single ingress controller for multiple ingress resources.
You can create multiple ingress resources as per path requirement and all will be managed by single ingress controller.
There are multiple ingress controller options also available you can use Nginx also that will create one LB and manage the paths.
Inside Kubernetes if you are creating a service with type LoadBalancer it will create the new LB resource in GCP so make sure your microservice type is ClusterIP and your all traffic goes inside K8s cluster via ingress path.
When you setup the ingress controller it will create one service with type LoadBalancer you can can use that IP in DNS servers to forward the subdomain and path to K8s cluster.
What is the best approach to create the ingress resource that interact with ELB into target deployment environment that runs on Kubernetes?
As we all know there are different cloud provider and many types of settings that are related to the deployment of your ingress resource which depends on your target environments: AWS, OpenShift, plain vanilla K8S, google cloud, Azure.
On cloud deployments like Amazon, Google, etc., ingresses need also special annotations, most of which are common to all micro services in need of an ingress.
If we deploy also a mesh like Istio on top of k8s then we need to use an Istio gateway with ingress. if we use OCP then it has special kind called “routes”.
I'm looking for the best solution that targets to use more standard options, decreasing the differences between platforms to deploy ingress resource.
So maybe the best approach is to create an operator to deploy the Ingress resource because of the many different setups here?
Is it important to create some generic component to deploy the Ingress while keeping cloud agnostic?
How do other companies deploy their ingress resources to the k8s cluster?
What is the best approach to create the ingress resource that interact with ELB into target deployment environment that runs on Kubernetes?
On AWS the common approach is to use ALB, and the AWS ALB Ingress Controller, but it has its own drawbacks in that it create one ALB per Ingress resource.
Is we deploy also a mesh like Istio then we need to use Istio gateway with ingress.
Yes, then the situation is different, since you will use VirtualService from Istio or use AWS App Mesh - that approach looks better, and you will not have an Ingress resource for your apps.
I'm looking for the best solution that targets to use more standard options, decreasing the differences between platforms to deploy ingress resource.
Yes, this is in the intersection between the cloud provider infrastructure and your cluster, so there are unfortunately many different setups here. It also depends on if your ingress gateway is within the cluster or outside of the cluster.
In addition, the Ingress resource, just become GA (stable) in the most recent Kubernetes, 1.19.
We have a microservices architecture. We are planning to move this to Kubernetes cluster with Docker as container Runtime.(On Premise, No cloud)
Now I am able to figure out everything but one thing is not clear.
Basically we have around 10 aggregators which we have exposed via Nginx. So we are planning to Use Nginx Ingress(Project which is maintained by Kubernetes).
My doubt is currently we have complex Nginx config like different log files for different domains, generate custom headers, using Nginx Caching with purging logic with Persistent Volumes etc. Currently, we have 5-6 config files for Nginx.
Is it all possible via Ingress? From what I have read, we cant directly provide Nginx conf, we have to provide all config via ingress only? Also is it possible to break the ingress config in multiple files?
If yes, can someone provide some reference?
Remember that you have to have an ingress controller to satisfy an Ingress. Only creating an Ingress resource has no effect. In your case you need to deploy an Ingress controller such as ingress-nginx.
You can have multiple ingress rules for the same hostname with different paths. You can spread the Ingress configuration for a common host across multiple Ingress resources using Mergeable Ingress resources. Such resources can belong to the same or different namespaces. This enables easier management when using a large number of paths. See the Mergeable Ingress Resources example on our GitHub.
As an alternative to Mergeable Ingress resources, you can use VirtualServer and VirtualServerRoute resources for cross-namespace configuration. See the Cross-Namespace Configuration example on our GitHub.
Take a look: cross-namespace-configuration/, ingress-controller-configmap.
Kubernetes has both Ingress (in front of a Service) and Service with type:
LoadBalancer. These seem to do identical things: allow public traffic into the pods matching the service's selector. What are the benefits and drawbacks of each? In what scenarios would I choose one over the other?
Ingress can be used to expose many services depending on the path or even multiple applications depending on the host or domain in the request.
A load balancer always exposes one service only.
Assume that AWS, GCP or Azure is where your infrastructure located
Ingress:
Only work if you have ingress controller such as nginx-ingress-controller, traefik,...
Many services could share the same ingress
Name based virtual hosting
path based routing
Only one AWS ELB (or GCP load balancer for Google Cloud) is needed
Recommend to follow this approach for most of use cases
serviceType LoadBalancer:
each service would create separated AWS ELB (cost inefficiency, would be super expensive if you have more and more services later)
Could be helpful in case you want to ensure maximum security / workload ( 1 ELB per service)