I have this application I'm toying with. It consists of a Deployment (web with 3 instances running) with a Service exposing it (web-service). Right now the Service is of type LoadBalancer. I also added a Ingress directive to route my traffic as desired.
My question is...with the Ingress in place, should I switch the service type from LoadBalancer to NodePort or I should leave it in the same state?
From the documentation I have read so far it seems to be the general concensus to avoid nodePort as it is allocating ports on the physical (or virtual) nodes and that can become crowded after a time. Having a LoadBalancer automatically exposes a free port that the Google L3 Loadbalancer routes traffic to so no need to switch this to Nodeport.
Also note - once you have a working Ingress you don't need to expose the backend service as a LoadBalancer or NodePort at all - a normal internal kind: Service is enough to receive traffic from the Ingress.
As discussed here and described here, the service needs to be NodePort
Related
Question is straightforward, but I've not been able to quite figure out which steps a request follows when it reaches kubernetes system.
Who first handle a received request? Ingress Controller, LoadBalancer, ClusterIP...
So, I know there are several ways to make pods externally accessible:
Creating a NodePort service.
Creating an LoadBalancer service.
Creating an Ingress rule.
Some questions here related with best-practices or mandatory facts?
Ingress is in front of a ClusterIP Service mandatory?
1.1 Could or shouldn't I create an Ingress in front of a NodePort or a LoadBalancer service?
Ingress Controllers are LoadBalancer Services? I mean, traefik or other Ingress Controllers are all of them deployed as LoadBalancer services?
Misunderstanding arises from several texts I've found over there:
image here: Seems LoadBalancer is placed first of Ingress Controllers.
image here: Seems Ingress is in front of a LoadBalancer.
Above questions arises from an attempt of expose externally a mongodb replicatset.
I've created a LoadBalancer for each node. Is this correct?
I'd like to create a domain using my Ingress Controller for those LoadBalancer. Can this be possible?
Is there point to create an Ingress in front of a headless service?
Ingress is in front of a ClusterIP Service mandatory?
If you want the service accessible externally, then you will need an externally accessible service. This can be a LoadBalancer service or an Ingress. A ClusterIP service is not accessible outside the cluster.
Could or shouldn't I create an Ingress in front of a NodePort or a LoadBalancer service?
You can create an Ingress in front of a NodePort or LoadBalancer, but there's no point in creating an Ingress in front of a LoadBalancer unless you want two different endpoints for accessing the same service (the LoadBalancer will get one IP and the Ingress Controller's own LoadBalancer will get another IP). However, using an Ingress will allow you to have additional functionality, such as SSL Certificates, which the standard LoadBalancer service resource does not (normally) provide
Ingress Controllers are LoadBalancer Services? I mean, traefik or other Ingress Controllers are all of them deployed as LoadBalancer services?
Correct. An Ingress controller opens an endpoint for traffic into the cluster, and then uses the ingress resources you create in the cluster to determine how and where to route the traffic.
The endpoint is a publicly accessible endpoint (unless you configure it to be an internal loadbalancer, in which case only machines within your corporate network will be able to access it).
The controller will normally update the Ingress resource in your cluster so you will see the IP of the loadbalancer belonging to the ingress
Before I begin, I would like to mention that I am on the free trial version of GKE.
I have a simple server running in a GKE cluster. I have a service that I use to expose the server. I am trying to configure an Ingress Controller and attach it to this service.
Everything works perfectly if my service is of type LoadBalancer, NodePort. However, if my service is of type ClusterIP, I get an error saying
error while evaluating the ingress spec: service "default/simple-server" is type "ClusterIP"
, expected "NodePort" or "LoadBalancer"
GKE then stops trying to provision an IP for the ingress. Why can't I provision a service of type clusterIP and is there a work around?
I tried using annotations.kubernetes.io/ingress.class: "nginx" and it still didn't work.
That's normal and it's pretty simple why you have the error.
A ClusterIP service is an internal only service inside of your Kubernetes cluster and what you are trying to create (from what I undestood) is a publicly exposed service using ingress, meaning you are going to create a Google Load balancer.
Now the why it doesn't support ClusterIP is because when you create the ingress, the LB resource created inside google cloud needs a target port on your cluster to call on And for that you need to expose an ip/port for it.
I would not recommend combining LB service (wich by default create a LB on cloud provider) and ingress, but stay on a nodeport/ingress duo that is cleaner.
Bonus: The annotation you used is for internal services with ingress to be linked to your ingress controller. This will allow the ingress-controller to list the hosts and proxy traffic to the right service.
The native GKE ingress controller does not support ClusterIP, but it works perfectly with LoadBalancer and NodePort type. Take a look at this issue
Non-native ingress controller Nginx works with ClusterIP.
You can use Ingress to access a ClusterIP in GKE by setting the annotation
cloud.google.com/neg: '{"ingress": true}' to the service.
apiVersion: v1
kind: Service
metadata:
name: myapp
annotations:
cloud.google.com/neg: '{"ingress": true}'
...
See also answer https://stackoverflow.com/a/68271142 in the related question Why does Google Cloud show an error when using ClusterIP
When reading through the kubernetes document I noticed that NodePort type service always automatically creates a ClusterIP and ingress traffic targeting NodePort will be routed to ClusterIP. My question is that why is this necessary? Why can't kubeproxy directly does load balancing for this NodePort through forwarding? ClusterIP doesn't seem to be necessary to support NodePort and it seems to introduce additional overhead.
Even Service of type NodePort does not directly contact Pods, they still go through the Service's cluster IP, and its associated Pod selector rules. With newer kubernetes I think you can also influence whether traffic is round-robin or weighted distributed, which wouldn't work if the NodePort directly contacted the Pods
Also, NodePorts are opened on every member of the cluster, but -- in most cases -- you don't have a Pod running on every member of the cluster, so it still has to use the Service IP to route to the actual Node upon which an in-service Pod can service that traffic.
Think of NodePorts an a mechanism to bridge the "outside world" with the "cluster world," rather than a shortcut mechanism to side-step iptables or ipvs
How can I expose a StatefulSet service (cassandra, mysql, etc...) with ClusterIP=None on Kubernetes in Google Cloud Platform?
I need to change the ClusterIP config? Or I need to configure Google Cloud NAT? Or I need to change other things?
Thanks
EDIT: I want to connect to cassandra from an external IP, from anyplace on the internet
EDIT2: I guess that the solution is to use LoadBalance instead of ClusterIP, but when I use LoadBalance, the Cassandra nodes can't find the seed node. Then I sill using ClusterIP=None to Cassandra cluster, and I created another POD with type=LoadBalance to connect to Cassandra and to have connections to exterior. And now it's working :)
If by "expose" you mean ability to reach your service endpoints without cluster IP , then just use selector in your headless service, i.e.
apiVersion: v1
kind: Service
metadata:
name: cassandra
spec:
clusterIP: None
selector:
app: cassandra
ports:
- port: 80
targetPort: 80
For more details refer to documentation
Otherwise, if you want to expose your deployments outside of the cluster, you won't be able to do it with headless service.
ClusterIP services are not exposed outside of the Kubernetes cluster. Perhaps you mean to use a NodePort or LoadBalancer service instead?
If you want to expose the service externally, you will need a service that is ClusterIP backed whether that be a NodePort or LoadBalancer; even if you use ingress, you will need to back it up with a ClusterIP service at the very least.
The ClusterIP is only internal and provides the Kubebernetes cluster a fixed endpoint to reference your deployment/pod internally. The simplest method to expose your services is to use a NodePort, in which case your service will take on the IP of the node externally with a high port number (30000+). On GCP, if you define a load-balancer, you will be given an external IP, and the traffic will be forwarded in order to your pods in the stateful sets. If you use an ingress, your external IP will be that of your ingress, and the packet forwarding to your services will be done based on the request URL (ie. you can have multiple FQDNs mapped to a single external IP in your DNS).
"Headless" services are mainly used to decouple your design from Kubernetes. The assumption is that you will be doing your own service discovery, and I don't believe that is a good use case for your application.
Hope this helps!
We are trying to find and implement best practises for Kubernetes and LoadBalancer typed services.
We found an entry that an open NodePort is a bad-practise if you use a LoadBalancer.
Kubernetes now always creates a NodePort too, when we use type: LoadBalancer
service:
type: LoadBalancer
port: 30111
I cannot find a hint in the documentation why this happens, if this is normal or how to avoid a NodePort here.
What's the reason Kubernetes always creates a NodePort too?
It's a normal behavior.
From documentation:
LoadBalancer: Exposes the service externally using a cloud provider’s load balancer. NodePort and ClusterIP services, to which the external load balancer will route, are automatically created.
The problem configuring an external load balancer and a NodePort service type, is that you would be balancing requests between the nodes twice: one time at the external load balancer and another time at svc Kubernetes level.
As far as I know, using the LoadBalancer service type Kubernetes avoids doing this and at least in EKS uses the NodePort service for health check purposes (target groups).