As various google articles(Example : this blog) states that this(connecting Kube cluster through proxy and clusterIP) method isn’t suitable for a production environment, but it’s useful for development.
My question is why it is not suitable for production ? Why connecting through nodeport service is better than proxy and clusterIP ?
Lets distinguish between three scenarios where connecting to the cluster is required
Connecting to Kubernetes API Server
Connecting to the API server is required for administrative purposes. The users of your application have no business with it.
The following options are available
Connect directly to Master IP via HTTPS
Kubectl Proxy Use kubectl proxy to to make the Kubernetes API available on your localhost.
Connecting external traffic to your applications running in the Kubernetes Cluster. Here you want to expose your applications to your users. You'll need to configure a Service and they can be of the following types
NodePort: Only accessible on the NodeIPs and ports > 30000
ClusterIP: Internal Only. External traffic cannot hit a service of type ClusterIP directly. Requires ingress resource & ingress controller to receive external traffic.
LoadBalancer: Allows you receive external traffic to one and only one service
Ingress: This isn't a type of service, it is another type of Kubernetes resource. By configuring NGINX Ingress for example, you can handle traffic to multiple ClusterIP services with only on external LoadBalancer.
A Developer needs to troubleshoot a pod/service: kubectl port-forward: Port forwarding example Requires kubectl to be configured on the system hence it cannot be used for all users of the application
As you can see from the above explanation, the proxy and port-forwarding option aren't viable options for connecting external traffic to the applications running because it requires your kubectl installed and configured with a valid kubeconfig which grants access into your cluster.
Related
Kuberenetes newbie here, we have a jms server outside the cluster thats only accessible through our cluster, how can I create a port forward proxy on the cluster so I can connect to it via my local pc?
Proxy is an application layer function or feature, whereas port forwarding is really just a manual entry in one of the NAPT tables. A proxy understands the application protocol and can be used as a single entry point for multiple exposed servers.
The NGINX Ingress Controller for Kubernetes (as a proxy) is compatible with the NGINX web server. If you want to access workloads that are already running on your cluster from outside of it. Creating an Ingress resource is the standard procedure. In your workload cluster, add an ingress controller. For installation instructions, see this page.
Kubernetes port forwarding:
This is especially useful when you want to directly communicate with a specific port on a Pod from your local machine, according to the official kubernetes Connect with Port Forwarding documentation. Additionally, you don't have to manually expose services to accomplish this. Kubectl port-forward, on the other hand, moves connections from a local port to a pod port. Kubectl port-forward is more general than kubectl proxy because it can forward TCP traffic while kubectl proxy can only forward HTTP traffic. Although Kubectl simplifies port forwarding, it should only be utilized for debugging.
You can learn more about how to use port-forward to access applications in a cluster and another similar info link & SO aids in better comprehension.
Finally, for more information, see Port-Forwarding and Proxy Server and Client Deployment.
I understand that, in Cloud scenarios, a LoadBalancer resource refers to and provisions an external layer 4 load balancer. There is some proprietary integration by which the cluster configures this load balancing. OK.
On-prem we have no such integration so we create our own load balancer outside of the cluster which distributes traffic to all nodes - effectively, in our case to the ingress.
I see no need for the cluster to know anything about this external load balancer.
What is a LoadBalancer* then in an on-prem scenario? Why do I have them? Why do I need one? What does it do? What happens when I create one? What role does the LoadBalancer resource play in ingress traffic. What effect does the LoadBalancer have outside the cluster? Is a LoadBalancer just a way to get a new IP address? How/why/ where does the IP point to and where does the IP come from?
All questions refer to the Service of type “LoadBalancer” inside cluster and not my load balancer outside the cluster of which the cluster has no knowledge.
As pointed out in the comments a kubernetes service of type LoadBalancer can not be used by default with on-prem setups. You can use metallb to setup a service of that type in an on prem environment.
Kubernetes does not offer an implementation of network load balancers (Services of type LoadBalancer) for bare-metal clusters. [...] If you’re not running on a supported IaaS platform (GCP, AWS, Azure…), LoadBalancers will remain in the “pending” state indefinitely when created. [...] MetalLB aims to redress this imbalance by offering a network load balancer implementation that integrates with standard network equipment, so that external services on bare-metal clusters also “just work” as much as possible.
You can for example use the BGP mode to advertise the service's IP to your router, read more on that in the docs.
The project is still in beta but is promoted as production ready and used by several bigger companies.
Edit
Regarding your question in the comments:
Can I just broadcast the MAC address of my node and manually add the IP I am broadcasting to the LoadBalancer service via kubectl edit?
Yes that would work too. That's basically what metallb does, announcing the IP and updating the service.
Why need a software then? Imaging having 500 hosts that come and go with thousends of services of type LoadBalancer that come and go. You need an automation here.
Why does Kubernetes need to know this IP?
It doesn't. If you don't use an external ip, the service is still usable via it's NodePort, see for example the istio docs (with a little more details added by me):
$ kubectl get svc istio-ingressgateway -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
istio-ingressgateway LoadBalancer 192.12.129.119 <Pending> [...],80:32123/TCP,443:30994/TCP,[...]
Here the external IP is not set and stays in <Pending>. You can still use the service by pointing your traffic to <Node-IP>:32123 for plain http and to <Node-IP>:30994 for https. As you can see above those ports are mapped to 80 and 443.
If the external ip is set you can direct traffic directly to port 80 and 443 on the external load balancer. Kube-Proxy will create an iptables chain with the destination of you external ip, that basically leads from the external IP over the service ip with a load balancer configuration to a pod ip.
To investigate that set up a service of type LoadBalancer, make sure it has an external ip, connect to the host and run the iptables-save command (e.g. iptables-save | less). Search for the external ip and follow the chain until you end up at the pod.
I want to deploy a website on my kubernetes cluster.
I followed this guide to set up my kubernetes cluster on my set of raspberries. Now I have tested it with some nginx containers and it works to a certain degree since I need to find the correct ip of the machine it is deployed on.
Now that I have a signed up a domain I like to forward the traffic to my deployed website on my kubernetes cluster.
I have done this before with nginx, certbot and letsencrypt without containerisation. Now I am just missing the part how kubernetes handles the network. I assumed it was similar to swarms network which forwards all the request to the correct machine. But kubernetes does it differently.
TLDNR: How to deploy a website on a self build raspberry pi kubernetes cluster?
You need to create Kubernetes Service (documentation) to expose the web service to the outside world.
There are two types of Services relevant to deployments outside of cloud providers:
ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster.
This is the default ServiceType.
NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service
routes, is automatically created. You'll be able to contact the NodePort Service, from outside the cluster, by requesting <NodeIP>:<NodePort>.
So what you probably want is a NodePort service, which will expose the service on some fixed port on each of your Nodes (documentation and examples)
Loadbalancer is used to handle a request and forward that request to a particular pod, but my question is how does the GKE LoadBalancer work, how is it different from the LoadBalancer we have in minikube, and how should we use LoadBalancer properly.
In GKE when you add a Service of type LoadBalancer, it makes the necessary Google Cloud API calls to create either an external network LB, or an internal TCP/UDP LB.
The cloud.google.com/load-balancer-type: "Internal" annotation denotes an internal LB; otherwise, GKE creates an external network load balancer. This type of LB operates at layer 3/4, and is not an application load balancer, and thus not aware of HTTP requests or headers.
The LB service was designed for deployments in external cloud providers. In Minikube you need to use the tunnel feature to expose it. tunnel runs as a process, creating a network route on the host to the service CIDR of the cluster using the cluster’s IP address as a gateway. The tunnel command exposes the external IP directly to programs running on the host OS.
I have deployment that hosts a website on port 9001 and a service attached to it. I want to allow anyone (from outside cluster) to be able to connect to that site.
Any help would be appreciated.
I want to allow anyone (from outside cluster) to be able to connect to that site
There are many ways to do this using kubernetes services to expose port 9001 of the website to the outside world:
Service type LoadBalancer if you have an external, cloud-provider's load-balancer.
ExternalIPs. The website can be hit at ExternalIP:Port.
Service type NodePort if the cluster's nodes are reachable from the users. The website can be hit at NodeIP:NodePort.
Ingress controller and ingress resource.
As you wrote that this is not a cloud deployment, you need to consider how to correctly expose this to the world in a decent fashion. First and formost, create a NodePort type service for your deployment. With this, your nodes will expose that service on a high port.
Depending on your network, at this point you either need to configure a loadbalancer in your network to forward traffic for some IP:80 to your node(s) high NodePort, or for example deploy HAProxy in a DeamonSet with hostNetwork: true that will proxy 80 to your NodePort.
A bit more complexity can be added by deployment of Nginx IngressController (exposed as above) and use of Ingress to make the Ingress Controller expose all your services without the need to fiddle with NodePort/LB/HAProxy for each of them individualy any more.