I've setup a kubernetes cluster using kubespray, and now I am trying to follow this guide
root#node1 ~ # kubectl get -n heptio-contour service contour -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
contour LoadBalancer 10.233.55.94 <pending> 80:32414/TCP,443:30149/TCP 42m app=contour
It seems EXTERNAL-IP is pending because I am on a bare metal machine (not AWS/GKE etc.)
What do I need to do in order to get an external ip showing there?
Kubernetes offers three ways to expose a service:
1) L4 LoadBalancer: Available only on cloud providers such as GCE and AWS
2) Expose Service via NodePort: The NodePort directive allocates a port on every worker node, which proxy the traffic to the respective Pod.
3) L7 Ingress: The Ingress is a dedicated load balancer (eg. nginx, HAProxy, traefik, vulcand) that redirects incoming HTTP/HTTPS traffic to the respective endpoints
Kubernetes does not offer implementation of network load-balancers (Services of type LoadBalancer) for bare metal clusters.
If you’re not running Kubernetes cluster on a supported IaaS platform (GCP, AWS, Azure…), LoadBalancers will remain in the “pending” state regardless of the time they were created.
The reason is the lack of support of IP routing between the external world and Kubernetes; there is no default implementation of transferring dns zones
used by Ingress to allocate communication to internal of the cluster.
There are external projects to provide bare-metal even in federation clusters mode to be part of standalone or hybrid solution.
It depends on the scale and the maturity of projects you have, so it should begin with choosing a proper load balancer or VIP provider:
https://github.com/google/metallb
https://docs.traefik.io/
https://github.com/kubernetes/contrib/tree/master/keepalived-vip
http://vulcand.github.io/
and deprecated:
http://www.linuxvirtualserver.org/software/ipvs.html
Please notice that in federation clusters (more than one bare metal Kubernetes
cluster) is needed to export IP address of each physical machine to central API
provider and probably it is not covered by the links I provided to you.
Related
I was making some research about how K8s resolves the services using the clusterIP services and how CNIs like WeaveNet or how service meshes like Istio provide additional features to this functionality. However, I'm new on the topic and I'd like to share here what I've found to see if somebody can expand and correct my points:
Istiod has a service registry. This service registry is filled with the entries coming from K8s services clusterIPs (which in turn is the service registry of K8s) and other possible external services defined with Kind: ServiceEntry
(see seciton 5.5 of book istio in action)
This service registry is then mixed with more information about virtualservices and destination rules. These new/added K8s kinds are CRDs from Istio. They are what give the features of L7 load balancing that allow to distribute traffic by HTTP headers or URI path.
Without Istio, K8s has different (3) ways to implement the clusterIPs services concept. This services provide load balancing at L4.
https://kubernetes.io/docs/concepts/services-networking/service/
The most extended one nowadays is the iptables proxy mode. The iptables of the Linux machine are populated in bases of what theh kube-proxy provides. Kube-proxy gets those data from the kube-apiserver and (problably the core-dns). The kube-apisever will in turn consult the etcd database to know about the k8s clusterIP services. The entry of the iptables is populated with a the clusterIP->pod IP with only one pod IP out of the many pod that a deployment behind the clusterIP could be.
Any piece of code/application inside of the container could make calls directy to the kube-apiserver if using the correct authentication and get the pod address but that would be not practic
K8s can use CNIs (container network interfaces). One example of this would be Weavenet.
https://www.weave.works/docs/net/latest/overview/
Wevenet creates a new layer 2 network using Linux kernel features. One daemon sets up this L2 network and manages the routing between machines and there are various ways to attach machines to the network.
In this network the containers can be exposed to the outside world.
Weavenet implements a micro DNS server at each node. You simply name containers and the routing just can work without the use of services, including the load balancing across multiple continers with the same name.
I understand that, in Cloud scenarios, a LoadBalancer resource refers to and provisions an external layer 4 load balancer. There is some proprietary integration by which the cluster configures this load balancing. OK.
On-prem we have no such integration so we create our own load balancer outside of the cluster which distributes traffic to all nodes - effectively, in our case to the ingress.
I see no need for the cluster to know anything about this external load balancer.
What is a LoadBalancer* then in an on-prem scenario? Why do I have them? Why do I need one? What does it do? What happens when I create one? What role does the LoadBalancer resource play in ingress traffic. What effect does the LoadBalancer have outside the cluster? Is a LoadBalancer just a way to get a new IP address? How/why/ where does the IP point to and where does the IP come from?
All questions refer to the Service of type “LoadBalancer” inside cluster and not my load balancer outside the cluster of which the cluster has no knowledge.
As pointed out in the comments a kubernetes service of type LoadBalancer can not be used by default with on-prem setups. You can use metallb to setup a service of that type in an on prem environment.
Kubernetes does not offer an implementation of network load balancers (Services of type LoadBalancer) for bare-metal clusters. [...] If you’re not running on a supported IaaS platform (GCP, AWS, Azure…), LoadBalancers will remain in the “pending” state indefinitely when created. [...] MetalLB aims to redress this imbalance by offering a network load balancer implementation that integrates with standard network equipment, so that external services on bare-metal clusters also “just work” as much as possible.
You can for example use the BGP mode to advertise the service's IP to your router, read more on that in the docs.
The project is still in beta but is promoted as production ready and used by several bigger companies.
Edit
Regarding your question in the comments:
Can I just broadcast the MAC address of my node and manually add the IP I am broadcasting to the LoadBalancer service via kubectl edit?
Yes that would work too. That's basically what metallb does, announcing the IP and updating the service.
Why need a software then? Imaging having 500 hosts that come and go with thousends of services of type LoadBalancer that come and go. You need an automation here.
Why does Kubernetes need to know this IP?
It doesn't. If you don't use an external ip, the service is still usable via it's NodePort, see for example the istio docs (with a little more details added by me):
$ kubectl get svc istio-ingressgateway -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
istio-ingressgateway LoadBalancer 192.12.129.119 <Pending> [...],80:32123/TCP,443:30994/TCP,[...]
Here the external IP is not set and stays in <Pending>. You can still use the service by pointing your traffic to <Node-IP>:32123 for plain http and to <Node-IP>:30994 for https. As you can see above those ports are mapped to 80 and 443.
If the external ip is set you can direct traffic directly to port 80 and 443 on the external load balancer. Kube-Proxy will create an iptables chain with the destination of you external ip, that basically leads from the external IP over the service ip with a load balancer configuration to a pod ip.
To investigate that set up a service of type LoadBalancer, make sure it has an external ip, connect to the host and run the iptables-save command (e.g. iptables-save | less). Search for the external ip and follow the chain until you end up at the pod.
I want to deploy a website on my kubernetes cluster.
I followed this guide to set up my kubernetes cluster on my set of raspberries. Now I have tested it with some nginx containers and it works to a certain degree since I need to find the correct ip of the machine it is deployed on.
Now that I have a signed up a domain I like to forward the traffic to my deployed website on my kubernetes cluster.
I have done this before with nginx, certbot and letsencrypt without containerisation. Now I am just missing the part how kubernetes handles the network. I assumed it was similar to swarms network which forwards all the request to the correct machine. But kubernetes does it differently.
TLDNR: How to deploy a website on a self build raspberry pi kubernetes cluster?
You need to create Kubernetes Service (documentation) to expose the web service to the outside world.
There are two types of Services relevant to deployments outside of cloud providers:
ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster.
This is the default ServiceType.
NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service
routes, is automatically created. You'll be able to contact the NodePort Service, from outside the cluster, by requesting <NodeIP>:<NodePort>.
So what you probably want is a NodePort service, which will expose the service on some fixed port on each of your Nodes (documentation and examples)
Loadbalancer is used to handle a request and forward that request to a particular pod, but my question is how does the GKE LoadBalancer work, how is it different from the LoadBalancer we have in minikube, and how should we use LoadBalancer properly.
In GKE when you add a Service of type LoadBalancer, it makes the necessary Google Cloud API calls to create either an external network LB, or an internal TCP/UDP LB.
The cloud.google.com/load-balancer-type: "Internal" annotation denotes an internal LB; otherwise, GKE creates an external network load balancer. This type of LB operates at layer 3/4, and is not an application load balancer, and thus not aware of HTTP requests or headers.
The LB service was designed for deployments in external cloud providers. In Minikube you need to use the tunnel feature to expose it. tunnel runs as a process, creating a network route on the host to the service CIDR of the cluster using the cluster’s IP address as a gateway. The tunnel command exposes the external IP directly to programs running on the host OS.
I am trying to deploy my sample micro service Docker image in Kubernetes cluster having 2 node. I explored everything about Pods, Services, Deployment, StatefulSets and Daemon-sets etc.
I am trying to create a sample deployment and Service for that. Here I explored about how deployment provides the scalability and load balancing functionality. And exploring about service discovery by providing Services ClusterIp.
I have two questions:
My scenario is that I am trying to deploy microservice on my on-premise Ubuntu machine. The machine has the IP address of 192.168.1.15. When I am referring Kubernetes, service will also have one clusterIP.
If my microservice end point is /api/v1/loadCustomer, how I can call this end point? Do I need to use clusterIP also ? Can I call simply 192.168.1.15:8080/api/v1/loadCustomers ?
What is the role of clusterIP when I am calling my end point ? Can I directly use port?
I am referring to the following link for exploration:
https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/
tldr:
you can not access the application using the clusterIP from the outside of the cluster. you can access the application using either loadbalancer's IP (type=LoadBalaner) or Node's IP (type=NodePort).
benefit of clusterIP:
As you know that pods can be created and terminated during its life-cycle consequently IP (endpoint IP)address created and terminated.Therefore, clusterIP is static which does not depends of the life-cycle of the pods.
Long Answer
In a Kubernetes cluster
an application or pod has following abstraction.
Endpoint IP and Port:It is provided by the CNI Plugins such as flannel, calico.
Each pod has an IP and tragetPort which is UNIQUE.
you can list and watch the endpoints by the following commands.
kubectl get endpoints --all-namespaces
clusterIP and port : It is provided by the kube-proxy component.
The replicated pods share a clusterIP and Port.
Load-balancing of request to the replicated pods.
internally expose so that other pod can discover it
you can list and watch clusterIP and port with the following command
kubectl get services --all-namespaces
externalIP and port: It can be layer 3-4 load balancer's IP and port or node's IP and Nodeport.
if you want to use loadbalancer's IP and port, you can use type=LoadBalaner in service file.
If you want to use node's IP, you need to use type=NodePort in service file.