I have two Kubernetes developpements: one with a Grafana pod, and the other with an InfluxDB pod.
I want:
1) to expose my Grafana:3000 to the outside word
2) Grafana to be able to connect to InfluxDB:8086
I've seen it done with two separate "services": one Grafana Service (type = NodePort, target port = 3000) and one InfluxDB service (type = ClusterIP, target port = 8086).
It works ok.
Yet can/should it be done with just one "service" ?
For north south traffic i.e exposing a service outside the cluster LoadBalancer or preferably ingress is better than NodePort because when NodePort is used if the Node IP or Port changes then the clients need to make changes in the endpoint that they are using to access the service.
For east-west traffic between services within the cluster clusterIP service is good enough.
Use ingress. You can use Nginx, Caddy, or other tool provide ingress service and define the routing of the ingress there.
Related
Exposing Service from a BareMetal(Kubeadm) Build Kubernetes Cluster to the outside world. I am trying to access my Nginx as a service outside of the cluster to get NGINX output in the web browser.
For that, I have created a deployment and service for NGINX as shown below,
As per my search, found that we have below to expose to outside world
MetalLb
Ingress NGINX
Some HELM resources
I would like to know all these 3 or any more approaches in such way it help me to learn new things.
GOAL
Exposing Service from a BareMetal(Kubeadm) Built Kubernetes Cluster to the outside world.
How Can I make my service has its own public IP to access from the outside cluster?
You need to set up MetalLB to get an external IP address for the LoadBalancer type services. It will give a local network IP address to the service.
Then you can do port mapping (configuration in the router) of incoming traffic of port 80 and port 443 to your external service IP address.
I have done a similar setup you can check it here in detail:
https://developerdiary.me/lets-build-low-budget-aws-at-home/
You need to deploy an ingress controller in your cluster so that it gives you an entrypoint where your applications can be accessed. Traditionally, in a cloud native environment it would automatically provision a LoadBalancer for you that will read the rules you define inside your Ingress object and route your request to the appropriate service.
One of the most commonly used ingress controller is the Nginx Ingress Controller. There are multiple ways you can use to deploy it (mainfests, helm, operators). In case of bare metal clusters, there are multiple considerations which you can read here.
MetalLB is still in beta stage so its your choice if you want to use. If you don't have a hard requirement to expose the ingress controller as a LoadBalancer, you can expose it as a NodePort Service that will accessible across all your nodes in the cluster. You can then map that NodePort Service in your DNS so that the ingress rules are evaluated.
I want to create a cluster of RESTful web APIs in AWS EKS and be able to access them through a single IP (allowing kubernetes to load balance requests to each). I have followed the procedure explained the this link and have set up an example nginx deployment as shown in the following image:
The problem is that when I access the example nginx deployment via 172.31.22.183 it works just fine, but when I try to use the cluster IP 10.100.145.181 it does not yield any response in such a way that it seems to be unreachable.
What's the purpose of that cluster ip then and how can I use it to achieve what I need?
What's the purpose of that cluster ip then and how can I use it to
achieve what I need?
ClusterIP is local IP that is used internally in the cluster, you can use it to access the application.
While i think Endpoint IP that you got, is might be external and you can access the application outside.
AWS EKS and be able to access them through a single IP (allowing
kubernetes to load balance requests to each)
For this best practice is to use the ingress, API gateway or service mesh.
Ingress is single point where all your request will be coming inside it will be load balancing and forwarding the traffic internally inside the cluster.
Consider ingress is like Loadbalancer single point to come inside the cluster.
Ingress : https://kubernetes.io/docs/concepts/services-networking/ingress/
AWS Example : https://aws.amazon.com/blogs/opensource/network-load-balancer-nginx-ingress-controller-eks/
ClusterIP is an IP that is only accessible inside the cluster. You cannot hit it from outside cluster unless you use kubectl port-forward
I need for a service in a K8 pod to be able to make HTTP calls to downstream services, load balanced by a NodePort, within the same cluster and namespace.
My constraints are these:
I can do this only through manipulation of deployment and service
entities (no ingress. I don't have that level of access to the
cluster)
I cannot add any K8 plugins
The port that the NodePort exposes must be randomized, not hard coded
This whole thing must be automated. I can't set the deployment with the literal value of
the exposed port. It needs to be set by some sort of variable, or
similar process.
Is this possible, and, if so, how?
It probably can be done but it will not be straight forward and you might have to add some custom automation. A NodePort service is meant to be used by an entity outside your cluster.
For inter-cluster communication, a regular service (with a ClusterIP) will work as designed. Your service can reach another service using DNS service discovery. For example. svc-name.mynamespace.svc.cluster.local would be the DNS entry for a svc-name in the mynamespace namespace.
If you can only do a NodePort which essentially is a port on your K8s nodes, you could create another Deployment or Pod of something like nginx or haproxy. Then have this deployment being serviced by regular K8s service with a ClusterIP. Then have nginx or haproxy point to the NodePort on all your nodes in your Kubernetes cluster. Also, have it configured so that it only forwards to listening NodePorts with some kind of healthcheck.
The above seems like an extra necessary step, but if NodePort from within the cluster is what you need (for some reason), it should do the trick.
I'm trying to use Kubernetes to make configurations and deployments explicitly defined and I also like Kubernetes' pod scheduling mechanisms. There are (for now) just 2 apps running on 2 replicas on 3 nodes. But Google's Kubernetes Engine's load balancer is extremely expensive for a small app like ours (at least for the moment) at the same time I'm not willing to change to a single instance hosting solution on a container or deploying the app on Docker swarm etc.
Using node's IP seemed like a hack and I thought that it might expose some security issues inside the cluster. Therefore I configured a Træfik ingress and an ingress controller to overcome Google's expensive flat rate for load balancing but turns out an outward facing ingress spins up a standart load balancer or I'm missing something.
I hope I'm missing something since at this rates ($16 a month) I cannot rationalize using kubernetes from start up for this app.
Is there a way to use GKE without using Google's load balancer?
An Ingress is just a set of rules that tell the cluster how to route to your services, and a Service is another set of rules to reach and load-balance across a set of pods, based on the selector. A service can use 3 different routing types:
ClusterIP - this gives the service an IP that's only available inside the cluster which routes to the pods.
NodePort - this creates a ClusterIP, and then creates an externally reachable port on every single node in the cluster. Traffic to those ports routes to the internal service IP and then to the pods.
LoadBalancer - this creates a ClusterIP, then a NodePort, and then provisions a load balancer from a provider (if available like on GKE). Traffic hits the load balancer, then a port on one of the nodes, then the internal IP, then finally a pod.
These different types of services are not mutually exclusive but actually build on each other, and it explains why anything public must be using a NodePort. Think about it - how else would traffic reach your cluster? A cloud load balancer just directs requests to your nodes and points to one of the NodePort ports. If you don't want a GKE load balancer then you can already skip it and access those ports directly.
The downside is that the ports are limited between 30000-32767. If you need standard HTTP port 80/443 then you can't accomplish this with a Service and instead must specify the port directly in your Deployment. Use the hostPort setting to bind the containers directly to port 80 on the node:
containers:
- name: yourapp
image: yourimage
ports:
- name: http
containerPort: 80
hostPort: 80 ### this will bind to port 80 on the actual node
This might work for you and routes traffic directly to the container without any load-balancing, but if a node has problems or the app stops running on a node then it will be unavailable.
If you still want load-balancing then you can run a DaemonSet (so that it's available on every node) with Nginx (or any other proxy) exposed via hostPort and then that will route to your internal services. An easy way to run this is with the standard nginx-ingress package, but skip creating the LoadBalancer service for it and use the hostPort setting. The Helm chart can be configured for this:
https://github.com/helm/charts/tree/master/stable/nginx-ingress
One option is to completely disable this feature on your GKE cluster. When creating the cluster (on console.cloud.google.com) under Add-ons disable HTTP load balancing. If you are using gcloud you can use gcloud beta container clusters create ... --disable-addons=HttpLoadBalancing.
Alternatively, you can also inhibit the GCP Load Balancer by adding an annotation to your Ingress resources, kubernetes.io/ingress.class=somerandomstring.
For newly created ingresses, you can put this in the yaml document:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: somerandomstring
...
If you want to do that for all of your Ingresses you can use this example snippet (be careful!):
kubectl get ingress --all-namespaces \
-o jsonpath='{range .items[*]}{"kubectl annotate ingress -n "}{.metadata.namespace}{" "}{.metadata.name}{" kubernetes.io/ingress.class=somerandomstring\n"}{end}' \
| sh -x
Now using Ingresses is pretty useful with Kubernetes, so I suggest you check out the nginx ingress controller and after deployment, annotate your Ingresses accordingly.
If you specify the Ingress class as an annotation on the Ingress object
kubernetes.io/ingress.class: traefik
Traefik will pick it up while the Google Load Balancer will ignore it. There is also a bit of Traefik documentation on this part.
You could deploy the nginx ingress controller using NodePort mode (e.g. if using the helm chart set controller.service.type to NodePort) and then load-balance amongst your instances using DNS. Just make sure you have static IPs for the nodes or you could even create a DaemonSet that somehow updates your DNS with each node's IP.
Traefik seems to support a similar configuration (e.g. through serviceType in its helm chart).
I have deployment that hosts a website on port 9001 and a service attached to it. I want to allow anyone (from outside cluster) to be able to connect to that site.
Any help would be appreciated.
I want to allow anyone (from outside cluster) to be able to connect to that site
There are many ways to do this using kubernetes services to expose port 9001 of the website to the outside world:
Service type LoadBalancer if you have an external, cloud-provider's load-balancer.
ExternalIPs. The website can be hit at ExternalIP:Port.
Service type NodePort if the cluster's nodes are reachable from the users. The website can be hit at NodeIP:NodePort.
Ingress controller and ingress resource.
As you wrote that this is not a cloud deployment, you need to consider how to correctly expose this to the world in a decent fashion. First and formost, create a NodePort type service for your deployment. With this, your nodes will expose that service on a high port.
Depending on your network, at this point you either need to configure a loadbalancer in your network to forward traffic for some IP:80 to your node(s) high NodePort, or for example deploy HAProxy in a DeamonSet with hostNetwork: true that will proxy 80 to your NodePort.
A bit more complexity can be added by deployment of Nginx IngressController (exposed as above) and use of Ingress to make the Ingress Controller expose all your services without the need to fiddle with NodePort/LB/HAProxy for each of them individualy any more.