I have a Kubernetes cluster with an on-demand auto-scaler to provide easily scaleable VM instances.
I want to provide SSH access (port 22) to the machines created within them behind a single ingress controller.
Example:
ssh poda.example.com:22 it should provide an ssh connection to pod A
ssh podb.example.com:22 it should provide an ssh connection to pod B
Another Example of UDP: There are two deployments of Redis within the cluster in two different namespaces. within the cluster, we are able to separate them using their service name. but outside the cluster we are not able to separate them
Expected Behaviour
redis-staging.example.com:6379 should connect to Redis in staging namespace
redis-prod.example.com:6379 it should connect to Redis in production namespace
I was using the Nginx ingress controller to open the port via a load-balancer to provide ssh access using the https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/ documentation.
Challenge is that this directly bypasses the requirement of the sub-domain part and connects one port to one pod only. even a non-existing sub-domain can connect to the one pod linked to the port exposed.
How can I implement the desired behavior with Nginx or any other Ingress controller?
Related
Kuberenetes newbie here, we have a jms server outside the cluster thats only accessible through our cluster, how can I create a port forward proxy on the cluster so I can connect to it via my local pc?
Proxy is an application layer function or feature, whereas port forwarding is really just a manual entry in one of the NAPT tables. A proxy understands the application protocol and can be used as a single entry point for multiple exposed servers.
The NGINX Ingress Controller for Kubernetes (as a proxy) is compatible with the NGINX web server. If you want to access workloads that are already running on your cluster from outside of it. Creating an Ingress resource is the standard procedure. In your workload cluster, add an ingress controller. For installation instructions, see this page.
Kubernetes port forwarding:
This is especially useful when you want to directly communicate with a specific port on a Pod from your local machine, according to the official kubernetes Connect with Port Forwarding documentation. Additionally, you don't have to manually expose services to accomplish this. Kubectl port-forward, on the other hand, moves connections from a local port to a pod port. Kubectl port-forward is more general than kubectl proxy because it can forward TCP traffic while kubectl proxy can only forward HTTP traffic. Although Kubectl simplifies port forwarding, it should only be utilized for debugging.
You can learn more about how to use port-forward to access applications in a cluster and another similar info link & SO aids in better comprehension.
Finally, for more information, see Port-Forwarding and Proxy Server and Client Deployment.
I have set up a kubernetes cluster using kubeadm on a server, which is using an ingress controller (nginx) and this is working as intended. However, I used to deploy a nginx reverse proxy when I was using docker and to forward traffic to the containers. I have read that the ingress controller embarks a reverse proxy but I am not sure if it is sufficient and how to configure it (like IP ban when too many requests are sent in 1 s, ...).
I am aware that it can be done by modifying the port of the cluster and forwarding the traffic from the reverse proxy to the ingress controller but I don't know if it has any utility.
If you have more control over your inbound traffic, you can test multiple ingresses, not only Nginx. It will depend on the purpose of your requirement, although Nginx supports rate-limit. I suggest test others ingresses but try to install metal-lb firstly. So you can assign a specific Loadbalancer IP for each ingress.
I have a Kubernetes cluster which doesn't need to expose ports to the public. I am installing monitoring and logging (Prometheus & Loki or Elastic) for in house use and would like to use their GUI. I could provision https ingress and limit IP access but port forwarding seems to work.
How Does port forwarding work, under the hood?
Is port forwarding as secure as my kubectl connection?
Is the connection as fast as an ingress load balancer based HTTPs connection?
In Kubernetes documentation you can find information that port-forward command allows you to access and interact with internal Kubernetes cluster processes from your localhost. Also it's one of the best tools to debugging.
Forward one or more local ports to a pod. This command requires the node to have 'socat' installed.
Use resource type/name such as deployment/mydeployment to select a pod. Resource type defaults to 'pod' if omitted.
If there are multiple pods matching the criteria, a pod will be selected automatically. The forwarding session ends when the selected pod terminates, and rerun of the command is needed to resume forwarding.
1. How Does port forwarding work, under the hood?
This information can be found in How Does Kubernetes Port Forwarding Work? article.
The whole process is simplified by the fact that kubectl already has a built-in port forwarding functionality.
A user interacts with Kubernetes using the kubectl command-line on their local machine.
The port-forward command specifies the cluster resource name and defines the port number to port-forward to.
As a result, the Kubernetes API server establishes a single HTTP connection between your localhost and the resource running on your cluster.
The user is now able to engage that specific pod directly, either to diagnose an issue or debug if necessary.
Port forwarding is a work-intensive method. However, in some cases, it is the only way to access internal cluster resources.
2. Is port forwarding as secure as my kubectl connection?
For this question, you can find answer in Is kubectl port-forward encrypted?. As pointed by #iomv
As far as I know when you port-forward the port of choice to your machine kubectl connects to one of the masters of your cluster so yes, normally communication is encrypted. How your master communicate to the pod though is dependent on how you set up internal comms.
or #neokyle
kubectl port-forward uses socat to make an encrypted TLS tunnel with port forwarding capabilities. The tunnel goes from you to the kube api-server to the pod so it may actually be 2 tunnels with the kube api-server acting as a pseudo router.
Kubecelt port-forward is encrypted.
3. Is the connection as fast as an ingress load balancer based HTTPs connection
As connection is inside the cluster, it should be faster than connection from outside the cluster to the cluster.
In addition, there was similar Stackoverflow thread about kubectl port-forward.
Basically I have Following Hdfs Cluster setup using docker-compose:
Node 1 with IP: 192.168.1.1 having service deployed as below:
Namenode1:9000
HMaster1: 8300
ZooKeeper1:1291
Node 2 with IP: 192.168.1.2 having service deployed as below:
Namenode2:9000
ZooKeeper2:1291
How does Traefik / Ngnix - (Ingress Controllers) forwards request to two different services having configured with same port number?
There are several great tutorials on how ingress and load balancing works in kubernetes, e.g. this one by Mark Betz. As a general rule, it helps to think in terms of services and workloads instead of specific nodes where your workloads are running on.
A workload deployed in Kubernetes (a so called Pod) has its own internal IP address, called a ClusterIP. That pod can have one or more ports open, just on that pod-owned ip address.
If you now have several pods to distribute the load, e.g. like 5 web server processes or backend logic, it would be hard for a client (inside the cluster) to keep track of all those pod IPs, because they also change when a pod is updated or just restarted due to a crash. This is why Kubernetes has a so called concept of services. Those provide a stable DNS name and IP which then transparently "forwards" to one of the healthy pods. So your client only needs to know the DNS name and not keep track of the specific pod IPs.
If you now want to expose such a service to the public, there are different ways. Either you set your service to type: LoadBalancer which then sets up some load balancer infrastructure on your cloud provider and routes traffic to the nodes and then to the pods - or - you already have an ingress controller in place and just define the routing based on host names and paths. An ingress controller itself is such a loadbalanced service with an attached cloud load balancer and also has some pods (with e.g. a traefik or nginx container) which then route your packets accordingly.
So coming back to your initial question: If you want to expose a service with several pods of the same kind, then you would first create a Service resource that matches your Pods using the selector and then you create one single ingress resource that provides a hostname/path and references this service. The ingress controller will pick up those ingress resources and configure the traefik or nginx accordingly. The ingress controller doesn't really care about the host IPs and port numbers, because it acts on the internal kubernetes ClusterIPs, so you even don't need (and shouldn't) expose such a service directly when you have an ingress in place.
I hope this answers your question regarding exposing two workloads over an ingress controller. For details, check the Kubernetes docs on Ingresses. Based on the services you named (zookeeper, hdfs) load balancing and ingresses might not be what you need for that case. Zookeeper instances should be internal in most cases and need to be adressed individually, so you might want to check out headless services, for this use case. Also check the Kubernetes docs for a way to run zookeeper.
I've been studying Kubernetes for a few weeks now, and using the kube-lego NGINX examples (https://github.com/jetstack/kube-lego) have successfully deployed services to Kubernetes cluster using Rancher on DigitalOcean.
I've deployed sample static sites, Wordpress, Laravel, Craft CMS, etc. All of which use custom Namespaces, Deployment, Secrets, Containers with external registries, Services, and Ingress Definitions.
Using the example (lego) NGINX Ingress Controller setup, I'm able to apply DNS to the exposed IP address of my K8s cluster, and have the resulting sites appear.
What I don't know, though, is how to allow for multiple hosts to have Ingress Controllers service the same deployments, and thus provide HA Ingress to the cluster. (by applying an external load balancer service, or geo-ip, or what-have-you).
Rancher (stable) allows me to add multiple hosts, I've spun up 3 to 5 at a time, and Kubernetes is configured and deployed across all Hosts. Furthermore, I'll define many replicas and/or deployments (listed above) and they will be spread over the cluster and accessible as would be expected. I've even specified multiple replicas of the Ingress Controller, but of course they all get scheduled on the same host, giving me only one IP address of Ingress.
So how do I allow multiple hosts (each with their own public facing IP address) to allow ingress into the cluster? I've also read about setting up multiple Ingress Controllers, but then you must specify what deployment/services are being serviced by what Ingress Controller, which then totally defeats the purpose.
Maybe I'm missing something, but if K8s multi-host is supposed to provide HA, and the Host with the Ingress Controller goes down, then the service will be rescheduled on the other Hosts, but the IP address that everything is pointing to will be dead, and thus an outage. Any way to have multiple IP Addresses to the same set of deployment/services?
I investigated my setup a bit more today, and I think I found out why I was having difficulty. The "LoadBalancer" is often mentioned as for use with Cloud Providers (in both docs, and what #fiunchinho describes). I was using it with a Rancher setup, which auto creates an HA-Proxy LoadBalancer ingress for you on the hosts.
By default, it will just schedule it on one of the hosts. You can specify that you want it scheduled globally buy providing an 'annotation' of io.rancher.scheduler.global: "true".
Like so:
annotations:
# Create load balancers on every host in the environment
io.rancher.scheduler.global: "true"
http://rancher.com/docs/rancher/v1.6/en/rancher-services/load-balancer/
I preferred LoadBalancer over NodePort because I wanted the ability to send port 80 (and in the future port 443) to any of the Nodes, and have them successfully fulfil my request by inspecting the Host header, and directing as-needed.
These LBs can also be setup in the Rancher UI under the "Infrastructure Stack" menu. I have successfully removed the single LB, and re-added one with an "Always run one instance of this container on every host" option enabled.
After this was configured, I could make a request to any of the Hosts for any of the Ingresses, and get a response, no matter what host the container was scheduled on.
https://rancher.com/docs/rancher/v1.6/en/rancher-services/load-balancer/
So cool!
The ingress controller is deployed like any regular pod. That means that you can have as many replicas as you'd like, which will be spread among all your nodes.
You need a Service object that group all the pods for the ingress controller.
Then you just need to expose that Service to outside the cluster. You can do that using a LoadBalancer service if you are on a cloud provider. Or you can use just a NodePort service.
The point is that the service will balance the traffic that your ingress controller receives between all the pods that are running on different kubernetes nodes. If one of the nodes goes down, it doesn't really matter, because there are other nodes containing ingress controller pods.