How to expose Minikube cluster to internet - kubernetes

I know minikube should be used for local only, but i'd like to create a test environment for my applications.
In order to do that, I wish to expose my applications running inside the minikube cluster to external access (from any device on public internet - like a 4G smartphone).
note : I run minikube with --driver=docker
kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
web8080 NodePort 10.99.39.162 <none> 8080:31613/TCP 3d1h
minikube ip
192.168.49.2
One way to do it is as follows :
firewall-cmd --add-port=8081/tcp
kubectl port-forward --address 0.0.0.0 services/web8080 8081:8080
then I can access it using :
curl localhost:8081 (directly from the machine running the cluster inside a VM)
curl 192.168.x.xx:8081 (from my Mac in same network - this is the private ip of the machine running the cluster inside a VM)
curl 84.xxx.xxx.xxx:8081 (from a phone connected in 4G - this is the public ip exposed by my router)
I don't want to use this solution because kubectl port-forward is weak and need to be run every time the port-forwarding is no longer active.
How can I achieve this ?
(EDITED) - USING LOADBALANCER
when using LoadBalancer type and minikube tunnel, I can expose the service only inside the machine running the cluster.
kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-service LoadBalancer 10.111.61.218 10.111.61.218 8080:31831/TCP 3d3h
curl 10.111.61.218:8080 (inside the machine running the cluster) is working
but curl 192.168.x.xx:8080 (from my Mac on same LAN) is not working
Thanks

Minikube as a development tool for a single node Kubernetes cluster provides inherent isolation layer between Kubernetes and the external devices (being specific the inbound traffic to your cluster from LAN/WAN).
Different --drivers are allowing for flexibility when it comes to the place where your Kubernetes cluster will be spawned and how it will behave network wise.
A side note (workaround)!
As your minikube already resides in a VM and uses --driver=docker you could try to use --driver=none (you will be able to curl VM_IP:NodePort from the LAN). It will spawn your Kubernetes cluster directly on the VM.
Consider checking it's documentation as there are some certain limitations/disadvantages:
Minikube.sigs.k8s.io: Docs: Drivers: None
As this setup is already basing on the VM (with unknown hypervisor) and the cluster is intended to be exposed outside of your LAN, I suggest you going with the production-ready setup. This will inherently eliminate the connectivity issues you are facing. Kubernetes cluster will be provisioned directly on a VM and not in the Docker container.
Explaining the --driver=docker used: It will spawn a container on a host system with Kubernetes inside of it. Inside of this container, Docker will be used once again to spawn the necessary Pods to run the Kubernetes cluster.
As for the tools to provision your Kubernetes cluster you will need to chose the option that suits your needs the most. Some of them are the following:
Kubeadm
Kubespray
MicroK8S
After you created your Kubernetes cluster on a VM you could forward the traffic from your router directly to your VM.
Additional resources that you might find useful:
Stackoverflow.com: Questions Expose Kubernetes cluster to the Internet (Virtualbox with minikube)

curl $(minikube ip):$NODE_PORT : Now we can test that the app is exposed outside of the cluster using curl, the IP of the Node and the externally exposed port.
For you : curl 192.168.49.2:31613

Use nginx reverse-proxy
https://www.zepworks.com/posts/access-minikube-remotely-kvm/
install nginx, then in nginx.conf add this
stream {
server {
listen 8081;
proxy_pass 192.168.49.2:8080;
}
}
restart nginx

One way that I use to get around the fact that the process of kubectl port-forward stops after a while is to create a detach session using tmux following this. With that, I haven't had any problems with the exact same Minikube cluster configuration that you have.

Related

browsing Istio/k8s services from internet

I have started to read Istio-in-action (by Manning) and Mastering-service-mesh (by Packt) and there are some examples where I cannot 'see' the right output.
I work on my laptop with Ubuntu 20.04 and I use [kind] for my local k8s cluster where I can create 3 or more worker-nodes.
When I deploy some Istio resources (e.g. virtual service) I would like to browse the service-mesh from my Ubuntu browser or from a different client (either a different laptop or cell phone) but it misses something in my 'infrastructure'- is it the external load balancer or some local Ubuntu configuration? Is it mandatory to work with a public cloud provider - GCP/AWS/Azure ; if Yes, which one is the most simple? I have tried with kubectl port-forward but without success.
Other resources are ok (e.g. istioctl dashboard kiali/jaeger/prometheus) even without an ExternalIP.
Could you help me to find a blog or a tutorial/hint/advice about the necessary elements for browsing the k8s/Istio services from the internet?
Thank you in advance!
When installing istio with the istio-ingressgateway enabled a service with that name is created in the istio-system namespace.
❯ kubectl get svc -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
istio-ingressgateway LoadBalancer 100.71.98.21 <pending> 80:32564:80/TCP,...
When deploying istio to a public cloud provider, that will create a load balancer (like AWS ELB) for you. When the setup is done the EXTERNAL-IP will switch from <pending> to an actual ip, the public ip of the load balancer. You can access your cluster by visiting that ip.
On your local setup you don't have this luxury. But the service still is created. In the PORT(S) column you can see a bunch of ports. That is actually a port mapping. So ports of your node machine are being mapped to that service.
You use this to get the port mapped to http (port 80): For me it would be the 32564. Or you can run this:
kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(#.name=="http2")].nodePort}'
Now just open your browser and use one of your worker's ip to access the cluster by visiting <NODE_IP>:<PORT> (where PORT is the one from above).
See docs

Access Minikube Services from public IP [duplicate]

I know minikube should be used for local only, but i'd like to create a test environment for my applications.
In order to do that, I wish to expose my applications running inside the minikube cluster to external access (from any device on public internet - like a 4G smartphone).
note : I run minikube with --driver=docker
kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
web8080 NodePort 10.99.39.162 <none> 8080:31613/TCP 3d1h
minikube ip
192.168.49.2
One way to do it is as follows :
firewall-cmd --add-port=8081/tcp
kubectl port-forward --address 0.0.0.0 services/web8080 8081:8080
then I can access it using :
curl localhost:8081 (directly from the machine running the cluster inside a VM)
curl 192.168.x.xx:8081 (from my Mac in same network - this is the private ip of the machine running the cluster inside a VM)
curl 84.xxx.xxx.xxx:8081 (from a phone connected in 4G - this is the public ip exposed by my router)
I don't want to use this solution because kubectl port-forward is weak and need to be run every time the port-forwarding is no longer active.
How can I achieve this ?
(EDITED) - USING LOADBALANCER
when using LoadBalancer type and minikube tunnel, I can expose the service only inside the machine running the cluster.
kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-service LoadBalancer 10.111.61.218 10.111.61.218 8080:31831/TCP 3d3h
curl 10.111.61.218:8080 (inside the machine running the cluster) is working
but curl 192.168.x.xx:8080 (from my Mac on same LAN) is not working
Thanks
Minikube as a development tool for a single node Kubernetes cluster provides inherent isolation layer between Kubernetes and the external devices (being specific the inbound traffic to your cluster from LAN/WAN).
Different --drivers are allowing for flexibility when it comes to the place where your Kubernetes cluster will be spawned and how it will behave network wise.
A side note (workaround)!
As your minikube already resides in a VM and uses --driver=docker you could try to use --driver=none (you will be able to curl VM_IP:NodePort from the LAN). It will spawn your Kubernetes cluster directly on the VM.
Consider checking it's documentation as there are some certain limitations/disadvantages:
Minikube.sigs.k8s.io: Docs: Drivers: None
As this setup is already basing on the VM (with unknown hypervisor) and the cluster is intended to be exposed outside of your LAN, I suggest you going with the production-ready setup. This will inherently eliminate the connectivity issues you are facing. Kubernetes cluster will be provisioned directly on a VM and not in the Docker container.
Explaining the --driver=docker used: It will spawn a container on a host system with Kubernetes inside of it. Inside of this container, Docker will be used once again to spawn the necessary Pods to run the Kubernetes cluster.
As for the tools to provision your Kubernetes cluster you will need to chose the option that suits your needs the most. Some of them are the following:
Kubeadm
Kubespray
MicroK8S
After you created your Kubernetes cluster on a VM you could forward the traffic from your router directly to your VM.
Additional resources that you might find useful:
Stackoverflow.com: Questions Expose Kubernetes cluster to the Internet (Virtualbox with minikube)
curl $(minikube ip):$NODE_PORT : Now we can test that the app is exposed outside of the cluster using curl, the IP of the Node and the externally exposed port.
For you : curl 192.168.49.2:31613
Use nginx reverse-proxy
https://www.zepworks.com/posts/access-minikube-remotely-kvm/
install nginx, then in nginx.conf add this
stream {
server {
listen 8081;
proxy_pass 192.168.49.2:8080;
}
}
restart nginx
One way that I use to get around the fact that the process of kubectl port-forward stops after a while is to create a detach session using tmux following this. With that, I haven't had any problems with the exact same Minikube cluster configuration that you have.

How do I use minikube's DNS?

How do I use minikube's (cluster's) DNS? I want to receive all IP addresses associated with all pods for selected headless service? I don’t want to expose it outside the cluster. I am currently creating back-end layer.
As stated in the following answer:
What exactly is a headless service, what does it do/accomplish, and what are some legitimate use cases for it?
„Instead of returning a single DNS A record, the DNS server will return multiple A records for the service, each pointing to the IP of an individual pod backing the service at that moment.”
Thus the pods in back-end layer can communicate to each other.
I can’t use dig command. It is not installed in minikube. Eventually how do I install it? There is no apt available.
I hope this explains more accurately what I want to achieve.
You mentioned that you want to receive IP addresses associated with pods for selected service name for testing how does headless service work.
For only testing purposes you can use port-forwarding. You can forward traffic from your local machine to dns pod in your cluster. To do this, you need to run:
kubectl port-forward svc/kube-dns -n kube-system 5353:53
and it will expose kubs-dns service on your host. Then all you need is to use dig command (or alternative) to query the dns server.
dig #127.0.0.1 -p 5353 +tcp +short <service>.<namespace>.svc.cluster.local
You can also test your dns from inside of cluster e.g. by running a pod with interactive shell:
kubectl run --image tutum/dnsutils dns -it --rm -- bash
root#dns:/# dig +search <service>
Let me know it it helped.
As illustrated in kubernetes/minikube issue 4397
Containers don't have an IP address by default.
You'll want to use minikube service or minikube tunnel to get endpoint information.
See "hello-minikube/ Create a service":
By default, the Pod is only accessible by its internal IP address within the Kubernetes cluster.
To make the hello-node Container accessible from outside the Kubernetes virtual network, you have to expose the Pod as a Kubernetes Service.
Expose the Pod to the public internet using the kubectl expose command:
kubectl expose deployment hello-node --type=LoadBalancer --port=8080
The --type=LoadBalancer flag indicates that you want to expose your Service outside of the cluster.
View the Service you just created:
kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-node LoadBalancer 10.108.144.78 <pending> 8080:30369/TCP 21s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP
On Minikube, the LoadBalancer type makes the Service accessible through the minikube service command.
Run the following command:
minikube service hello-node

How to expose service outside k8s cluster?

I have run a Hello World application using the below command.
kubectl run hello-world --replicas=2 --labels="run=load-balancer-example" --image=gcr.io/google-samples/node-hello:1.0 --port=8080
Created a service as below
kubectl expose deployment hello-world --type=NodePort --name=example-service
The pods are running
NAME READY STATUS RESTARTS AGE
hello-world-68ff65cf7-dn22t 1/1 Running 0 2m20s
hello-world-68ff65cf7-llvjt 1/1 Running 0 2m20s
Service:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
example-service NodePort 10.XX.XX.XX <none> 8080:32023/TCP 66s
Here, I am able to test it through curl inside the cluster.
curl http://10.XX.XX.XX:8080
Hello Kubernetes!
How can I access this service outside my cluster? (example, through laptop browser)
you shoud try
http://IP_OF_KUBERNETES:32023
IP_OF_KUBERNETES can be your master IP your worker IP
when you expose a port in kubernetes .It expose that port in all of your server in cluster.Imagine you have two worker node with IP1 and IP2
and one pode is running in IP1 and in worker2 there is no pods but you can access your pod by
http://IP1:32023
http://IP2:32023
You should be able to access it outside the cluster using NodePort assingned(32023). Please paste following http://<IP>:<Port> in your browser and you will able to access your app:
http://<MASTER/WORKER_IP>:32023
There are answers already provided, but I felt like this topic needed some consolidation.
This seems to be fairly easy. NodePort actually exposes your application as the name says on the port of each node. So all you have to do is just find the IP address of the Node on which the pod is. You can do it by running:
kubectl get pods -o wide so you can find the IP or name of the node on which the pod is, then just follow what previous answers state: so http://<MASTER/WORKER_IP>:PORT
There is more methods:
You can deploy Ingress Controller and configure Ingress so the application will be reachable through the internet.
You can also use kubectl proxy to expose ClusterIP service outside of the cluster. Like in this example with Dashboard.
Another way is to use LoadBalancer type, which requires underlying cloud infrastructure.
If you are using minikube you can try to run minikube service list to check your exposed services and their IP.
You can access your service using MasterIP or WorkerIP. If you are planning to use it in production or in a more reliable way you should create a service with type LoadBalancer. And use load balancers IP to access it.
If you are using any cloud env, make sure the firewall rules allow incoming traffic.
This will take care of redirecting request to which ever node the pod is running on. Else you will have to manually hit masterIP or workerIP depending on where the pod is running. If the pod gets moved to different node, you will have to change the ip you are hitting
Service Load Balancer

External IP assignment with Minihube ingress add-on enabled

For development purposes I try to use Minikube. I want to test how my application will catch an event of exposing a service and assigning an External-IP.
When I exposed a service in Google Container Engine quick start tutorial I could see an event of External IP assignment with:
kubectl get services --watch
I want to achieve the same with Minikube (if possible).
Here is how I try to set things up locally on my OSX development machine:
minikube start --vm-driver=xhyve
minikube addons enable ingress
kubectl run echoserver --image=gcr.io/google_containers/echoserver:1.4 --port=8080
kubectl expose deployment echoserver --type="LoadBalancer"
kubectl get services --watch
I see the following output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
echoserver LoadBalancer 10.0.0.138 <pending> 8080:31384/TCP 11s
kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 4m
External-Ip field never gets updated and shows pending phase. Is it possible to achieve external IP assignment with Minikube?
On GKE or AWS installs, the external IP comes from the cloud support that reports back to kube API the address that the created LB was assigned.
To have the same on minikube you'd have to run some kind of an LB controller, ie. haproxy one, but honestly, for minikube it makes little sense, as you have single IP that you know in advance by minikube ip so you can use NodePort with that knowledge. LB solution would require setting some IP rangethat can be mapped to particular nodeports, as this is effectively what LB will do - take traffic from extIP:extPort and proxy it to minikubeIP:NodePort.
Unless your use case prevents you from it, you should consider Ingress as the way of ingesting traffic to your minikube.
If you want to emulate external IP assignment event (like the one you can observe using GKE or AWS), this can be achieved by applying the following patch on your sandbox kubernetes:
kubectl run minikube-lb-patch --replicas=1 --image=elsonrodriguez/minikube-lb-patch:0.1 --namespace=kube-system
https://github.com/elsonrodriguez/minikube-lb-patch#assigning-external-ips