Ingress Niginx on Multi-Node Virtualbox Driver Minikube - kubernetes

Question

I am following this tutorial for setting up Ingress with Ingress-Nginx on Minikube. But I can't seem to get it to work. I get a connection refused when I try to connect to port 80 on the VM IP address returned by minikube ip
My setup is this:
Minikube version: v1.25.1
VirtualBox version: 6.1
Kubernetes version: v1.22.5
The ingress-nginx namespace has the below resources:
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-controller-85f4c5b458-2dhqh 1/1 Running 0 49m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller NodePort 10.102.88.109 <none> 80:30551/TCP,443:31918/TCP 20h
service/ingress-nginx-controller-admission ClusterIP 10.103.134.39 <none> 443/TCP 20h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ingress-nginx-controller 1/1 1 1 20h
NAME DESIRED CURRENT READY AGE
replicaset.apps/ingress-nginx-controller-85f4c5b458 1 1 1 20h
NAME COMPLETIONS DURATION AGE
job.batch/ingress-nginx-admission-create 1/1 6s 20h
job.batch/ingress-nginx-admission-patch 1/1 6s 20h
The default namespace has the below resources
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/web-79d88c97d6-rvp2r 1/1 Running 0 47m 10.244.1.4 minikube-m02 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 20h <none>
service/web NodePort 10.104.20.14 <none> 8080:31613/TCP 20h app=web
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/example-ingress nginx hello-world.info localhost 80 20h
Minikube is exposing these services:
|---------------|------------------------------------|--------------|-----------------------------|
| NAMESPACE | NAME | TARGET PORT | URL |
|---------------|------------------------------------|--------------|-----------------------------|
| default | kubernetes | No node port |
| default | web | 8080 | http://192.168.59.106:31613 |
| ingress-nginx | ingress-nginx-controller | http/80 | http://192.168.59.106:30551 |
| | | https/443 | http://192.168.59.106:31918 |
| ingress-nginx | ingress-nginx-controller-admission | No node port |
| kube-system | kube-dns | No node port |
| kube-system | registry | No node port |
|---------------|------------------------------------|--------------|-----------------------------|
In step 4 of the Create an Ingress section The tutorial mentions this:
Add the following line to the bottom of the /etc/hosts file on your computer (you will need administrator access):
172.17.0.15 hello-world.info
Note: If you are running Minikube locally, use minikube ip to get the external IP. The IP address displayed within the ingress list will be the internal IP.
It's a three node cluster using VirtualBox. I've tried adding the Minikube ingress-nginx-controller service's IP (192.168.59.106, which is also the result of minikube ip) to my hosts file, but it doesn't work. And as far as I know, I can't include the service's node port 30551 in the hosts file to test that.
Some guidance on how to get this working would be much appreciated

Answer 1

You are correct. You cannot include the port in the /etc/hosts file. To get there, you would need to specify the full path in your browser or some other application as following (assuming no connectivity issues):
hello-world.info:30551
I'd recommend you to tell specifically what type of issue you have. There can be multiple issues and each one will have different solution.
For example there will be a difference between the inability to access the Service and getting the 404 message.
I'm not sure if it's related but I had connectivity issues when I created a cluster in a following way:
minikube start --driver="virtualbox"
minikube node add
minikube node add
However, when I ran below command, I encountered noone:
minikube start --driver="virtualbox" --nodes=3
Assuming that you would like to expose your Nginx Ingress controller to be available on the ports 80 and 443 instead of NodePort's you can do:
Spawn your cluster
Deploy:
Metallb.universe.tf
Configure your address pool similar to:
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 192.168.59.200-192.168.59.210"
Change the Service of your: ingress-nginx-controller to LoadBalancer instead of NodePort (kubectl edit svc -n ingress-nginx ingress-nginx-controller)
Check on the Service:
kubectl get svc -n ingress-nginx ingress-nginx-controller
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.106.63.253 192.168.59.201 80:30092/TCP,443:30915/TCP 23m
Put the EXTERNAL-IP of your Ingress controller to your /etc/hosts file.
Create an Ingress resource that is matching what you've inputted as a name to /etc/hosts and has some backend.
Additional resources:
Kubernetes.github.io: Ingress nginx
Kubernetes.io: Docs: Concepts: Services networking: Service

Answer 2

When following the tutorial, I enabled the ingress addon after creating my cluster by running minikube addons enable ingress
This appeared to succeed, but when trying to connect to port 80 on the IP address returned by minikube ip (which is also the ingress-nginx-controller minikube service address), I got a connection refused. This can be validated by running:
nc -zv $(minikube ip) 80
However, when I enabled ingress at the time of initial cluster creation with this command:
minikube start --driver=virtualbox \
--kubernetes-version=v1.22.5 --nodes 3 \
--addons=ingress
and then ran nc -zv $(minikube ip) 80, the connection was accepted. I'm not sure if this is an issue with Minikube or with VirtualBox, but enabling ingress at the initial cluster creation time rather then subsequently worked for me
I was then able to update my hosts file with just the IP of the minikube node and the hello-world.info host
One thing that might also be worth noting if you create and delete your clusters a lot, I found sometimes when updating the hosts file on a Mac that old IPs were being cached. Running sudo dscacheutil -flushcache may help with this