I have a Kubernetes Cluster running on Azure. I use the nginx-ingress to handle incoming requests. To set up the ingress I used the official guide https://kubernetes.github.io/ingress-nginx/deploy/#azure .
I also created a public static IP which I want to use for the Ingress.
Unfortunately, I´m not able to find the ingress service (generic-deployment.yaml). Also, my ingress is not describable.
How I installed Ingress:
$ sudo kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
...
deployment.apps/nginx-ingress-controller created
$ sudo kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/cloud-generic.yaml
service/ingress-nginx created
Additionally, I installed some routing configs by ingress.yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path:
backend:
serviceName: app0-service
servicePort: 80
- path: /app1
backend:
serviceName: app1-service
servicePort: 80
$sudo kubectl apply -f ingress.yaml
ingress.extensions/myingress created
What confuses me
Unfortunately, I´m not able to find my ingress-nginx service.
$ sudo kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
app0-service ClusterIP 10.0.28.3 <none> 80/TCP 3m48s
app1-service ClusterIP 10.0.226.249 <none> 80/TCP 3m47s
kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 39m
But my ingress is running:
$ sudo kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
myingress * 23.97.xxx.xxx 80 54m
In browser 23.97.xxx.xxx works partly.
1) If I proxy a domain name to 23.97.xxx.xxx, the domain in a browser will be rewritten by the IP.
2) If I try to browse directly to subroute like 23.97.xxx.xxx/app1/page1. I get every time the main page of app1.
I expected to get an IP from my ingress-service. Because I want to update this IP address by adding loadbalancerIP to spec in cloud-generic.yaml.
(like https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/static-ip/static-ip-svc.yaml).
Is my IP from ingress the right one to use? And why I can´t find my ingress-service?
Looking at service yaml at https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/cloud-generic.yaml you can see it's get created in namespace ingress-nginx.
You should be able to get your service by running:
kubectl get service -n ingress-nginx
You can also get all services by running kubectl get service --all-namespaces.
Related
I am following this tutorial for how to expose your local cluster for external access.
I only need to be able to check my application from browser, without exposing the app to the Internet.
> kubectl get service web
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
web NodePort 10.98.217.114 <none> 8080:32718/TCP 10m
> minikube service web --url
http://192.168.49.2:32718
Followed the guide until the etc/hosts part. I set up the ingress:
> kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
example-ingress nginx hello-world.info 192.168.49.2 80 96s
For various reasons I cannot edit the etc/hosts file on my Windows machine, it says another process is using it. However, neither 192.168.49.2 nor http://192.168.49.2:32718 in the browser returns anything, as well as curl 192.168.49.2 (and with :32718). I don't think that should be expected, as the hosts file merely forwards hello-world.info to the IP, I should be able to access my app with just the IP. What am I missing here?
Kubectl v1.24.1 (kustomize v4.5.4, server v1.23.3), Minikube v1.25.2, Windows 10, Minikube with the Docker driver.
Okay my solution to the problem was this: port-forward to the ingress-controller pod (not to the ingress itself object, because it doesn't seem to be possible)
Sample ingress file for a service named "web":
# example-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 8080
Run it with
kubectl apply -f example-ingress.yaml
Check that it's running
> kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
example-ingress nginx * 192.168.49.2 80 23h
If you ssh into minikube (minikube ssh), you can curl 192.168.49.2:80 and it returns the proper output.
Output nginx-controller pods:
> kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS
AGE
ingress-nginx-admission-create-56gbc 0/1 Completed 0
46h
ingress-nginx-admission-patch-fqf92 0/1 Completed 0
46h
ingress-nginx-controller-cc8496874-7znt5 1/1 Running 4 (39m ago)
46h
Forward port to it:
> kubectl port-forward ingress-nginx-controller-cc8496874-7znt5 8080:80 -n ingress-nginx
Then check out localhost:8080. If it returns nginx 404, then your Ingress.yaml setup is probably wrong. Otherwise works for me.
I have installed ingress controller via helm as a daemonset. I have configured the ingress as follows:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: webapp-ingress
namespace: rcc
annotations:
haproxy.org/check: 'true'
haproxy.org/check-http: /serviceCheck
haproxy.org/check-interval: 5s
haproxy.org/cookie-persistence: SERVERID
haproxy.org/forwarded-for: 'true'
haproxy.org/load-balance: leastconn
kubernetes.io/ingress.class: haproxy
spec:
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: webapp-frontend
port:
number: 8080
kubectl get ingress -n rcc
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
webapp-ingress <none> example.com 10.110.186.170 80 11h
The type chosen was loadbalancer.
I can ping from any node the ip address of the ingress on port 80 also can curl it just fine. I can also browse any of the ingress pods ip address from the node just fine. But when I browse the node ip o port 80 I get connection refused. Anything that I am missing here?
I installed last haproxy ingress which is 0.13.4 version using helm.
By default it's installed with LoadBalancer service type:
$ kubectl get svc -n ingress-haproxy
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
haproxy-ingress LoadBalancer 10.102.166.149 <pending> 80:30312/TCP,443:32524/TCP 3m45s
Since I have the same kubeadm cluster, EXTERNAL-IP will be pending. And as you correctly mentioned in question, CLUSTER-IP is accessible on the nodes when cluster is set up using kubeadm.
There are two options how to access your ingress:
Using NodePort:
From output above there's a NodePort 30312 for internally exposed port 80. Therefore from outside the cluster it should be accessed by Node_IP:NodePort:
curl NODE_IP:30312 -IH "Host: example.com"
HTTP/1.1 200 OK
Set up metallb:
Follow installation guide and second step is to configure metallb. I use layer 2. Be careful to assign not used ip range!
After I installed and set up the metallb, my haproxy has EXTERNAL-IP now:
$ kubectl get svc -n ingress-haproxy
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
haproxy-ingress LoadBalancer 10.102.166.149 172.16.1.241 80:30312/TCP,443:32524/TCP 10m
And now I can access ingress by EXTERNAL-IP on port 80:
curl 172.16.1.241 -IH "Host: example.com"
HTTP/1.1 200 OK
Useful to read:
Kubernetes service types
I am trying to create an application in Kubernetes (Minikube) and expose its service to other applications in same clusters, but i get connection refused if i try to access this service in Kubernetes node.
This application just listen on HTTP 127.0.0.1:9897 address and send response.
Below is my yaml file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: exporter-test
namespace: datenlord-monitoring
labels:
app: exporter-test
spec:
replicas: 1
selector:
matchLabels:
app: exporter-test
template:
metadata:
labels:
app: exporter-test
spec:
containers:
- name: prometheus
image: 34342/hello_world
ports:
- containerPort: 9897
---
apiVersion: v1
kind: Service
metadata:
name: exporter-test-service
namespace: datenlord-monitoring
annotations:
prometheus.io/scrape: 'true'
prometheus.io/port: '9897'
spec:
selector:
app: exporter-test
type: NodePort
ports:
- port: 8080
targetPort: 9897
nodePort: 30001
After I apply this yaml file, the pod and the service deployed correctly, and I am sure this pod works correctly, since when I login the pod by
kubectl exec -it exporter-test-* -- sh, then just run curl 127.0.0.1:9897, I can get the correct response.
Also, if I run kubectl port-forward exporter-test-* -n datenlord-monitoring 8080:9897, I can get correct response from localhost:8080. So this application should work well.
However, when I trying to access this service from other application in same K8s cluster by exporter-test-service.datenlord-monitoring.svc:30001 or just run curl nodeIp:30001 in k8s node or run curl clusterIp:8080 in k8s node, I got Connection refused
Anyone had same issue before? Appreciate for any help! Thanks!
you are mixing two things here. NodePort is the port the application is available from outside your cluster. Inside your cluster you need to access your service via the service port, not the NodePort.
Try changing exporter-test-service.datenlord-monitoring.svc:30001 to exporter-test-service.datenlord-monitoring.svc:8080
Welcome to the community!
There are no issues with behaviour you observed.
In short words kubernetes cluster (which is minikube in this case) has its own isolated network with internal DNS.
One way to access your service on the node: you specified nodePort for your service and this made the service accessible on the localhost:30001. You can check it by running on your host:
$ kubectl get svc -n datenlord-monitoring
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
exporter-test-service NodePort 10.111.191.159 <none> 8080:30001/TCP 2m45s
# Test:
curl -I localhost:30001
HTTP/1.1 200 OK
Another way to expose service to the host network is to use minikube tunnel (run in the another console). You'll need to change service type from NodePort to LoadBalancer:
$ kubectl get svc -n datenlord-monitoring
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
exporter-test-service LoadBalancer 10.111.191.159 10.111.191.159 8080:30001/TCP 18m
# Test:
$ curl -I 10.111.191.159:8080
HTTP/1.1 200 OK
Why some of options doesn't work.
Connection to the service by its DNS + NodePort. NodePort is used to link host IP and NodePort to service port inside kubernetes cluster. Internal DNS is not accessible outside kubernetes cluster (unless you don't add IPs to /etc/hosts on your host machine)
Inside the cluster you should use internal DNS with internal service port which is 8080 in your case. You can check how this works with a separate container in the same namespace (e.g. image curlimages/curl) and get following:
$ kubectl exec -it curl -n datenlord-monitoring -- curl -I exporter-test-service:8080
HTTP/1.1 200 OK
Or from the pod in a different namespace:
$ kubectl exec -it curl-default-ns -- curl -I exporter-test-service.datenlord-monitoring.svc:8080
HTTP/1.1 200 OK
I've attached useful links which help you to understand this difference.
Edit: DNS inside deployed pod
$ kubectl exec -it exporter-test-xxxxxxxx-yyyyy -n datenlord-monitoring -- bash
root#exporter-test-74cf9f94ff-fmcqp:/# cat /etc/resolv.conf
nameserver 10.96.0.10
search datenlord-monitoring.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
Useful links:
DNS for pods and services
Service types
Accessing apps in Minikube
you need to change 127.0.0.1:9897 to 0.0.0.0:9897 so that application listens to all incoming requests
It might be hard to explain so sorry if ı can not explain correctly.
In our k8s cluster we have two OpenStack-Load Balancer because we would like to expose our application through ingress which has to be internet facing. In same cluster we also deployed pgadmin4 which has to be intranet facing.(only reachable from internal network.)
So in front of these OpenStack-LB, we have also f5 Load Balancer which handle https connection,ssl .. and also logic to expose via intranet or internet.
MyApp is internet facing and needs to reachable with host.internet.net
PgAdmin4 is intranet and needs to reachable via host.intranet.net/pgadmin4
So the issue is, when I try to expose my application through ingress using host.internet.net it won't works and ı received below error cause probably it can not able to communicate with correct openStack-LB. When ı tried to expose via openStack-lb IP everything works properly.
{"level":"error","msg":"Service not found for
dev/oneapihub-ui-dev","time":"2020-03-26T05:20:05Z"}
{"level":"error","msg":"endpoints not found for
dev/oneapihub-ui-dev","time":"2020-03-26T05:20:05Z"}
And the question is , how can I handle this issue via ingress controller? Should I intall another traefik ingress controller?
capel0068340585:~ semural$ kubectl get ingress -n ingress
NAME HOSTS ADDRESS PORTS AGE
ingress-traefik-dashboard * 80 21d
kubectl get tenantSpec -o yaml
loadBalancers:
- ip: <IP1>
name: LBaaS2
ports:
- extPort: 80
name: "80"
nodePort: 30001
- ip: <IP2>
name: LBaaS1
ports:
- extPort: 80
name: "80"
nodePort: 30000
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/oneapihub-ui-dev ClusterIP 10.254.173.130 <none> 80/TCP 15m
NAME ENDPOINTS AGE
endpoints/oneapihub-ui-dev 10.6.24.136:3000 15m
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: traefik
hosts:
- host: host.internet.net -> example
paths: [/]
tls: []
ingress:
enabled: ingress
annotations:
kubernetes.io/ingress.class: traefik
hosts:
- host: host.intranet.net
paths:
- /pgadmin4
You error state "Service not found for dev/oneapihub-ui-dev", which means traefik is trying to connect to a Service in the dev namespace called "oneapihub-ui-dev" which it cannot find.
You need to make sure that both the Service exists and that it has endpoints. You can check if the Service exists with kubectl -n dev get service oneapihub-ui-dev. If it exists, check if it has endpoints with kubectl -n dev get ep oneapihub-ui-dev.
EDIT: If the Service exists and has Endpoints, than you may want to look into the RBAC permissions of traefik to see if it has enough permissions to look in the dev namespace and if you do not deploy any NetworkPolicies on the dev namespace that prevent the ingress namespace from connecting.
I solved this issue using via using labelSelector for traefik.. The the services that I'd expose only for internal networking has a label such as traffic-type=internal.. You could also provide a namespace for RBAC permissions.
kubernetes:
namespaces:
- default
- database
- monitoring
- logging
- ingress
labelSelector: "traffic-type=internal"
I'm trying to get an ingress controller working in Minikube and am following the steps in the K8s documentation here, but am seeing a different result in that the IP address for the ingress controller is different than that for Minikube (the example seems to indicate they should be the same):
$ kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
example-ingress hello-world.info 10.0.2.15 80 12m
$ minikube ip
192.168.99.101
When I try to connect to the Minikube IP address (using the address directly vs. adding it to my local hosts file), I'm getting a "Not found" response from NGINX:
$ curl http://`minikube ip`/
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>openresty/1.15.8.1</center>
</body>
</html>
When I try to connect to the IP address associated with the ingress controller, it just hangs.
Should I expect the addresses to be the same as the K8s doc indicates?
Some additional information:
$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
minikube Ready master 2d23h v1.16.0 10.0.2.15 <none> Buildroot 2018.05.3 4.15.0 docker://18.9.9
$ kubectl get ingresses example-ingress -o yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.k8s.io/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/$1"},"name":"example-ingress","namespace":"default"},"spec":{"rules":[{"host":"hello-world.info","http":{"paths":[{"backend":{"serviceName":"web","servicePort":8080},"path":"/"}]}}]}}
nginx.ingress.kubernetes.io/rewrite-target: /$1
creationTimestamp: "2019-10-28T15:36:57Z"
generation: 1
name: example-ingress
namespace: default
resourceVersion: "25609"
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/example-ingress
uid: 5e96c378-fbb1-4e8f-9738-3693cbce7d9b
spec:
rules:
- host: hello-world.info
http:
paths:
- backend:
serviceName: web
servicePort: 8080
path: /
status:
loadBalancer:
ingress:
- ip: 10.0.2.15
Here’s what worked for me:
minikube start
minikube addons enable ingress
minikube addons enable ingress-dns
Wait until you see the ingress-nginx-controller-XXXX is up and running using Kubectl get pods -n ingress-nginx
Create an ingress using the K8s example yaml file
Update the service section to point to the NodePort Service that you already created
Append
127.0.0.1 hello-world.info
to your /etc/hosts file on MacOS (NOTE:
Do NOT use the Minikube IP)
Run minikube tunnel ( Keep the window open. After you entered the password there will be no more messages, and the cursor just blinks)
Hit the hello-world.info ( or whatever host you configured in the yaml file) in a browser and it should work
I've reproduced your scenario in a Linux environment (on GCP) and I also have different IPs:
user#bf:~$ minikube ip
192.168.39.144
user#bf:~$ kubectl get ingresses
NAME HOSTS ADDRESS PORTS AGE
example-ingress * 192.168.122.173 80 30m
Your problem is not related to the fact you have different IPs. The guide instructs us to create an ingress with the following rule:
spec:
rules:
- host: hello-world.info
This rule is telling the ingress service that a DNS record with hello-world.info name is expected.
If you follow the guide a bit further, it instructs you to create an entry on your hosts file pointing to your ingress IP or Minikube IP.
Note: If you are running Minikube locally, use minikube ip to get the external IP. The IP address displayed within the ingress list
will be the internal IP.
Source: Set up Ingress on Minikube with the NGINX Ingress Controller
(if you want to curl the IP instead of DNS name, you need to remove the host rule from your ingress)
It should look like this:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: web
servicePort: 8080
Apply your changes:
user#bf:~$ kubectl apply -f example-ingress.yaml
And curl the IP using -Lk options to surpass problems related to secure connections.
user#bf:~$ curl -Lk 192.168.39.144
Hello, world!
Version: 1.0.0
Hostname: web-9bbd7b488-l5gc9
In addition to the accepted answer, minikube now has a tunnel command which allows you generate external ip addresses for your services which can be accessed directly on your host machine without using the general minikube ip.
Run minikube tunnel in a separate terminal. This runs in the foreground as a daemon.
In a different terminal, execute your kubectl apply -f <file_name> command to deploy your desired service. It should generate an ip address for you that is routed directly to your service and available on port 80 on that address.
More here on the minikube documentation: https://minikube.sigs.k8s.io/docs/tasks/loadbalancer/
I got Minikube on Windows 11 to work for me
minikube start --vm-driver=hyperv
Install minikube Ingress Controller
minikube addons enable ingress
minikube addons enable ingress-dns
Deploy Helm Chart
helm install ...
Get Kubernetes IP Address
nslookup <host-found-in-ingress> $(minikube ip)
Add to etc/host
<minikube-ip> <domain-url>
Live!
curl <domain-url>