Access kubernetes-dashboard using ingess ( 404 Not Found ) - kubernetes

I'm relatively new to k8s and was following an tutorial to get familiar with it. There was a example on exposing kubernetes-dashboard via ingress and I tried to try it.
Configured kubernetes-dashboard by running following. As per its documentation.
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.1/aio/deploy/recommended.yaml
But different from the tutorial kubernetes-dashboard was exposed via port 443
service/dashboard-metrics-scraper ClusterIP 10.108.119.138 <none> 8000/TCP 50m
service/kubernetes-dashboard ClusterIP 10.100.58.17 <none> 443/TCP 50m
So I changed the ingress configuration yaml accordingly.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
name: ingress-dashboard
namespace: kubernetes-dashboard
spec:
rules:
- host: k8s-dashboard.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: kubernetes-dashboard
port:
number: 443
Then I describe the ingress and get the ip and added an entry in /etc/hosts for it
kubectl describe ingress ingress-dashboard -n kubernetes-dashboard
Name: ingress-dashboard
Labels: <none>
Namespace: kubernetes-dashboard
Address: 192.168.49.2
Ingress Class: <none>
Default backend: <default>
Rules:
Host Path Backends
---- ---- --------
k8s-dashboard.com
/ kubernetes-dashboard:443 (172.17.0.6:8443)
Annotations: kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 24m (x2 over 25m) nginx-ingress-controller Scheduled for sync
/etc/hosts change
192.168.49.2 k8s-dashbaord.com
When tried to access k8s-dashbaord.com. I get a 404 Not Found from nginx. So it seems like ingress is running but it cannot reach the service.
The ip mapped to ingress rule seems to be wrong though. (172.17.0.6:8443). Because that is not the ip of the service.
What am I doing wrong here?
P.S
If I just to a proxy ( kubectl proxy ) and access dashboard it works fine.

Related

Application not accessible using ingress but works with LoadBalancer GKE

I am trying to configure a hello world application using ingress in GKE. I have been referring a GCP official documentation to deploy an application using Ingress.
Deploying an app using ingress
But this does not work i have tried to refer several documents but none of those work. I have installed the ingress controller in my kubernetes cluster.
kubectl get svc -n ingress-nginx returns below output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
AGE
ingress-nginx-controller LoadBalancer 10.125.177.232 35.232.139.102 80:31835/TCP,443:31583/TCP 7h24m
kubectl get pods-n ingress-nginx returns
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-jj72r 0/1 Completed 0 7h24m
ingress-nginx-admission-patch-pktz6 0/1 Completed 0 7h24m
ingress-nginx-controller-5cb8d9c6dd-vptkh 1/1 Running 0 7h24m
kubectl get ingress returns below output
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-resource <none> 35.232.139.102.nip.io 34.69.2.173 80 7h48m
kubectl get pods returns below output
NAME READY STATUS RESTARTS AGE
hello-app-6d7bb985fd-x5qpn 1/1 Running 0 43m
kubect get svc returns below output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-app ClusterIP 10.125.187.239 <none> 8080/TCP 43m
Ingress resource yml file used
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-resource
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- host: 35.232.139.102.nip.io
http:
paths:
- pathType: Prefix
path: "/hello"
backend:
service:
name: hello-app
port:
number: 8080
Can someone tell me what i am doing wrong ? When i try to reach the application its not working.
So I have installed Ingress-controller and used ingress controller ip as the host in my ingress file.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-resource
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- host: "35.232.139.102.nip.io"
http:
paths:
- pathType: Prefix
path: "/hello"
backend:
service:
name: hello-app
port:
number: 8080
Issue here was I forgot to add the IP from which I was accessing the application. When you create a GKE cluster there will be a firewall with the cluster-name-all in this firewall you will need to add your IP address of the machine from which you are trying to access the application. Also ensure that the port number is also exposed in my case both were not provided hence it was failing.

ingress nginx 404 not found

Created a kubernetes cluster, and installed ingress-nginx controller. I am getting a 404 not found if i go to the ingress-nginx-controller load balancer external ip that is aa************.us-east-1.elb.amazonaws.com
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
kubectl create namespace ingress-nginx
helm install ingress-nginx -n ingress-nginx ingress-nginx/ingress-nginx
to get the service:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.100.219.162 aa************.us-east-1.elb.amazonaws.com 80:32091/TCP,443:32305/TCP 154m
ingress-nginx-controller-admission ClusterIP 10.100.208.135 <none> 443/TCP 154m
my ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: factory
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: factory.**.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: factory
port:
number: 80
- host: api.factory.**.com # myfactoryapi-factorydomain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: factory
port:
number: 8082
all my namespace
kubectl get namespace
NAME STATUS AGE
default Active 34d
ingress-nginx Active 160m
kerberos-factory Active 34d
kube-node-lease Active 34d
kube-public Active 34d
kube-system Active 34d
mongodb Active 8d
to get my ingress
kubectl get ing -n kerberos-factory
NAME CLASS HOSTS ADDRESS PORTS AGE
factory <none> factory.**.com,api.factory.**.com a****.us-east-1.elb.amazonaws.com 80 65m
to describe the ingress
kubectl describe ing -n kerberos-factory
Name: factory
Namespace: kerberos-factory
Address: a********.us-east-1.elb.amazonaws.com
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
factory.***.com
/ factory:80 (192.168.34.220:80)
api.factory.***.com
/ factory:8082 (192.168.34.220:8082)
Annotations: kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 3m8s (x3 over 4m51s) nginx-ingress-controller Scheduled for sync
why am i getting 404 not found

GKE - exposing Grafana externally not working using GCP Ingress

I've Prometheus/Grafana enabled on GKE (in namespace - monitoring)
Karans-MacBook-Pro:ingress-ns karanalang$ kc get svc -n monitoring
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
grafana ClusterIP 10.80.9.14 <none> 3000/TCP 7d4h
prometheus-operated ClusterIP None <none> 9090/TCP 7d4h
prometheus-operator ClusterIP None <none> 8080/TCP 7d4h
I'm trying to expose Grafana using Ingress, below is the Ingress description,
the path is '/grafana'
Karans-MacBook-Pro:ingress-ns karanalang$ kc describe ingress ingress-grafana -n monitoring
Name: ingress-grafana
Namespace: monitoring
Address: 34.117.119.113
Default backend: default-http-backend:80 (10.76.0.8:8080)
Rules:
Host Path Backends
---- ---- --------
*
/grafana grafana:3000 (10.76.0.5:3000)
Annotations: ingress.kubernetes.io/backends: {"k8s-be-31823--45a575f79c8f25d8":"HEALTHY","k8s1-45a575f7-monitoring-grafana-3000-2fa5518a":"HEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s2-fr-tkept4vc-monitoring-ingress-grafana-y11p7u0i
ingress.kubernetes.io/target-proxy: k8s2-tp-tkept4vc-monitoring-ingress-grafana-y11p7u0i
ingress.kubernetes.io/url-map: k8s2-um-tkept4vc-monitoring-ingress-grafana-y11p7u0i
kubernetes.io/ingress.class: gce
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 23m loadbalancer-controller UrlMap "k8s2-um-tkept4vc-monitoring-ingress-grafana-y11p7u0i" created
Normal Sync 23m loadbalancer-controller TargetProxy "k8s2-tp-tkept4vc-monitoring-ingress-grafana-y11p7u0i" created
Normal Sync 23m loadbalancer-controller ForwardingRule "k8s2-fr-tkept4vc-monitoring-ingress-grafana-y11p7u0i" created
Normal IPChanged 23m loadbalancer-controller IP is now 34.117.119.113
Normal Sync 5m12s (x7 over 23m) loadbalancer-controller Scheduled for sync
when i do a curl /grafana, it shows that login is found
However - when i use the same on the browser, it gives '404 Not Found'
Karans-MacBook-Pro:ingress-ns karanalang$ curl 34.117.119.113/grafana
Found.
what needs to be done to debug/fix this ?
tia!
Based on the description provided for your ingress-grafana Ingress resource, it is a normal behavior. You only use /grafana path in the rules. But the screenshot shows a redirect to the /login page. And since you don't have other rules, you get this message.
To solve this problem, you can change path in the ingress-grafana Ingress resource to /* as shown below:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grafana-ingress
spec:
rules:
- http:
paths:
- path: /*
pathType: ImplementationSpecific
backend:
service:
name: grafana
port:
number: 3000
This will allow to use redirections for the Grafana service.

Kubernetes Dashboard & Ingress on Docker Desktop

I am trying to access kubernetes dashboard on my local PC through Ingress. The steps I've done so far are:
Install Nginx Ingress by:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/cloud/deploy.yaml
PS D:\dev\kubernetes-dashboard-ingress> kubectl get all -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-admission-create-7rzdl 0/1 Completed 0 148m
pod/ingress-nginx-admission-patch-295pf 0/1 Completed 0 148m
pod/ingress-nginx-controller-7fc74cf778-jz6ts 1/1 Running 0 148m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller LoadBalancer 10.106.183.115 localhost 80:30673/TCP,443:32591/TCP 148m
service/ingress-nginx-controller-admission ClusterIP 10.103.188.122 <none> 443/TCP 148m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ingress-nginx-controller 1/1 1 1 148m
NAME DESIRED CURRENT READY AGE
replicaset.apps/ingress-nginx-controller-7fc74cf778 1 1 1 148m
NAME COMPLETIONS DURATION AGE
job.batch/ingress-nginx-admission-create 1/1 16s 148m
job.batch/ingress-nginx-admission-patch 1/1 16s 148m
Install kubernetes dashboard:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0/aio/deploy/recommended.yaml
When I inspect kubernetes dashboard namespace, I notice that the service is running on port 443:
PS D:\dev\kubernetes-dashboard-ingress> kubectl get service -n kubernetes-dashboard -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
dashboard-metrics-scraper ClusterIP 10.110.109.6 <none> 8000/TCP 135m k8s-app=dashboard-metrics-scraper
kubernetes-dashboard ClusterIP 10.110.230.166 <none> 443/TCP 135m k8s-app=kubernetes-dashboard
So I created Ingress rule:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dashboard-ingress
namespace: kubernetes-dashboard
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- host: "my-dashboard.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: kubernetes-dashboard
port:
number: 443
and after applying this rule:
PS D:\dev\kubernetes-dashboard-ingress> kubectl get ingress -n kubernetes-dashboard -o wide
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
dashboard-ingress <none> my-dashboard.com localhost 80 121m
I just add the following entry in my windows host file:
127.0.0.1 my-dashboard.com
However, I am getting nothing when I tried to access the dashboard through my browser (http://my-dashboard.com). Have I missed anything?
I was following the tutorial here: https://www.youtube.com/watch?v=X48VuDVv0do. The tutorial was done using minikube - so the dashboard there was available on port 80. Whereas the one i installed directly from github above was available on port 443. Do I need to configure some certificate / secret? I noticed that a few stuffs were created in the Secret by kubernetes-dashboard:
PS D:\dev\kubernetes-dashboard-ingress> kubectl get secret -n kubernetes-dashboard -o wide
NAME TYPE DATA AGE
default-token-97skl kubernetes.io/service-account-token 3 140m
kubernetes-dashboard-certs Opaque 0 140m
kubernetes-dashboard-csrf Opaque 1 140m
kubernetes-dashboard-key-holder Opaque 2 140m
kubernetes-dashboard-token-rwgs4 kubernetes.io/service-account-token 3 140m
and if i tried to describe Ingress:
PS D:\dev\kubernetes-dashboard-ingress> kubectl describe ingress dashboard-ingress -n kubernetes-dashboard
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
Name: dashboard-ingress
Namespace: kubernetes-dashboard
Address: localhost
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
my-dashboard.com
/ kubernetes-dashboard:443 (10.1.0.106:8443)
Annotations: kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/ssl-passthrough: true
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 7m4s (x10 over 144m) nginx-ingress-controller Scheduled for sync
I know I can access the dashboard using kubectl proxy - but I would like to test out Ingress (learning it). Thank you in advance!
I'm running the following:
Docker Desktop 3.2.2 (61853)
Engine: 20.10.5
Compose: 1.28.5
Kubernetes: v1.19.7
Your service name seems to be wrong:
You listed your services:
PS D:\dev\kubernetes-dashboard-ingress> kubectl get service -n kubernetes-dashboard -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
dashboard-metrics-scraper ClusterIP 10.110.109.6 <none> 8000/TCP 135m k8s-app=dashboard-metrics-scraper
kubernetes-dashboard ClusterIP 10.110.230.166 <none> 443/TCP 135m k8s-app=kubernetes-dashboard
In your ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dashboard-ingress
namespace: kubernetes-dashboard
annotations:
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- host: "my-dashboard.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: my-dashboard # <<< This line should be kubernetes-dashboard
port:
number: 443
Ok. Figured out the issue. My request (in chrome) went through the corporate proxy, and that did not forward the request further to my kubernetes cluster. After adding 'my-dashboard.com' to the no proxy list, I can access it through browser.
Thank you thomas for the pointer !

Google Kubernetes Engine Ingress doesn't work

Create ingress followed the guide of 'Kubernetes in Action' book on GKE, but the ingress doesn't work, can' be accessed from the public IP address of Ingress.
Create the replicaset to create pod.
Create Service. (followed the nodeport method on 'Kubernetes in Action').
Create ingress.
ReplicaSet, Service, Ingress are created successfully, nodeport can be accessed from the public IP address, no UNHEALTHY in ingress.
replicaset:
apiVersion: apps/v1beta2
kind: ReplicaSet
metadata:
name: kubia
spec:
replicas: 3
selector:
matchLabels:
app: kubia
template:
metadata:
labels:
app: kubia
spec:
containers:
- name: kubia
image: sonyfaye/kubia
Service:
apiVersion: v1
kind: Service
metadata:
name: kubia-nodeport
spec:
type: NodePort
ports:
- port: 80
targetPort: 8080
nodePort: 30123
selector:
app: kubia
Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: kubia
spec:
rules:
- host: kubia.example.com
http:
paths:
- path: /
backend:
serviceName: kubia-nodeport
servicePort: 80
The nodeport itself can be accessed from public IP addresses.
C:\kube>kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.59.240.1 <none> 443/TCP 8d
kubia-nodeport NodePort 10.59.253.10 <none> 80:30123/TCP 20h
C:\kube>kubectl get node
NAME STATUS ROLES AGE VERSION
gke-kubia-default-pool-08dd2133-qbz6 Ready <none> 8d v1.12.8-gke.6
gke-kubia-default-pool-183639fa-18vr Ready <none> 8d v1.12.8-gke.6
gke-kubia-default-pool-42725220-43q8 Ready <none> 8d v1.12.8-gke.6
C:\kube>kubectl get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
gke-kubia-default-pool-08dd2133-qbz6 Ready <none> 8d v1.12.8-gke.6 10.140.0.17 35.201.224.238 Container-Optimized OS from Google 4.14.119+ docker://17.3.2
gke-kubia-default-pool-183639fa-18vr Ready <none> 8d v1.12.8-gke.6 10.140.0.18 35.229.152.12 Container-Optimized OS from Google 4.14.119+ docker://17.3.2
gke-kubia-default-pool-42725220-43q8 Ready <none> 8d v1.12.8-gke.6 10.140.0.16 34.80.225.64 Container-Optimized OS from Google 4.14.119+ docker://17.3.2
C:\kube>curl http://34.80.225.64:30123
You've hit kubia-j2lnr
But the ingress can't be accessed from outside.
hosts file:
34.98.92.110 kubia.example.com
C:\kube>kubectl describe ingress
Name: kubia
Namespace: default
Address: 34.98.92.110
Default backend: default-http-backend:80 (10.56.0.7:8080)
Rules:
Host Path Backends
---- ---- --------
kubia.example.com
/ kubia-nodeport:80 (10.56.0.14:8080,10.56.1.6:8080,10.56.3.4:8080)
Annotations:
ingress.kubernetes.io/backends: {"k8s-be-30123--c4addd497b1e0a6d":"HEALTHY","k8s-be-30594--c4addd497b1e0a6d":"HEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-kubia--c4addd497b1e0a6d
ingress.kubernetes.io/target-proxy: k8s-tp-default-kubia--c4addd497b1e0a6d
ingress.kubernetes.io/url-map: k8s-um-default-kubia--c4addd497b1e0a6d
Events:
<none>
C:\kube>curl http://kubia.example.com
curl: (7) Failed to connect to kubia.example.com port 80: Timed out
C:\kube>telnet kubia.example.com 80
Connecting To kubia.example.com...
C:\kube>telnet 34.98.92.110 80
Connecting To 34.98.92.110...Could not open connection to the host, on port 80: Connect failed
Tried from intranet.
curl 34.98.92.110 IP can get some resule, and 80 port of 34.98.92.110 is accessible from intranet.
C:\kube>kubectl exec -it kubia-lrt9x bash
root#kubia-lrt9x:/# curl http://kubia.example.com
curl: (6) Could not resolve host: kubia.example.com
root#kubia-lrt9x:/# curl http://34.98.92.110
default backend - 404root#kubia-lrt9x:/# curl http://34.98.92.110
default backend - 404root#kubia-lrt9x:/#
root#kubia-lrt9x:/# curl http://10.56.0.7:8080
default backend - 404root#kubia-lrt9x:/#
Does anybody know how to debug this?
The nodeport is been added to the firewall, or else nodeport is not accessible. The Ingress IP seems don't need to be added to the firewall.
Try to expose replicaset to be able to connect from the outside:
$ kubectl expose rs hello-world --type=NodePort --name=my-service
remember to first delete service kubia-nodeport and delete selector and section with service in Ingress configuration file and then apply changes using kubectl apply command.
More information you can find here: exposing-externalip.
Useful doc: kubectl-expose.