I am trying to expose kubernetes-dashboard app externally using Ingress resource. I have installed Nginx Controller and a service called Kubernetes-dashboard is clusterIP type service with port 443.
I have created Ingress resource with YAML file and pointing to backend service which is kubernetes-dashboard but somehow I am not getting the IP address of my host (dashboard.com) so that I can add this entry in /etc/hosts file. what is the resolution here. I am not able to paste the yaml file here as this website complain about code formatting.
I tried to put YAML file here in various ways but it does not work.
yaml file of kubernetes-dashboard as below:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: dashboard-ingress namespace: kubernetes-dashboard spec: tls: - hosts: - dashboard.com secretName: kubernetes-dashboard-certs rules: - host: dashboard.com http: paths: - pathType: ImplementationSpecific path: / backend: service: name: kubernetes-dashboard port: number: 443
Kubernetes-dashboard service config as below: Name: kubernetes-dashboard Namespace: kubernetes-dashboard Labels: k8s-app=kubernetes-dashboard Annotations: Selector: k8s-app=kubernetes-dashboard Type: ClusterIP IP Family Policy: SingleStack IP Families: IPv4 IP: 10.106.1.186 IPs: 10.106.1.186 Port: 443/TCP TargetPort: 8443/TCP Endpoints: 10.44.0.3:8443 Session Affinity: None Events:
I am not getting the IP address of my host
You have to use the Nginx ingress controller service IP everywhere so traffic gets forwarded and managed via Nginx ingress.
you can check the IP of Nginx controller using the
Kubectl get svc -n ingress-nginx
Nginx controller service will be exposed as the type LoadBalancer you can use this IP into the DNS route as A or CNAME record.
Any request coming to your domain will get forwarded to
ingress > Nginx ingress controller > K8s service > K8s PODs
Related
I am trying to access the Kubernetes Dashboard through an Istio Gateway + Virtual Service.
However, all I get is 404 page not found when I try to access the dashboard with browser. Accessing the Dashboard through k8s NodePort or k8s LoadBalancer service works just as expected. The pod, however, complains in the logs about http: TLS handshake error from 127.0.0.6:52483: remote error: tls: bad certificate.
Running httpbin through Istio (as given in their documentation) works as expected, so Istio seem to be working fine as well.
I am using the official Kubernetes Dashboard YAML-s. I am giving the service below (with type: LoadBalancer added, although it doesn't seem to make a difference for Istio, although it allows me to access the Dashboard through a separate IP).
Just for the record, my k8s cluster is comprised of VirtualBox machines running MetalLB.
kubectl get services --all-namespaces returns the following:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 11d
httpbin httpbin ClusterIP 10.100.186.188 <none> 8000/TCP 47h
istio-system istio-egressgateway ClusterIP 10.109.231.163 <none> 80/TCP,443/TCP 5d3h
istio-system istio-ingressgateway LoadBalancer 10.111.188.94 192.168.56.46 15021:31440/TCP,80:31647/TCP,443:32715/TCP 5d3h
istio-system istiod ClusterIP 10.104.236.247 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 5d3h
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 11d
kubernetes-dashboard dashboard-metrics-scraper ClusterIP 10.101.131.136 <none> 8000/TCP 43h
kubernetes-dashboard kubernetes-dashboard-service LoadBalancer 10.103.130.244 192.168.56.47 443:30041/TCP 43h
kubernetes-dashboard kubernetes-dashboard-service-np NodePort 10.100.49.224 <none> 8443:30002/TCP 43h
If I try to access the LoadBalancer directly via the IP from above and through browser, I get the usual Kubernetes Dashboard login page. The browser url is https://192.168.56.47.
YAML-s:
istio-gateway.yaml:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: kubernetes-dashboard-gateway
namespace: kubernetes-dashboard
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: PASSTHROUGH
hosts:
- "*"
istio-virtual-service.yaml:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kubernetes-dashboard-virtual-service
namespace: kubernetes-dashboard
spec:
hosts:
- "*"
gateways:
- kubernetes-dashboard-gateway
tls:
- match:
- sniHosts: ["*"]
route:
- destination:
host: kubernetes-dashboard-service
port:
number: 443
dashboard-service.yaml:
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-service
namespace: kubernetes-dashboard
spec:
ports:
- port: 443
targetPort: 8443
# - port: 8000
# targetPort: 9090
selector:
k8s-app: kubernetes-dashboard
type: LoadBalancer
User suren has mentioned:
your gateway is listening 443. not 80
Yes, this could be a problem. You are trying to reach port 80, but you are exposing only port 443. Try to change your configuration or change your port during request.
See albo documentation about Deploy and Access the Kubernetes Dashboard.
Hm, I got it working with the configuration as above and with explicitly specifying a host in all places where I have previously placed a "*". I had to add that host in /etc/hosts to be able to access it in browser.
It seems that this last part was key, as well as specifying the sniHost in the Virtual Service. The other problems were mostly configuration issues with the TLS. Setting it to PASSTHROUGH seems to work, because it forces Istio to sort of forward the HTTPS request to the Kubernetes Dashboard, which is responsible for decrypting etc.
Hope you are all well,
I am currently trying to rollout the awx-operator on to a Kubernetes Cluster and I am running into a few issues with going to the service from outside of the cluster.
Currently I have the following services set up:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
awx NodePort 10.102.30.6 <none> 8080:32155/TCP 110m
awx-operator NodePort 10.110.147.152 <none> 80:31867/TCP 125m
awx-operator-metrics ClusterIP 10.105.190.155 <none> 8383/TCP,8686/TCP 3h17m
awx-postgres ClusterIP None <none> 5432/TCP 3h16m
awx-service ClusterIP 10.102.86.14 <none> 80/TCP 121m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17h
I did set up a NodePort which is called awx-operator. I did attempt to create an ingress to the application. You can see that below:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: awx-ingress
spec:
rules:
- host: awx.mycompany.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: awx
port:
number: 80
When I create the ingress, and then run kubectl describe ingress, I get the following output:
Name: awx-ingress
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
awx.mycompany.com
/ awx:80 (10.244.1.8:8080)
Annotations: <none>
Events: <none>
Now I am not too sure whether the default-http-backend:80 error is a red-herring as I have seen this in a number of places and they don't seem too worried about it, but please correct me if I am wrong.
Please let me know whether there is anything else I can do to troubleshoot this, and I will get back to you as soon as I can.
You are right and the blank address is the issue here. In traditional cloud environments, where network load balancers are available on-demand, a single Kubernetes manifest suffices to provide a single point of contact to the NGINX Ingress controller to external clients and, indirectly, to any application running inside the cluster.
Bare-metal environments on the other hand lack this option, requiring from you a slightly different setup to offer the same kind of access to external consumers:
This means you have to do some additional gymnastics to make the ingress work. And you have basically two main options here (all well described here):
A pure software solution: MetalLB
Over the NodePort service.
What is happening here is that you basically creating a service type NodePort with selector that matches your ingress controller pod and then it's routes the traffic accordingly to your ingress object:
# Source: ingress-nginx/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
annotations:
labels:
helm.sh/chart: ingress-nginx-3.30.0
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 0.46.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
Full nginx deployment that conatains that service can be found here.
If you wish to skip the ingress you might be just using the nodePort service awx and reach it directly.
I am using Kubernetes 1.22 and the operator version 0.14.0.
I have a Kubernetes baremetal installation and I have to use ingress. The ingress provided with the operator is not compatible with the version of kubernetes I am using so I had to define it myself.
I am using Ansible but you could work out the values for the variables :)
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ awx_deployment_name }}-ingress-unmanaged
namespace: {{ awx_namespace }}
annotations:
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
ingressClassName: nginx
rules:
- host: {{ awx_host }}
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: {{ awx_deployment_name }}-service
port:
number: 80
tls:
- hosts:
- {{ awx_host }}
secretName: {{ awx_tls_secret}}
you can simply expose the deployment to a service type LoadBalancer
the following command creates a service with a type loadBalancer
kubectl expose deployment awx-demo --port=80 --target-port=8052 --name=awx-lb --type=LoadBalancer
Cluster information:
Installation Method: kubeadm
Kubernetes: 1.19.2
Master & Nodes: Ubuntu 20.04.1 (Oracle Virutalbox)
Docker: 19.03.12
Calico: 3.16.1
Ingress : Bare-metal - 0.40.1
I want to access the Kubernetes dashboard from my laptop using ingress without proxy?
Can anyone help me with the steps... ( I tried multiple ways with the help of the internet... not sure where I am missing?)
Note: As per discussion forums I have added "hostNetwork: true" under the deployment section in ingress YAML to resolve "not working without host parameter" and commented "type: NodePort".
Updated info:
I have created ingress-controller as daemon instead of deployments/pod - this helps in accessing directly with worker IPs. (this is what I am expecting - but unable to access kubernetes dashboard as it is in different namespace)
Ingress yaml: this is running in default namespace
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kdash-in-ns
port:
number: 443
kdash-in-ns yaml - svc with External Name
kind: Service
apiVersion: v1
metadata:
name: kdash-in-ns
namespace: default
spec:
type: ExternalName
externalName: kubernetes-dashboard.kubernetes-dashboard.svc.cluster.local
ports:
- name: https
port: 443
Below details about kdash-in-ns svc with ExternalName
dockeras#ubuntu3:~/simplek8s/kubernetes/yamls/ingress-demo$ kubectl describe svc kdash-in-ns
Name: kdash-in-ns
Namespace: default
Labels: <none>
Annotations: <none>
Selector: <none>
Type: ExternalName
IP:
External Name: kubernetes-dashboard.kubernetes-dashboard.svc.cluster.local
Port: https 443/TCP
TargetPort: 443/TCP
Endpoints: <none>
Session Affinity: None
Events: <none>
kubectl describe for the updated ingress route: in this i have ngnix - which is working fine (i guess both ingress and nginx are in same namespace.. getting error for dashboard - as it in different namespace (kubernetes-dasbhoard))
dockeras#ubuntu3:~$ kubectl describe ing nginx-ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
Name: nginx-ingress
Namespace: default
Address: 192.168.1.31,192.168.1.32
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
*
/nginx nginx-deploy-main:80 )
/foo kubernetes-dashboard:443 (<error: endpoints "kubernetes-dashboard" not found>)
/dashboard kdash-in-ns:443 (<error: endpoints "kdash-in-ns" not found>)
Annotations: kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 4m40s nginx-ingress-controller Ingress default/nginx-ingress
When I tried the same URLs in browser below are the responses (One of my worker iP - 192.168.1.31)
192.168.1.31/nginx - responds with nginx default page (pod - nginx-deploy-main)
192.168.1.31/foo - error page - 503 service temporarily Unavailable (default nginx)
192.168.1.31/dashboard - 504 Gateway Time-out (default nginx)
running svc, pods:
All Pods and svcs
If I understand correctly, you want to access kubernetes service (dashboard) from outside cluster. You may deploy metallb LoadBalancer and manage a pull of IPs from external cluster network, assigned to your cluster.
So, you can assign an IP and a LoadBalancer through which you will access your service. Below is an example for mssql server, but you can easily adapt it to your needs with dashboard:
apiVersion: v1
kind: Service
metadata:
name: sql-server-lb
namespace: database-server
annotations:
metallb.universe.tf/address-pool: default
spec:
selector:
app: sql-server
ports:
- port: 1433
targetPort: 1433
type: LoadBalancer
https://metallb.universe.tf/
I have a service object of type LoadBalancer:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default user-api LoadBalancer 10.106.120.69 <pending> 9411:31622/TCP,3000:31878/TCP,4000:30202/TCP,3001:31656/TCP,4001:31455/TCP 96m
I can reach to the service internally with 3000 port via internal_ip:31878.
Supposing I have a public_ip and ingress, how should I correctly configure it to let the internal service to reach to the internet?
NAME HOSTS ADDRESS PORTS AGE
app-ingress user-api.my-domain.com 80 95m
again, I'm guessing that this was a cluster setup by kubeadm because of your other questions and again the solution is to install an external load-balancer plugin such as metalLB. After that your loadbalancers will get IPs assigned automatically and all ports of your services/ingress will be open on this IP and will lead to your pods.
Installing metalLB is very easy:
kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.8.3/manifests/metallb.yaml
create and apply config of IP pools to be used for your load-balancers similarly to this one:
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 92.18.1.200-92.18.1.250
If you have a service object of type LoadBalancer, you should already be able to reach to the service from the Internet. If you have a Load Balancer that is attached to an Ingress Controller, you should change the type of your service object (to something like ClusterIP) and register the service in the Ingress Controller manifest. Example is:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /<context-path>
backend:
serviceName: <service-name>
servicePort: <service-port>
I've deployed a series of deployments and services to a Kubernetes cluster with a load balancer. When I try to access my app this does not work as my application is exposed on port 80 but the URL is always redirected to port 443 (HTTPS). I suspect this is to do with the fact that the cluster IP is on port 443.
Any ideas on how I can fix this?
db NodePort 10.245.175.203 <none> 5432:30029/TCP 25m
kubernetes ClusterIP 10.245.0.1 <none> 443/TCP 8m
redis NodePort 10.245.197.157 <none> 6379:31277/TCP 25m
web LoadBalancer 10.245.126.122 123.12.123.123 80:31430/TCP 25m
This is likely due to your application itself redirecting to port 443. What type of application is it?
This service exposed on port 443 has nothing to do with your application:
kubernetes ClusterIP 10.245.0.1 <none> 443/TCP 8m
It's basically an internal service that allows you to access the kube-apiserver within your cluster.
You could try just setting up the LoadBalancer to listen on port 443 directly. Only you would have to port 80 traffic wouldn't work. If you want the port 80 redirects to work I suggest you use an Ingress controller like nginx. Something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: your-ingress
annotations:
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
tls:
- hosts:
- yourhostname.com
secretName: tls-secret
rules:
- host: yourhostname.com
http:
paths:
- path: /
backend:
serviceName: web
servicePort: 443
You will also have to create a TLS secret holding your cert and key:
$ kubectl create secret tls tls-secret --key /tmp/tls.key --cert /tmp/tls.crt