How to make My First ingress work on baremetal NodeIP? - kubernetes

I have pod:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-app
namespace: dev
spec:
selector:
matchLabels:
app: hello
replicas: 3
template:
metadata:
labels:
app: hello
spec:
containers:
- name: hello
image: "gcr.io/google-samples/hello-app:2.0"
Make service:
---
apiVersion: v1
kind: Service
metadata:
name: hello-service
namespace: dev
labels:
app: hello
spec:
type: ClusterIP
selector:
app: hello
ports:
- port: 80
targetPort: 8080
protocol: TCP
Check it:
---
apiVersion: v1
kind: Service
metadata:
name: hello-node-service
namespace: dev
spec:
type: NodePort
selector:
app: hello
ports:
- port: 80
targetPort: 8080
$ kubectl get svc -n dev
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-node-service NodePort 10.233.3.50 <none> 80:31263/TCP 9h
hello-service ClusterIP 10.233.45.159 <none> 80/TCP 44h
$ curl -I http://cluster.local:31263
HTTP/1.1 200 OK
Date: Sat, 11 Sep 2021 07:31:28 GMT
Content-Length: 66
Content-Type: text/plain; charset=utf-8
I have verified that the service is working.
Install ingress with NodeIP (https://kubernetes.github.io/ingress-nginx/deploy/):
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/baremetal/deploy.yaml
$ kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx --watch
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-7gsft 0/1 Completed 0 10h
ingress-nginx-admission-patch-qj57b 0/1 Completed 1 10h
ingress-nginx-controller-8cf5559f8-mh6fr 1/1 Running 0 10h
$ kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.233.52.118 <none> 80:30377/TCP,443:31682/TCP 10h
ingress-nginx-controller-admission ClusterIP 10.233.51.175 <none> 443/TCP 10h
Check it:
$ curl -I http://cluster.local:30377/healthz
HTTP/1.1 200 OK
Date: Sat, 11 Sep 2021 07:39:04 GMT
Content-Type: text/html
Content-Length: 0
Connection: keep-alive
Make ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-hello
namespace: dev
spec:
rules:
- host: cluster.local
http:
paths:
- backend:
service:
name: hello-service
port:
number: 80
path: "/hello"
pathType: Prefix
Check It:
$ curl -I http://cluster.local:30377/hello
HTTP/1.1 404 Not Found
Date: Sat, 11 Sep 2021 07:40:43 GMT
Content-Type: text/html
Content-Length: 146
Connection: keep-alive
It's doesn't work. I spend few days, tried add ExternalIP to ingress controller.
Can you please tell me who had the experience of setting up ingress, what am I doing wrong?
=(((
INFO about cluster:
$ kubectl get ingress -n dev
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-hello <none> cluster.local 80 10h
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
kuber-ingress-01 Ready worker 10d v1.21.3
kuber-master1 Ready control-plane,master 10d v1.21.3
kuber-master2 Ready control-plane,master 10d v1.21.3
kuber-master3 Ready control-plane,master 10d v1.21.3
kuber-node-01 Ready worker 10d v1.21.3
kuber-node-02 Ready worker 10d v1.21.3
kuber-node-03 Ready worker 10d v1.21.3
Inventory:
kuber-master1 10.0.57.31
kuber-master2 10.0.57.32
kuber-master3 10.0.57.33
kuber-node-01 10.0.57.34
kuber-node-02 10.0.57.35
kuber-node-03 10.0.57.36
kuber-ingress-01 10.0.57.30
$ ping cluster.local
PING cluster.local (10.0.57.30) 56(84) bytes of data.
64 bytes from ingress.example.com (10.0.57.30): icmp_seq=1 ttl=62 time=0.603 ms

The solution is to add the following content to the ingress - annotation.
Then the ingress controller starts to see the DNS addresses.
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$1
Also, for convenience, changed path: / to a regular expression:
- path: /v1(/|$)(.*)

Related

ingress-nginx working but nginx-ingress not

I have Keyclock installed on my Kubernetes cluster.
Default ingress which Keycloak creates looks like this.
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
route.openshift.io/termination: passthrough
creationTimestamp: "2022-11-09T13:08:00Z"
generation: 1
labels:
app: keycloak
app.kubernetes.io/managed-by: keycloak-operator
name: keycloak-kc-ingress
namespace: default
ownerReferences:
- apiVersion: k8s.keycloak.org/v2alpha1
blockOwnerDeletion: true
controller: true
kind: Keycloak
name: keycloak-kc
uid: 67a18d00-4bee-4587-b330-cdaf21b39084
resourceVersion: "155002"
uid: 87c2aff4-1489-4ba9-bdf6-9fe1a288c800
spec:
defaultBackend:
service:
name: keycloak-kc-service
port:
number: 8443
rules:
- host: keycloak.example.com
http:
paths:
- backend:
service:
name: keycloak-kc-service
port:
number: 8443
pathType: ImplementationSpecific
status:
loadBalancer:
ingress:
- ip: 10.0.0.3
After installing ingress-nginx and adding kubernetes.io/ingress.class=nginx annotation, everything works.
For some reasons, however, I need to use nginx-ingress.
My new ingress looks like this.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
kubernetes.io/ingress.class: nginx
# nginx.ingress.kubernetes.io/backend-protocol: HTTPS
# route.openshift.io/termination: passthrough
labels:
app: keycloak
app.kubernetes.io/managed-by: keycloak-operator
# target: keycloak-kc-service
name: keycloak-kc-ingress
namespace: default
spec:
defaultBackend:
service:
name: keycloak-kc-service
port:
number: 8443
rules:
- host: accounts.example.com
http:
paths:
- backend:
service:
name: keycloak-kc-service
port:
number: 8443
path: /
pathType: Prefix
tls:
- hosts:
- accounts.example.com
secretName: keycloak-tls-secret
Unfortunately, this ingress returns the error "502 Bad Gateway".
We can't handle it. Please help.
Information for debugging
kubectl get deployments -A
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
default keycloak-operator 2/2 2 2 141m
kube-system cilium-operator 1/1 1 1 148m
kube-system coredns 2/2 2 2 148m
kube-system konnectivity-agent 2/2 2 2 148m
kube-system metrics-server 2/2 2 2 148m
kubernetes-dashboard dashboard-metrics-scraper 2/2 2 2 148m
nginx-ingress nginx-ingress-nginx-ingress-nginx-ingress 1/1 1 1 127m
olm catalog-operator 1/1 1 1 142m
olm olm-operator 1/1 1 1 142m
olm packageserver 2/2 2 2 142m
kubectl get services -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default keycloak-kc-discovery ClusterIP None <none> 7800/TCP 114m
default keycloak-kc-service ClusterIP 10.240.18.67 <none> 8443/TCP 114m
default keycloak-operator ClusterIP 10.240.24.103 <none> 80/TCP 141m
default kubernetes ClusterIP 10.240.16.1 <none> 443/TCP 149m
default postgres-db ClusterIP 10.240.18.157 <none> 5432/TCP 140m
kube-system hcloud-csi-controller-metrics ClusterIP 10.240.30.190 <none> 9189/TCP 149m
kube-system hcloud-csi-node-metrics ClusterIP 10.240.26.123 <none> 9189/TCP 149m
kube-system kube-dns ClusterIP 10.240.16.10 <none> 53/TCP,53/UDP 149m
kube-system metrics-server ClusterIP 10.240.31.184 <none> 443/TCP 149m
kubernetes-dashboard dashboard-metrics-scraper ClusterIP 10.240.25.29 <none> 8000/TCP 149m
nginx-ingress nginx-ingress-nginx-ingress-nginx-ingress LoadBalancer 10.240.26.173 10.0.0.3,167.235.123.123,2a01:4f8:1c1f:6484::1 80:31670/TCP,443:30557/TCP 128m
olm operatorhubio-catalog ClusterIP 10.240.22.30 <none> 50051/TCP 142m
olm packageserver-service ClusterIP 10.240.23.246 <none>
Unfortunately, this ingress returns the error "502 Bad Gateway".
We can't handle it. Please help.

How to create ingress-nginx for my kubernetes deployment and service?

I am able to access my django app deployment using LoadBalancer service type but I'm trying to switch to ClusterIP service type and ingress-nginx but I am getting 503 Service Temporarily Unavailable when I try to access the site via the host url. Describing the ingress also shows error: endpoints "django-service" not found and error: endpoints "default-http-backend" not found. What am I doing wrong?
This is my service and ingress yaml:
---
apiVersion: v1
kind: Service
metadata:
name: django-service
spec:
type: ClusterIP
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: django-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/force-ssl-redirect: 'true'
nginx.ingress.kubernetes.io/ssl-redirect: 'true'
spec:
tls:
- hosts:
- django.example.com
rules:
- host: django.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: django-service
port:
number: 80
ingressClassName: nginx
kubectl get all
$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/django-app-5bdd8ffff9-79xzj 1/1 Running 0 7m44s
pod/postgres-58fffbb5cc-247x9 1/1 Running 0 7m44s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/django-service ClusterIP 10.233.29.58 <none> 80/TCP 7m44s
service/pg-service ClusterIP 10.233.14.137 <none> 5432/TCP 7m44s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/django-app 1/1 1 1 7m44s
deployment.apps/postgres 1/1 1 1 7m44s
NAME DESIRED CURRENT READY AGE
replicaset.apps/django-app-5bdd8ffff9 1 1 1 7m44s
replicaset.apps/postgres-58fffbb5cc 1 1 1 7m44s
describe ingress
$ kubectl describe ing django-ingress
Name: django-ingress
Labels: <none>
Namespace: django
Address: 10.10.30.50
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
SNI routes django.example.com
Rules:
Host Path Backends
---- ---- --------
django.example.com
/ django-service:80 (<error: endpoints "django-service" not found>)
Annotations: nginx.ingress.kubernetes.io/force-ssl-redirect: true
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: true
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 5m28s (x2 over 6m5s) nginx-ingress-controller Scheduled for sync
Normal Sync 5m28s (x2 over 6m5s) nginx-ingress-controller Scheduled for sync
I think you forgot to make the link with your deployment in your service.
apiVersion: v1
kind: Service
metadata:
name: django-service
spec:
type: ClusterIP
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8000
selector:
app: your-deployment-name
Your label must be set in your deployment as well:
spec:
selector:
matchLabels:
app: your-deployment-name
template:
metadata:
labels:
app: your-deployment-name

404 Not Found error after configuring the Nginx Ingress Controller

UPDATE:
The issue persists but I used another way (sub-domain name, instead of the path) to 'bypass' the issue:
ubuntu#df1:~$ cat k8s-dashboard-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: k8s-dashboard-ingress
namespace: kubernetes-dashboard
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/secure-backends: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- dashboard.XXXX
secretName: df1-tls
rules:
- host: dashboard.XXXX
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kubernetes-dashboard
port:
number: 443
This error bothers me for some time and I hope with your help I can come down to the bottom of it.
I have one K8S cluster (single node so far, to avoid any network related issues). I installed Grafana on it.
All pods are running fine:
ubuntu:~$ k get po -A
NAMESPACE NAME READY STATUS RESTARTS AGE
default grafana-646c8874cb-h6tc5 1/1 Running 0 11h
default nginx-1-7bdc99b884-xh7kl 1/1 Running 0 36h
kube-system coredns-64897985d-4sk6l 1/1 Running 0 2d16h
kube-system coredns-64897985d-dx5h6 1/1 Running 0 2d16h
kube-system etcd-df1 1/1 Running 1 3d14h
kube-system kilo-kb52f 1/1 Running 0 2d16h
kube-system kube-apiserver-df1 1/1 Running 1 3d14h
kube-system kube-controller-manager-df1 1/1 Running 4 3d14h
kube-system kube-flannel-ds-fjkxv 1/1 Running 0 3d13h
kube-system kube-proxy-bd2xt 1/1 Running 0 3d14h
kube-system kube-scheduler-df1 1/1 Running 10 3d14h
kubernetes-dashboard dashboard-metrics-scraper-799d786dbf-5skdw 1/1 Running 0 2d16h
kubernetes-dashboard kubernetes-dashboard-6b6b86c4c5-56zp2 1/1 Running 0 2d16h
nginx-ingress nginx-ingress-5b467c7d7-qtqtq 1/1 Running 0 2d15h
As you saw, I installed nginx ingress controller.
Here is the ingress:
ubuntu:~$ k describe ing grafana
Name: grafana
Labels: app.kubernetes.io/instance=grafana
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=grafana
app.kubernetes.io/version=8.3.3
helm.sh/chart=grafana-6.20.5
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
kalepa.k8s.io
/grafana grafana:80 (10.244.0.14:3000)
Annotations: meta.helm.sh/release-name: grafana
meta.helm.sh/release-namespace: default
Events: <none>
Here is the service that is defined in above ingress:
ubuntu:~$ k get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
grafana ClusterIP 10.96.148.1 <none> 80/TCP 11h
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d14h
If I do a curl to the cluster ip of the service, it goes through without an issue:
ubuntu:~$ curl 10.96.148.1
Found.
If I do a curl to the hostname with the path to the service, I got the 404 error:
ubuntu:~$ curl kalepa.k8s.io/grafana
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.21.5</center>
</body>
</html>
The hostname is resolved to the cluster ip of the nginx ingress service (nodeport):
ubuntu:~$ grep kalepa.k8s.io /etc/hosts
10.96.241.112 kalepa.k8s.io
This is the nginx ingress service definition:
ubuntu:~$ k describe -n nginx-ingress svc nginx-ingress
Name: nginx-ingress
Namespace: nginx-ingress
Labels: <none>
Annotations: <none>
Selector: app=nginx-ingress
Type: NodePort
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.96.241.112
IPs: 10.96.241.112
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 31803/TCP
Endpoints: 10.244.0.6:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 31913/TCP
Endpoints: 10.244.0.6:443
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
What am I missing? Thanks for your help!
This is happening as you are using /grafana and this path does not exist in the grafana application - hence 404. You need to first configure grafana to use this context path before you can forward your traffic to /grafana.
If you use / as path, it will work. That's why curl 10.96.148 works as you are not adding a route /grafana. But most likely that path is already used by some other service, that's why you were using /grafana to begin with.
Therefore, you need to update your grafana.ini file to set the context root explicitly as shown below.
You may put your grafana.ini in a configmap, mount it to the original grafana.ini location and recreate the deployment.
[server]
domain = kalepa.k8s.io
root_url = http://kalepa.k8s.io/grafana/
I can see there is no ingressClassName specified for your ingress. It looks something like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-ingress
spec:
ingressClassName: nginx
tls:
- hosts:
- kalepa.k8s.io
secretName: secret_name
rules:
- host: kalepa.k8s.io
http:
paths:
...

kubernetes ingress 502 bad gateway

I installed a Kubernetes Cluster on bare metal (using VMware virtual machines) with the following nodes
master-01 Ready control-plane,master 5d3h v1.21.3
master-02 Ready control-plane,master 5d3h v1.21.3
master-03 Ready control-plane,master 5d3h v1.21.3
worker-01 Ready <none> 5d2h v1.21.3
worker-02 Ready <none> 5d2h v1.21.3
worker-03 Ready <none> 5d2h v1.21.3
Metallb is installed as loadbalancer for the cluster and calico as CNI
I also installed nginx-ingress-controller with helm
$ helm repo add nginx-stable https://helm.nginx.com/stable
$ helm repo update
$ helm install ingress-controller nginx-stable/nginx-ingress
I deployed a simple nginx server for testing
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx-app
spec:
replicas: 2
selector:
matchLabels:
app: nginx-app
template:
metadata:
labels:
app: nginx-app
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector:
app: nginx-app
#type: LoadBalancer
ports:
- protocol: TCP
port: 80
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-myapp
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myapp.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-service
port:
number: 80
My deployments with loadbalancer types get their IP from metallb and works fine but when I add ingress although an IP is assigned I get error 502 bad gateway as shown below:
firewall is enabled but required ports are opened
6443/tcp 2379-2380/tcp 10250-10252/tcp 179/tcp 7946/tcp 7946/udp 8443/tcp on master nodes
10250/tcp 30000-32767/tcp 7946/tcp 7946/udp 8443/tcp 179/tcp on worker nodes
My services and pods works fine
kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
ingress-controller-nginx-ingress LoadBalancer 10.101.17.180 10.1.210.100 80:31509/TCP,443:30004/TCP 33m app=ingress-controller-nginx-ingress
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6d <none>
nginx-service ClusterIP 10.101.48.198 <none> 80/TCP 31m app=nginx-app
My ingress logs gives me error with no route to the internal IP
2021/07/29 07:46:24 [error] 42#42: *8 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:07:46:24 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
W0729 07:50:16.416830 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] "GET / HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" "-"
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://10.1.210.100/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" "-"
W0729 07:56:43.420282 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:05:28.422594 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:10:45.425329 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
2021/07/29 08:13:59 [error] 42#42: *12 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:08:13:59 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
2021/07/29 08:14:09 [error] 42#42: *14 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:08:14:09 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
Any idea please ?
EDIT : As asked here description of services and pods
$ kubectl describe pod nginx-deployment-6f7d8d4d55-sncdr
Name: nginx-deployment-6f7d8d4d55-sncdr
Namespace: default
Priority: 0
Node: worker-01/10.1.210.63
Start Time: Thu, 29 Jul 2021 08:43:59 +0100
Labels: app=nginx-app
pod-template-hash=6f7d8d4d55
Annotations: cni.projectcalico.org/podIP: 192.168.171.17/32
cni.projectcalico.org/podIPs: 192.168.171.17/32
Status: Running
IP: 192.168.171.17
IPs:
IP: 192.168.171.17
Controlled By: ReplicaSet/nginx-deployment-6f7d8d4d55
Containers:
nginx:
Container ID: docker://fc61b73f8a833ad13b8956d8ce151b221b75a58a9a2fbae928464f3b0a77cca2
Image: nginx
Image ID: docker-pullable://nginx#sha256:8f335768880da6baf72b70c701002b45f4932acae8d574dedfddaf967fc3ac90
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Thu, 29 Jul 2021 08:44:01 +0100
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wkc48 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-wkc48:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 16m default-scheduler Successfully assigned default/nginx-deployment-6f7d8d4d55-sncdr to worker-01
Normal Pulling 16m kubelet Pulling image "nginx"
Normal Pulled 16m kubelet Successfully pulled image "nginx" in 1.51808376s
Normal Created 16m kubelet Created container nginx
Normal Started 16m kubelet Started container nginx
$ kubectl describe svc ingress-controller-nginx-ingress
Name: ingress-controller-nginx-ingress
Namespace: default
Labels: app.kubernetes.io/instance=ingress-controller
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ingress-controller-nginx-ingress
helm.sh/chart=nginx-ingress-0.10.0
Annotations: meta.helm.sh/release-name: ingress-controller
meta.helm.sh/release-namespace: default
Selector: app=ingress-controller-nginx-ingress
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.101.17.180
IPs: 10.101.17.180
LoadBalancer Ingress: 10.1.210.100
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 31509/TCP
Endpoints: 192.168.37.202:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 30004/TCP
Endpoints: 192.168.37.202:443
Session Affinity: None
External Traffic Policy: Local
HealthCheck NodePort: 31108
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal IPAllocated 18m metallb-controller Assigned IP "10.1.210.100"
Normal nodeAssigned 3m21s (x182 over 18m) metallb-speaker announcing from node "worker-02"
$ kubectl describe svc nginx-service
Name: nginx-service
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=nginx-app
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.101.48.198
IPs: 10.101.48.198
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 192.168.171.17:80
Session Affinity: None
Events: <none>
$ kubectl exec -it ingress-controller-nginx-ingress-dd5db86dc-gqdpm -- /bin/bash
nginx#ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17:80
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx#ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx#ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 10.101.48.198
curl: (7) Failed to connect to 10.101.48.198 port 80: Connection timed out
nginx#ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl nginx-deployment-6f7d8d4d55-sncdr
curl: (6) Could not resolve host: nginx-deployment-6f7d8d4d55-sncdr
nginx#ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$
To be honest I don't understand why curl svcip doesn't work anymore; yesterday it worked.
The problem was a firewall issue I disabled firewalld and it works now, I thought that had to open port 8443 but it seems to be another port if anyone can tell me which one
Thank you
I had a similar issue with a traefik ingress in k3s. I enabled masquerade in firewalld
firewall-cmd --permanent --add-masquerade && firewall-cmd --reload
Credit to this post for the idea: https://github.com/k3s-io/k3s/issues/1646#issuecomment-881191877

how to get pod to pod ping on different nodes working?

I would like to be able to ping from one pod to another. That works if the pods are on the same host. It does not work if the pods are on different hosts.
$ kubectl get pod,svc -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
pod/bbtest-5949c4d8c5-259wx 1/1 Running 1 2d 192.168.114.158 gordon-dm1.sdsc.edu <none>
pod/busybox-7cd98849ff-m75qv 0/1 Running 0 3m 192.168.78.30 gordon-dm3.sdsc.edu <none>pod/nginx-64f497f8fd-j4qml 1/1 Running 0 20m 192.168.114.163 gordon-dm1.sdsc.edu <none>
pod/nginx-64f497f8fd-tw4vb 1/1 Running 0 22m 192.168.209.32 gordon-dm4.sdsc.edu <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17d <none>
$ kubectl run busybox --rm -ti --image busybox /bin/sh
/ # ping 192.168.114.163
PING 192.168.114.163 (192.168.114.163): 56 data bytes
^C
--- 192.168.114.163 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss
/ #
I set up flannel but it doesn't make a change. I tried felixconfiguration only to get an error : resource does not exist: FelixConfiguration(default)
Any help to get pod to pod communication to work ?
Best practice is to use a service and open the nginx specific ports that require to receive connections and use the service hostname.
Use curl -I <service-name>.<namespace> for testing.
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
ports:
- port: 80
protocol: TCP
selector:
run: my-nginx
Result:
/ # curl -I my-nginx.default
HTTP/1.1 200 OK
Server: nginx/1.19.6
Date: Sun, 03 Jan 2021 17:44:26 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 15 Dec 2020 13:59:38 GMT
Connection: keep-alive
ETag: "5fd8c14a-264"
Accept-Ranges: bytes
P.S. I used kubectl run alpine --rm -ti --image alpine /bin/sh and apk add curl