kubernetes: can't ping pods - kubernetes

Question

I created a k8s cluster which network configuration to pod is podSubnet: 172.168.0.0/12
Then, I find that can't ping those pod's IP.
for example, the deployment of metrics
# on k8s-master01 node:
$ kubectl get po -n kube-system -o wide
metrics-server-545b8b99c6-r2ql5 1/1 Running 0 5d1h 172.171.14.193 k8s-node02 <none> <none>
# ping 172.171.14.193 -c 2
PING 172.171.14.193 (172.171.14.193) 56(84) bytes of data.
^C
--- 172.171.14.193 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1016ms
# this is route table
# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.180.104.1 0.0.0.0 UG 0 0 0 eth0
10.180.104.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.161.125.0 10.180.104.110 255.255.255.192 UG 0 0 0 tunl0
172.162.195.0 10.180.104.109 255.255.255.192 UG 0 0 0 tunl0
172.169.92.64 10.180.104.108 255.255.255.192 UG 0 0 0 tunl0
172.169.244.192 0.0.0.0 255.255.255.255 UH 0 0 0 cali06e1673851f
172.169.244.192 0.0.0.0 255.255.255.192 U 0 0 0 *
172.171.14.192 10.180.104.111 255.255.255.192 UG 0 0 0 tunl0
that shows metric pod host on k8s-node02. This is k8s-node02's route table
# route -n
Kernel IP routing table of k8s-master01
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.180.104.1 0.0.0.0 UG 0 0 0 eth0
10.180.104.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.169.254 10.180.104.11 255.255.255.255 UGH 0 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.161.125.0 10.180.104.110 255.255.255.192 UG 0 0 0 tunl0
172.162.195.0 10.180.104.109 255.255.255.192 UG 0 0 0 tunl0
172.169.92.64 10.180.104.108 255.255.255.192 UG 0 0 0 tunl0
172.169.244.192 10.180.104.107 255.255.255.192 UG 0 0 0 tunl0
172.171.14.192 0.0.0.0 255.255.255.192 U 0 0 0 *
172.171.14.193 0.0.0.0 255.255.255.255 UH 0 0 0 cali872eed170f4
172.171.14.194 0.0.0.0 255.255.255.255 UH 0 0 0 cali7d7625dd37e
172.171.14.203 0.0.0.0 255.255.255.255 UH 0 0 0 calid4e258f95f6
172.171.14.204 0.0.0.0 255.255.255.255 UH 0 0 0 cali5cf96eb1028
in fact, all pods can't access. I created a service based on example deployment.
# kubectl describe svc my-service
Name: my-service
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=demo-nginx
Type: ClusterIP
IP Families: <none>
IP: 10.100.75.139
IPs: 10.100.75.139
Port: http 80/TCP
TargetPort: 80/TCP
Endpoints: 172.161.125.14:80,172.161.125.15:80,172.171.14.203:80
Session Affinity: None
Events: <none>
# ping 10.100.75.139 -c 1
PING 10.100.75.139 (10.100.75.139) 56(84) bytes of data.
64 bytes from 10.100.75.139: icmp_seq=1 ttl=64 time=0.077 ms
# nc -vz 10.100.75.139 80
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connection timed out.
I suppose that the root cause is route table but not sure. Would you please help to fix this issue??
please feel free to let me know if you need more information.
Thanks a lot in advance.
BR//

Answer 1

In connect mode, Ncat initiates a connection (or sends UDP data) to a service that is listening somewhere. For those familiar with socket programming, connect mode is like using the connect function. In listen mode, Ncat waits for an incoming connection (or data receipt), like using the bind and listen to functions.
A connection timed out response indicates that your connection is not working, which could mean your firewall is blocking the port. Test the connection status by adding a rule that accepts connections on the required port.
There might be issues with traffic rules, try adding outbound traffic rules in security groups if you face the error again. Even after adding the traffic rules, if you face the same error then there must be some restricting rules on the node which is blocking you from accessing the port.
Refer to Testing Network services for more information.