how to get pod to pod ping on different nodes working? - kubernetes

Question

I would like to be able to ping from one pod to another. That works if the pods are on the same host. It does not work if the pods are on different hosts.
$ kubectl get pod,svc -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
pod/bbtest-5949c4d8c5-259wx 1/1 Running 1 2d 192.168.114.158 gordon-dm1.sdsc.edu <none>
pod/busybox-7cd98849ff-m75qv 0/1 Running 0 3m 192.168.78.30 gordon-dm3.sdsc.edu <none>pod/nginx-64f497f8fd-j4qml 1/1 Running 0 20m 192.168.114.163 gordon-dm1.sdsc.edu <none>
pod/nginx-64f497f8fd-tw4vb 1/1 Running 0 22m 192.168.209.32 gordon-dm4.sdsc.edu <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17d <none>
$ kubectl run busybox --rm -ti --image busybox /bin/sh
/ # ping 192.168.114.163
PING 192.168.114.163 (192.168.114.163): 56 data bytes
^C
--- 192.168.114.163 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss
/ #
I set up flannel but it doesn't make a change. I tried felixconfiguration only to get an error : resource does not exist: FelixConfiguration(default)
Any help to get pod to pod communication to work ?

Answer 1

Best practice is to use a service and open the nginx specific ports that require to receive connections and use the service hostname.
Use curl -I <service-name>.<namespace> for testing.
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
ports:
- port: 80
protocol: TCP
selector:
run: my-nginx
Result:
/ # curl -I my-nginx.default
HTTP/1.1 200 OK
Server: nginx/1.19.6
Date: Sun, 03 Jan 2021 17:44:26 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 15 Dec 2020 13:59:38 GMT
Connection: keep-alive
ETag: "5fd8c14a-264"
Accept-Ranges: bytes
P.S. I used kubectl run alpine --rm -ti --image alpine /bin/sh and apk add curl