Kubernetes : Cannot interconnect pod in microservice application - kubernetes

Question

I am working on a microservice application and I am unable to connect my React to my backend api pod.
The request will be internal as I am using ServerSideRendering, so when the page load first, the client pod connects directly to the backend pod. I am using ingress-nginx to connect them internally as well.
Endpoint(from React pod --> Express pod):
http://ingress-nginx-controller.ingress-nginx.svc.cluster.local
Ingress details:
$ kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.245.81.11 149.69.37.110 80:31702/TCP,443:31028/TCP 2d1h
Ingress-Config:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
rules:
- host: cultor.dev
http:
paths:
- path: /api/users/?(.*)
backend:
serviceName: auth-srv
servicePort: 3000
- path: /?(.*)
backend:
serviceName: client-srv
servicePort: 3000
Ingress log:
[error] 1230#1230: *1253654 broken header: "GET /api/users/currentuser HTTP/1.1
Also, I am unable to ping ingress-nginx-controller.ingress-nginx.svc.cluster.local from inside of client pod.
EXTRA LOGS
$ kubectl get ns
NAME STATUS AGE
default Active 2d3h
ingress-nginx Active 2d1h
kube-node-lease Active 2d3h
kube-public Active 2d3h
kube-system Active 2d3h
#####
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
auth-mongo-srv ClusterIP 10.245.155.193 <none> 27017/TCP 6h8m
auth-srv ClusterIP 10.245.1.179 <none> 3000/TCP 6h8m
client-srv ClusterIP 10.245.100.11 <none> 3000/TCP 6h8m
kubernetes ClusterIP 10.245.0.1 <none> 443/TCP 2d3h
UPDATE:
Ingress logs:
[error] 1230#1230: *1253654 broken header: "GET /api/users/currentuser HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
host: cultor.dev
x-request-id: 5cfd15996dc8481114b39a16f0be5f06
x-real-ip: 45.248.29.8
x-forwarded-for: 45.248.29.8
x-forwarded-proto: https
x-forwarded-host: cultor.dev
x-forwarded-port: 443
x-scheme: https
cache-control: max-age=0
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36
sec-fetch-site: none
sec-fetch-mode: navigate
sec-fetch-user: ?1
sec-fetch-dest: document
accept-encoding: gzip, deflate, br
accept-language: en-US,en-IN;q=0.9,en;q=0.8,la;q=0.7

Answer 1

This is a bug in using Ingress loadbalancer with Digitalocean for proxy to connect pods internally via load balancer:
Workaround:
DNS record for a custom hostname (at a provider of your choice) must be set up that points to the external IP address of the load-balancer. Afterwards, digitalocean-cloud-controller-manager must be instructed to return the custom hostname (instead of the external LB IP address) in the service ingress status field status.Hostname by specifying the hostname in the service.beta.kubernetes.io/do-loadbalancer-hostname annotation. Clients may then connect to the hostname to reach the load-balancer from inside the cluster.
Full official explaination of this bug