Why does the pods hostname don't resolve? - kubernetes

Question

i have a problem with pods in kubernetes.
i have an app pod (invoice) with init container that checks if mysql pod (invoice-mysql) is running
invoice-mysql is running and ready but the init container in the invoice pod is not seeing it
here is the logs of the init container
DB is not yet reachable;sleep for 10s before retry
DB is not yet reachable;sleep for 10s before retry
nc: bad address 'invoice-mysql'
nc: bad address 'invoice-mysql'
DB is not yet reachable;sleep for 10s before retry
nc: bad address 'invoice-mysql'
DB is not yet reachable;sleep for 10s before retry
nc: bad address 'invoice-mysql'
here is the yaml of invoice
apiVersion: apps/v1
kind: Deployment
metadata:
name: invoice
namespace: jhipster
spec:
replicas: 1
selector:
matchLabels:
app: invoice
version: 'v1'
template:
metadata:
labels:
app: invoice
version: 'v1'
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- invoice
topologyKey: kubernetes.io/hostname
weight: 100
initContainers:
- name: init-ds
image: busybox:latest
command:
- '/bin/sh'
- '-c'
- |
while true
do
rt=$(nc -z -w 1 invoice-mysql 3306)
if [ $? -eq 0 ]; then
echo "DB is UP"
break
fi
echo "DB is not yet reachable;sleep for 10s before retry"
sleep 10
done
containers:
- name: invoice-app
image: docker.pkg.github.com/morsi84/kubernetes/invoice
env:
- name: SPRING_PROFILES_ACTIVE
value: prod
- name: SPRING_CLOUD_CONFIG_URI
value: http://admin:${jhipster.registry.password}#jhipster-registry.Jhipster.svc.cluster.local:8761/config
- name: JHIPSTER_REGISTRY_PASSWORD
valueFrom:
secretKeyRef:
name: registry-secret
key: registry-admin-password
- name: EUREKA_CLIENT_SERVICE_URL_DEFAULTZONE
value: http://admin:${jhipster.registry.password}#jhipster-registry.Jhipster.svc.cluster.local:8761/eureka/
- name: SPRING_DATASOURCE_URL
value: jdbc:mysql://invoice-mysql.Jhipster.svc.cluster.local:3306/invoice?useUnicode=true&characterEncoding=utf8&useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC&createDatabaseIfNotExist=true
- name: SPRING_SLEUTH_PROPAGATION_KEYS
value: 'x-request-id,x-ot-span-context'
- name: JAVA_OPTS
value: ' -Xmx256m -Xms256m'
resources:
requests:
memory: '512Mi'
cpu: '500m'
limits:
memory: '1Gi'
cpu: '1'
ports:
- name: http
containerPort: 8081
readinessProbe:
httpGet:
path: /management/health
port: http
initialDelaySeconds: 20
periodSeconds: 15
failureThreshold: 6
livenessProbe:
httpGet:
path: /management/health
port: http
initialDelaySeconds: 120
imagePullSecrets:
- name: regcred
here is the yaml of invoice-mysql
apiVersion: apps/v1
kind: Deployment
metadata:
name: invoice-mysql
namespace: jhipster
spec:
replicas: 1
selector:
matchLabels:
app: invoice-mysql
template:
metadata:
labels:
app: invoice-mysql
spec:
volumes:
- name: data
emptyDir: {}
containers:
- name: mysql
image: mysql:8.0.20
env:
- name: MYSQL_USER
value: root
- name: MYSQL_ALLOW_EMPTY_PASSWORD
value: 'yes'
- name: MYSQL_DATABASE
value: invoice
args:
- --lower_case_table_names=1
- --skip-ssl
- --character_set_server=utf8mb4
- --explicit_defaults_for_timestamp
ports:
- containerPort: 3306
volumeMounts:
- name: data
mountPath: /var/lib/mysql/
resources:
requests:
memory: '512Mi'
cpu: '500m'
limits:
memory: '1Gi'
cpu: '1'
---
apiVersion: v1
kind: Service
metadata:
name: invoice-mysql
namespace: jhipster
spec:
selector:
app: invoice-mysql
ports:
- port: 3306
here is a description of the invoice-mysql service
Name: invoice-mysql
Namespace: jhipster
Labels: <none>
Annotations: <none>
Selector: app=invoice-mysql
Type: ClusterIP
IP: 10.98.220.110
Port: <unset> 3306/TCP
TargetPort: 3306/TCP
Endpoints: 10.244.1.66:3306
Session Affinity: None
Events: <none>
working environment:
faulty environment:
Can you help me please

Answer 1

The problem in fact was due to the firwall on my centos hosts, i disabled the firewall and the names started resolving.

Answer 2

I think you should use a full service domain instead of invoice-mysql. Can you try using invoice-mysql.Jhipster.svc.cluster.local instead?