In my cluster I have a nodejs application pod and a redis pod, and I am trying to connect to redis from nodejs, but I am getting the following error:
[ioredis] Unhandled error event: Error: getaddrinfo ENOTFOUND redis at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:60:26)
It is worth noting that in my nodejs app I am pointing at redis:6379 and if I change the pointer to localhost:6379, I will get a ECONNREFUSED error.
My redis deployment yaml looks like this:
apiVersion: v1
kind: Pod
metadata:
name: redis
spec:
containers:
- name: redis
image: redis:5.0.4
command:
- redis-server
- '/redis-master/redis.conf'
env:
- name: MASTER
value: 'true'
ports:
- containerPort: 6379
resources:
limits:
cpu: '0.1'
volumeMounts:
- mountPath: /redis-master-data
name: data
- mountPath: /redis-master
name: config
volumes:
- name: data
emptyDir: {}
- name: config
configMap:
name: redis-config
items:
- key: redis-config
path: redis.conf
Mu redis service yaml is the following:
apiVersion: v1
kind: Service
metadata:
name: redis-master
labels:
app: redis
role: master
tier: backend
spec:
ports:
- name: redis
port: 6379
targetPort: 6379
selector:
app: redis
role: master
tier: backend
And the service of my app looks like this:
apiVersion: v1
kind: Service
metadata:
name: bff
annotations:
prometheus.io/scrape: 'true'
prometheus.io/port: '8080'
labels:
app: bff
spec:
ports:
- name: external
port: 80
targetPort: web
protocol: TCP
- name: metrics
port: 8080
targetPort: metrics
protocol: TCP
selector:
app: bff
I have tried following other answers given to similar questions, but they do not seem to work in my case.
The root cause is your service selector definition doesn't match your pods's lables.
Quote from Kubernetes service document:
The controller for the Service selector continuously scans for Pods that match its selector, and then POSTs any updates to an Endpoint object also named “my-service”.
See this link for full document.
In short, by following this Kubernetes official example, you should deploy the deployment "redis-master-deployment.yaml" instead of bare pod.
Related
I would like to access my pods created by my backend via subdomain, i already setup a pod yaml and a service yaml
I followed the official k8s docs but its not working, i just get connection refused when i visit http://playout-6297bbdceab3f039170509ee.default-subdomain.default.svc.domain.com
Here are both yaml files:
Service
apiVersion: v1
kind: Service
metadata:
name: default-subdomain
spec:
selector:
name: playout
clusterIP: None
ports:
- name: http
port: 80
targetPort: 7999
Pod
apiVersion: v1
kind: Pod
metadata:
name: playout-6297bbd9eab3f039170509e8
namespace: default
labels:
name: playout
spec:
hostname: playout-6297bbd9eab3f039170509e8
subdomain: default-subdomain
containers:
- image: eli4n/playout:playout-latest
ports:
- containerPort: 7999
env:
- name: PLAYOUT_STATION
value: 6297bbceeab3f039170509df
- name: PLAYOUT_CHANNEL
value: 6297bbd9eab3f039170509e8
name: playout
imagePullSecrets:
- name: regcred`
I use k8s with rancher via hetzner cloud driver
Btw i'm really new to k8s
Thanks!
I've been using Google Cloud Platform to deploy my Magento
application with Google Kubernetes Engine.
It used to work but now I got that error. Here's the architecture:
and this are the services:
dial tcp IP:80: connect: operation timed out
I'm trying to expose my nginx pod using load balancer.
My nginx pod configuration file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
tier: backend
spec:
replicas: 1
selector:
matchLabels:
app: nginx
tier: backend
template:
metadata:
labels:
app: nginx
tier: backend
spec:
volumes:
- name: dir
persistentVolumeClaim:
claimName: dir
- name: config
configMap:
name: nginx-config
items:
- key: config
path: site.conf
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
volumeMounts:
- name: dir
mountPath: /var/www/
- name: config
mountPath: /etc/nginx/conf.d
and to expose it I run this command :
kubectl expose deployment nginx --port=80 --type=LoadBalancer
Is there anyone familiar with this error ?
Learning Kubernetes by setting up two pods, each running an elastic-search and a kibana container respectively.
My configuration file is able to setup both pods as well as create two services to access these applications on host machine's web browser.
Issue is that i don't know how to make Kibana container communicate with ES application/pod.
Earlier while learning Docker i crafted a docker-compose app configuration and now basically trying to do the same using Kubernetes ( docker-compose config pasted below ) .
Came across a blog that suggested using Deployment instead of Pod. Again not sure how would one make Kibana talk to ES
Kubernetes configuation yaml:
apiVersion: v1
kind: Pod
metadata:
name: pod-elasticsearch
labels:
app: myapp
spec:
hostname: "es01-docker-local"
containers:
- name: myelasticsearch-container
image: myelasticsearch
imagePullPolicy: Never
volumeMounts:
- name: my-volume
mountPath: /home/newuser
volumes:
- name: my-volume
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: myelasticsearch-service
spec:
type: NodePort
ports:
- targetPort: 9200
port: 9200
nodePort: 30015
selector:
app: myapp
---
apiVersion: v1
kind: Pod
metadata:
name: pod-kibana
labels:
app: myapp
spec:
containers:
- name: mykibana-container
image: mykibana
imagePullPolicy: Never
volumeMounts:
- name: my-volume
mountPath: /home/newuser
volumes:
- name: my-volume
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: mykibana-service
spec:
type: NodePort
ports:
- targetPort: 5601
port: 5601
nodePort: 30016
selector:
app: myapp
For reference below is the docker-compose that i am trying to replicate on Kubernetes
version: "2.2"
services:
elasticsearch:
image: myelasticsearch
container_name: myelasticsearch-container
restart: always
hostname: 'es01.docker.local'
ports:
- '9200:9200'
- '9300:9300'
volumes:
- myVolume:/home/newuser/
environment:
- discovery.type=single-node
kibana:
depends_on:
- elasticsearch
image: mykibana
container_name: mykibana-container
restart: always
ports:
- '5601:5601'
volumes:
- myVolume:/home/newuser/
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: http://es01:9200
volumes:
myVolume:
networks:
myNetwork:
ES Pod description:
% kubectl describe pod/pod-elasticsearch
Name: pod-elasticsearch
Namespace: default
Priority: 0
Node: docker-desktop/192.168.65.3
Start Time: Sun, 10 Jan 2021 23:06:18 -0800
Labels: app=myapp
Annotations: <none>
Status: Running
IP: 10.x.0.yy
IPs:
IP: 10.x.0.yy
In kubernetes Pod/Deployment/DaemonSet... in the same cluster can communicate with each other with no problem because it has a flat network architecture .One way for these resources to call each other directly is by the name of Kubernetes service of each resource.
For example any resource in the cluster can call your kibana-app directly by service name you give it to it mykibana-service.name-of-namespace.
So for kibana pod to communicate with elasticsearch it can use http://name-of-service-of-elasticsearch.name-of-namespace:9200 namespace is be default if you dont specify where you create your service => http://name-of-service-of-elasticsearch.default:9200 or http://name-of-service-of-elasticsearch:9200
The concern you raised on what type of your resource you have to create (pod, deployment,daemonset or statefulSet) is not important for these resources to communicate with each other.
If you re having problem converting docker-compose to manifest file you can start with Kompose you can do kompose convert where is your docker-compose is located .
Here sample
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: elasticsearch
name: elasticsearch
namespace: default
spec:
selector:
matchLabels:
app: elasticsearch
template:
metadata:
labels:
app: elasticsearch
spec:
containers:
- image: myelasticsearch:yourtag #fix this
name: elasticsearch
ports:
- containerPort: 9200
- containerPort: 9300
volumeMounts:
- mountPath: /home/newuser/
name: my-volume
volumes:
- name: my-volume
emptyDir: {} # I wouldnt use emptydir
---
apiVersion: v1
kind: Service
metadata:
labels:
app: elasticsearch
name: elasticsearch
namespace: default
spec:
ports:
- port: 9200
name: "9200"
targetPort: 9200
- port: 9300
name: "9300"
targetPort: 9300
selector:
app: elasticsearch
type: ClusterIP #you dont need to make expose your service publicly
#####################################
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: kibana
name: kibana
namespace: default
spec:
selector:
matchLabels:
app: kibana
template:
metadata:
labels:
app: kibana
spec:
containers:
- env:
- name: ELASTICSEARCH_URL
value: http://elasticsearch:9200/ #elasticsearch is the same name as service resrouce name
- name: ELASTICSEARCH_HOSTS
value: http://elasticsearch:9200
image: mykibana:yourtagname #fix this
name: kibana
---
apiVersion: v1
kind: Service
metadata:
labels:
app: kibana
name: kibana
namespace: default
spec:
ports:
- port: 5601
protocol: TCP
targetPort: 5601
selector:
app: kibana
type: NodePort
You can choose whats adequate for your app , for example in elasticsearch you can use StatefulSet ,Deployment, in ElasticSearch, and you can you use Deployment for Kibana , Also you can change the type of volume .
Also the mynetwork that you created in docker-compose can be translated network policy where you can isolate your resources (for example isolated mynetwork namespace) because these resources are not isolated if they are created in the same cluster by default.
Hope I helped
If you want to deploy Elasticsearch and Kibana in Kubernetes the usual way then you have to take care of some core Elasticsearch cluster configuration like:
cluster.initial_master_nodes [7.0] Added in 7.0.
network.host
network.publish_host
Also you would have to carefully setup the network.host so that even after accidental pod restarts the network.host remains the same.
While deploying Kibana you need provide Elasticsearch service and also manually configure the SSL certificates if Elasticsearch has SSL enabled.
So to install Elastic Stack on Kubernetes then you should probably prefer
Elastic Cloud on Kubernetes (ECK). The documentation provided by Elastic is easy to understand.
Elastic Cloud on Kubernetes (ECK) uses Kubernetes Operators to make installation easier and it automatically takes care of core cluster configuration.
ECK installation will create a default user called "elastic" and you can retrieve its password from secrets. It also creates self-signed certificates which can be found in secrets.
For deploying Kibana you can just provide "elasticsearchRef" in your YAML file and it will automatically configure the Elasticsearch endpoints. You can use the default "elastic" user to login to Kibana.
I have a sample application running on a Kubernetes cluster. Two microservices, one is a mongodb container and the other is a java springboot container.
The springboot container interacts with the mongodb container thro a service and stores data into the mongodb container.
The specs are provided below.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: empappdepl
labels:
name: empapp
spec:
replicas: 1
template:
metadata:
labels:
name: empapp
spec:
containers:
-
resources:
limits:
cpu: 0.5
image: 11.168.xx.xx:5000/employee:latest
imagePullPolicy: IfNotPresent
name: wsemp
ports:
- containerPort: 8080
name: wsemp
command: ["java","-Dspring.data.mongodb.uri=mongodb://mongoservice/microservices", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
imagePullSecrets:
- name: myregistrykey
---
apiVersion: v1
kind: Service
metadata:
labels:
name: empwhatever
name: empservice
spec:
ports:
- port: 8080
nodePort: 30062
type: NodePort
selector:
name: empapp
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: mongodbdepl
labels:
name: mongodb
spec:
replicas: 1
template:
metadata:
labels:
name: mongodb
spec:
containers:
- resources:
limits:
cpu: 1
image: mongo
imagePullPolicy: IfNotPresent
name: mongodb
ports:
- containerPort: 27017
---
apiVersion: v1
kind: Service
metadata:
labels:
name: mongowhatever
name: mongoservice
spec:
ports:
- port: 27017
targetPort: 27017
protocol: TCP
type: NodePort
selector:
name: mongodb
I would like to know how this communication can be accomplished in spinnaker since it creates its own labels and selectors.
Thanks,
This is how it needs to be done.
Each loadbalancer created for the application is the service. So for mongodb application, after a loadbalancer is created with the nodeport settings, get the name of the service eg: mongodb-dev. The server group for mongodb also needs to be created.
Then when creating the employee server group, you need to specify the commands one by one in a separate line for that container as mentioned here
https://github.com/spinnaker/spinnaker/issues/2021#issuecomment-334885467
"java","-Dspring.data.mongodb.uri=mongodb://name-of-mongodb-service/microservices", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"
Now when the employee and mongodb pod starts, it is able to get its mapping and able to communicate properly.
I would like to connect my Kubernetes cluster to Google Cloud SQL.
I have at least 10 different deployed pods which presently connect to MySQL [docker image deployed to k8s] using a JDBC url + username/password.
It it possible to use a single instance of the Google Cloud SQL Proxy and connect all the pods through this proxy to the Cloud SQL database? Ideally I would like to replace the mysql running in the container with the proxy.
I would prefer not having to run the proxy inside each deployment. The only samples I found seem to indicate the proxy needs to be declared in each deployment.
I found a solution.
Deploy the proxy with the yml below, and expose the deployment as a service. Most importantly, make the proxy listen on 0.0.0.0, instead of default 127.0.0.1. All the secrets as per the Google Cloud sql documentation
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: mysql
spec:
replicas: 1
template:
metadata:
name: mysql
labels:
name: mysql
spec:
containers:
- image: b.gcr.io/cloudsql-docker/gce-proxy:1.05
name: cloudsql-proxy
command: ["/cloud_sql_proxy", "--dir=/cloudsql",
"-instances=MYSQL:ZONE:DATABASE_INSTANCE=tcp:0.0.0.0:3306",
"-credential_file=/secrets/cloudsql/credentials.json"]
volumeMounts:
- name: cloudsql-oauth-credentials
mountPath: /secrets/cloudsql
readOnly: true
- name: ssl-certs
mountPath: /etc/ssl/certs
ports:
- containerPort: 3306
name: mysql
volumes:
- name: cloudsql-oauth-credentials
secret:
secretName: cloudsql-oauth-credentials
- name: ssl-certs
hostPath:
path: /etc/ssl/certs
The solution is slightly more expensive than having the proxy in the same deployment as the client software, since there is an extra TCP connection.
However there are many benefits:
Much simpler and doesn't require modifying existing K8S deployment files
Allows switching the implementation to a MySQL Docker container or using the Google Cloud SQL proxy without any modifications to the client configuration.
You can create a deployment and a service to expose the cloudsql proxy to other pods like so:
apiVersion: v1
kind: Service
metadata:
name: cloudsqlproxy
spec:
ports:
- port: 3306
targetPort: database-port
selector:
app: cloudsqlproxy
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: cloudsqlproxy
spec:
template:
metadata:
labels:
app: cloudsqlproxy
spec:
volumes:
- name: service-account-token
secret:
secretName: service-account-token
containers:
- name: cloudsql-proxy
image: gcr.io/cloudsql-docker/gce-proxy:1.11
imagePullPolicy: Always
command:
- /cloud_sql_proxy
- -instances=<project>:<cloudsqlinstance>=tcp:0.0.0.0:3306
- -credential_file=/secrets/cloudsql/credentials.json
ports:
- name: database-port
containerPort: 3306
volumeMounts:
- name: service-account-token
mountPath: /secrets/cloudsql
readOnly: true
So within any of your pods, the database your MYSQL_HOST:MYSQL_PORT will be cloudsqlproxy:3306
For multiple databases through the same proxy, you'd have the same deployment structure for the proxy, except that you will now expose 2 ports from the pod, like so:
apiVersion: extensions/v1beta1
...
spec:
template:
...
spec:
volumes:
...
containers:
- name: cloudsql-proxy
...
ports:
- name: database-port1
containerPort: 3306
- name: database-port2
containerPort: 3307
...
Then you'd create 2 services to for discovery on those ports like so:
apiVersion: v1
kind: Service
metadata:
name: cloudsqlproxy-db1
spec:
ports:
- port: 3306
targetPort: database-port1
selector:
app: cloudsqlproxy
---
apiVersion: v1
kind: Service
metadata:
name: cloudsqlproxy-db2
spec:
ports:
- port: 3306
targetPort: database-port2
selector:
app: cloudsqlproxy
So, with both services set to port 3306, you can connect to each database on that port:
mysql --host=cloudsqlproxy-db1 --port=3306 ...
mysql --host=cloudsqlproxy-db2 --port=3306 ...
Reference: https://github.com/GoogleCloudPlatform/cloudsql-proxy/blob/master/Kubernetes.md
With Google "Private IP" the cloud proxy is now irrelevant!