I created four services, two ClusterIP and two NodePort services. Against each service I spin up two containers as shown below.
However, the problem is some services work fine and some while calling from inside the container not able to resolve the service hostname.
While narrowing down a problem, I created a below matrix:-
TYPES NodePort ClusterIp
NodePort Pass Fail
ClusterIp Pass Fail
which explains:-
A request (curl –v http://order-service-ip/swagger/index.html) from inside the container of aggregator-service(nodePort) fails and throws
could not resolve hostname error but vice-versa works. That is hitting
a request (curl –v http://aggregator-service/swagger/index.html) from
inside the container of order-service-ip works.
In the same way, calling of nodeport service from another nodeport container works.
But, calling of clusterIP service from a ClusterIp container fails and not able to resolve the hostname.
Surprisingly, a cluster IP container is able to resolve the hostname of Nodeport service and traversing back from the same nodePort container to the same ClusterIp service does not work.
Any suggestions are appreciated. I am stuck with this problem for more than four days now.
Here are the details of pods and services and end points are also working fine.
NAME ........ READY STATUS
nodeport-aggegator-deployment-64497699d-6jqz4 1/1 Running
nodeport-aggegator-deployment-64497699d-jx8n6 1/1 Running
clusterip-order-deployment-ip-594ff6b59b-pb4bp 1/1 Running
clusterip-order-deployment-ip-594ff6b59b-rbhj4 1/1 Running
nodeport-resources-deployment-6b98d47b5b-qvd59 1/1 Running
nodeport-resources-deployment-6b98d47b5b-zjrh7 1/1 Running
clusterip-product-deployment-ip-7589c74bfc-dx8l4 1/1 Running
clusterip-product-deployment-ip-7589c74bfc-mbqs5 1/1 Running
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
aggregator-service NodePort 10.100.66.74 <none> 8081:30392/TCP,443:30891/TCP
order-service-ip ClusterIP 10.100.118.19 <none> 8010/TCP,443/TCP
resources-service NodePort 10.100.81.65 <none> 8001:31076/TCP,443:30429/TCP
product-service-ip ClusterIP 10.100.66.14 <none> 8011/TCP,443/TCP
kubernetes ClusterIP 10.100.0.1 <none> 443/TCP
Thanks
The problem was with in a code which had a port conflict. The one being used to redirect from Nodeport to ClusterIp was not correct.
Related
I run a local kubernetes cluster (Minikube) and I try to connect pgAdmin to postgresql, bot run in Kubernetes.
What would be the connection string? Shall I access by service ip address or by service name?
kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dbpostgresql NodePort 10.103.252.31 <none> 5432:30201/TCP 19m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d21h
pgadmin-service NodePort 10.109.58.168 <none> 80:30200/TCP 40h
kubectl get ingress:
NAME CLASS HOSTS ADDRESS PORTS AGE
pgadmin-ingress <none> * 192.168.49.2 80 40h
kubectl get pod
NAME READY STATUS RESTARTS AGE
pgadmin-5569ddf4dd-49r8f 1/1 Running 1 40h
postgres-78f4b5db97-2ngck 1/1 Running 0 23m
I have tried with 10.103.252.31:30201 but without success.
Inside the cluster, services can refer to each other by DNS based on Service object names. So in this case you would use dbpostgresql or dbpostgresql.default.svc.cluster.local as the hostname.
Remember minikube is running inside its' own container, the NodePort clusterIPs you're getting back are open inside of minikube. So to get minikube's resolution of port and ip, run: minikube service <your-service-name> --url
This will return something like http://127.0.0.1:50946 which you can use to create an external DB connection.
Another option would be to use kubectl to forward a local port to the service running on localhost ex. kubectl port-forward service/django-service 8080:80
I'm currently learning kubernetes and started to deploy ELK stack on a minikube cluster (running on a linux EC2 instance), though i was able to run all the objects successfully, I'm not able to access any of the tool from my windows browser, looking for some inputs on how to access all below exposed ports from my windows browser.
Cluster details:
NAME READY STATUS RESTARTS AGE
pod/elasticsearch-deployment-5c7d5cb5fb-g55ft 1/1 Running 0 3m43s
pod/kibana-deployment-76d8744864-ddx4h 1/1 Running 0 3m43s
pod/logstash-deployment-56849fcd7b-bjlzf 1/1 Running 0 3m43s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elasticsearch-service ClusterIP XX.XX.XX.XX <none> 9200/TCP 3m43s
service/kibana-service ClusterIP XX.XX.XX.XX <none> 5601/TCP 3m43s
service/kubernetes ClusterIP XX.XX.XX.XX <none> 443/TCP 5m15s
service/logstash-service ClusterIP 10.XX.XX.XX <none> 9600/TCP,5044/TCP 3m43s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/elasticsearch-deployment 1/1 1 1 3m43s
deployment.apps/kibana-deployment 1/1 1 1 3m43s
deployment.apps/logstash-deployment 1/1 1 1 3m43s
NAME DESIRED CURRENT READY AGE
replicaset.apps/elasticsearch-deployment-5c7d5cb5fb 1 1 1 3m43s
replicaset.apps/kibana-deployment-76d8744864 1 1 1 3m43s
replicaset.apps/logstash-deployment-56849fcd7b 1 1 1 3m43s
Note: I also tried to run all the above services as NodePort and using the minikube ip i was able hit curl commands to check the status of the application, but still not able to access any of it via my browser
Generally if you want expose anything outside the cluster you need to user service type:
NodePort, LoadBalancer or use Ingress. If you will check Minikube documentaton, you will find that Minikube supports all those types.
If you thought about LoadBalancer, you can use minikube tunnel.
When you are using cloud environment and non standard ports, you should check firewall rules to check if port/traffic is open.
Regarding error from comment, it seems that you have issue with Kibana port 5601.
Did you check similar threads like this or this? If this won't be helpful, please provide Kibana configuration.
did you check just a normal port-forward instead of minikube ip, and expose. Those didnt work for me neither.
something like this may would help.
kubectl port-forward deployment/kibana-kibana 5601
I deployed my application using deployment construct. State of my pod is Running and making curl against pod's IP returns application content. However when I created service using kubectl expose deployment and I curl service's IP then curl throws Connection refused error. Why is that?
My pod
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
cge-frontend-5d4595469b-qvcsd 0/1 Running 0 19s 10.40.0.4 compute04 <none> <none>
My service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cge-frontend ClusterIP 10.98.212.184 <none> 80/TCP 16m
Error
$ curl 10.98.212.184
curl: (7) Failed connect to 10.98.212.184:80; Connection refused
After investigating my service with kubectl describe svc command. I fogure out that my service has no Endpoints - endpoints section should list pod's IP.
$ kubectl describe svc cge-frontend
Name: cge-frontend
Namespace: default
Labels: app=cge-frontend
Annotations: <none>
Selector: app=cge-frontend
Type: ClusterIP
IP: 10.98.212.184
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints:
Session Affinity: None
It turned out that, the error was caused by one of my probe that was keeping my pod in Running state but not in Readystate. Fixing probes, fixed my pods, and that fixed the service.
My pod after fixing probes is now in correct state READY 1/1
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
cge-frontend-5d4595469b-qvcsd 1/1 Running 0 19s 10.40.0.5 compute04 <none> <none>
I have a StatefulSet that matchs the one described in the
StatefulSet Kubernetes tutorial which creates a mysql master/slaves structure. I have the following kubernetes objects:
NAME READY STATUS RESTARTS AGE
pod/mysql-0 2/2 Running 7 23h
pod/mysql-1 2/2 Running 6 23h
pod/mysql-2 2/2 Running 6 23h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 21d
service/mysql ClusterIP None <none> 3306/TCP 23h
service/mysql-read ClusterIP 10.152.183.89 <none> 3306/TCP 23h
NAME READY AGE
statefulset.apps/mysql 3/3 23h
EDIT: The mysql and mysql-read services connect with all the mysql pods.
The node is on my computer, I'm using microk8s.
Now I have a little program runnning on my computer (not in the cluster) that I would like to connect to the master (mysql-0) inside the StatefulSet. Id need something like a service that only connect with mysql-0. Any suggestions on how to do it?
EDIT: The idea is to find a solution that allow me to deploy the cluster just with the .yaml files. It isn't interesting to find one that involves more commands than kubectl apply
The program is the following one:
import mysql.connector
mydb = mysql.connector.connect(
host="localhost??",
user="root",
passwd="",
database="testtable"
)
mycursor = mydb.cursor()
sql = "INSERT INTO testtable (name) VALUES (%s)"
val = ("holaquetal")
mycursor.execute(sql, val)
mydb.commit()
I thought I could add a different label to the mysql-0 pod and add a NodePort service that looks for that label, but I dont want to do it by command line. Is it possible to add one label to mysql-0 inside the StatefulSet yaml?
Another idea could be to do a query DNS to the DNS server that microk8s provides, looking for "mysql.mysql-0", but I don't know how to connect the program to the DNS server, so that I can use host=mysql.mysql-0 or the whole CNAME.
If the node is on your computer so can use port-forward to forward the port of mysql from the pod to your local. Try this:
kubectl port-forward mysql-0 3306:3306
or forward directly to your master service
kubectl port-forward svc/mysql-read 3306:3306
I am using kubespray to run a kubernetes cluster on my laptop. The cluster is running on 7 VMs and the roles of the VM's spread as follows:
NAME STATUS ROLES AGE VERSION
k8s-1 Ready master 2d22h v1.16.2
k8s-2 Ready master 2d22h v1.16.2
k8s-3 Ready master 2d22h v1.16.2
k8s-4 Ready master 2d22h v1.16.2
k8s-5 Ready <none> 2d22h v1.16.2
k8s-6 Ready <none> 2d22h v1.16.2
k8s-7 Ready <none> 2d22h v1.16.2
I've installed https://istio.io/ to build a microservices environment.
I have 2 services running and like to access from outside:
k get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
greeter-service ClusterIP 10.233.50.109 <none> 3000/TCP 47h
helloweb ClusterIP 10.233.8.207 <none> 3000/TCP 47h
and the running pods:
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default greeter-service-v1-8d97f9bcd-2hf4x 2/2 Running 0 47h 10.233.69.7 k8s-6 <none> <none>
default greeter-service-v1-8d97f9bcd-gnsvp 2/2 Running 0 47h 10.233.65.3 k8s-2 <none> <none>
default greeter-service-v1-8d97f9bcd-lkt6p 2/2 Running 0 47h 10.233.68.9 k8s-7 <none> <none>
default helloweb-77c9476f6d-7f76v 2/2 Running 0 47h 10.233.64.3 k8s-1 <none> <none>
default helloweb-77c9476f6d-pj494 2/2 Running 0 47h 10.233.69.8 k8s-6 <none> <none>
default helloweb-77c9476f6d-tnqfb 2/2 Running 0 47h 10.233.70.7 k8s-5 <none> <none>
The problem is, I can not access the services from outside, because I do not have the EXTERNAL IP address(remember the cluster is running on my laptop).
k get svc istio-ingressgateway -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway LoadBalancer 10.233.61.112 <pending> 15020:31311/TCP,80:30383/TCP,443:31494/TCP,15029:31383/TCP,15030:30784/TCP,15031:30322/TCP,15032:30823/TCP,15443:30401/TCP 47h
As you can see, the column EXTERNAL-IP the value is <pending>.
The question is, how to assign an EXTERNAL-IP to the istio-ingressgateway.
First of all, you can't make k8s to assign you an external IP address, as LoadBalancer service is Cloud Provider specific. You could push your router external IP address to be mapped to it, I guess, but it is not trivial.
To reach the service, you can do this:
kubectl edit svc istio-ingressgateway -n istio-system
Change the type of the service from LoadBalancer to ClusterIp. You can also do NodePort. Actually you can skip this step, as LoadBalancer service already contains NodePort and ClusterIp. It is just to get rid of that pending status.
kubectl port-forward svc/istio-ingressgateway YOUR_LAPTOP_PORT:INGRESS_CLUSTER_IP_PORT -n istio-system
I don't know to which port you want to access from your localhost. Say 80, you can do:
kubectl port-forward svc/istio-ingressgateway 8080:80 -n istio-system
Now port 8080 of your laptop (localhost:8080) will be mapped to the port 80 of istio-ingressgateway service.
By default, there is no way Kubernetes can assign external IP to LoadBalancer service.
This service type needs infrastructure support which works in cloud offerings like GKE, AKS, EKS etc.
As you are running this cluster inside your laptop, deploy MetalLB Load Balancer to get EXTERNAL-IP
It's not possible as Suresh explained.
But if you want to access from your laptop you can use in your service type: NodePort, which gives you access from outside the cluster.
You should first obtain the IP of your cluster, then create your service with something like this:
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
selector:
app: my-app
type: NodePort
ports:
- name: http
protocol: TCP
port: 3000
targetPort: 3000
nodePort: 30000
After that, you can access from your laptop with: http://cluster-ip:30000
There is no need to create an ingress for that.
You should use a port in range (30000-32767), as stated below:
If you set the type field to NodePort, the Kubernetes control plane allocates a port from a range specified by --service-node-port-range flag (default: 30000-32767).
If you are using minikube, just run:
$ minikube tunnel
$ k get svc -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway LoadBalancer 10.111.187.167 127.0.0.1 15021:31949/TCP,80:32215/TCP,443:30585/TCP 9m48s