I've developed a python script, using python kubernetes-client to harvest Pods' internal IPs.
But when I try to make an http request to these IPs, from another pod, I get Connection refused error.
I spin up a temporary curl container:
kubectl run curl --image=radial/busyboxplus:curl -it --rm
And having the internal IP of one of the pods, I try to make a GET request:
curl http://10.133.0.2/stats
and the response is:
curl: (7) Failed to connect to 10.133.0.2 port 80: Connection refused
Both pods are in the same default namespace and use the same default ServiceAccount.
I know that I can call the Pods thru the ClusterIP service by which they're load-balanced, but this way I will only access a single Pod at random (depending which one the service forwards the call to), when I have multiple replicas of the same Deployment.
I need to be able to call each Pod of a multi-replica Deployment separately. That's why I'm going for the internal IPs.
I guess you missed the port number here
It should be like this
curl POD_IP:PORT/stats
Related
We have a rancher based kubernetes cluster with calico on openstack.
A spring config server (server pod here onwards) is deployed as a service.
Service is exposed on nodeport.
Another pod is deployed with curl (client pod here onwards)
Doing a curl from client pod on server pod nodeport with its node ip on which server pod is running gives proper result.
Doing curl from client pod on server nodeport with another node's ip where server pod is not running pod gives curl(56) connection reset by peer for bigger response
Doing curl from client pod on service and it's port gives results for small data but for bigger response again gives curl(56)
If both client and server pods are running on same node, response is fine.
My understanding is:
No issues in server pod, as able to get response on nodeport
No issues in client pod/curl as able to get response from nodeport
Service and pod linkage is fine as it works well with small response size
When I say bigger response, I mean just 1 kb+
It will give an error when there is no reply from the server.
Ex:
$ curl http://youtube.com:443
Curl: (52) Empty reply from server
Please recheck your proxy, firewall settings.
Error while getting config map appconfig
Get "https://xxx.xx.x.x:443/api/v1/namespaces/app/configmaps/appconfig": dial tcp xxx.xx.x.x:443: connect: connection refused"
But when istio sidecar is not injected, there is no error
Try this:
oc patch deploy <deployment-name> -p '{"spec":{"template":{"metadata":{"annotations":{"traffic.sidecar.istio.io/excludeOutboundIPRanges": "'$(oc get svc kubernetes -n default -o jsonpath='{.spec.clusterIP}')/32'"}}}}}'
Not sure if it is a bug or not, but apparently istio sidecar proxy does not allow for application containers to communicate with kubernetes API server when data plane is in strict mtls mode.
The above patch introduces an IP range in which the kubernetes API server resides and allows connections to those addresses go outside the sidecar proxy, thus avoiding network rules it enforces.
I am deploy kubernetes UI using this command:
kubectl apply -f kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml
And it response "Unable to connect to the server: dial tcp 185.199.110.133:443: i/o timeout"
I behind proxy, how can i fix it?
All the services that you deployed via the supplied url don't have a kind specified. This means they will be using the default service type which is ClusterIP.
Services of Kind ClusterIP are only accessible from inside your Kubernetes Cluster.
If you want the Dashboard to be accessible from outside your Cluster, you will need a service of type NodePort. A NodePort Service will assign a random high number port on all your nodes on which your application, in this case the k8s dashboard, will be accessible via ${ip-of-any-node}:${assigned-nodeport}.
For more information, please take a look at the official k8s documentation.
If your cluster is behind a proxy, also make sure, that you can reach your clusters node's external ip from wherever you are trying to send the request from.
In order to find out what port number has been assigned to your NodePort service use kubectl describe service ${servicename} or kubectl get service ${servicename} -o yaml
I am deploy kubernetes UI using this command:
kubectl apply -f kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml
And it response "Unable to connect to the server: dial tcp 185.199.110.133:443: i/o timeout"
I behind proxy, how can i fix it?
All the services that you deployed via the supplied url don't have a kind specified. This means they will be using the default service type which is ClusterIP.
Services of Kind ClusterIP are only accessible from inside your Kubernetes Cluster.
If you want the Dashboard to be accessible from outside your Cluster, you will need a service of type NodePort. A NodePort Service will assign a random high number port on all your nodes on which your application, in this case the k8s dashboard, will be accessible via ${ip-of-any-node}:${assigned-nodeport}.
For more information, please take a look at the official k8s documentation.
If your cluster is behind a proxy, also make sure, that you can reach your clusters node's external ip from wherever you are trying to send the request from.
In order to find out what port number has been assigned to your NodePort service use kubectl describe service ${servicename} or kubectl get service ${servicename} -o yaml
I am developing microservices using SpringBoot and using Kubernetes for deployment. For that I have two services Order and Customer.
Then Order service calls the Customer service to get some data on http protocol. It call Kubernetes service. I tried both name of Customer service and ip as well but during this communication it throw time out exception.
Following is piece of code.
Customer Service :
I tried to use call both using IP address and service name as well, something like below code.
but it does not work.
It throws following error. In screen shot I attach with name but It gives me same error with IP address as well.
Its Minikube single node cluster.
What wrong I am doing here?
Please use below link to troubleshoot Kubernetes Services
https://kubernetes.io/docs/tasks/debug-application-cluster/debug-service/
Please use below commands to check if you can curl to Kubernetes Service customer-service on Port 8080 from any pod inside the same namespace
kubectl run -it --rm=true --restart=Never curl --image=radial/busyboxplus -- curl http://customer-service:8080
kubectl run -it --rm=true --restart=Never curl --image=radial/busyboxplus -- curl http://10.104.255.198:8080
Also order-service Pods should call Customer Service Pods using customer-service and not using Ingress. Ingress is for sending requests to Customer Service Pods from outside the kubernetes cluster
You can use the following steps to check:
1.Use the telnet command to check access to the POD mapping port;
2.Check whether your Service rules are configured correctly, here you can debug by creating a Node port type;
3.Check iptables or ipvs in your underlying communication forwarding rules;