My question is similar to the following question in this link: See Inbound Outbound communication of pods
I a running a Kubernetes cluster (with kubeadm and CRI-O engine) on EC2 instances (1 master, 1-2 workers, a very simple application- such as WordPress + MySQL DB).
What I'm looking for is a topology tool to map all pods' communication with other pods in the cluster, with output that I can parse later for further use.
My goal is to generate a JSON/CSV file (or any other type that I can parse later) with the following information:
<Pod i name, inbound communications list, protocol, port>, and the inbound communications list will include <Pod j name, protocol, port> for each pod in communication with the specific pod (i).
I also would like the tool to be not-Web-UI based, but an API/CLI based tool.
So far I came across WeaveScope, which can be scraped for the information I'm interested in (as mentioned in the question I referred to at the beginning). But this is a web-UI based tool and therefore does not fit for my purposes.
Thanks in advance!
Related
I have read this question which is very similar to what I am asking, but still wanted to write a new question since the accepted answer there seems very incomplete and also potentially wrong.
Basically, it seems like there is some missing or contradictory information regarding built in load-balancing for regular Kubernetes Services (I am not talking about LoadBalancer services). For example, the official Cilium documentation states that "Kubernetes doesn't come with an implementation of Load Balancing". In addition, I couldn't find any information in the official Kubernetes documentation about load balancing for internal services (there was only a section discussing this under ingresses).
So my question is - how does load balancing or distribution of requests work when we make a request from within a Kubernetes cluster to the internal address of a Kubernetes service?
I know there's a Kubernetes proxy on each node that creates the DNS records for such services, but what about services that span multiple pods and nodes? There's got to be some form of request distribution or load-balancing, or else this just wouldn't work at all, no?
A standard Kubernetes Service provides basic load-balancing. Even for a ClusterIP-type Service, the Service has its own cluster-internal IP address and DNS name, and forwards requests to the collection of Pods specified by its selector:.
In normal use, it is enough to create a multiple-replica Deployment, set a Service to point at its Pods, and send requests only to the Service. All of the replicas will receive requests.
The documentation discusses the implementation of internal load balancing in more detail than an application developer normally needs. Unless your cluster administrator has done extra setup, you'll probably get round-robin request routing – the first Pod will receive the first request, the second Pod the second, and so on.
... the official Cilium documentation states ...
This is almost certainly a statement about external load balancing. As a cluster administrator (not a programmer) a "plain" Kubernetes installation doesn't include an external load-balancer implementation, and a LoadBalancer-type Service behaves identically to a NodePort-type Service.
There are obvious deficiencies to round-robin scheduling, most notably if you do wind up having individual network requests that take a long time and a lot of resource to service. As an application developer the best way to address this is to make these very-long-running requests run asynchronously; return something like an HTTP 201 Created status with a unique per-job URL, and do the actual work in a separate queue-backed worker.
Maybe this question is very wrong but my research so far hasn't been very helpful.
My plan is to deploy a server app to multiple pods , as replicas (same code running in multiple pods) and I want each pod to be able to communicate with the rest pods.
More specifically I need to broadcast a message to all the rest pods every x minutes.
I cannot find examples of how I could do that with Python code or anything helpful related to the communication internally between the pods. I can see some instructions for the yaml configurations that I should use to make that possible , but no practical examples , which makes me think that maybe using Kubernetes is not the best technology service for what I am trying to do (?).
Any advice/suggestion/documentation is more than needed.
Thank you
Applications is typically deployed as Deployment to Kubernets, however in use-cases where you want stable network identity for your Pods, it is easier to deploy your app as StatefulSet.
When your app is deployed as StatefulSet the pods will be named e.g.: appname-0, appname-1, appname-2 if your StatefulSet is named appname and your replicas is replicas: 3
I cannot find examples of how I could do that with Python code
This is just plain network programming between the pods. You can use any UDP or TCP protocol, e.g. you can use http for this. The network address is the pod name (since your replicas are Pods within the same namespace) e.g. http://appname-0 or http://appname-1.
I am new to the Kubernetes, and I'm trying to understand that how can I apply it for my use-case scenario.
I managed to install a 3-node cluster on VMs within the same network. Searching about K8S's concepts and reading related articles, still I couldn't find answer for my below question. Please let me know if you have knowledge on this:
I've noticed that internal DNS service of K8S applies on the pods and this way services can find each other with hostnames instead of IPs.
Is this applicable for communication between pods of different nodes or this is only within the services inside a single node? (In other words, do we have a dns service on the node level in the K8S, or its only about pods?)
The reason for this question is the scenario that I have in mind:
I need to deploy a micro-service application (written in Java) with K8S. I made docker images from each service in my application and its working locally. Currently, these services are connected via pre-defined IP addresses.
Is there a way to run each of these services within a separate K8S node and use from its DNS service to connect the nodes without pre-defining IPs?
A service serves as an internal endpoint and (depending on the configuration) load balancer to one or several pods behind it. All communication typically is done between services, not between pods. Pods run on nodes, services don't really run anything, they are just routing traffic to the appropriate pods.
A service is a cluster-wide configuration that does not depend on a node, thus you can use a service name in the whole cluster, completely independent from where a pod is located.
So yes, your use case of running pods on different nodes and communicate between service names is a typical setup.
Let us say I've two zones zone1 and zone2, having 2 apps deployed in each zone. Let us say App1 is a client which fetches information from App2, App1 connects to App2 using k8s service, Now how can I configure app1 of zone1 to connect to app2 of zone1(preferably, if app2 of zone1 is loaded or down connect to app2 of zon2).
Though this can be achieved by application layer using zuul and ribbon with headless service, I want to move this to infra layer. Is there any possibility to do in K8s.
I see IPVS supports Locality-Based Least Connection algorithm, but not sure k8s supports this algorithm, I see supported algos are rr, wrr, lc, sed. but no documentation regarding support for lblc. if lblc is supported is this better solution to prefer same node/pod in dc/pod in zone.
NOTE: This is solution is purely for on-prem k8s cluster.
I will answer only to part of your question, have no experience with "best practices" configuration in this area.
But what I want to share with you - is that kubernetes definitely supports Locality-Based Least Connection algorithm.
You can find this in the source code:
LocalityBasedLeastConnection IPVSSchedulerMethod = "lblc"
// LocalityBasedLeastConnectionWithReplication with Replication assigns jobs destined for the same IP address to the
// least-connection node in the server set for the IP address. If all the node in the server set are overloaded,
// it picks up a node with fewer jobs in the cluster and adds it to the sever set for the target.
// If the server set has not been modified for the specified time, the most loaded node is removed from the server set,
// in order to avoid high degree of replication.
You can find info on how to enable IPVS here: https://kubernetes.io/blog/2018/07/09/ipvs-based-in-cluster-load-balancing-deep-dive/
P.S. Above article doesnt contains no info about lblc but as per source code - k8s supports it.
I currently have two clusters on GKE - one in eu-west1-b and another in us-east1-b. The pods deployed to the nodes in these clusters need to make location-based requests (for latency testing purposes).
I also need to connect to my postgres instance on RDS, which uses IP-based whitelisting for external connections. The nodes in my clusters have ephemeral IPs so I can't use them.
I have done a lot of research and gone through lots of SO answers and docs and tutorials and come to the solution that routing traffic through a NAT is pretty much the best/only way to do this right now on GKE.
https://serverfault.com/questions/835425/kubernetes-external-connection-through-single-ip
Similar to that question above, I don't want to route all of my traffic through the NAT. My reason is because I need my requests to come from the internet gateway associated with the current node so it is coming from a particular region.
The above question has some answers that almost get me there, but doesn't include any kube-specific configuaration. This is a great tutorial:
https://docs.tenable.com/pvs/deployment/Content/GoogleCloudInstructionsNatGateway.htm
But again, is not based on kube.
My thinking is that I need to define a service for postgres in my kube cluster, and then tell that to route to the external service through the NAT. Not entirely sure where to start and would appreciate help.
A solution:
Tag your instances in different zones/regions with different tags
Create static IP addresses for each zone/region
Create NAT exit nodes (GCE instances or instance groups) using the external address from above
Create a route trough each of the NAT exit nodes. Restrict each route with destination IP range for your RDS ingress IP/32 and network tags from Step 1 (so the instances use the correct gateway)