I have 2 Deployment - A(1 replica) and B(4 replica)
I have scheduled job in the POD A and on successful completion it hits endpoint present in one of the Pods from Deployment B through the service for Deployment B.
Is there a way I can hit all the endpoints of 4 PODS from Deployment B on successful completion of job?
Ideally one of the pod is notified!! But is this possible as I don't want to use pub-sub for this.
Is there a way I can hit all the endpoints of 4 PODS from Deployment B on successful completion of job?
But is this possible as I don't want to use pub-sub for this.
As you say, a pub-sub solution is best for this problem. But you don't want to use it.
Use stable network identity for Service B
To solve this without pub-sub, you need a stable network-identity for the pods in Deployment B. To get this, you need to change to a StatefulSet for your service B.
StatefulSets are valuable for applications that require one or more of the following.
Stable, unique network identifiers.
When B is deployed with a StatefulSet, your job or other applications can reach your pods of B, with a stable network identity that is the same for every version of service B that you deploy. Remember that you also need to deploy a Headless Service for your pods.
Scatter pattern: You can have an application aware (e.g. aware of number of pods of Service B) proxy, possibly as a sidecar. Your job sends the request to this proxy. The proxy then sends a request to all your replicas. As described in Designing Distributed Systems: Patterns and Paradigms
Pub-Sub or Request-Reply
If using pub-sub, the job only publish an event. Each pod in B is responsible to subscribe.
In a request-reply solution, the job or a proxy is responsible for watching what pods exists (unless it is a fixed number of pods) in service B, in addition it need to send request to all, if requests fails to any pod (it will happen on deployments sometime) it is responsibly to retry the request to those pods.
So, yes, it is a much more complicated problem in a request-reply way.
Kubernetes service is an abstraction to provide service discovery and load balancing. So if you are using service your request will be sent to one of the backend pods.
To achieve what you want I suggest you create 4 different services each having only one backend pod or use a message queue such as rabbitmq between service A and service B.
You can use headless service. That way Kubernetes won't allocate separate IP address and instead will setup DNS record with IP addresses of all the pods. Then in your application just resolve the record and send notification to all endpoints. But really, this is ideal use case for pub-sub or service discovery system. DNS is too unreliable for this.
PUB-SUB is the best option here. I have similar use case and I am using pub-sub , which is in production for last 6 months.
Related
Maybe this question is very wrong but my research so far hasn't been very helpful.
My plan is to deploy a server app to multiple pods , as replicas (same code running in multiple pods) and I want each pod to be able to communicate with the rest pods.
More specifically I need to broadcast a message to all the rest pods every x minutes.
I cannot find examples of how I could do that with Python code or anything helpful related to the communication internally between the pods. I can see some instructions for the yaml configurations that I should use to make that possible , but no practical examples , which makes me think that maybe using Kubernetes is not the best technology service for what I am trying to do (?).
Any advice/suggestion/documentation is more than needed.
Thank you
Applications is typically deployed as Deployment to Kubernets, however in use-cases where you want stable network identity for your Pods, it is easier to deploy your app as StatefulSet.
When your app is deployed as StatefulSet the pods will be named e.g.: appname-0, appname-1, appname-2 if your StatefulSet is named appname and your replicas is replicas: 3
I cannot find examples of how I could do that with Python code
This is just plain network programming between the pods. You can use any UDP or TCP protocol, e.g. you can use http for this. The network address is the pod name (since your replicas are Pods within the same namespace) e.g. http://appname-0 or http://appname-1.
In Kubernetes, is it possible to have 2 services for a single deployment, one which is "standard" and proxies in front of all ready pods, and a second service which sends traffic only the elected leader? If so how? I am using client-go for leader election. These are layer 4 services.
I know that a service can use labels for a selector, but client-go uses an annotation to mark the leader. Using a service without selectors and creating/removing an endpoint in the leader callbacks seems hacky/buggy. Thank you.
In Kubernetes, is it possible to have 2 services for a single deployment, one which is "standard" and proxies in front of all ready pods, and a second service that sends traffic only the elected leader?
Yes, but it seems a bit hacky. The way services work is like this:
Service -> Service labels select endpoints -> Endpoint -> Endpoint labels select pods -> Pods (PodIP)
So you could have your regular "Service" that points to all the pods on your Deployment or StatefulSet which automatically provisions all the Endpoints.
You could also have another set of "Headless Service" + "Endpoint" each manually created individually that you make their labels match with each other and then have that Endpoint manually match with the label of the pod of your choice.
Now with respect to client-go/leaderelection. It seems like it works using an Endpoint or ConfigMap lock for the leader (The example shows a ConfigMap lock). But, looks like you want to use the Endpoint lock. So this package doesn't work with services or labels, it looks like it just works on Endpoint resources. So essentially if you have 3 nodes and want to find the leader you would have to use 3 manually created Endpoint resources. The one that is the leader will always have the annotation.
Now how do you tie it to 2) above? As your client elects or selects the leader then you also have to change the endpoint labels so that they match your manually created headless service. (It could be done in your code too)
You could also just elect to just use Endpoints instead 2) above (No headless service) and have client-go/leaderelection talk directly to the endpoints.
Another option is to take advantage of StatefulSets and its required headless service. So that service will resolve to the IP addresses of all the replicas in your quorum-based cluster. The leader election would be up to the client package (client-go doesn't seem to support this) which is pretty the case for most quorum based applications (K8s, Zookeeper, Kafka, Etcd, etc, etc); the client is the one that finds who the leader is.
✌️
I am currently switching from Service Fabric to Kubernetes and was wondering how to do custom and more complex load balancing.
So far I already read about Kubernetes offering "Services" which do load balancing for pods hidden behind them, but this is only available in more plain ways.
What I want to rewrite right now looks like the following in Service Fabric:
I have this interface:
public interface IEndpointSelector
{
int HashableIdentifier { get; }
}
A context keeping track of the account in my ASP.Net application e.g. inherits this. Then, I wrote some code which would as of now do service discovery through the service fabric cluster API and keep track of all services, updating them when any instances die or are being respawned.
Then, based on the deterministic nature of this identifier (due to the context being cached etc.) and given multiple replicas of the target service of a frontend -> backend call, I can reliably route traffic for a certain account to a certain endpoint instance.
Now, how would I go about doing this in Kubernetes?
As I already mentioned, I found "Services", but it seems like their load balancing does not support custom logic and is rather only useful when working with stateless instances.
Is there also a way to have service discovery in Kubernetes which I could use here to replace my existing code at some points?
StatefulSet
StatefulSet is a building block for stateful workload on Kubernetes with certain guarantees.
Stable and unique network identity
StatefulSet Pods have a unique identity that is comprised of an ordinal, a stable network identity, and stable storage.
As an example, if your StatefulSet has the name sharded-svc
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: sharded-svc
And you have e.g. 3 replicas, those will be named by <name>-<ordinal> where ordinal starts from 0 up to replicas-1.
The name of your pods will be:
sharded-svc-0
sharded-svc-1
sharded-svc-2
and those pods can be reached with a dns-name:
sharded-svc-0.sharded-svc.your-namespace.svc.cluster.local
sharded-svc-1.sharded-svc.your-namespace.svc.cluster.local
sharded-svc-2.sharded-svc.your-namespace.svc.cluster.local
given that your Headless Service is named sharded-svc and you deploy it in namespace your-namespace.
Sharding or Partitioning
given multiple replicas of the target service of a frontend -> backend call, I can reliably route traffic for a certain account to a certain endpoint instance.
What you describe here is that your stateful service is what is called sharded or partitioned. This does not come out of the box from Kubernetes, but you have all the needed building blocks for this kind of service. It may happen that it exists an 3rd party service providing this feature that you can deploy, or it can be developed.
Sharding Proxy
You can create a service sharding-proxy consisting of one of more pods (possibly from Deployment since it can be stateless). This app need to watch the pods/service/endpoints in your sharded-svc to know where it can route traffic. This can be developed using client-go or other alternatives.
This service implements the logic you want in your sharding, e.g. account-nr modulus 3 is routed to the corresponding pod ordinal
Update: There are 3rd party proxies with sharding functionallity, e.g. Weaver Proxy
Sharding request based on headers/path/body fields
Recommended reading: Weaver: Sharding with simplicity
Consuming sharded service
To consume your sharded service, the clients send request to your sharding-proxy that then apply your routing or sharding logic (e.g. request with account-nr modulus 3 is routed to the corresponding pod ordinal) and forward the request to the replica of sharded-svc that match your logic.
Alternative Solutions
Directory Service: It is probably easier to implement sharded-proxy as a directory service but it depends on your requirements. The clients can ask your directory service to what statefulSet replica should I send account-nr X and your serice reply with e.g. sharded-svc-2
Routing logic in client: The probably most easy solution is to have your routing logic in the client, and let this logic calculate to what statefulSet replica to send the request.
Services generally run the proxy in kernel space for performance reasons so writing custom code is difficult. Cillium does allow writing eBPF programs for some network features but I don't think service routing is one of them. So that pretty much means working with a userspace proxy instead. If your service is HTTP-based, you could look at some of the existing Ingress controllers to see if any are close enough or allow you to write your own custom session routing logic. Otherwise you would have to write a daemon yourself to handle it.
We have a service with multiple replicas which works with storage without transactions and blocking approaches. So we need somehow to synchronize concurrent requests between multiple instances by some "sharding" key. Right now we host this service in Kubernetes environment as a ReplicaSet.
Don't you know any simple out-of-the-box approaches on how to do this to not implement it from scratch?
Here are several of our ideas on how to do this:
Deploy the service as a StatefulSet and implement some proxy API which will route traffic to the specific pod in this StatefulSet by sharding key from the HTTP request. In this scenario, all requests which should be synchronized will be handled by one instance and it wouldn't be a problem to handle this case.
Deploy the service as a StatefulSet and implement some custom logic in the same service to re-route traffic to the specific instance (or process on this exact instance). As I understand it's not possible to have abstract implementation and it would work only in Kubernetes environment.
Somehow expose each pod IP outside the cluster and implement routing logic on the client-side.
Just implement synchronization between instances through some third-party service like Redis.
I would like to try to route traffic to the specific pods. If you know standard approaches how to handle this case I'll be much appreciated.
Thank you a lot in advance!
Another approach would be to put a messaging queue (like Kafka and RabbitMq) in front of your service.
Then your pods will subscribe to the MQ topic/stream. The pod will decide if it should process the message or not.
Also, try looking into service meshes like Istio or Linkerd.
They might have an OOTB solution for your use-case, although I wasn't able to find one.
Remember that Network Policy is not traffic routing !
Pods are intended to be stateless and indistinguishable from one another, pod-networking.
I recommend to Istio. It has special component which is responsible or routing- Envoy. It is a high-performance proxy developed in C++ to mediate all inbound and outbound traffic for all services in the service mesh.
Useful article: istio-envoy-proxy.
Istio documentation: istio-documentation.
Useful Istio explaination https://www.youtube.com/watch?v=e2kowI0fAz0.
But you should be able to create a Deployment per customer group, and a Service per Deployment. The Ingress nginx should be able to be told to map incoming requests by whatever attributes are relevant to specific customer group Services.
Other solution is to use kube-router.
Kube-router can be run as an agent or a Pod (via DaemonSet) on each node and leverages standard Linux technologies iptables, ipvs/lvs, ipset, iproute2.
Kube-router uses IPVS/LVS technology built in Linux to provide L4 load balancing. Each ClusterIP, NodePort, and LoadBalancer Kubernetes Service type is configured as an IPVS virtual service. Each Service Endpoint is configured as real server to the virtual service. The standard ipvsadm tool can be used to verify the configuration and monitor the active connections.
How it works: service-proxy.
I have a service which is long-running (in a while 1 loop) and processes payloads via GCloud pub/sub, after which it writes the result to a DB.
The service doesn't need to listen on any port.
What would the declarative YAML config look like for Kind=Deployment?
I understand ClusterIP is the default type, and the docs go on to say that a headless service just has to define spec.clusterIP as None.
(A better practice would probably be to modify the worker to exit after a successful payload processing, and change the Kind to Job, but this is in the backlog)
What you're describing sounds more like a job or a deployment than a service. You can run a deployment (which creates a replicaset, which ensures a certain number of replicas are running) without creating a service.
If your pod isn't exposing any network services for others to consume, there's very little reason to create a service.