I'm looking at consensus-type tools like ZooKeeper, Consul and Eureka and they all seem to market the same set of solutions:
Service discovery
Dynamic, centralized configuration management
Synchronization primitives
Consensus algorithms
However the more I read about these things, the more I struggle to see how service discovery is really any different than a dynamic, centralized configuration management (KV pair) system.
My understanding (so far) of service discovery is that it allows nodes to dynamically search for, find and connnect to remote services. So if an application uses an AuthService for authentication, authorization, it would use service discovery to find an AuthService node at, say, http://auth103.example.org:9103 and use it.
My understanding of dynamic config systems is that they provide a centralized infrastructure for nodes to dynamically receive updates from, as well as publish updates to, config servers. So if an app instance decides it needs to update a configuration for all its other instances, it would contact the config service and update, say, the numPurgerThreads config. The config service would then update all other app instances so that they updated their respective configs properly.
But aren't these exactly the same problem?
In both cases, you:
Connect to a lookup service of some sort
Query it for data; or
Publish data to it, which then ripples out to other nodes
Service discovery is dynamic configuration, right?!?!
What I'm really driving at is: Couldn't I just implement one config service with one of these tools, which coincidentally also solves service discovery? Or is there a reason why I would need to have, say, 1 Consul cluster for config/KV management, and another, say, Consul cluster for service discovery?
Well, if you look at it like that, these are all just flavors of databases, or data stores if you will, as you describe it:
Connect to a lookup service of some sort
Query it for data; or
Publish data to it, which then ripples out to other nodes
All of the use cases you mentioned require some sort of a data store that multiple clients connect to. What makes some of them more optimal then others for different use cases is their interface, data model, consistency guaranties etc.
Specifically, in case of service discovery, a nice feature can be failure detection - e.g. a k/v store that allows removing service data when the service is no longer connected. This way, you can register a service to your service discovery tool and know that when the service goes down or looses connectivity, it will no longer be present in the stored data.
Just to clarify a few terms, (imo) dynamic config systems can update their state at runtime, in contrast to static where everything is defined beforehand (i.e. config files). Centralized CM means there's a single place to store all config data. So centralized CM can both be static or dynamic, right?
IMO, service discovery has a component which CM doesn't and that's a protocol for automatic detection of services.
Guess you'd need a dynamic CM (KV store) to implement service discovery, but you couldn't implement a CM store just using a service discovery (protocol). Take DHCP for example (hope we agree it is a service discovery protocol?) - Dynamic Host Configuration Protocol', so it has a configuration aspect, but also a protocol, which is a bit more than a simple KV store. Btw, it's decentralized, just to illustrate the previous point that CM doesn't have to be centralized..
So service discovery has a configuration aspect, but CM doesn't (necessarily) have a service discovery one. Does this mean SD is a subset of CM? I'd say no.
Related
I am new to distributed systems, and came to this problem once needed to deploy a gRPC service to kubernetes (GKE). As far as I know, when a client initiate an rpc, it creates a long lasting http2 connection and further calls are multiplexed on it. I like to send/push notifications or similar messages to the client through this connection. If I deploy to multiple pod, then the connections are spread across them, and not sure what is the best way to locate the instance where the channel is registered to the client. A possible solution could be, as soon as user initiate a connection, keep a reference of clientId and pod ip (or some identification) in a centralized service and other pods lookup the pod and forward the message to it. Is something like is advisable or is there an existing solution for this? I am unfamiliar with this space and any suggestion is highly appreciated.
Edit: (response to #mebius99)
While looking at deploying option, I stumbled upon GKE, and other cloud deployment options were limited because of my use of gRPC/http2. Thanks for mentioning service discovery , and that or service mesh might be an option. With gRPC, client maintains a long lived connection to a single pod. So, I want every pod to be able to query, based on unique clientId (clients can do an initial register rpc call), which pod is it connected, so can make use of this connection and also a way pods to forward the message between them. So, something like when I get a registration call from client, I update the central registry about the client and pod ip, then look it up from any pod and forward package to it so it further forward to client through the existing streaming connection. You guiding me to the right direction, please let me know above is possible in container environment.
thank you.
Another idea, You can use Envoy proxy.
If you are using GKE, these posts are helpful.
https://cloud.google.com/solutions/exposing-grpc-services-on-gke-using-envoy-proxy
https://github.com/GoogleCloudPlatform/grpc-gke-nlb-tutorial
I'd suggest to start from the Kubernetes Service concept and Service discovery. The External HTTP(S) Load Balancing should fit your needs.
In case you need something more sophisticated, Envoy proxy + Network Load Balancing could be a solution, as is mentioned here.
It sounds like you want to implement some kind of Pub-Sub system.
You must do some back-of-envelop calculation of the scale, such as how many clients, how many messages per second first.
Then you can choose whether to implement yourself or pick an off-the-shelf system, such as https://doc.akka.io/docs/alpakka/current/google-cloud-pub-sub-grpc.html
I just want to add more explanations to the existing answers here.
Since requests in HTTP/2 is multiplexed (multiple requests can be active on the same connection at any point in time), requests will be just pinned to a single Kubernetes pod. Hence, we need to configure a service mesh to shift from connection-based balancing to request-based balancing. Envoy Proxy mentioned here is one example.
I'd recommend everyone to read this good article from Kubernetes blog https://kubernetes.io/blog/2018/11/07/grpc-load-balancing-on-kubernetes-without-tears.
(Apology in advance for the noob question; I have zero experience with DevOps.)
In my recent project I stumbled upon this problem that I don’t know if service discovery tools (such as Consul/Istio/etc.) can address.
Our use case is this: we have a VoIP app similar in idea to Discord. Users can join a voice channel and start talking. However, to forward the voice packets between users in a same voice channel, their WebRTC voice connections need to be handled by a same server process, so that we can process & forward all the voice packets in a voice channel in-memory.
In order to do this, we have a separate service (call it service X) in front of our voice service (service V) that receives a user request to join channel N, and based on N, assign a server in service V to the user. We need to guarantee that for the same channel N, X always picks the same server in V.
We implemented this in a non-scalable way just for quick prototyping. Now that we want to implement this properly, I’m wondering if tools like Consul/Istio/etc. can help us in this scenario. Is there a common approach to address this kind of problems?
Istio won't necessarily help you since it's more about [controlling traffic](Like you mentioned you can use Consul as a service discovery tool, or ). For example, doing canary deployment or applying security to your service. Quoted from the docs:
Istio doesn’t provide DNS resolution. Applications can try to resolve the FQDN by using the DNS service present in their platform of choice, for example kube-dns.
You can use the standard Kubernetes service discovery using DNS for Services and Pods. Or like you mentioned you can use Consul as a service discovery tool, the added benefit of using something like Consul is that since it's not Kubernetes specific you could potentially also use for services outside your Kubernetes cluster or in other Kubernetes clusters.
Since sounds like your initial connections come and go, it sounds like that in order to who joins what channel and what channel talks to what backend you will need to keep state somewhere like a database, or key-value store.
I have some servers I want to deploy in Kubernetes. The clients of those servers will also be in Kubernetes. Clients and servers can independently be deployed or scaled.
The clients must know the list of the servers (IPs). I have an HTTP endpoint on the clients to update the list of the servers while the clients are running (hot config reload).
All this is currently running outside of Kubernetes. I want to migrate to GCP.
What's the industry standard regarding pods updates and notifications? I want to get notified when servers are updated to call the endpoints on the clients to update the list of the servers.
Can't use a LoadBalancer since the clients really need to call a specific server (business logic are in the clients).
Thanks
The standard for calling a group of pods that offer a functionality is services. If you don't want automated load-balancing or a single IP address, which regular services do, you should look into headless services. Calling headless services returns a list of DNS A records that point to the pods behind the service. This list is automatically updated as pods become available/unavailable.
While I think modifying an existing script to just pull a list from a headless is much simpler, it might be worth mentioning CRDs (Custom Resource Definitions) as well.
You could build a custom controller that listens to service events and then posts the data from that event to an HTTP endpoint of another Service or Ingress. The custom resource would define which service to watch and where to post the results.
Though, this is probably much heavier weight solution that just having a sidecar / separate container in a pod polling the service for changes (which sounds closer to you existing model).
I upvoted Alassane answer as I think it is the correct first path to something like this before building a CRD.
Recently I started working with Microservices, I wrote a library for service discovery using Redis to store every service's url and port number, along with a TTL value for the entry. It turned out to be an expensive approach since for every cross service call to any other service required one call to Redis. Caching didn't seem to be a good idea, since the services won't be up all the times, there can be possible downtimes as well.
So I wanted to write a separate microservice which could take care of the orchestration part. For this I need to figure out a really low level network protocol to take care of the exchange of heartbeats(which would help me figure out if any of the service instance goes unavailable). How do applications like zookeeperClient, redisClient take care of heartbeats?
Moreover what is the industry's preferred protocol for cross service calls?
I have been calling REST Api's over HTTP and eliminated every possibility of Joins across different collections.
Is there a better way to do this?
Thanks.
I think the term "Orchestration" is not good for what you are asking. From what I've encountered so far in microservices world the term "Orchestration" is used when a complex business process is involved and not for service discovery. What you need is a Service registry combined with a Load balancer. You can find here all the information you need. Here are some relevant extras that great article:
There are two main service discovery patterns: client‑side discovery and server‑side discovery. Let’s first look at client‑side discovery.
The Client‑Side Discovery Pattern
When using client‑side discovery, the client is responsible for determining the network locations of available service instances and load balancing requests across them. The client queries a service registry, which is a database of available service instances. The client then uses a load‑balancing algorithm to select one of the available service instances and makes a request.
The network location of a service instance is registered with the service registry when it starts up. It is removed from the service registry when the instance terminates. The service instance’s registration is typically refreshed periodically using a heartbeat mechanism.
Netflix OSS provides a great example of the client‑side discovery pattern. Netflix Eureka is a service registry. It provides a REST API for managing service‑instance registration and for querying available instances. Netflix Ribbon is an IPC client that works with Eureka to load balance requests across the available service instances. We will discuss Eureka in more depth later in this article.
The client‑side discovery pattern has a variety of benefits and drawbacks. This pattern is relatively straightforward and, except for the service registry, there are no other moving parts. Also, since the client knows about the available services instances, it can make intelligent, application‑specific load‑balancing decisions such as using hashing consistently. One significant drawback of this pattern is that it couples the client with the service registry. You must implement client‑side service discovery logic for each programming language and framework used by your service clients.
The Server‑Side Discovery Pattern
The client makes a request to a service via a load balancer. The load balancer queries the service registry and routes each request to an available service instance. As with client‑side discovery, service instances are registered and deregistered with the service registry.
The AWS Elastic Load Balancer (ELB) is an example of a server-side discovery router. An ELB is commonly used to load balance external traffic from the Internet. However, you can also use an ELB to load balance traffic that is internal to a virtual private cloud (VPC). A client makes requests (HTTP or TCP) via the ELB using its DNS name. The ELB load balances the traffic among a set of registered Elastic Compute Cloud (EC2) instances or EC2 Container Service (ECS) containers. There isn’t a separate service registry. Instead, EC2 instances and ECS containers are registered with the ELB itself.
HTTP servers and load balancers such as NGINX Plus and NGINX can also be used as a server-side discovery load balancer. For example, this blog post describes using Consul Template to dynamically reconfigure NGINX reverse proxying. Consul Template is a tool that periodically regenerates arbitrary configuration files from configuration data stored in the Consul service registry. It runs an arbitrary shell command whenever the files change. In the example described by the blog post, Consul Template generates an nginx.conf file, which configures the reverse proxying, and then runs a command that tells NGINX to reload the configuration. A more sophisticated implementation could dynamically reconfigure NGINX Plus using either its HTTP API or DNS.
Some deployment environments such as Kubernetes and Marathon run a proxy on each host in the cluster. The proxy plays the role of a server‑side discovery load balancer. In order to make a request to a service, a client routes the request via the proxy using the host’s IP address and the service’s assigned port. The proxy then transparently forwards the request to an available service instance running somewhere in the cluster.
The server‑side discovery pattern has several benefits and drawbacks. One great benefit of this pattern is that details of discovery are abstracted away from the client. Clients simply make requests to the load balancer. This eliminates the need to implement discovery logic for each programming language and framework used by your service clients. Also, as mentioned above, some deployment environments provide this functionality for free. This pattern also has some drawbacks, however. Unless the load balancer is provided by the deployment environment, it is yet another highly available system component that you need to set up and manage.
I have many services connected to zookeeper, and I want that service A can get service B's IP, when service B connected to zookeeper, is there any API can do that? Or I have to use other config file to write down all services's IP?
Take a look if this solves your problem:
http://curator.apache.org/curator-x-discovery/
Zookeeper doesn't provide service discovery out of the box, but it is easy to implement it yourself.
You won't be able to get the IP addresses of other connected clients (services, in your case) straight from the Zookeeper API. In order to get other services connected to the cluster, each service has to individually create an ephemeral znode under a specific path, e.g. /services, and set the necessary addressing and naming info as znode's data (IP, port, etc). This way, you can list that path and discover active services, or watch the /services path for any changes in your service configuration.
Since services are creating ephemeral nodes, they will automatically be removed once they are disconnected and their session expires. Of course, once you start doing something like this, you will see that there are many small details and design decisions you have to make, ergo the already mentioned Curator recipe.