Currently in our project, we have few Stateless and Stateful services and then we have an API (which is again a Stateless service). Our API service is exposed over http and run of frontEnd Nodes in cluster. Any client from outside hits the WebAPi stateless service, which inturn can call other services via SF remoting. But other services are not exposed over HTTP. and individual services also can call each other via SF remoting.
As a part of a new requirement, there are some other services hosted in other cloud (Openshift) needs to access our Stateless and Stateful services directly (i.e. without the WebAPI service) over REST. I understand that we can expose our Stateless and Stateful services over http, by writing our own custom HttpCommunicationListener (which should implement "ICommunicationListener"). But apart from this I guess, we would need to configure some reverse proxy and Load Balancer stuff etc, to ensure that one URL works for all the requests.
Is this something, which can be achieved. If yes, can somebody points me to any documentation or code sample ?
I recommend having a look at Traefik as a reverse proxy and load balancer.
You can run it as a (containerized) ingress routing service inside the cluster, and direct HTTP calls to your services.
Here's the documentation.
Here's how to get started.
Here's an example.
Related
why k8s uses RestAPI in NodeStats/PodStats summary, but uses gRPC in CRI( RunPodSandbox/CreateContainer/StartContainer)?
why doesn't k8s use gRPC in the whole project?
Thank you.
Without knowing anything about the internals of k8s, it's reasonable to assume that it would serve generic REST endpoints for services it provides to unknown clients, but would be a gRPC client for other internal k8s services, and possibly serve gRPC to other internal k8s services.
gRPC is certainly more efficient to use for cooperating services, but it would be quite awkward for a random client not fully integrated with k8s to form a gRPC client call, so it uses a simplified REST interface for those external clients.
I have a Service Fabric Application which consists of two services. Both are stateless services and have a single instance of the services.
In Service A, I would like to define a Endpoint in the Resources section of the ServiceManifest.xml. I don't care what port I get, I just need one and I'd like to get it from Service Fabric so it can ensure it's accessible between the vms managed by the Service Fabric cluster.
In Service B, I'd like to pass the port created for Service A so it can use it to interact with the Service A. I will be defining both services with Service Fabric DNS names, so Service B will know the host of Service A (regardless of where it's running). But Service B also needs to the port that was created for Service A (via it's Endpoint declaration). Is that information passed to the Services? Can it be passed as a parameter, or is there another mechanism.
Thanks for any help
You can discover information about the endpoints of the other service by using the QueryManager on the FabricClient. Example here.
Out of curiosity, can't you use SF remoting for this?
Say you are using Microservices with Docker Containers and Kubernetes.
If you use an API Gateway (e.g. Azure API Gateway) in front of your microservices to handle composite UI and authentication, do you still need a Service Mesh to handle Service Discovery and Circuit Breaker? Is there any functionality in Azure API Gateway to handle these kind of challenges? How?
API gateways are applied on Layer 7 of OSI model or you can say to manage traffic coming from outside network ( sometimes also called north/south traffic ) , whereas Service Mesh is applied to Layer 4 of OSI model or to manager inter-services communications ( sometimes also called as east/west traffic). Some examples of API Gateway features are Reverse Proxy,Load Balancing , Authentication and Authorization , IP Listing , Rate-Limiting etc.
Service Mesh, on the other hand, works like a proxy or a side-car pattern which de-couples the communication responsibility of the service and handles other concerns such as Circuit breaker , timeouts , retries , service-discovery etc.
If you happen to use Kubernetes and Microservices then you might want to explore other solutions such as Ambassador + Istio Or Kong which works as Gateway as well as Service Mesh.
An API Gateway only handles the entry point into your Kubernetes clusters, e.g. it sends a request to your frontend microservice. However, it can do nothing after the request enters your cluster. There might still be multiple calls between microservices. You still want to verify authentication for those requests, you still want to make sure that there are circuit breakers in between the services, etc. Theoretically, you could make sure all your microservices call each other via the API gateway, however I do not think that is what you want.
In short: No, because an API Gateway is only an entry point, any service to service communication is better handled with a Service Mesh.
you can use an API gateway to handle service discovery and circuit breaker - but that would make it a central point in your deployment i.e. all calls external and internal will have to be routed via the gateway.
A service mesh deploy an additional edge component ("sidecar") alongside each service making the overall behavior distributed (but also more complex)
Depending on your particular requirements you may use one, the other, both or none
Nicely explained by fatcook above.. See Azure-Frontdoor
as this is attempting to do the same as Kong on Azure. API gateway + handling control plane level features
Recently I started working with Microservices, I wrote a library for service discovery using Redis to store every service's url and port number, along with a TTL value for the entry. It turned out to be an expensive approach since for every cross service call to any other service required one call to Redis. Caching didn't seem to be a good idea, since the services won't be up all the times, there can be possible downtimes as well.
So I wanted to write a separate microservice which could take care of the orchestration part. For this I need to figure out a really low level network protocol to take care of the exchange of heartbeats(which would help me figure out if any of the service instance goes unavailable). How do applications like zookeeperClient, redisClient take care of heartbeats?
Moreover what is the industry's preferred protocol for cross service calls?
I have been calling REST Api's over HTTP and eliminated every possibility of Joins across different collections.
Is there a better way to do this?
Thanks.
I think the term "Orchestration" is not good for what you are asking. From what I've encountered so far in microservices world the term "Orchestration" is used when a complex business process is involved and not for service discovery. What you need is a Service registry combined with a Load balancer. You can find here all the information you need. Here are some relevant extras that great article:
There are two main service discovery patterns: client‑side discovery and server‑side discovery. Let’s first look at client‑side discovery.
The Client‑Side Discovery Pattern
When using client‑side discovery, the client is responsible for determining the network locations of available service instances and load balancing requests across them. The client queries a service registry, which is a database of available service instances. The client then uses a load‑balancing algorithm to select one of the available service instances and makes a request.
The network location of a service instance is registered with the service registry when it starts up. It is removed from the service registry when the instance terminates. The service instance’s registration is typically refreshed periodically using a heartbeat mechanism.
Netflix OSS provides a great example of the client‑side discovery pattern. Netflix Eureka is a service registry. It provides a REST API for managing service‑instance registration and for querying available instances. Netflix Ribbon is an IPC client that works with Eureka to load balance requests across the available service instances. We will discuss Eureka in more depth later in this article.
The client‑side discovery pattern has a variety of benefits and drawbacks. This pattern is relatively straightforward and, except for the service registry, there are no other moving parts. Also, since the client knows about the available services instances, it can make intelligent, application‑specific load‑balancing decisions such as using hashing consistently. One significant drawback of this pattern is that it couples the client with the service registry. You must implement client‑side service discovery logic for each programming language and framework used by your service clients.
The Server‑Side Discovery Pattern
The client makes a request to a service via a load balancer. The load balancer queries the service registry and routes each request to an available service instance. As with client‑side discovery, service instances are registered and deregistered with the service registry.
The AWS Elastic Load Balancer (ELB) is an example of a server-side discovery router. An ELB is commonly used to load balance external traffic from the Internet. However, you can also use an ELB to load balance traffic that is internal to a virtual private cloud (VPC). A client makes requests (HTTP or TCP) via the ELB using its DNS name. The ELB load balances the traffic among a set of registered Elastic Compute Cloud (EC2) instances or EC2 Container Service (ECS) containers. There isn’t a separate service registry. Instead, EC2 instances and ECS containers are registered with the ELB itself.
HTTP servers and load balancers such as NGINX Plus and NGINX can also be used as a server-side discovery load balancer. For example, this blog post describes using Consul Template to dynamically reconfigure NGINX reverse proxying. Consul Template is a tool that periodically regenerates arbitrary configuration files from configuration data stored in the Consul service registry. It runs an arbitrary shell command whenever the files change. In the example described by the blog post, Consul Template generates an nginx.conf file, which configures the reverse proxying, and then runs a command that tells NGINX to reload the configuration. A more sophisticated implementation could dynamically reconfigure NGINX Plus using either its HTTP API or DNS.
Some deployment environments such as Kubernetes and Marathon run a proxy on each host in the cluster. The proxy plays the role of a server‑side discovery load balancer. In order to make a request to a service, a client routes the request via the proxy using the host’s IP address and the service’s assigned port. The proxy then transparently forwards the request to an available service instance running somewhere in the cluster.
The server‑side discovery pattern has several benefits and drawbacks. One great benefit of this pattern is that details of discovery are abstracted away from the client. Clients simply make requests to the load balancer. This eliminates the need to implement discovery logic for each programming language and framework used by your service clients. Also, as mentioned above, some deployment environments provide this functionality for free. This pattern also has some drawbacks, however. Unless the load balancer is provided by the deployment environment, it is yet another highly available system component that you need to set up and manage.
I have a scenario where one of our services exposes WCF hosts that receive callbacks from an external service.
These hosts are dynamically created and there may be hundreds of them. I need to ensure that they are all up and running on the node before the node starts receiving requests so they don't receive failures, this is critical.
Is there a way to ensure that the service doesn't receive requests until I say it's ready? In cloud services I could do this by containing all this code within the OnStart method.
My initial thought is that I might be able to bootstrap this before I open the communication listener - in the hope that the fabric manager only sends requests once this has been done, but I can't find any information on how this lifetime is handled.
There's no "fabric manager" that controls network traffic between your services within the cluster. If your service is up, clients or other services inside the cluster can choose to try to connect to it if they know its address. With that in mind, there are two things you have control over here:
The first is whether or not your service's endpoint is discoverable by other services or clients. This is the point at which your service endpoint is registered with Service Fabric's Naming Service, which occurs when your ICommunicationListener.OpenAsync method returns. At that point, the service endpoint is registered and others can discover it and attempt to connect to it. Of course you don't have to use the Naming Service or the ICommunicationListener pattern if you don't want to; your service can open up an endpoint whenever it feels like it, but if you don't register it with the Naming Service, you'll have to come up with your own service discovery mechanism.
The second is whether or not the node on which your service is running is receiving traffic from the Azure Load Balancer (or any load balancer if you're not hosting in Azure). This has less to do with Service Fabric and more to do with the load balancer itself. In Azure, you can use a load balancer probe to determine whether or not traffic should be sent to nodes.
EDIT:
I added some info about the Azure Load Balancer to our documentation, hope this helps: https://azure.microsoft.com/en-us/documentation/articles/service-fabric-connect-and-communicate-with-services/