I'm wondering, is it possible to have many servers with the same web service deployed and then put them to communicate together?
Can you make a distributed system (having transparency, being failsafe and such things)
on top of web services, instead of TCP?
Is this a poor idea?
This is very possible to do. You can either purchase a hardware load balancer that will direct traffic to one of its configured host or, if you are on Windows, you can use Network Load Balancing.
There are many hardware load balancers on the market, the one I have used and have been pleased with is made by Coyote Point: http://www.coyotepoint.com/
It's not a poor idea, and is a commonly used to distribute traffic.
Related
I am new to Kubernetes. I am planning to build/deploy an application to EKS. This will likely be deployed on azure and gcp as well.
I want to separate data plane and control plane in my application deployed in EKS.
Is there anyway EKS/kubernetes allows to accomplish this?
Or should we go for two EKS with one for data plane and another for control plane?
Here is the problem(copied from the answer below)
I have an application, built using the microservice architecture
(meaning you will have it split into components that will communicate
with eachother).
I want to deploy this application on a public cloud (EKS, GCP, AWS).
I want a separation of the APPLICATION control plane (decision making
components like authentication APIs, internal service routing) from
the APPLICATION data plane (the serving of your application data to
the clients through the egress).
My Understanding
What I understand from your description is:
You have an application, built using the microservice architecture (meaning you will have it split into components that will communicate with eachother).
You want to deploy this application on a public cloud (EKS, GCP, AWS).
You want a separation of the APPLICATION control plane (decision making components like authentication APIs, internal service routing) from the APPLICATION data plane (the serving of your application data to the clients through the egress).
If those 3 points above are true, then the answer is yes, every cloud platform has these capabilities. You will need to architect your application in a manner that your control microservices are isolated from your data microservices. There are many ways in which you can do it (most/all of them are available in all public clouds).
Some Design Ideas
Here is a conceptual description:
By using authentication and authorization mechanisms to ensure authorized communication between control and data plane applications. Think kubernetes ServiceAccounts.
By using network policies to restrict unauthorized traffic flow between microservices. You will require a networking overlay that supports these. Think calico CNI.
By using separate namespaces for your application services as necessary for better isolation.
At one level below, in your cloud, you can limit the traffic using security groups. Or even gateways.
You can use different instance types that match your various workload types, this will not only ensure optimal performance but also separation of failure domains. For example, if a dedicated database instance crashed, the event streaming service will still be running.
Some Notes
Also understand that in a public cloud solution (even EKS) a cluster->cluster traffic is more expensive for you than the traffic inside a single cluster. This will be a very important cost factor and you should consider using a single cluster for your application. (k8s clusters can typically scale to 1000s of nodes).
I hope this somewhat answers your question. There are a lot of decisions you need to make but in short, yes, it is possible to do this separation, and your entire application will have to be designed in this way.
I great open-source observability control plane for you apps is Odigos. The installation is super easy and within a few minutes you can get traces, metrics and logs. You get auto-instrumentation for all languages (including GO) as well as a manager of your opentelemetry collectors.
Check it out: https://github.com/keyval-dev/odigos
I read this article about the API Gateway pattern. I realize that API Gateways typically serve as reverse proxies, but this forces a bottleneck situation. If all requests to an application's public services go through a single gateway, or even a single load balancer across multiple replicas of a gateway (perhaps a hardware load balancer which can handle large amounts of bandwidth more easily than an API gateway), then that single access point is the bottleneck.
I also understand that it is a wide bottleneck, as it simply has to deliver messages in proxy, as the gateways and load balancers themselves are not responsible for any processing or querying. However, imagining a very large application with many users, one would require extremely powerful hardware to not notice the massive bandwidth traveling over the gateway or load balancer, given that every request to every microservice exposed by the gateway travels through that single access point.
If the API gateway instead simply redirected the client to publicly exposed microservices (sort of like a custom DNS lookup), the hardware requirements would be much lower. This is because the messages traveling to and from the API Gateway would be very small, the requests consisting only of a microservice name, and the responses consisting only of the associated public IP address.
I recognize that this pattern would involve greater latency due to increased external requests. It would also be more difficult to secure, as every microservice is publicly exposed, rather than providing authentication at a single entrypoint. However, it would allow for bandwidth to be distributed much more evenly, and provide a much wider bottleneck, thus making the application much more scalable. Is this a valid strategy?
A DNS/Public IP based approach not good from a lot of perspectives:
Higher attack surface area as you have too many exposed points and each need to be protected
No. of public IPs needed is higher
You may need more DNS settings with subdomains or domains to be there for these APIs
A lot of times your APIs will run on a root path but you may want to expose them on a folder path example.com/service1, which requires
you to use some gateway for the same
Handling SSL certificates for these public exposures
Security on a focussed set of nodes vs securing every publically exposed service becomes a very difficult task
While theoretically it is possible to redirect clients directly to the nodes, there are a few pitfalls.
Security, Certificate and DNS management has been covered by #Tarun
Issues with High Availability
DNS's cache the domains vs IP's they serve fairly aggressively because these seldom change. If we use DNS to expose multiple instances of services publicly, and one of the servers goes down, or if we're doing a deployment, DNS's will continue routing the requests to the nodes which are down for a fair amount of time. We have no control over external DNS's and their policies.
Using reverse proxies, we avoid hitting those nodes based on health checks.
I m studying Microservices architecture and I m actually wondering something.
I m quite okay with the fact of using (back) service discovery to make request able on REST based microservices. I need to know where's the service (or at least the front of the server cluster) to make requests. So it make sense to be able to discover an ip:port in that case.
But I was wondering what could be the aim of using service registry / discovery when dealing with AMQP (based only, without HTTP possible calls) ?
I mean, using AMQP is just like "I need that, and I expect somebody to answer me", I dont have to know who's the server that sent me back the response.
So what is the aim of using service registry / discovery with AMQP based microservice ?
Thanks for your help
AMQP (any MOM, actually) provides a way for processes to communicate without having to mind about actual IP addresses, communication security, routing, among other concerns. That does not necessarily means that any process can trust or even has any information about the processes it communicates with.
Message queues do solve half of the process: how to reach the remote service. But they do not solve the other half: which service is the right one for me. In other words, which service:
has the resources I need
can be trusted (is hosted on a reliable server, has a satisfactory service implementation, is located in a country where the local laws are compatible with your requirements, etc)
charges what you want to pay (although people rarely discuss cost when it comes to microservices)
will be there during the whole time window needed to process your service -- keep in mind that servers are becoming more and more volatile. Some servers are actually containers that can last for a couple minutes.
Those two problems are almost linearly independent. To solve the second kind of problems, you have resource brokers in Grid computing. There is also resource allocation in order to make sure that the last item above is correctly managed.
There are some alternative strategies such as multicasting the intention to use a service and waiting for replies with offers. You may have reverse auction in such a case, for instance.
In short, the rule of thumb is that if you do not have an a priori knowledge about which service you are going to use (hardcoded or in some configuration file), your agent will have to negotiate, which includes dynamic service discovery.
So let's say I have the following (obviously simplified) architecture: I have 10 servers running the same REST API endpoint. I have an intermediate API which fields requests, and then forwards it to one of the servers (a load balancer).
Now let's imagine that this is a really big, streaming response. As such, I obviously don't want the data to have to go back through the load balancer -- because wouldn't this bog down and defeat the purpose of the load balancing server?. What would be the proper way to implement a load balancing system that would delegate a request to node but not force the response back through the load balancing server?
Further, are there any REST frameworks on the JVM that implement this?
What you are looking for is called DSR (direct server return). you can attempt to google it a bit. AFAIK most hardware load balancers have this option.
The question is what load balancer are you using? Is it hardware, ELB on AWS, HAProxy?
For example:
http://blog.haproxy.com/2011/07/29/layer-4-load-balancing-direct-server-return-mode/
If you're not really into load balancers, you could attempt to set this up in 2 stages: first - the client hits the API and gets the ip of the server, second the client talks to the servers. The hard part will be not to overload some servers when leaving others idle (both initial setup and rebalancing workloads as time goes by)
This is my case:
I have 6 servers across US and Europe. All servers are on a load balancer. When you visit the website (www.example.com) its pointing on the load balancer IP address and from their you are redirect to one of the servers. Currently, if you visit the website from Germany for example, you are transfered randomly in one of the server. You could transfer to the Germany server or the server in San Fransisco.
I am looking for a way to redirect users to the nearest server based on their location but without changing url. So I am NOT looking of having many url's such as www.example.com, www.example.co.uk, www.example.dk etc
I am looking for something like a CDN where you retrieve your files from the nearest server (?) so I can get rid of the load balancer because if it crashes, the website does not respond (?)
For example:
If you are from uk, redirect to IP 53.235.xx.xxx
If you are from west us, redirect to IP ....
if you are from south europe, redirect to IP ... etc
DNSMadeeasy offers a feature similar to this but they are charging a 600 dollars upfront price and for a startup that doesnt know if that feature will work as expected or there is no trial version we cannot afford: http://www.dnsmadeeasy.com/enterprise-dns/global-traffic-director/
What is another way of doing this?
Also another question on the current setup. Even with 6 servers all connected to the load balancer, if the load balancer has lag issues, it takes everything with it, right? or if by any change it goes down, the website does not respond. So what is the best way to eliminate that downtime so that if one server IP address does not respond, move to the next (as a load balancer would do but load balancers can have issues themselves)
Would help to know what type of application servers you're talking about; i.e. J2EE (like JBoss/Tomcat), IIS, etc?
You can use a hardware or software load balancer with Sticky IP and define ranges of IPs to stick to different application servers. Each country's ISPs should have it's own block of IPs.
There's a list at the website below.
http://www.nirsoft.net/countryip/
Here's also a really, really good article on load balancing in general, with many high availability / persistence issues addressed. That should answer your second question on the single point of failure at your load balancer; there's many different techniques to provide both high availability and load distribution. Alot depends on what kind of application your run and whether you require persistent sessions or not. Load balancing by sticky IP, if persistence isn't required and you're LB does health checks properly, can provide high availability with easy failover. The downside is that load isn't evenly distributed, but it seems you're looking for distribution based on proximity, not on load.
http://1wt.eu/articles/2006_lb/index.html