This is my case:
I have 6 servers across US and Europe. All servers are on a load balancer. When you visit the website (www.example.com) its pointing on the load balancer IP address and from their you are redirect to one of the servers. Currently, if you visit the website from Germany for example, you are transfered randomly in one of the server. You could transfer to the Germany server or the server in San Fransisco.
I am looking for a way to redirect users to the nearest server based on their location but without changing url. So I am NOT looking of having many url's such as www.example.com, www.example.co.uk, www.example.dk etc
I am looking for something like a CDN where you retrieve your files from the nearest server (?) so I can get rid of the load balancer because if it crashes, the website does not respond (?)
For example:
If you are from uk, redirect to IP 53.235.xx.xxx
If you are from west us, redirect to IP ....
if you are from south europe, redirect to IP ... etc
DNSMadeeasy offers a feature similar to this but they are charging a 600 dollars upfront price and for a startup that doesnt know if that feature will work as expected or there is no trial version we cannot afford: http://www.dnsmadeeasy.com/enterprise-dns/global-traffic-director/
What is another way of doing this?
Also another question on the current setup. Even with 6 servers all connected to the load balancer, if the load balancer has lag issues, it takes everything with it, right? or if by any change it goes down, the website does not respond. So what is the best way to eliminate that downtime so that if one server IP address does not respond, move to the next (as a load balancer would do but load balancers can have issues themselves)
Would help to know what type of application servers you're talking about; i.e. J2EE (like JBoss/Tomcat), IIS, etc?
You can use a hardware or software load balancer with Sticky IP and define ranges of IPs to stick to different application servers. Each country's ISPs should have it's own block of IPs.
There's a list at the website below.
http://www.nirsoft.net/countryip/
Here's also a really, really good article on load balancing in general, with many high availability / persistence issues addressed. That should answer your second question on the single point of failure at your load balancer; there's many different techniques to provide both high availability and load distribution. Alot depends on what kind of application your run and whether you require persistent sessions or not. Load balancing by sticky IP, if persistence isn't required and you're LB does health checks properly, can provide high availability with easy failover. The downside is that load isn't evenly distributed, but it seems you're looking for distribution based on proximity, not on load.
http://1wt.eu/articles/2006_lb/index.html
Related
How it's possible that Cloudflare proxy has always US IP address or Google, Netflix and etc when i access on their websites from Asia, why i don't see some Asia IP (I expected the most close server to me and not in US).
Why i connect like first to US IP, ok, they will have some kind of load balancer server, but for the most low latency for TTFB why it's works like that or is the US IP address associated to some Asia server?? What is the trick?
I tried find some info how it's works but it's always same answer and doesn't explain anything for me..it's just saying that it's in power of DNS..I know about GEO DNS, but there is my end, but simple GEO DNS will always show different IP then is there another layer of something?
I'm just lost..
As far as I know, the "proxy" feature of Cloudflare will always display and "use" an US based IP-Address.
Your request is still sent to the nearest Cloudflare data center (somewhere in Asia, in your case).
But the "proxy" IP (which is static), is always the same.
Not entirely sure though, might as well be worth contacting their support team and asking for exact information on this.
I read this article about the API Gateway pattern. I realize that API Gateways typically serve as reverse proxies, but this forces a bottleneck situation. If all requests to an application's public services go through a single gateway, or even a single load balancer across multiple replicas of a gateway (perhaps a hardware load balancer which can handle large amounts of bandwidth more easily than an API gateway), then that single access point is the bottleneck.
I also understand that it is a wide bottleneck, as it simply has to deliver messages in proxy, as the gateways and load balancers themselves are not responsible for any processing or querying. However, imagining a very large application with many users, one would require extremely powerful hardware to not notice the massive bandwidth traveling over the gateway or load balancer, given that every request to every microservice exposed by the gateway travels through that single access point.
If the API gateway instead simply redirected the client to publicly exposed microservices (sort of like a custom DNS lookup), the hardware requirements would be much lower. This is because the messages traveling to and from the API Gateway would be very small, the requests consisting only of a microservice name, and the responses consisting only of the associated public IP address.
I recognize that this pattern would involve greater latency due to increased external requests. It would also be more difficult to secure, as every microservice is publicly exposed, rather than providing authentication at a single entrypoint. However, it would allow for bandwidth to be distributed much more evenly, and provide a much wider bottleneck, thus making the application much more scalable. Is this a valid strategy?
A DNS/Public IP based approach not good from a lot of perspectives:
Higher attack surface area as you have too many exposed points and each need to be protected
No. of public IPs needed is higher
You may need more DNS settings with subdomains or domains to be there for these APIs
A lot of times your APIs will run on a root path but you may want to expose them on a folder path example.com/service1, which requires
you to use some gateway for the same
Handling SSL certificates for these public exposures
Security on a focussed set of nodes vs securing every publically exposed service becomes a very difficult task
While theoretically it is possible to redirect clients directly to the nodes, there are a few pitfalls.
Security, Certificate and DNS management has been covered by #Tarun
Issues with High Availability
DNS's cache the domains vs IP's they serve fairly aggressively because these seldom change. If we use DNS to expose multiple instances of services publicly, and one of the servers goes down, or if we're doing a deployment, DNS's will continue routing the requests to the nodes which are down for a fair amount of time. We have no control over external DNS's and their policies.
Using reverse proxies, we avoid hitting those nodes based on health checks.
Assume I'm working on a multiplayer online game. Each group of players may start an instance of the game to play. Take League Of Legends as an example.
At any moment of time, there are many game matches being served at the same time. My question is about the architecture of this case. Here are my suggestions:
Assume we have a cloud with a gateway. Any game instance requires a game server behind this gateway to serve the game. For different clients outside the cloud to access different game servers in the cloud, the gateway may differentiate between connections according to ports. It is like we have one machine with many processes each of them listening on a different port.
Is this the best we can get?
Is there another way for the gateway to differentiate connections and forward them to different game instances?
Notice that these are socket connections NOT HTTP requests to an API gateway.
EDIT 1: This question is not about Load Balancing
The keyword is ports. Will each match be served on a different port? or is there another way to serve multiple services on the same host (host = IP)?
Elaboration: I'm using client-server model for each match instance. So multiple clients may connect to the same match server to participate in the same match. Each match need to be server by a match server.
The limitation in mind is: For one host (=IP) to serve multiple services it need to provide them on different ports. Match 1 on port 1234. So clients participating in match 1 will connect to and communicate with the match server on port 1234.
EDIT 2: Scalability is the target
My match server does not calculate and maintain the world of many matches. It maintains the world of one match. This is why each match need another instance of the match server. It is not scalable to have all clients communicating about different matches to connect to one process and to be processed by one process.
My idea is to serve the world of each match by different process. This will require each process to be listening on a different port.
Example: Any client will start a TCP connection with a server listening on port A. Is there is a way to serve multiple MatchServers on the same port A (so that more simultaneous MatchServers won't result in more ports)?
Is there a better scalable way to serve the different worlds of multiple matches?
Short answer: you probably shouldn't use proxy-gateway to handle user connections unless you are absolutely sure there's no other way - you are severely limiting your scaling ability.
Long answer:
What you've described is just a load balancing problem. You can find plenty of solutions based on given restrictions via google.
For League Of Legends it can be quite simple: using some health-check find server with lowest amount of load and stick (kinda like sticky sessions) current game to this server - until the game is finished any computations for particular game are made there. You could use any kind of caching mechanism to store game - server relation for subsequent requests on gateway side.
Another, a bit more complicated example could be data storage for statistics for particular game - it's usually solved via sharding which is a usual consequence of distributed computing. It could be solved this way: use some kind of hashing function (for example, modulo) with game ID as parameter to calculate server number. For example 18283 mod 15 = 13 for game ID = 18283 and 15 available shards - so 13th server should store/serve this data.
Main problem here would be "rebalancing" - adding/remove a shard from cluster, for example.
Those are just two examples, you can google more of them using appropriate keywords. Just keep in mind that all of this is just a subset of problems of distributed computing.
From https://developers.google.com/maps/documentation/staticmaps/#Limits I read:
The Google Static Maps API has the following usage limits:
25 000 free static map requests per application per day.
If I'm not providing an API key in the URL, how does it determine the limit? IP of the referring page? domain of the referring URL? IP of the client?
We used the static maps on our website, we've discovered it uses the IP address of the client. So someone who looked at our website a lot would find the big your quota has been exceeded image would appear for them but not for me.
I start by declaring that I do not know this, but the logical choice is the domain.
With ip restriction multiple clients on the same web server would consume each others quotas which they should have thought of.
*Client ip would be useless in every metric.
*Server ip would mean multiple clients on one host would consume each others quota.
Whats left is the domain. However with that said, Google is know to use their brains and I would not be surprised if they have a combination running to find abuse. Like so.
If domainA.com uses up 25 000 in one day and them immediately domainB.com comes online and starts asking for images from the same ip that might ring some bells.
Of course the same would be true even for different ip if they all request the same location.
So in summary, I think if you randomize which domain asks for the map at any given cient request and only locally mark a domain as spent (for the day) when you get error back I think you can request infinite amount (if you have infinite domains). With the possible caveat of detection if all request the same location.
Of course spreading the different domains over different servers/ip would make it impossible to detect, however unlikely it's needed.
There is no clarity in the pricing model and the usage limits that Google has posted for their web service API's but I guess the accepted answer is wrong and misleading - refer Two conflicting statements for google static map usage
The 25k usage limit will be for the application and not for its client.
I am using Haproxy with two different nodes having different machines 'geographically scattered'
Load-balancer-one having dns = http1.example.com
Load-balancer-two having dns = http2.example.com
The service is listening on DNS main site with original hostname --haproxy
My question is how to maintain a static URL? i.e. it must not show the back-end server domain's or IPs, I want to show only original hostname.
The simplest method is to setup a round robin DNS entry that returns the IP addresses of both servers.
You likely however want to use a GSLB (global server load balancing) solution that can remove failed load balancers from responses based on a health check. If you are in multiple data centers, some GSLB solutions can route users to the most performant location for them.
F5 and Netscaler have hardware GSLB solutions. Dyn, Akamai, UltraDNS and others offer GSLB as a service. AWS' Route53 has a weighted round robin solution. They do not currently offer health checking or routing based on geographic location.