I am developing a chat back-end application on aws cloud. In order to make a scalable architecture for the chat back-end I must ensure that the one who is opening a connection is the real one.
To be more accurate that chat ofcourse must keep a tcp connection open with the server all the time and I have the following problems:
1 - the back-end has a load balancer elastic load balancer.
2 - the tcp connection between the client app and the back-end server must stay open and alive. which mean the app must keep the connection alive with the server not the elb load balancer.
3 - the elb load balancer must send the connection and load through a session table sticky session to the same server the app connected to before.
unfortunately, the load balancer only support l4 and l7 layers and I think I need to use the l3 layer.
the main problem here is most people operate behind proxy server so I can't maintain a connection with them because the tcp connection will be made with the proxy and not their app.
I don't know how to solve this but the only solution that I know now is:
I must prevent the users from operating behind any proxy servers to make sure the tcp connection is direct with them not the proxy, how do I do that?
If there is a way to let them operate behind a proxy and a solution can be made on the back-end tell me.
I'm not sure I understand your concern. If you are using web sockets, most proxies would allow this type of communication but they can cause you troubles as well if they have timeouts and such.
You cannot control whether someone is behind a proxy. In many cases the proxy will be completely transparent so you'd have no way to know it is there without inspecting all of the network hops. You may want to read up further on this. A good start is this article -
https://www.infoq.com/articles/Web-Sockets-Proxy-Servers
If you are attempting to use the IP address as an authentication mechanism, I suggest instead using a standard authentication mechanism. Once authenticated, you should manage the session using either session cookies, JWT, or another standard session management solution. Note that JWT is typically stateless (doesn't use a session) but can be used to authorize a user to session type data.
Related
So let's say I have the following (obviously simplified) architecture: I have 10 servers running the same REST API endpoint. I have an intermediate API which fields requests, and then forwards it to one of the servers (a load balancer).
Now let's imagine that this is a really big, streaming response. As such, I obviously don't want the data to have to go back through the load balancer -- because wouldn't this bog down and defeat the purpose of the load balancing server?. What would be the proper way to implement a load balancing system that would delegate a request to node but not force the response back through the load balancing server?
Further, are there any REST frameworks on the JVM that implement this?
What you are looking for is called DSR (direct server return). you can attempt to google it a bit. AFAIK most hardware load balancers have this option.
The question is what load balancer are you using? Is it hardware, ELB on AWS, HAProxy?
For example:
http://blog.haproxy.com/2011/07/29/layer-4-load-balancing-direct-server-return-mode/
If you're not really into load balancers, you could attempt to set this up in 2 stages: first - the client hits the API and gets the ip of the server, second the client talks to the servers. The hard part will be not to overload some servers when leaving others idle (both initial setup and rebalancing workloads as time goes by)
The server consists of several services with which a user interacts: profiles, game logics, physics.
I heard that it's a bad practice to have multiple client connections to the same server.
I'm not sure whether I will use UDP or TCP.
The services are realtime, they should reply as fast as possible so I don't want to include any additional rerouting if there are no really important reasons. So are there any reasons to rerote traffic through one external endpoint service to specific internal services in my case?
This seems to be multiple questions in one package. I will try to answer the ones I can identify as separate...
UDP vs TCP: You're saying "real-time", this usually means UDP is the right choice. However, that means having to deal with lost packets and possible re-ordering of packets. But, using UDP leaves a couple of possible delay-decreasing tricks open.
Multiple connections from a single client to a single server: This consumes resources (end-points, as it were) on both the client (probably ignorable) and on the server (possibly a problem, possibly ignorable). The advantage of using separate connections for separate concerns (profiles, physics, ...) is that when you need to separate these onto separate servers (or server farms), you don't need to update the clients, they just need to connect to other end-points, using code that's already tested.
"Re-router" (or "load balancer") needed: Probably not going to be an issue initially. However, it will probably become an issue later. Depending on your overall design and server OS, using UDP may actually become an asset here. UDP packet arrives at the load balancer, dispatched to the right backend and that could then in theory send back a reply with the source IP of the load balancer.
An alternative would be to have a "session broker". The client makes an initial connection to a well-known endpoint, says "I am a client, tell me where my profile, physics, what-have0-you servers are", the broker considers the current load, possibly the location of the client and other things that may make sense and the client then connects to the relevant backends on its own. The downside of this is that it's harder (not impossible, but harder) to silently migrate an ongoing session to a new backend, when there's a load-balancer in the way, this can be done essentially-transparently.
I'm using third party service that has own notion of session and expects all requests in session to come from same IP. They claim that it's a required security measure and suggest to use proxy, LOL.
Is there any way to use same IP for several requests with Socket API?
Interval between requests in session is ~10 seconds, so keeping connection alive and reusing it should work. I've tried to setup proxy module that runs single instance and uses HttpClient with connection pool. Logging shows that after first request connection is properly released and stored in pool. While doing second request I see that pool has 1 connection, but HttpClient says that there in no free connection for route and opens new. Probably route changes somehow?
It goes against the way App Engine is meant to work, scale-ability. Spawning instances closets to the consumers and multiple instance will mean different IP's. If you want a static IP you will need to switch to Compute Engine. Its a server VM that can have a static IP address. Or if your 3rd party service can take range you get get them from the link.
App Engine does not currently provide a way to map static IP addresses to an application. In order to optimize the network path between an end user and an App Engine application, end users on different ISPs or geographic locations might use different IP addresses to access the same App Engine application. DNS might return different IP addresses to access App Engine over time or from different network locations.
Actually, I solved this issue. Solution that I described in question was missing one step. As my connection was SSL authenticated, I had to use same context for all requests that I make.
I have a perl web application (CGI::Application with ModPerl::Registry) which connects to a authenticated custom server over a socket and exchanges data (command/response) with it. Currently the web application connects to the server, authenticates and disconnects on every page request - even for the same user.
Is there some way I can use the same socket over multiple page requests which share a common session id? Creating a separate daemon that proxies connections and makes them persistent is an option I am exploring, but would like to know if there are any simpler solutions.
I have no control over the design of the custom server unfortunately.
Looks like the same question was asked on PerlMonks. The responses there point in the right direction, but the issue seems to be that you want one cached connection per session, not one cached connection per session per httpd thread/process. You might have to resort to a separate proxy process to get the behaviour you want.
I apologize for the weird question wording... here's the design problem:
I am developing a server (on Linux using C++, FWIW) that provides a service to many instances of a client application running on consumer PCs.
I want the following:
1) All clients first identify themselves to a "gatekeeper" server application. Consider this a login procedure, with credentials like a user name and password being passed in. Call the gatekeeper program "gserver". (for gatekeeper.)
2) Once each client has been validated, it is then placed into a long term connection with one of several instances of a different server application running on the same physical server box bound to the same server address. Call any of these instances "wserver" (for "working" server.)
So, what the client sees is that a "gatekeeper" application gives it passworded access to one of several "working" servers running on the same box.
Here is the "real" challenge: we want to exclusively use a "well known" port number for the inbound server connections (like port 80 or 443, say.) Or, our own "well known" port.
We would prefer not to have to make the client talk to a second port on the server for the long term connection phase with wserver(n). The problem with this, of course, is that only one server process at a time can be bound to the same port and server address.
This implies that a connection made by the client with gserver must also fill the role of the long term connection. The only way I see to accomplish this is that gserver must, after login, act like a proxy and copy traffic between itself and the client to the particular wserver(n) that the client is bound to logically.
It would be ideal if a TCP/IP connection first made between client(n) and gserver could be somehow "transported" to another application on the same server, intact, and could then be sustained by one of the wserver(n) instances for the long term connection.
I know that web servers do something like this for spreading out server loads. "Load balancing". The main difference here is that the "balancing" is the allocation of a particular user to a particular wserver(n) instance. But I also have the impression that load balancing is a kind of proxying - which I am trying to avoid (since it complicates the architecture and adds overhead as well as a single point of failure.)
This is a conceptual and design question. Don't worry about source code examples, unless they are absolutely essential to get the ideas across. If we pin down an approach, I can code it up.
Thanks!
What you are looking for is file descriptor passing. See UNP 15.7. One well-known heavy user of this facility is postfix.
I developed such an application long time ago. Since multiple servers can't listen on the same port. What you need is to have gserver listening on the well-known port. Once connection is established, pass the connection to the other servers via an Unix socket. Once the connection is passed to other server, gserver is out of picture. It can die and the other server will be still serving the connection.
I dont' know if this applies to your design, but the usual solution (as implemmented by the xinetd daemon) is to fork() and then exec() the process. For example, xinetd may serve services like rlogin, rsh, tftp, telnet, etc. which are actually served by different programs. This will not be useful to you if your wservers are processes already running in the system.