I'm trying to create a server app in node.js, where multiple clients connect, and then one sends data, and that data gets send to another specific client. Which client it gets sent to is determined by a 'user id' that all clients will send after they connect.
How can I keep track of clients as they connect? How can I find my specific client? I realize this is a very broad question, but any pointers would be appreciated...
Thanks!
Have a look at some of the existing open source node servers like Socket.IO.
Socket.IO basically assigns each client a unique id. Ids are stored in a hash which is then used as a lookup to identify specific clients - you can create channels as well as broadcast to all connected clients.
Related
I am working on a personal project in which I want to be able to send one message from a producer to an end-user.
Each message will have a key that identifies the user that has to receive the message.
This is the overall structure I have imagined:
I cannot figure out how I can tell the load balancer that whenever a user with key 2 for example contacts the load balancer, then we have to set up a connection (possibly with a WebSocket) with the consumer handling partitions with key 2 in them. Probably something can be done by using the same technique Kafka uses whenever it has to assign a partition to the key, or by keeping track of the keys each consumer manages.
I do not know whether this is possible, but even if it was, the technique I described would probably make the code too coupled with the architecture.
Could you please help me out with how I can achieve this? I do not want to store messages on a remote data store and retrieve them from a random consumer. I want the consumer to be able to serve the user as soon as possible whenever a connection is established with it. If there is no connection with that user, then I can store the message and deliver it when the connection is ready.
I eventually found useful the Push Messaging technique they use at Netflix. The trick is to add another level of indirection, made of web servers. Whenever a new client connects to one of the web servers, a tuple <client_id, webserver_id> is saved in an external data store. When the consumer needs to send the message to the client having that specific key, it looks it up on the external registry to find where the client is connected. Once it finds it, it sends the message to the right web server that will push the message to the client.
I'm trying to build a entirely contained trading simulator using quickfix/J. The systems ought to consist of 2 client applications (a market/exchange and a broker) as well as a router (server/acceptor). In particular I'd like to know:
Client-Client communication
How the two clients can communicate to each other, but the server handling all the messaging logic, ie. messages should go through server and it should decide where and how to forward messages. I ought to be able to pass a targetID in FIX message, and the server app should handle routing to desired client.
Multiple clients on same port
Have multiple clients connected on same port but messages should only go to a particular sender comp Id ie. clients should not be privy of communication from other clients.
I've already set up the acceptor, and 2 clients. I know I could do this programmaticaly using plain old Java but I'd like to leverage the quickfix library and would like a relativly out of the box solution.
MVP: client (broker) sends fix message through the acceptor(router), message is processed and forwarded to a particular market, market recieves message through server and does some business logic, market sends fix message back to client through acceptor.
ps: I like the quickfix library but I'm very flexible if there any other library/languages you'd recommend
Short answer: QuickFIX/J (as far as I can tell similarly QuickFIX or quickfix/n) will not route messages based on tags. This has to be implemented in your application code.
Edit: with regard to your second point. There is no problem having your FIX server listening for multiple FIX connections on the same port (This applies for QuickFIX/J and I guess also the other language variants.) Sessions are addressed via the SessionID so it is ensured that only the correct FIX Session gets its messages.
I'm trying to build instant messaging app. Clients will not only send messages but also often send audios. And I've decided to use websocket connection to communicate with clients. It is fast and allows to send binary data.
The main idea is to receive from client1 message and notify about it client2. But here's the thing. My app will be running on GAE. And what if client1's socket is opened on server1 and client2's is opened on server2. This servers don't know about each others clients.
I have one idea how to solve it, but I am sure it is shitty way. I am going to use some sort of communication between servers(for example JMS or open another websocket connection between servers, doesn't matter right now).
But it surely will lead to a disaster. I can't even imagine how often those servers will speak to each other. For each message server1 should notify server2, server2 should notify client2. But things become even worse when serverN comes into play.
Another way I see this to work is Firebase. But it restricts message size to 4KB. So I can't send audios via it. As a solution I can notify client about new audio and he goes to my server for it.
Hope I clearly explained the problem. Does anyone know how to solve it? Or maybe there are another ways to build such apps?
If you are building a messaging cluster and expect communicating clients to connect to different instances of the server then server-server communication is inevitable. Usually it's not a problem though.
First, if you don't use any load balancing your clients will connect to the same server 50% of time on average (in case of 2 servers).
Second, intra-datacenter links are fast and free in all known public clouds.
Third, you can often do something smart on the frontend to make sure two likely to communicate clients connect to the same server. For instance direct all clients from the same country to the same server using DNS load balancing.
The second part of the question is about passing large media files. It's a common best practice to send it out of band - store on the server and only pass the reference to it. Like someone suggested in the comment, save the audio on the server and just send a message like "audio is available, fetch it from here ...". You don't need to poll the server for that. Just fetch it once when the receiving client requests it.
In general, it seems like you are trying to reinvent the wheel. Just use something off the shelf.
Let all client get connected to multiple servers and each server keeps this metadata
A centralized system like zookeeper stores active servers details
When a client c1 sends a message to client c2:
the message is received by a server (say s1, we can add a load balancer to distribute incoming requests)
s1 will broadcast this information to all other servers to get which server the client c2 is connected to OR a better approach to use consistent hashing which decides which server the client can connect to & in this approach message broadcast is not required
the corresponding server responses to server s1 (say s2)
now s1 sends the message m to s2 and server s2 to client c2
Cons of the above approach:
Each server will have a connection with the n-1 servers, creating a mesh topology
Centralized system (zookeeper) becomes a single point of failures (which is solvable)
Apps like Whatsapp, G-Talk uses XMPP and TCP/IP.
The challenge: We have a number of clients in distributed outposts that I have to manage with a central server. As some clients are located in DMZ or behind proxies, they should be connecting to the server!
As I only have to deal with one client at a time, the server doesn't necessarily have to be able to handle multiple clients simultaniously, however, I would like to see a list of the clients that are trying to connect to the server. Plus, I would like to see more information about the clients than just the IP address, for example the geographic location and some information, if the client has some files in a specific directory that the central server is interested in. My question is, how I best do smth like that.
Sure, I could simply show every client trying to connect in a listbox and accept only the one that I want to connect with, but is that really the way to go? I doubt I can get more information about the client than it's IP address?
I was wondering, if this calls for UDP. The clients send UDP datagrams that just inform the server that they are alive and that they want to connect. On the server, I see all these clients listed with the data they sent. I can then select one client, send an answer/"connection request" with UDP so that this particular client will connect via TCP to the server?
Is that possible?
This sounds like using a hammer to crack a nut. Just have them all connect via TCP. Then you get their presence, their IP address, anything else they care to send you. Deal with them all at once. It's not hard.
I'm currently looking into XMPP and I would like to know if there is a way to create dynamic XMPP rosters. I want the contact list of any user be automatically generated by the server / component / plugin.
Can components access and modify rosters?
I know that some servers (like OpenFire) use an external database to store these information but if I modify the database, I don't think the users will be notified in realtime.
Are some people already doing that or do you have an idea how to create it?
Thank you for your time.
In most XMPP servers this is called "shared roster groups".
I can't comment on Openfire in particular because I haven't used it for a long time, but I don't think I know of a server currently in which external roster changes are instantly transmitted to clients. Usually the client will simply pick up the new roster when it next logs in.
It would be possible to push the updates instantly to clients using what XMPP calls "roster pushes". This would require quite some work on the server though to identify what the changes are between what each client knows the roster to be, and what the new roster is, and then transmit just the changes.
If you really need this then a server plugin would probably be the way to go, or pester your server developers for the feature (I know that as a Prosody developer I've already been pestered, and this is something I'm planning to work on).
As for whether components can access users' rosters - this is dependent upon the server implementation and configuration.