I'm very new to Node.JS and asynchronous programming and have a challenging question. I want to fork a process from Node and then shoot that output back to the browser with Websockets, specifically the Sockets.io library. What is the best and most robust way to handle this?
The data isn't mission critical, it's just for updating the user on status. So if they leave the page, the socket can close and the child process can continue to run. It'd also be neat if there was some way to access the socket via a specific URL in Express and come back to it later (but that may be another days work).
Use the Redis Store support of socket.io:
var RedisStore = require('socket.io').RedisStore;
var io = require('socket.io').listen(app);
io.set('store', new RedisStore());
The socket.io library use redis server to storage the data and the events.
Related
I'm new to Akka and Scala and self learning this to do a small project with websockets. End goal is simple, make a basic chat server that publishes + subscribes messages on some webpage.
In fact, after perusing their docs, I already found the pages that are relevant to my goal, namely this and this.
Using dynamic junctions (aka MergeHub & BroadcastHub), and the Flow.fromSinkAndSource() method, I was able to acheive a very basic example of what I wanted. We can even get a kill switch using the example from the akka docs which I have shown below. Code is like:
private lazy val connHub: Flow[Message, Message, UniqueKillSwitch] = {
val (sink, source) = MergeHub.source[Message].toMat(BroadcastHub.sink[Message])(Keep.both).run()
Flow.fromSinkAndSourceCoupled(sink, source).joinMat(KillSwitches.singleBidi[Message, Message])(Keep.right)
}
However, I now see one issue. The above will return a Flow that will be used by Akka's websocket directive: akka.http.scaladsl.server.Directives.handleWebSocketMessages(/* FLOW GOES HERE */)
That means the akka code itself will materialize this flow for me so long as I provide it the handler.
But let's say I wanted to arbitrarily kill one user's connection through a KillSwitch (maybe because their session has expired on my application). While a user's websocket would be added through the above handler, since my code would not be explicitly materializing that flow, I won't get access to a KillSwitch. Therefore, I can't kill the connection, only the user can when they leave the webpage.
It's strange to me that the docs would mention the kill switch method without showing how I would get one using the websocket api.
Can anyone suggest a solution as to how I could obtain the kill switch per connection? Do I have a fundamental misunderstanding of how this should work?
Thanks in advance.
I'm very happy to say that after a lot of time, research, and coding, I have an answer for this question. In order to do this, I had to post in the Akka Gitter as well as the Lightbend discussion forum. Please refer to the amazing answer I got there for some perspective on the problem and some solutions. I'll summarize that here.
In order to get the UniqueKillSwitch from the code that I was using, I needed to use the mapMaterializeValue() method on the Flow that I was returning. Here is the code that I'm using to return a Flow to the handleWebSocketMessages directive now:
// note - state will not be updated properly if cancellation events come through from the client side as user->killswitch mapping may still remain in concurrent map even if the connections are closed
Flow.fromSinkAndSourceCoupled(mergeHubSink, broadcastHubSource)
.joinMat(KillSwitches.singleBidi[Message, Message])(Keep.right)
.mapMaterializedValue { killSwitch =>
connections.put(user, killSwitch) // add kill switch in side effect once value is ready from materialization
NotUsed.notUsed()
}
The above code lives in a Chatroom class I've created that has access to the mergehub and broadcast hub materialized sink and source. It also has access to a concurrent hashmap that persists the kill switch to a user. In this way, we now have access to the Kill Switch through querying it through a map. From there, you can call switch.shutdown() to kill the user's connection from the server side.
My main issue was that I originally thought I could get the switch directly even though I didn't control the materialization. This doesn't seem possible. I suggest this method for when you know that the caller that requires your Flow doesn't care about the materialized value (aka the kill switch).
Please reference the answer I've linked for more scenarios and ways to handle this problem.
I have an application (Laravel + MongoDB running on Nginx) where I pull some data from the database and render it on the screen. The application focusses on multiple real life objects. Once an object is turned on (is_on equals to true in the database), the timer on the screen needs to start ticking. Once the object is turned off (is_on equals to false in the database) the clock stops ticking and resets to 0. The format of the clock is HH:MM:SS. So it shows how long the real life object is turned on.
My problem is that I don't really now how to save/implement such timer. When the user request the page, I pull the necessary data from the database. If I also save the timer in the database, you have to make a query every second which is very bad practice.
I remembered something about WebSockets and tried to look into them. I actually managed to build a basic Hello World chat application, but don't really know how to implement this in my project. There is no place for it in the database (because of the queries), so I don't really know where to save that timer on the server. I'm also doubting if WebSockets are the way to go.
So are WebSockets the way to go and if it is, can you guys point me in the right direction on how to implement this? If not, can you advise me what I should do?
Thanks in advance!
From your question:
I understand that the objects you print in the screen are modified by
users in the application, and your aim is to live forward those
modifications to other active client instances of your application.
In that case, as you mention, I would point you to websockets. They are a great way to feed information directly to the client, so the client receives the update signals and modify the interface, with no need of user action.
In order to implement the logic to notify the client, I recommend using a push approach, but this is really depending on what kind of clients you'd like to notify, since the push world is still a bit tricky.
Further readings for this websocket based push implementation:
Question about Push Flags:
Difference between push and urgent flags in TCP
If your client runs in browser or mobile this question is nice to read:
How to send push notification to web browser?
Also html5 websockets:
http://www.websocket.org/aboutwebsocket.html
As a sidenote:
A nice architecture for client-server live communication is based on node.js together with socket.io library offering good performance and not really complex implementation if you know what you do.
My scenario is this:
I have multiple webservers that:
need to communicate with the backend (IBus.Publish/IBus.Subscribe)
need to communicate with each-other (IBus.Publish/IBus.Subscribe)
Aside from the webservers, I have a number of windows services that consume the same messages.
In order to make this work, I have the webservers send messages to a central hub, which sole responsebility it is to wrap the message in a new message type and publish it to all subscribers.
Can I somehow avoid this, so I can publish the messages directly from the webservers?
EDIT (Added some code) - Current situation:
... WebServer
_bus.Send(new Message{Body="SomethingChanged"});
... Hub
public void Handle(Message message){
_bus.Publish(new WrappedMessage{Message = message})
}
... Handlers (WebServers, WindowsServices etc)
public void Handle(WrappedMessage message){
//Actually do important stuff
}
Wanted situation:
... WebServer
_bus.Publish(new Message{Body="SomethingChanged"};
... Handlers (WebServers, WindowsServices etc)
public void Handle(Message message){
//Do important stuff
}
Well, there isn't anything that technically prevents you from publishing messages inside your web application, and likewise there's nothing that prevents you from subscribing to those messages in all instances of the same web application. The question is whether you should :)
Without knowing the details of your problem, my immediate feeling is that you would be better off using some kind of shared persistent storage for whatever it is that you're trying to synchronize (a cache?), possibly using some kind of read replication if you'd like to scale out and make reads really fast.
Again, without knowing the details of your problem, I'll try and suggest something, and then you can see if that could inspire you into an even better solution... here goes:
Use MongoDB (possible as a replica set if you want to scale out your read operations) as the persistent storage of the thing you're caching
Whenever something happens in the web application, bus.Send a message to your backend
In your backend message handler, you update Mongo (which automatically will replicate to read slaves)
Whenever you need to query your data, you just query your Mongo set (using slaveOk=true whenever you can accept slightly stale values)
The reason I'm suggesting this alternative solution, is that web applications (at least in .NET land) have this funny transient nature where the IIS will dictate its lifecycle, and at any given time you can have n instances of it. This complicates matters if you keep state in it. This makes me think of the web application as a client, not a publisher.
A simpler solution is to keep state in something that does not come & go, e.g. a database. And the reason I'm suggesting Mongo is that my guess is that you're worried about being able to serve web requests fast, but since MongoDB is fairly easy to install as a replica set where read operations will be pretty fast (and, more importantly: horisontally scaleable), my guess is that this setup would make everything much simpler.
How does that sound?
When one player makes a move that is sent to the server. And that move is pushed by the server to the second player. As far as I know, the server pushing the move to the second player goes against being a RESTful api.
From what little I know about backbone.js it is meant really for RESTful setups. Is there a way to use backbone.js with websockets to allow the server to push data down to the clients at any time?
Is there even an idiomatic way of implementing chess with backbone.js and websockets? And if not then what would be the correct way to implement chess?
You can definitely do it. Instead of fetching your collection/model, you will just set or update/reset the json data from the websocket into the proper model or collection.
Somewhat pseudo-code example:
var board = new Backbone.Collection(); // this would probably be your own extended Collection instead.
function boardChange(jsonFromServer){
// Take the json array from server,
// and update the collection with it.
// This would trigger 'change' events in each model in the collection (if changed).
board.update(jsonFromServer);
}
Implementing a chess app doesn't really require a Backbone architecture. As long as your server supports Asynchronous API, WebSockets, or even long-polling (anything real-time), it's possible. There's tons of APIs out there on the web already that does this (ie FireBase) as well as frameworks (ie, Meteor) comes into mind.
Also check out Socket IO if you're using Node JS for your server-side. There's tons of open source projects on GitHub that takes advantage of some of these web technologies already, Backbone in particular. Backbone with Socket IO. Backbone.ioBind also looks like a promising project with code samples that you can look at.
To make it work with Backbone, the data API just needs to notify any client-side listeners that an update has been made on the server which in turn triggers a change event on your Backbone Model.
You can even set a timer that performs a request to the server every n amount of time just to test out your code prototypes.
You could overload the Backbone.sync method to use websockets. The de-facto To-Do example (http://addyosmani.github.com/todomvc/) does this to use localstorage instead of a RESTful datastore, and you could do the same for web sockets. In fact if you look around Github/Google you may be able to find someone who's already done it.
I have a perl app which processes text files from the local filesystem (think about it as an overly-complicated grep).
I want to design a webapp which allows remote users to invoke the perl app by setting the required parameters.
Once it's running it would be desirable some sort of communication between the perl app and the webapp about the status of the process (running, % done, finished).
Which would be a recommended way of communication between the two processes? I was thinking in a database table, but I'm not really sure it's a good idea.
any suggestions are appreciated.
Stackers, go ahead and edit this answer to add code examples or links to them.
DrNoone, two approaches come to mind.
callback
Your greppy app needs to offer a callback function that returns the status and which is periodically called by the Web app.
event
This makes sense if you are already using a Web server/app framework which exposes an event loop usable from external applications (rather unlikely in Perl land). The greppy app fires events on status changes and the Web app attaches/listens to them and acts accordingly.
For IPC as you envision it, a plain database is not so suitable. Look into message queues instead. For great interop, pick AMPQ compliant implementation.
If you run the process using open($handle, "cmd |") you can read the results in real time and print them straight to STDOUT while your response is open. That's probably the simplest approach.