I'm new to Akka and Scala and self learning this to do a small project with websockets. End goal is simple, make a basic chat server that publishes + subscribes messages on some webpage.
In fact, after perusing their docs, I already found the pages that are relevant to my goal, namely this and this.
Using dynamic junctions (aka MergeHub & BroadcastHub), and the Flow.fromSinkAndSource() method, I was able to acheive a very basic example of what I wanted. We can even get a kill switch using the example from the akka docs which I have shown below. Code is like:
private lazy val connHub: Flow[Message, Message, UniqueKillSwitch] = {
val (sink, source) = MergeHub.source[Message].toMat(BroadcastHub.sink[Message])(Keep.both).run()
Flow.fromSinkAndSourceCoupled(sink, source).joinMat(KillSwitches.singleBidi[Message, Message])(Keep.right)
}
However, I now see one issue. The above will return a Flow that will be used by Akka's websocket directive: akka.http.scaladsl.server.Directives.handleWebSocketMessages(/* FLOW GOES HERE */)
That means the akka code itself will materialize this flow for me so long as I provide it the handler.
But let's say I wanted to arbitrarily kill one user's connection through a KillSwitch (maybe because their session has expired on my application). While a user's websocket would be added through the above handler, since my code would not be explicitly materializing that flow, I won't get access to a KillSwitch. Therefore, I can't kill the connection, only the user can when they leave the webpage.
It's strange to me that the docs would mention the kill switch method without showing how I would get one using the websocket api.
Can anyone suggest a solution as to how I could obtain the kill switch per connection? Do I have a fundamental misunderstanding of how this should work?
Thanks in advance.
I'm very happy to say that after a lot of time, research, and coding, I have an answer for this question. In order to do this, I had to post in the Akka Gitter as well as the Lightbend discussion forum. Please refer to the amazing answer I got there for some perspective on the problem and some solutions. I'll summarize that here.
In order to get the UniqueKillSwitch from the code that I was using, I needed to use the mapMaterializeValue() method on the Flow that I was returning. Here is the code that I'm using to return a Flow to the handleWebSocketMessages directive now:
// note - state will not be updated properly if cancellation events come through from the client side as user->killswitch mapping may still remain in concurrent map even if the connections are closed
Flow.fromSinkAndSourceCoupled(mergeHubSink, broadcastHubSource)
.joinMat(KillSwitches.singleBidi[Message, Message])(Keep.right)
.mapMaterializedValue { killSwitch =>
connections.put(user, killSwitch) // add kill switch in side effect once value is ready from materialization
NotUsed.notUsed()
}
The above code lives in a Chatroom class I've created that has access to the mergehub and broadcast hub materialized sink and source. It also has access to a concurrent hashmap that persists the kill switch to a user. In this way, we now have access to the Kill Switch through querying it through a map. From there, you can call switch.shutdown() to kill the user's connection from the server side.
My main issue was that I originally thought I could get the switch directly even though I didn't control the materialization. This doesn't seem possible. I suggest this method for when you know that the caller that requires your Flow doesn't care about the materialized value (aka the kill switch).
Please reference the answer I've linked for more scenarios and ways to handle this problem.
Related
I recently started developing using akka event sourcing/cluster sharding, and thanks to the online resources I think I understood the basic concepts and how to create a simple application with it. I am however struggling to apply this methodology in a slightly more complex data structure:
As an example, let's think about webpages and URLs.
Each Page can be represented with an actor in the cluster (having its unique id as the path of the page, e.g. /questions/60037683).
On each page I can issue commands such as
Create page (so if the page does not exist, it will be created)
Edit page (editing the details of the page)
Get page content (and children)
Etc.
When issuing commands to single pages, everything is easy as it's "written on the manual". But I have the added the complexity that a web page can have children, so when creating a "child page" I need the parent to update references to its children.
I thought of some possible approaches, but they feel incomplete.
Sending all events to the single WebPage and when creating a page, finding the parent page (if any) and communicate that a new child has been added
Sending all events to the single WebPage, and when creating a page, the message is sent to the parent, and then it will create a new command that will tell the child to initialize
Creating an infrastructure as WebPageRepository that will keep track of the page tree and will relay CRUD commands to all web page actors.
My real problem is, I think, handling the return of Futures properly when relaying messages to other actors that have to actually perform the job.
I'm making a lot of confusion and some reading resources would be greatly appreciated.
Thanks for your time.
EDIT: the first version was talking about a generical hierarchical file-system-like structure. I updated with the real purpose, webpages and urls and tried to clarify better my issues
After some months of searching, I reached the conclusion that what I'm doing is trying to have actors behave transactionally, so that when something is created, the parent is also updated in a safe manner, meaning that if one operation fails, all operations who completed successfully are rolled back.
The best pattern for this, in my opinion, proved to be the saga pattern, which adds a bit of complexity to the whole process, but in the long run it does what I needed.
Basically I ended up implementing the main actor as a stand alone piece (as it should be) that can receive create commands and add children commands.
There is then a saga actor which takes care of creating the content, adding the child to the parent and rolling back everything if something fails during the process.
If someone else has a better solution, I'll be glad to hear it out
I have an user interface and provide a button to the user, which executes the function longComputation(x: A): A and updates then the user interface (particularly the model) with the new result. This function may take longer to compute the result and should therefore compute in parallel.
Diode provides me with Effect, PotAction, and AsyncAction. I read the documentation about Effects and PotActions/AsyncActions, but I cannot even get a simple example to work.
Can someone point me to or provide an simple working example?
I created a ScalaFiddle based on the SimpleCounter example. There is a LongComputation button, which should run in parallel; but is not.
In JavaScript you cannot run things in parallel without using Web Workers because the JS engine is single-threaded. Web Workers are more like separate processes than threads, as they don't share memory and you need to send messages to communicate between workers and the main thread.
Have less than 50 reputation to comment, so I have to create a new answer instead of commenting #ochrons answer:
As mentioned Web Workers communicate via message passing and share no state. This concept is somehow similar to Akka - even Akka.js exists which enables you to use actor systems in ScalaJS and therefore the browser.
As far as I understand, in a CQRS-oriented API exposed through a RESTful HTTP API the commands and queries are expressed through the HTTP verbs, the commands being asynchronous and usually returning 202 Accepted, while the queries get the information you need. Someone asked me the following: supposing they want to change some information, they would have to send a command and then a query to get the resulting state, why to force the client to make two HTTP requests when you can simply return what they want in the HTTP response of the command in a single HTTP request?
We had a long conversation in DDD/CRQS mailing list a couple of months ago (link). One part of the discussion was "one way command" and this is what I think you are assuming. You can find out that Greg Young is opposed to this pattern. A command changes the state and therefore prone to failure, meaning it can fail and you should support this. REST API with POST/PUT requests provide perfect support for this but you should not just return 202 Accepted but really give some meaningful result back. Some people return 200 success and also some object that contains a URL to retrieve the newly created or updated object. If the command handler fails, it should return 500 and an error message.
Having fire-and-forget commands is dangerous since it can give a consumer wrong ideas about the system state.
My team also recently had a very heated discussion about this very thing. Thanks for posting the question. I have usually been the defender of the "fire and forget" style commands. My position has always been that, if you want to be able to move to an async command dispatcher some day, then you cannot allow commands to return anything. Doing so would kill your chances since an async command doesn't have much of a way to return a value to the original http call. Some of my team mates really challenged this thinking so I had to start thinking if my position was really worth defending.
Then I realized that async or not async is JUST an implementation detail. This led me to realize that, using our frameworks, we can build in middleware to accomplish the same thing our async dispatchers are doing. So, we can build our command handlers the way we want to, returning what ever makes sense, and then let the framework around the handlers deal with the "when".
Example: My team is building an http API in node.js currently. Instead of requiring a POST command to only return a blank 202, we are returning details of the newly created resource. This helps the front-end move on. The front-end POSTS a widget and opens a channel to the server's web socket using the same command as the channel name. the request comes to the server and is intercepted by middleware which passes it to the service bus. When the command is eventually processed synchronously by the handler, it "returns" via the web socket and the front-end is happy. The middleware can be disabled easily, making the API synchronous again.
There is nothing stopping you from doing that. If you execute your commands synchronously and create your projections synchronously, then it will be easy for you to just make a query directly after executing the command and returning that result. If you do this asynchronously via the rest-api, then you have no query result to send back. If you do it asynchronously within your system, then you can wait for the projection to be created and then send the response to the client.
The important thing is that you separate your write and read models in classic CQRS style. That does not mean that you cannot do a read in the same request as you do the command. Sure, you can send a command to the server and then with SignalR (or something) wait for a notification that your projection have been created/updated. I do not see a problem with waiting for the projection to be created on the server side instead for on the client.
How you do this will affect you infrastructure and error handling. Also, you will hold the HTTP request open for a longer time if you return the result at once.
I want to use elements of CQRS pattern in my project. I wonder if i do it right with Command and Events.
The thing that I'm not sure is if event can invoke command. To better show what i want to do I will use diagram and example.
This is an example:
User invoke TripCreateCommand. TripCreateCommandHandler do his job and after success publish TripCreatedEvent.
Now we have two listener to TripCreatedEvent (the order of listener execution does not matter)
First listener (can be execute after the second listener):
for each user in trip.author.friends invoke two Command (the order of commands is important)
PublishTripOnUserWallCommand
SendNewTripEmailNotificationCommand
SendNewTripPlatformNotification
Second listener (can be execute before the first listener):
PublishTripOnUserSocials
And this is sample diagram:
Is this a good way ? Can EventListener invoke Command, or maybe I should do it in some other way ?
Your question is about Mesage Driven Architecture which works together with but otherwise unrelated to CQRS.
Anyway, your diagram is almost correct. The event subscriber/handler (I prefer this terminology) can send new Commands via the service bus, but it's not a rule that you should always do this. I implement quite a lot of functionality directly in the event handler, although probalby would be more clean and reliable to send a new command. It really depends on what I want to do.
Note that the message handlers (commands or events) should not know about other handlers. They should know about the bus and the bus takes care of handling. This means that in your app, the event handlers would take the bus as dependency, create the command and send it via the bus. The event handler itself doesn't know what command handler generated the event and can 'reply' to it.
Usually the commands would be handled independently and you can't guarantee the order (unless they're handled synchronously) so maybe you want the second command to be issued as a result of the first command's handling. Indeed, it can be the case for a Saga.
AFAIK you are talking only about doing things synchronously, so your approach works in this case but it's probably not scalable. Moving to async handling will break this execution flow. However your application can be fine with it, not everyhting needs to be twitter.
A message driven architecture is not that straightforward and for some cases (like you want an immediate response from the backend) it's quite complicated to implement, at least more complicated than with the 'standard' approach. So maybe for those particular cases you might want to do it the 'old' way.
If you're worried about decoupling and testing, you can still design the services as they were message handlers but use them directly, instead of a service bus.
Not sure why you would need Commands for performing the updating the information on the user's wall. Why would you choose not to use a View Model Updater for that task.
Sending an email can be considered a Command but could also easily be viewed as just another View Model update.
Not clear on what the purpose of the SendNewTripPlatformNotification is, so I cannot give any suggestions there...
Some of this could also be a candidate for a Saga. Secondly I'm missing your Domain in the diagram, that is what should be responsible for publishing any events, or do you consider the CommandHandler to be the Domain?
In the old days, when each request to a web application was handled by one thread, it was fairly easy to understand the logs. One could, for example, use a servlet filter to name the thread that was handling a request with some sort of request id. This request id then could be output in the logs. In this world, a simple grep was all it took to collect the log lines for a given request.
In my current position, I'm building web applications with Scala (we're using Scalatra but that isn't specifically relevant to my question). Each request creates a scala.concurrent.Future and is then parked until that future has completed. The important bit here is that the thread that actually handles the business logic is different from the thread that handled the request which is different (I think) from the thread that completes the request and so the context of that request is lost during processing. The business logic can log all it likes but it is hard to associate that logging with the specific request it relates to.
Now from the standpoint of supporting my web services in production, the old approach was great and I'd like to come up with something similar for my asynchronous services. I've been trying to come up with a way to do it but have come up empty. That is, I haven't come up with anything nearly as light weight as the old, name-the-thread model. Does the Stack Overflow crowd have any suggestions?
Thanks
As you have written, assign an id to each request, and pass that to the business logic function. You can also do this with implicit parameter, so your code won't be cluttered.
This should be possible with MDC logging available with SLF4j which uses Thread local storage to store the context of the each request.
Also you will have to create a MDC Context Propagating execution context, to move the context across threads.
This post describes it well:
http://code.hootsuite.com/logging-contextual-info-in-an-asynchronous-scala-application/