I'd like to observe a log file which is continuously updated by the system. I thought about using a Comet actor to do that but I need advice for the right direction. I want to achieve similar functionality to tail -F for the WebApp. If something new is written to the log file the Comet actor should pick it up continually. I want to show this information on a webpage. How can I achieve this with the Lift framework and Scala?
You need to solve 2 problems separately: log tailing, and comet page updates and then put everything together:
The way I would do it is by creating an Akka/Scala/Lift actor that tails the log. Look at this question for example on how to tail a log file in Java.
Then whenever this actor detects some changes in the log it should send a message to the comet actor which in turn will update the web page. You can easily find general examples on how to use Comet, for example here.
If you don't want to use an actor then just schedule a thread that runs and tails the log.
Note that if your log rotates and/or archives you'll have to deal with it too.
A completely alternative solution would be to hook up custom log appender or custom logger, for example this and make it send you log events. I think if you use Akka Logger you can simply subscribe to the event which your actor will receive on each log event, which you just forward to the Comet actor in a similar manner to the solution above.
Related
I'm new to Akka and Scala and self learning this to do a small project with websockets. End goal is simple, make a basic chat server that publishes + subscribes messages on some webpage.
In fact, after perusing their docs, I already found the pages that are relevant to my goal, namely this and this.
Using dynamic junctions (aka MergeHub & BroadcastHub), and the Flow.fromSinkAndSource() method, I was able to acheive a very basic example of what I wanted. We can even get a kill switch using the example from the akka docs which I have shown below. Code is like:
private lazy val connHub: Flow[Message, Message, UniqueKillSwitch] = {
val (sink, source) = MergeHub.source[Message].toMat(BroadcastHub.sink[Message])(Keep.both).run()
Flow.fromSinkAndSourceCoupled(sink, source).joinMat(KillSwitches.singleBidi[Message, Message])(Keep.right)
}
However, I now see one issue. The above will return a Flow that will be used by Akka's websocket directive: akka.http.scaladsl.server.Directives.handleWebSocketMessages(/* FLOW GOES HERE */)
That means the akka code itself will materialize this flow for me so long as I provide it the handler.
But let's say I wanted to arbitrarily kill one user's connection through a KillSwitch (maybe because their session has expired on my application). While a user's websocket would be added through the above handler, since my code would not be explicitly materializing that flow, I won't get access to a KillSwitch. Therefore, I can't kill the connection, only the user can when they leave the webpage.
It's strange to me that the docs would mention the kill switch method without showing how I would get one using the websocket api.
Can anyone suggest a solution as to how I could obtain the kill switch per connection? Do I have a fundamental misunderstanding of how this should work?
Thanks in advance.
I'm very happy to say that after a lot of time, research, and coding, I have an answer for this question. In order to do this, I had to post in the Akka Gitter as well as the Lightbend discussion forum. Please refer to the amazing answer I got there for some perspective on the problem and some solutions. I'll summarize that here.
In order to get the UniqueKillSwitch from the code that I was using, I needed to use the mapMaterializeValue() method on the Flow that I was returning. Here is the code that I'm using to return a Flow to the handleWebSocketMessages directive now:
// note - state will not be updated properly if cancellation events come through from the client side as user->killswitch mapping may still remain in concurrent map even if the connections are closed
Flow.fromSinkAndSourceCoupled(mergeHubSink, broadcastHubSource)
.joinMat(KillSwitches.singleBidi[Message, Message])(Keep.right)
.mapMaterializedValue { killSwitch =>
connections.put(user, killSwitch) // add kill switch in side effect once value is ready from materialization
NotUsed.notUsed()
}
The above code lives in a Chatroom class I've created that has access to the mergehub and broadcast hub materialized sink and source. It also has access to a concurrent hashmap that persists the kill switch to a user. In this way, we now have access to the Kill Switch through querying it through a map. From there, you can call switch.shutdown() to kill the user's connection from the server side.
My main issue was that I originally thought I could get the switch directly even though I didn't control the materialization. This doesn't seem possible. I suggest this method for when you know that the caller that requires your Flow doesn't care about the materialized value (aka the kill switch).
Please reference the answer I've linked for more scenarios and ways to handle this problem.
I recently started developing using akka event sourcing/cluster sharding, and thanks to the online resources I think I understood the basic concepts and how to create a simple application with it. I am however struggling to apply this methodology in a slightly more complex data structure:
As an example, let's think about webpages and URLs.
Each Page can be represented with an actor in the cluster (having its unique id as the path of the page, e.g. /questions/60037683).
On each page I can issue commands such as
Create page (so if the page does not exist, it will be created)
Edit page (editing the details of the page)
Get page content (and children)
Etc.
When issuing commands to single pages, everything is easy as it's "written on the manual". But I have the added the complexity that a web page can have children, so when creating a "child page" I need the parent to update references to its children.
I thought of some possible approaches, but they feel incomplete.
Sending all events to the single WebPage and when creating a page, finding the parent page (if any) and communicate that a new child has been added
Sending all events to the single WebPage, and when creating a page, the message is sent to the parent, and then it will create a new command that will tell the child to initialize
Creating an infrastructure as WebPageRepository that will keep track of the page tree and will relay CRUD commands to all web page actors.
My real problem is, I think, handling the return of Futures properly when relaying messages to other actors that have to actually perform the job.
I'm making a lot of confusion and some reading resources would be greatly appreciated.
Thanks for your time.
EDIT: the first version was talking about a generical hierarchical file-system-like structure. I updated with the real purpose, webpages and urls and tried to clarify better my issues
After some months of searching, I reached the conclusion that what I'm doing is trying to have actors behave transactionally, so that when something is created, the parent is also updated in a safe manner, meaning that if one operation fails, all operations who completed successfully are rolled back.
The best pattern for this, in my opinion, proved to be the saga pattern, which adds a bit of complexity to the whole process, but in the long run it does what I needed.
Basically I ended up implementing the main actor as a stand alone piece (as it should be) that can receive create commands and add children commands.
There is then a saga actor which takes care of creating the content, adding the child to the parent and rolling back everything if something fails during the process.
If someone else has a better solution, I'll be glad to hear it out
In are app, we are quite heavily using Akka Event Stream to handle logic that is not related to the main business flow. Things like: send emails, sync records, etc... All of these events are currently fired and forgotten.
system.eventStream.publish(<event>)
And they are handled by listeners, asynchronously, in most cases.
However, I am now investigating an option of extending class functionality through events system and that, sometimes, requires a return value from an event?
Is it even possible to get some result back from an event? I could not find anything specific in this regard.
Thanks,
It's a feature of event based systems: you know who triggered the event, you don't know who will handle it, nor when (can be asynchronous).
So, the most idiomatic solution, if your listener should generate an answer, is to do it in an asynchronous way. And as such, the listener should send an event with it's answer.
Another way to do it is to bypass the event bus and register you handlers as listeners of your event source (you should handle your self the registe/unregister functions through akka messages and a simple list). And when your event source triggers the listeners, it can do it using the ask pattern.
I want to use elements of CQRS pattern in my project. I wonder if i do it right with Command and Events.
The thing that I'm not sure is if event can invoke command. To better show what i want to do I will use diagram and example.
This is an example:
User invoke TripCreateCommand. TripCreateCommandHandler do his job and after success publish TripCreatedEvent.
Now we have two listener to TripCreatedEvent (the order of listener execution does not matter)
First listener (can be execute after the second listener):
for each user in trip.author.friends invoke two Command (the order of commands is important)
PublishTripOnUserWallCommand
SendNewTripEmailNotificationCommand
SendNewTripPlatformNotification
Second listener (can be execute before the first listener):
PublishTripOnUserSocials
And this is sample diagram:
Is this a good way ? Can EventListener invoke Command, or maybe I should do it in some other way ?
Your question is about Mesage Driven Architecture which works together with but otherwise unrelated to CQRS.
Anyway, your diagram is almost correct. The event subscriber/handler (I prefer this terminology) can send new Commands via the service bus, but it's not a rule that you should always do this. I implement quite a lot of functionality directly in the event handler, although probalby would be more clean and reliable to send a new command. It really depends on what I want to do.
Note that the message handlers (commands or events) should not know about other handlers. They should know about the bus and the bus takes care of handling. This means that in your app, the event handlers would take the bus as dependency, create the command and send it via the bus. The event handler itself doesn't know what command handler generated the event and can 'reply' to it.
Usually the commands would be handled independently and you can't guarantee the order (unless they're handled synchronously) so maybe you want the second command to be issued as a result of the first command's handling. Indeed, it can be the case for a Saga.
AFAIK you are talking only about doing things synchronously, so your approach works in this case but it's probably not scalable. Moving to async handling will break this execution flow. However your application can be fine with it, not everyhting needs to be twitter.
A message driven architecture is not that straightforward and for some cases (like you want an immediate response from the backend) it's quite complicated to implement, at least more complicated than with the 'standard' approach. So maybe for those particular cases you might want to do it the 'old' way.
If you're worried about decoupling and testing, you can still design the services as they were message handlers but use them directly, instead of a service bus.
Not sure why you would need Commands for performing the updating the information on the user's wall. Why would you choose not to use a View Model Updater for that task.
Sending an email can be considered a Command but could also easily be viewed as just another View Model update.
Not clear on what the purpose of the SendNewTripPlatformNotification is, so I cannot give any suggestions there...
Some of this could also be a candidate for a Saga. Secondly I'm missing your Domain in the diagram, that is what should be responsible for publishing any events, or do you consider the CommandHandler to be the Domain?
I'm trying to implement a basic but flexible "event queue" so that my web app can go run subs asynchronously. The way I have working for me now is the app writes a record to an "event queue" table in my database with info on what to do and when to fire. Then I have a daemon script that queries that table periodically and if something needs to be done, it fires it off.
I'd like to start moving towards watchers using something like AnyEvent or EV so I'm not hitting my database so often, but try as I might, I can't find good info on "best practices" for setting something like this up. AnyEvent's documentation is pretty good, but it seems to assume you know how your events should be passed around...which I don't.
What should my watcher be watching? A file? If so, what should be in that file? I don't need to send a whole bunch of data around, I just basically need something that says, "go off and run this sub right now"
I would greatly appreciate someone pointing me in the right direction.
EDIT:
It's been requested that I be more specific: The events I'm trying to fire are various. Sometimes it's an email that needs sending, sometimes it's some DB work, sometimes I just need an action to be delayed for a few hours or days. In all cases, I have some sort of backend script to handle the action, I just need a way for my frontend (web app) to tell my backend, "hey I need you to go do this in x minutes" or "I need you to do this now"
You might look at Mojolicious which is built to be non-blocking and respond asynchronously. It even uses EV internally if you have it installed. To get started read the doc for Mojolicious::Lite and then the Guides (in order) from here: http://mojolicio.us/perldoc
So what is it that you are actually trying to do? Until you describe what it is you actually need, I can't help you any further that that.
There are some CPAN modules in order to run asynchronous tasks via message queues. Examples:
Queue::DBI
POE::Component::MessageQueue
Any::MQ