I have an Actor, which acts as a Supervisor, but also needs to "return" data to the caller, wether this is an actor or not shouldn't matter.
I am asking my Supervisor, let's call him SV.
SV Processes the message I send to him, and sends back a response.
val system = ActorSystem("ActorSystem")
val sv = system.actorOf(Props[SV], name = "SV")
sv ? msg
And SV's recieve method looks like this:
def receive = {
case msg => (someChild ? msg).pipeTo(sender)
...
}
This all works jolly fine.
The problem is when the child throws an exception, this exception is caught by the supervisor strategy.
override def supervisorStrategy = OneForOneStrategy () {
case e : Throwable => {
val newResponse = someNewResponse
sender ! newResponse
...
}
}
sender is not longer a reference to whoever called SV in the first place, and I can't seem to work out how to send a message back to the askee, and get back to my original flow.
One of the three Actor rules is: “An Actor can send a finite number of messages to other Actors it knows.” The last two words are critical here: if the supervisor has not been introduced to the original sender somehow and the failure (exception) itself does not contain the sender’s reference either, then the supervisor cannot possibly send a message to the sender. Either you catch all exceptions in the child actor, wrap them together with the sender in your own exception type and then rethrow, or the original message needs to travel via the supervisor to the child and back, so that the supervisor can see that a reply is outstanding when a failure occurs.
Are you using the supervisor strategy and exceptions to control the flow or your data? Consider using the type system (Option type or a Failure on the response Future) for "exception" cases in your child actors and handle the response flow without exceptions.
The supervisor strategy is for unhandled exceptions.
When an unhandled exception occurs you lose the ability to respond with a message to the sender. Unless you wrap the sender in the unhandled exception like Roland Kuhn suggests.
Let the supervisor strategy instead handle how the actor system should respond to your unhandled exceptions by mapping them to a Directive.
Related
Let's say I have a very simple actor class which receives any message and prints to the console.
class SimpleActor extends Actor{
def receive: Receive = {
case message =>
println(s"[${this}][${self}] received message: ${message}")
}
}
val simpleActor = actorSystem.actorOf(Props[SimpleActor], "simpleActor")
simpleActor ! "Hey"
As you can see I am using both this and self here and both has different values. Its output is something like:
[pkg.ActorRunner$SimpleActor#65cca69][Actor[akka://ActorDemo/user/simpleActor#934141660]] received message: Hey
I want to understand the difference between self and this because in complex scenarios(production system) if the actor breaks, for example: throws an exception than I think value for this gets changed.
this is classic java reference to the object extending Actor trait while self is a reference to the ActorRef that is what you need to send messages (! or tell and ? or ask)
You can't send messages to this
You shouldn't pass reference to this outside actor while passing reference to self is perfectly fine, in fact it is sent implicitly when you send a message to an actor from another actor. If you pass this to another object you will be risking actor's state encapsulation. Remember that the only way to communicate with an actor is via messages, i.e. with its ActorRef
self will remain valid after actor restart, aka you can keep sending messages to the same ActorRef (self). Only when the actor is stopped the reference to ActorRef is no longer valid and messages sent to that address will end in Dead Letters.
this will no longer be valid after actor restart. A new object of type Actor is instantiated to clear the actor state that could be compromised due to failure.
What restarting means
Unless the failure is specifically recognizable, the third cause cannot be ruled out, which leads to the conclusion that the internal state needs to be cleared out. If the supervisor decides that its other children or itself is not affected by the corruption—e.g. because of conscious application of the error kernel pattern—it is therefore best to restart the child. This is carried out by creating a new instance of the underlying Actor class and replacing the failed instance with the fresh one inside the child’s ActorRef; the ability to do this is one of the reasons for encapsulating actors within special references. The new actor then resumes processing its mailbox, meaning that the restart is not visible outside of the actor itself with the notable exception that the message during which the failure occurred is not re-processed.
Actor Reference and Path Equality
Note that a restart of an actor caused by a failure still means that it is the same actor incarnation, i.e. a restart is not visible for the consumer of the ActorRef.
What is the Difference Between Actor Reference and Path?
An actor reference designates a single actor and the life-cycle of the reference matches that actor’s life-cycle; an actor path represents a name which may or may not be inhabited by an actor and the path itself does not have a life-cycle, it never becomes invalid. You can create an actor path without creating an actor, but you cannot create an actor reference without creating corresponding actor.
You can create an actor, terminate it, and then create a new actor with the same actor path. The newly created actor is a new incarnation of the actor. It is not the same actor. An actor reference to the old incarnation is not valid for the new incarnation. Messages sent to the old actor reference will not be delivered to the new incarnation even though they have the same path.
I'm doing something like this:
...lots of surrounding context...
val actor = context.actorOf(Props(new Actor(...))
(actor ? message) onComplete {
case Done => {
println("done")
Do stuff with the surrounding context of this actor
}
case _ => {
println("got wildcard") // try to handle error message here
}
}
Actor is failing at initialization with
akka.actor.ActorInitializationException (in that actors context)
In the parent actor I am getting **dead letters encountered.**
since actor was never created properly and "message" couldn't be delivered.
In this scenario the "ask?" pattern never returns properly.
I could create a SupervisorStrategy in the parent global level, but I need to understand the context of the actor that generated the exception and it isn't possible (or convenient) handling it at the global level
Is there a way to "catch" the initialization exception in the "ask" context actor ? message?
According to akka philosophy ask should be used only when you interact with the actor from the outside of the actor system.
Since you have a reference to the context seems that you are inside the actor code. So you should try to get rid of ask.
If I'm wrong and you call ask from non-akka code then you shouldn't try to rely on anything that happens inside the actor system. All that you have is an ActorRef with no guaranties what is on the other side of this reference.
I'm trying to respond to the sender of message when the receiving actor dies due to that message. If I Restart the actor on failure I get
preRestart(reason: Throwable, message: Option[Any])
but now I'm committed to restarting.
If I Stop the actor, i only get
postStop()
with no knowledge what stopped myself.
Meanwhile in the supervisor, I only get the Throwable and no indication of what caused it.
I suppose, I can dig through the DeadLetters post actor termination, but that seems like a noisy approach, since I'd have to listen to all of dead letters and somewhere correlate the termination with the deadletter event stream.
UPDATE: DeadLetter really doesn't seem to be an option. The message that caused the death doesn't even go to DeadLetters, it just disappears.
Is there a mechanism I am overlooking?
According to this thread on the Akka Users List, there isn't a mechanism in the actor supervision death cycle to accomplish this. Moreover, the documentation explicitly states that the message is dropped:
What happens to the Message
If an exception is thrown while a message is being processed (i.e. taken out of its mailbox and handed over to the current behavior), then this message will be lost. It is important to understand that it is not put back on the mailbox. So if you want to retry processing of a message, you need to deal with it yourself by catching the exception and retry your flow.
The ideal solution would be to use a dedicated actor for dangerous operations and have the initiator monitor the death of that actor to determine failure.
As my scenario arose from something considered safe but that had a bug in it, the separate actor option would have been after the fact. To avoid wrapping all code paths in try/catch but be able to guard more complicated and critical flows, I ended up with creating a wrapper for receive that let's me intercept exceptions:
object SafeReceive {
def apply(receive: Receive)(recover: Any => PartialFunction[Throwable, Unit]): Receive =
new Receive {
override def isDefinedAt(x: Any): Boolean = receive.isDefinedAt(x)
override def apply(v1: Any): Unit = try {
receive(v1)
} catch recover(v1)
}
}
which I can use for select actors like this:
def receive = SafeReceive {
case ... => ...
} {
msg => {
case e: Exception =>
sender ! OperationFailed(msg, e)
throw e
}
}
I have some actors that when their preRestart() is invoked, the "message" is None. I don't understand why this is the case.
Context
My application is a Play web application that uses Akka. My error handling strategy is to send an error notification to the origin of the request, so details can be sent down to the user. I do this by hooking into preRestart() as follows:
trait NotifyRequesterErrorHandling { this: Actor with ActorLogging =>
override def preRestart(reason: Throwable, message: Option[Any]) {
notifyRequester(message, reason)
}
protected def notifyRequester(message: Option[Any], reason: Throwable) {
message match {
case Some(m) =>
val who = getRequester(m)
who ! SearchError(reason, m)
case None =>
val error = if (reason == null) "No exception" else reason toString()
ReportError("NotificationError", Some(reason), "", s"${this.context.self.path}: Attempted to notify requester but no previous message")
}
}
def getRequester(message: Any): ActorRef
}
Problem
In my logs, I am seeing a lot of the "Attempted to notify requester but no previous message" error logs. Usually it occurs for all the actors in my system. This is likely because I have one top-level actor, which is responsible for all the other actors (they are all children).
In the logs, the reason parameter does contain a throwable, though.
I am also using a one-for-all strategy. So, basically, whenever all the actors are restarted, I get lots of these errors.
Possible Explanations aka guesses
After all actors are restarted a new instance of each actor is created, and thus there is no previous message
When actors are restarted as one-for-all, all of their messages have been processed and their queue is empty. This means that there is no previous message
The documentation for preRestart states: "message - optionally the current message the actor processed when failing, if applicable"
I.e. it is only applicable to the failing actor. Not the other ones who are just restarted and not failing.
I have a supervising Akka actor which uses a router to forward messages to worker actors.
I have a class which wraps the supervisor and when I call a method on that class it "asks" the supervisor to do something and then I use Await.result(theFuture) to wait for the result (I cannot continue without the result).
If the workers throw an exception, I want to restart the worker which threw the exception, and I want the exception to be caught by the code which calls the wrapper class.
I passed a OneForOneStrategy to the router constructor, which returns RESTART in the case of an Exception. In the postRestart method of the worker, I log the restart, so I can validate that the worker is actually restarted.
When the worker throws an exception, it gets restarted, but the exception disappears. The Future which is the result of asking the supervisor, contains an exception, but it is a akka.pattern.AskTimeoutException, which is thrown after just 5 seconds rather than 20 seconds, which is the implicit timeout that I have lingering around. The exception actually occurs less than a second after the worker starts.
Question 1: how can I get the exception from the worker in the code which calls my wrapper class?
Also, the receive method of the worker is like this:
def receive = {
case r: Request =>
val response = ??? //throws an exception sometimes
sender ! response
}
Something is logging the exception to the console, but it isn't my code. The stack trace is:
[ERROR] [02/11/2013 21:34:20.093] [MySystem-akka.actor.default-dispatcher-9]
[akka://MySystem/user/MySupervisor/MyRouter/$a] Something went wrong!
at myApp.Worker.$$anonfun$receive$1.applyOrElse(Source.scala:169)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:425)
at akka.actor.ActorCell.invoke(ActorCell.scala:386)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:230)
at akka.dispatch.Mailbox.run(Mailbox.scala:212)
at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:502)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:262)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1478)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
Line 169 of Source.scala is the line val response = ??? shown in the listing of the receive method above.
Question 2: who is logging that exception to the console, and how can I stop it?
1)
try somethingThatCanFail() catch {
case e: Exception => sender ! Status.Failure(e); throw e
}
The "tell failure" causes the caller to get a Failure containing the exception. Throwing "e" causes the oneForOneStrategy to be called which restarts the worker.
2)
It is the actor system itself that logs the failure, and the only way
to quiet it down is to filter out things by creating and configuring
your own LoggingAdapter as described here
http://doc.akka.io/docs/akka/2.1.0/scala/logging.html There is a
ticket for changing this
https://www.assembla.com/spaces/akka/tickets/2824 but it is targeted
for Akka 2.2
Answered by https://groups.google.com/forum/#!topic/akka-user/fenCvYu3HYE
In order to be notified of one of your children failing, you need to
First watch the child
Then you will be sent a Terminated() when the actor dies with a reference to it.
Something like:
class ParentActor extends Actor {
// this is sample of how to watch for death of one of your children
val childActor = context.actorOf(Props[SomeService], "SomeService")
val dyingChild = context.watch(context.actorOf(childActor))
def receive = {
case Terminated(`dyingChild`) =>
println("dyingChild died")
case Terminated(terminatedActor) =>
println(s"This child just died $terminatedActor")
}
}
Hope this helps.