How to restart a killed actor? - scala

I have a supervisor and a child actor that looks as the following:
SupervisorActor.scala
import akka.actor.{Actor, ActorLogging, Props, Terminated}
object SupervisorActor {
def props: Props = Props(new SupervisorActor)
object KillFoo
object TerminateFoo
object RestartFoo
}
final class SupervisorActor extends Actor with ActorLogging {
import com.sweetsoft.SupervisorActor._
log.info(s"Start the actor ${self}")
private val foo = context.actorOf(FooActor.props, "FOOCHILD")
context.watch(foo)
override def receive: Receive = {
case KillFoo =>
context.stop(foo)
case Terminated(actor) =>
log.info(s"Kill actor $actor")
case TerminateFoo =>
log.info("Terminate FooActor")
foo ! new IllegalArgumentException
case RestartFoo =>
}
}
FooActor.scala
import akka.actor.SupervisorStrategy.{Escalate, Restart, Resume, Stop}
import akka.actor.{Actor, ActorLogging, OneForOneStrategy, Props}
import scala.concurrent.duration._
object FooActor {
def props: Props = Props(new FooActor)
}
final class FooActor extends Actor with ActorLogging {
log.info(s"Start the actor ${ self }")
override val supervisorStrategy: OneForOneStrategy =
OneForOneStrategy(maxNrOfRetries = 1, withinTimeRange = 1.minute) {
case _: ArithmeticException => Resume
case _: NullPointerException => Restart
case _: IllegalArgumentException =>
println("STOP exception raised.")
Stop
case _: Exception => Escalate
}
override def receive: Receive = {
case _ => log.info("I got killed.")
}
}
and Main.scala
import akka.actor.ActorSystem
import com.sweetsoft.SupervisorActor.{TerminateFoo}
import scala.concurrent.Future
import scala.io.StdIn
object Main extends App {
val system = ActorSystem("monarch")
implicit val dispatcher = system.dispatcher
try {
// Create top level supervisor
val supervisor = system.actorOf(SupervisorActor.props, "mupervisor")
// Exit the system after ENTER is pressed
Future {
Thread.sleep(2000)
supervisor ! TerminateFoo
}
StdIn.readLine()
} finally {
system.terminate()
}
}
After the FooActor got killed, I would like to restart it manually again, like I did:
private val foo = context.actorOf(FooActor.props, "FOOCHILD")
How to do it?
I am thinking about to create a function, that is going to create the FooActor and after it got killed, just call the function to start a new FooActor.

There are a few problems with the code. supervisorStrategy should be in the SupervisorActor as it is responsible for supervision, not the child actor itself.
foo ! new IllegalArgumentException won't cause the child actor to terminate as actors can accept any object as a message and Exception derived objects are not treated specially. It'll just print "I got killed." but otherwise ignore the message and keep running. In particular, it won't invoke supervisorStrategy
handler.
You can either:
send the pre-defined system PoisonPill message to the child actor
to gracefully stop it (i.e. after processing all pending messages)
send the pre-defined system Kill message to the child actor to stop
it immediately after processing the current message and ignore the
other queued messages if any.
forward TerminateFoo message to it
and throw an exception in its handler. In this case the child
object's destiny is decided by supervisorStrategy handler for
IllegalArgumentException exception type (i.e. Stop in your case).
.
override def receive: Receive = {
case TerminateFoo =>
log.info("Stopping FooActor by throwing an unhandled exception for supervisorStrategy to process it")
throw new IllegalArgumentException
case m => log.info(s"I received a message $m")
}
See https://doc.akka.io/docs/akka/current/actors.html#stopping-actors for details

Related

How to stop an Actor reloading on exception

In a scenario where an exception is thrown in an Actor receive I want to prevent this actor from being reloaded. I understood that the correct way to do this is by overriding supervisorStrategy but this does not work as shown in the example below:
class MyActor extends Actor {
println("Created new actor")
def receive = {
case msg =>
println("Received message: " + msg)
throw new Exception()
}
override val supervisorStrategy = OneForOneStrategy() {
case _: Exception => Stop
}
}
val system = ActorSystem("Test")
val actor = system.actorOf(Props(new MyActor()))
actor ! "Hello"
When I run this code "Created new actor" is output twice showing that the Actor is reloaded again after the exception.
What is the correct way to prevent the Actor being reloaded?
When an actor overrides the default supervisor strategy, that strategy applies to that actor's children. Your actor is using the default supervisor strategy, which restarts actors when they throw an exception. Define a parent for your actor and override the supervisor strategy in that parent.
class MyParent extends Actor {
override val supervisorStrategy = OneForOneStrategy() {
case _: Exception => Stop
}
val child = context.actorOf(Props[MyActor])
def receive = {
case msg =>
println(s"Parent received the following message and is sending it to the child: $msg")
child ! msg
}
}
class MyActor extends Actor {
println("Created new actor")
def receive = {
case msg =>
println(s"Received message: $msg")
throw new Exception()
}
}
val system = ActorSystem("Test")
val actor = system.actorOf(Props[MyParent])
actor ! "Hello"
In the above example, a MyActor is created as a child of MyParent. When the latter receives the "Hello" message, it sends the same message to the child. The child is stopped when it throws the exception, and "Created new actor" is therefore printed only once.

neat way to test become unbecome swtichover in scala

I am trying to test state switch over of receive method.
Found this stackoverflow post, but that also not clearly give a solution.
Please find below simplified code snippet :-
package become_unbecome_basics
import akka.actor.{Actor, ActorSystem}
import akka.testkit.{ImplicitSender, TestActorRef, TestKit}
import become_unbecome_basics.BasicBecomeUnbecomeActor.{SWITCH_TO_MASTER, SWITCH_TO_STANDBY}
import com.typesafe.scalalogging.LazyLogging
import org.scalatest.FlatSpecLike
import org.scalatest.Matchers._
class BecomUnbecomeSwitchoverTest extends TestKit(ActorSystem("testSystem")) with ImplicitSender with FlatSpecLike{
"initially receive" should "points to master" in {
val aRef = TestActorRef[BasicBecomeUnbecomeActor]
val actor = aRef.underlyingActor
//not sure, weather we have something like this to assert
//actor.receive should be(actor.master)
}
}
object BasicBecomeUnbecomeActor{
case object SWITCH_TO_MASTER
case object SWITCH_TO_STANDBY
}
class BasicBecomeUnbecomeActor extends Actor with LazyLogging{
override def receive: Receive = master
def master: Receive = {
case SWITCH_TO_STANDBY =>
context.become(standBy)
case msg => logger.debug(s"master : $msg received")
}
def standBy: Receive = {
case SWITCH_TO_MASTER =>
context.unbecome()
case msg => logger.debug(s"standBy : $msg received")
}
}
The StackOverflow post you mentioned contains two suggested ways to test your actor.
Emit the state changes to some other actor.
Don't test state change, but the actor's behaviour.
In the first example, you would have some way of sending information out of your actor on every state change. In Akka, sending state change information as actor messages is natural way to implement this.
import akka.actor._
import akka.testkit._
class ExampleActor(notify: ActorRef) extends Actor with ActorLogging {
import ExampleActor.{Master, StandBy}
def receive: Receive = master
def master: Receive = {
case StandBy =>
notify ! StandBy
context.become(standby)
case msg =>
log.debug("received msg in master: {}", msg)
}
def standby: Receive = {
case Master =>
notify ! Master
context.become(master)
case msg =>
log.debug("received msg in standby: {}", msg)
}
}
object ExampleActor {
def props(notify: ActorRef): Props = Props(new ExampleActor(notify))
case object Master
case object StandBy
}
class ExampleActorTest extends TestKit(ActorSystem("testSystem")) with FlatSpecLike {
"ExampleActor" should "move to stand by state" in {
val probe = TestProbe()
val actor = system.actorOf(ExampleActor.props(probe.ref))
actor ! ExampleActor.StandBy
probe.expectMsg(ExampleActor.StandBy)
}
}
(I haven't run the code yet so apologies for any errors in the code)
In the code above, the ExampleActor is a stateful actor which notifies the given actor reference of any state changes. Note that this doesn't allow inspecting the current state, but instead a log of state transitions. Also, it is possible to introduce a bug in the state notification code because the notification code is manually added to the actor instead of the actor doing it automatically.
I changed the testing style to asynchronous testing style to get more realistic tests.
The state change notifications allows you to get information about what specific state the actor transitions to, but it doesn't tell you if it works the way it should. Instead of testing what state changes the actor goes through, how about testing what the actor itself does.
class Accumulator extends Actor with ActorLogging {
import Accumulator._
def receive: Receive = accumulatorReceive(0)
def accumulatorReceive(x: Int): Receive = {
case Add(i) => next(x + i)
case Remove(i) => next(x - i)
case Multiply(i) => next(x * i)
case Divide(i) => next(x / i)
case Get => sender() ! x
}
def next(x: Int) = context.become(accumulatorReceive(x))
}
object Accumulator {
def props: Props = Props(new Accumulator)
case class Add(i: Int)
case class Remove(i: Int)
case class Multiply(i: Int)
case class Divide(i: Int)
case object Get
}
class AccumulatorTest extends TestKit(ActorSystem("testSystem")) with FlatSpecLike {
import Accumulator._
"Accumulator" should "accumulate" in {
val probe = TestProbe()
val actor = system.actorOf(Accumulator.props)
actor ! Add(3)
actor ! Remove(1)
actor ! Multiply(4)
actor ! Divide(2)
probe.send(actor, Get)
probe.expectMsg(5)
}
}
In this example, Accumulator does state changes, but it doesn't notify when its state has changed. Instead, it has a specific get command for inspecting interesting parts about its state. In the test, we send multiple messages that cause state changes in the accumulator actor. Finally, we inspect the result of these messages by querying the accumulator.

Akka - test supervision strategy

I have the following scenario: the parent supervisor actor creates a child for each message using a factory (function) passed in the constructor.
class supervisorActor(childActorMaker: ActorRefFactory => ActorRef)
extends Actor with ActorLogging{
def receive: Receive = {
case "testThis" =>
val childActor = childActorMaker(context)
childActor!"messageForChild"
}
override val supervisorStrategy =
OneForOneStrategy() {
case _ => Stop
}
}
class childActor extends Actor {
def receive:Receive = {
case _ => /** whatever **/
}
}
In the test I override the child Receive to force an exception. My expectation is the child actor to be stopped every time because of the Supervision Strategy i set.
"When the child Actor throws an exception the Supervisor Actor " should " " +
" stop it" in {
val childActorRef = TestActorRef(new childActor() {
override def receive = {
case msg: String => throw new IllegalArgumentException("kaboom")
}
})
watch(childActorRef)
val maker = (_: ActorRefFactory) => childActorRef
val supervisorActorRef = system.actorOf(Props(new supervisorActor(maker)))
supervisorActorRef!"testThis"
expectTerminated(childActorRef, 1.second)
}
I would expect the child actor to be stoped because of the supervisorStrategy Stop.
Instead I get this error:
java.lang.AssertionError: assertion failed: timeout (1 second) during expectMsg: Terminated
Any idea of why is this happening? Thank you
It seems that the childActorRef is not created with supervisorActorRef's context (I mean ActorContext of supervisorActor you created in the test code).
So childActor(childActorRef) is not a child of supervisorActor(supervisorActorRef) in your test code. That's why supervisor strategy of supervisorActor doesn't serve the purpose.

Handling exceptions in preStart in actors

I have a service that has a supervisor responsible for build children actors and handling their exceptions.
ServiceMain -> Supervisor -> DiscoveryActor
To build the DiscoveryActor I call the following
Await.result(supervisor ? (Props[Discovery], "Discovery"), Duration.create(60, SECONDS)) match {
case actor: ActorRef =>
discoveryActor = actor
case ex: Exception =>
logger.error("Failed to initialize discovery actor", ex)
sys.exit(1)
}
Which gets handled by this code in Supervisor
def receive = {
case p: Props => sender() ! context.actorOf(p)
case (p: Props, s: String) => sender() ! context.actorOf(p, s)
}
It is expected of the DiscoveryActor to throw an exception in preStart() if it can't research a host as configured. The exception ApiConnectionException is thrown by the actor and is captured by akka and turned into a ActorInitializationException.
I have tired catching this exception in the Await and with the superviseStrategy as below
override val supervisorStrategy =
AllForOneStrategy() {
case _: Exception => Escalate
}
But neither of these have managed to catch it, what I am attempting to do is catch such exception and exit the application.
If anyone can point out where I am going wrong or what I have missed then I shall be very grateful!
I simplified your code a bit just to get straight to the root of the problem. You can copy it and paste it into your editor. It uses ScalaTest suite.
SupervisorStrategy defined in Supervisor actor does catch an exception thrown by Discovery actor in preStart method. You might want to have a closer look at your own code.
Your Await block is trying to catch an exception, but it's not possible in this context. The Exception is thrown by Discovery actor, not send as a message. Ask pattern (?) you use simply awaits for a message to arrive though. Only use of SupervisorStrategy can get you back a thrown exception. Instead of escalating exception in your Supervisor you could send a message to your application Guardian actor saying that initialization failed so the application should exit. Or do it directly in your Supervisor.
import java.util.concurrent.TimeUnit
import akka.actor.SupervisorStrategy.Escalate
import akka.actor._
import akka.pattern.ask
import akka.testkit.{ImplicitSender, TestKit}
import akka.util.Timeout
import org.scalatest.{BeforeAndAfterAll, FunSuiteLike, Matchers}
import scala.concurrent.Await
abstract class ActorSuite(systemName: String)
extends TestKit(ActorSystem(systemName))
with FunSuiteLike
with ImplicitSender
with Matchers
with BeforeAndAfterAll {
override def afterAll {
TestKit.shutdownActorSystem(system)
}
}
class FailingActorInitializationSuite extends ActorSuite("failing-system") {
test("run it") {
val supervisor = system.actorOf(Props[Supervisor])
var discoveryActor: ActorRef = null
implicit val timeout = Timeout(60, TimeUnit.SECONDS)
Await.result(
supervisor ?(Props[Discovery], "Discovery"), timeout.duration) match {
case actor: ActorRef =>
discoveryActor = actor
}
}
}
class Supervisor extends Actor with ActorLogging {
override val supervisorStrategy =
AllForOneStrategy() {
case e: Exception =>
log.error(s"Caught an exception [${e.getCause.getMessage}] and escalating")
Escalate
}
override def receive: Receive = {
case (p: Props, s: String) => sender() ! context.actorOf(p, s)
}
}
class Discovery extends Actor {
override def preStart(): Unit = {
super.preStart()
throw new RuntimeException("Can't create")
}
override def receive: Actor.Receive = {
case _ =>
}
}

Resolving Akka futures from ask in the event of a failure

I am calling an Actor using the ask pattern within a Spray application, and returning the result as the HTTP response. I map failures from the actor to a custom error code.
val authActor = context.actorOf(Props[AuthenticationActor])
callService((authActor ? TokenAuthenticationRequest(token)).mapTo[LoggedInUser]) { user =>
complete(StatusCodes.OK, user)
}
def callService[T](f: => Future[T])(cb: T => RequestContext => Unit) = {
onComplete(f) {
case Success(value: T) => cb(value)
case Failure(ex: ServiceException) => complete(ex.statusCode, ex.errorMessage)
case e => complete(StatusCodes.InternalServerError, "Unable to complete the request. Please try again later.")
//In reality this returns a custom error object.
}
}
This works correctly when the authActor sends a failure, but if the authActor throws an exception, nothing happens until the ask timeout completes. For example:
override def receive: Receive = {
case _ => throw new ServiceException(ErrorCodes.AuthenticationFailed, "No valid session was found for that token")
}
I know that the Akka docs say that
To complete the future with an exception you need send a Failure message to the sender. This is not done automatically when an actor throws an exception while processing a message.
But given that I use asks for a lot of the interface between the Spray routing actors and the service actors, I would rather not wrap the receive part of every child actor with a try/catch. Is there a better way to achieve automatic handling of exceptions in child actors, and immediately resolve the future in the event of an exception?
Edit: this is my current solution. However, it's quite messy to do this for every child actor.
override def receive: Receive = {
case default =>
try {
default match {
case _ => throw new ServiceException("")//Actual code would go here
}
}
catch {
case se: ServiceException =>
logger.error("Service error raised:", se)
sender ! Failure(se)
case ex: Exception =>
sender ! Failure(ex)
throw ex
}
}
That way if it's an expected error (i.e. ServiceException), it's handled by creating a failure. If it's unexpected, it returns a failure immediately so the future is resolved, but then throws the exception so it can still be handled by the SupervisorStrategy.
If you want a way to provide automatic sending of a response back to the sender in case of an unexpected exception, then something like this could work for you:
trait FailurePropatingActor extends Actor{
override def preRestart(reason:Throwable, message:Option[Any]){
super.preRestart(reason, message)
sender() ! Status.Failure(reason)
}
}
We override preRestart and propagate the failure back to the sender as a Status.Failure which will cause an upstream Future to be failed. Also, it's important to call super.preRestart here as that's where child stopping happens. Using this in an actor looks something like this:
case class GetElement(list:List[Int], index:Int)
class MySimpleActor extends FailurePropatingActor {
def receive = {
case GetElement(list, i) =>
val result = list(i)
sender() ! result
}
}
If I was to call an instance of this actor like so:
import akka.pattern.ask
import concurrent.duration._
val system = ActorSystem("test")
import system.dispatcher
implicit val timeout = Timeout(2 seconds)
val ref = system.actorOf(Props[MySimpleActor])
val fut = ref ? GetElement(List(1,2,3), 6)
fut onComplete{
case util.Success(result) =>
println(s"success: $result")
case util.Failure(ex) =>
println(s"FAIL: ${ex.getMessage}")
ex.printStackTrace()
}
Then it would properly hit my Failure block. Now, the code in that base trait works well when Futures are not involved in the actor that is extending that trait, like the simple actor here. But if you use Futures then you need to be careful as exceptions that happen in the Future don't cause restarts in the actor and also, in preRestart, the call to sender() will not return the correct ref because the actor has already moved into the next message. An actor like this shows that issue:
class MyBadFutureUsingActor extends FailurePropatingActor{
import context.dispatcher
def receive = {
case GetElement(list, i) =>
val orig = sender()
val fut = Future{
val result = list(i)
orig ! result
}
}
}
If we were to use this actor in the previous test code, we would always get a timeout in the failure situation. To mitigate that, you need to pipe the results of futures back to the sender like so:
class MyGoodFutureUsingActor extends FailurePropatingActor{
import context.dispatcher
import akka.pattern.pipe
def receive = {
case GetElement(list, i) =>
val fut = Future{
list(i)
}
fut pipeTo sender()
}
}
In this particular case, the actor itself is not restarted because it did not encounter an uncaught exception. Now, if your actor needed to do some additional processing after the future, you can pipe back to self and explicitly fail when you get a Status.Failure:
class MyGoodFutureUsingActor extends FailurePropatingActor{
import context.dispatcher
import akka.pattern.pipe
def receive = {
case GetElement(list, i) =>
val fut = Future{
list(i)
}
fut.to(self, sender())
case d:Double =>
sender() ! d * 2
case Status.Failure(ex) =>
throw ex
}
}
If that behavior becomes common, you can make it available to whatever actors need it like so:
trait StatusFailureHandling{ me:Actor =>
def failureHandling:Receive = {
case Status.Failure(ex) =>
throw ex
}
}
class MyGoodFutureUsingActor extends FailurePropatingActor with StatusFailureHandling{
import context.dispatcher
import akka.pattern.pipe
def receive = myReceive orElse failureHandling
def myReceive:Receive = {
case GetElement(list, i) =>
val fut = Future{
list(i)
}
fut.to(self, sender())
case d:Double =>
sender() ! d * 2
}
}