Actor supervised by BackoffSupervisor loses stashed messages after restart - scala

Question

I have an actor with stash usage. Sometimes, when it crashes, it loses all stashed messages. I found that it depends on what supervision logic I use.
I wrote a simple example.
An actor with the stash:
case object WrongMessage
case object TestMessage
case object InitialMessage
class TestActor extends Actor with Stash {
override def receive: Receive = uninitializedReceive
def uninitializedReceive: Receive = {
case TestMessage =>
println(s"stash test message")
stash()
case WrongMessage =>
println(s"wrong message")
throw new Throwable("wrong message")
case InitialMessage =>
println(s"initial message")
context.become(initializedReceive)
unstashAll()
}
def initializedReceive: Receive = {
case TestMessage =>
println(s"test message")
}
}
In the following code, TestActor never receives stashed TestMessage:
object Test1 extends App {
implicit val system: ActorSystem = ActorSystem()
val actorRef = system.actorOf(BackoffSupervisor.props(Backoff.onFailure(
Props[TestActor], "TestActor", 1 seconds, 1 seconds, 0
).withSupervisorStrategy(OneForOneStrategy()({
case _ => SupervisorStrategy.Restart
}))))
actorRef ! TestMessage
Thread.sleep(5000L)
actorRef ! WrongMessage
Thread.sleep(5000L)
actorRef ! InitialMessage
}
But this code works well:
class SupervisionActor extends Actor {
val testActorRef: ActorRef = context.actorOf(Props[TestActor])
override def supervisorStrategy: SupervisorStrategy = OneForOneStrategy()({
case _ => SupervisorStrategy.Restart
})
override def receive: Receive = {
case message => testActorRef forward message
}
}
object Test2 extends App {
implicit val system: ActorSystem = ActorSystem()
val actorRef = system.actorOf(Props(classOf[SupervisionActor]))
actorRef ! TestMessage
Thread.sleep(5000L)
actorRef ! WrongMessage
Thread.sleep(5000L)
actorRef ! InitialMessage
}
I looked into sources and found that actor supervision uses
LocalActorRef.restart method which backed by system dispatcher logic, but BackoffSupervisor simply creates a new actor after termination of the old one. Is there any way to work around it?

Answer 1

I'm not sure one can make restart under BackoffSupervisor properly send stashed messages without some custom re-implementation effort.
As you've already pointed out that BackoffSupervisor does its own restart that bypasses the standard actor lifecycle. In fact, it's explicitly noted in the BackoffOnRestartSupervisor source code:
Whatever the final Directive is, we will translate all Restarts to our
own Restarts, which involves stopping the child.
In case you haven't read about this reported issue, it has a relevant discussion re: problem with Backoff.onFailure.
Backoff.onStop would also give the wanted BackoffSupervisor feature, but unfortunately it has its own use cases and won't be triggered by an exception.