In Akka Cookbook by Héctor Veiga Ortiz, the reader is told that
When an actor throws an exception, it sends a message to the supervisor, and the supervisor handles the failure by restarting that actor. It clears out the accumulated state of the actor, and creates a fresh new actor, means, it then restores the last value assigned to the state of old actor to the preRestart value.
However, I tried testing the following code, which suggests that what the author says isn't true.
import akka.actor._
import akka.actor.SupervisorStrategy._
import akka.util.Timeout
import scala.concurrent.Await
import scala.concurrent.duration._
import akka.pattern.ask
case object Error
case class StopActor(actorRef: ActorRef)
case object Inc
class LifeCycleActor extends Actor {
var sum = 1
override def preRestart(reason: Throwable, message: Option[Any]):Unit =
println(s"sum in preRestart is $sum")
override def preStart(): Unit = println(s"sum in preStart is $sum")
def receive = {
case Inc => sum += 1
case Error => throw new ArithmeticException()
case _ => println("default msg")
}
override def postStop(): Unit =
println(s"sum in postStop is ${sum * 3}")
override def postRestart(reason: Throwable): Unit = {
sum = sum * 2
println(s"sum in postRestart is $sum")
}
}
class Supervisor extends Actor {
override val supervisorStrategy = OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 minute){
case _: ArithmeticException => Restart
case t =>
super.supervisorStrategy.decider.applyOrElse(t, (_:Any)=>Escalate)
}
def receive = {
case (props: Props, name: String) => sender ! context.actorOf(props, name)
case StopActor(actorRef) => context.stop(actorRef)
}
}
object ActorLifeCycle extends App {
implicit val timeout = Timeout(2 seconds)
val actorSystem = ActorSystem("Supervision")
val supervisor = actorSystem.actorOf(Props[Supervisor], "supervisor")
val childFuture = supervisor ? (Props(new LifeCycleActor), "LifeCycleActor")
val child = Await.result(childFuture.mapTo[ActorRef], 2 seconds)
child ! Inc
child ! Error
Thread.sleep(1000)
supervisor ! StopActor(child)
}
The output I get is as follows.
sbt:chpt2_ActorLifeCycle> runMain ActorLifeCycle
sum in preStart is 1
sum in preRestart is 2
[ERROR] [11/08/2018 20:06:01.423]
[Supervision-akka.actor.default-dispatcher-4]
[akka://Supervision/user/supervisor/LifeCycleActor] null
java.lang.ArithmeticException
sum in postRestart is 2
sum in postStop is 6
If what the author says is true, the final value should be double what it is.
First I guess you forget to add sum += 1 when receive message Inc in child actor when post the question, please modify. Otherwise, as I test, you cannot get your output.
Next explain your code:
From next diagram, you can see the preReStart is called on old instance, not on new instance.
There is also a desc about this, detail here
The old actor is informed by calling preRestart with the exception which caused the restart and the message which triggered that exception; the latter may be None if the restart was not caused by processing a message, e.g. when a supervisor does not trap the exception and is restarted in turn by its supervisor, or if an actor is restarted due to a sibling’s failure. If the message is available, then that message’s sender is also accessible in the usual way (i.e. by calling sender). This method is the best place for cleaning up, preparing hand-over to the fresh actor instance, etc. By default it stops all children and calls postStop.
The initial factory from the actorOf call is used to produce the fresh instance.
The new actor’s postRestart method is invoked with the exception which caused the restart. By default the preStart is called, just as in the normal start-up case.
So, for your example:
when call preRestart, it print the sum of old actor, that is 2, notice: you have inc it.
when call postRestart, it print the sum of new actor, that is the initial value 1 with the calculation sum = sum * 2, finally print 2, not 4. The Inc message just receive on old instance, not on new instance.
Finally, the content of the book:
When an actor throws an exception, it sends a message to the supervisor, and the supervisor handles the failure by restarting that actor. It clears out the accumulated state of the actor, and creates a fresh new actor, means, it then restores the last value assigned to the state of old actor to the preRestart value.
I think what you concern is it then restores the last value assigned to the state of old actor to the preRestart value. I don't quite know what it means, if you just think it assign the last value of old actor to preRestart function, then it is correct, as it just run on old instance, otherwise, it seems conflict with akka official guide & experiment; And, if want to restore the value, we may had to use resume not restart.
Anyway, I think we should use the akka offical document as a standard, and understand the correct logic.
Related
I'm handling a Future in an Actor with the pipeTo pattern which seems to be working ok.
In the example I've mocked up below UserProxyActor asks UserActivityActor with a Get(userId) message.
I want to include the parameters of the Get message in the response so that the receiving actor has everything it needs to process the message. For example, insert the activities into a DB with the related userId.
Is the userId available in the map call or does it get "closed over"?
Is this going to work because the ask pattern will block?
Is there some much nicer way to do this that I haven't come across?
class UserActivityActor(repository: UserActivityRepository) extends Actor {
import akka.pattern.pipe
import UserActivityActor._
implicit val ec: ExecutionContext = context.dispatcher
def receive = {
case Get(userId) =>
// user's historical activities are retrieved
// via the separate repository
repository.queryHistoricalActivities(userId)
.map(a => UserActivityReceived(userId, a)) // wrap the completed future value in a message
.recover{case ex => RepoFailure(ex.getMessage)} // wrap failure in a local message type
.pipeTo(sender())
class UserProxyActor(userActivities: ActorRef) extends Actor {
import UserProxyActor._
import akka.pattern.{ ask, pipe }
implicit val ec: ExecutionContext = context.dispatcher
implicit val timeout = Timeout(5 seconds)
def receive = {
case GetUserActivities(user) =>
(userActivities ? UserActivityActor.Get(user))
.pipeTo(sender())
}
}
Is the userId available in the map call or does it get "closed over"?
Get should be immutable if yes userId will be available.
Is this going to work because the ask pattern will block?
The actor receives a Get message, creates a Future and then process another message. No blocking at all. Ask's Future is not completed until the Future is completed or a timeout occurs.
Is there some much nicer way to do this that I haven't come across?
Looks nice if repository.queryHistoricalActivities(userId) is not blocking call.
I don't think there's anything wrong with what you did.
The only thing I want to mention is that I personally prefer not to use ask when communicating between actors (and keep in mind, this is largely a personal preference), so I'd do something like this:
class UserActivityActor(repository: UserActivityRepository) extends Actor {
import akka.pattern.pipe
import UserActivityActor._
implicit val ec: ExecutionContext = context.dispatcher
def receive = {
case Get(userId) =>
// user's historical activities are retrieved
// via the separate repository
repository.queryHistoricalActivities(userId)
.map(a => UserActivityReceived(userId, a)) // wrap the completed future value in a message
.recover{case ex => RepoFailure(ex.getMessage)} // wrap failure in a local message type
.pipeTo(sender())
class UserProxyActor(userActivities: ActorRef) extends Actor {
import UserProxyActor._
def receive = {
case GetUserActivities(user, sender()) =>
userActivities.forward(UserActivityActor.Get(user))
}
}
This does remove the time out behavior though (in your original implementation, the requesting actor will wait at most 5 seconds, or receive a failure, in mine, it may wait indefinitely).
I'm testing an actor that uses an asnychronous future-based API. The actor uses the pipe pattern to send a message to itself when a future completes:
import akka.pattern.pipe
// ...
// somewhere in the actor's receive method
futureBasedApi.doSomething().pipeTo(self)
In my test I mock the API so I control future completion via promises. However, this is interleaved with other messages sent directly to the actor:
myActor ! Message("A")
promiseFromApiCall.success(Message("B"))
myActor ! Message("C")
Now I'm wondering how I can guarantee that the actor receives and
processes message B between message A and C in my test because message B is actually sent in another thread, so I can't control the order
in which the actor's mailbox receives the messages.
I thought about several possible solutions:
sleep after each message for a few milliseconds to make another
order very unlikely
wait for the actor to acknowledge each message, although
acknowledgement is only required for testing
send message B directly to the actor to simulate completion of the
future and write a separate test that ensures that the pipe pattern
is properly used (the test above would not fail if the actor would
not pipe the result message to itself)
I don't really like either of these options but I tend to use the last
one. Is there another better way I can enforce a certain message order in the tests?
Clarification: The question is not how to deal with the fact that messages might be received in random order in production. Controlling the order in the test is essential to make sure that the actor can actually deal with different message orders.
One idea is to define a flag in your actor that indicates whether the actor has received message B. When the actor receives message C, the actor can stash that message C if the flag is false, then unstash it once the actor receives message B. For example:
class MyActor extends Actor with Stash {
def receiveBlock(seenMsgB: Boolean, seenMsgC: Boolean): Receive = {
case MakeApiCall =>
callExternalApi().mapTo[MessageB].pipeTo(self)
case m: MessageB if seenMsgC => // assume msg C has been stashed
unstashAll()
// ...do something with msg B
become(receiveBlock(true, seenMsgC)) // true, true
case m: MessageB if !seenMsgC =>
// ...do something with message B
become(receiveBlock(true, seenMsgC)) // true, false
case m: MessageC if seenMsgB =>
// ...do something with message C
context.become(receiveBlock(seenMsgB, true)) // true, true
case m: MessageC if !seenMsgB =>
stash()
context.become(receiveBlock(seenMsgB, true)) // false, true
case ...
}
def receive = receiveBlock(false, false)
}
After reading a lot more about akka, I finally found a better solution: Replacing the actor mailbox with one I can observe in the tests. This way I can wait until the actor receives a new message after I complete the promise. Only then the next message is sent. The code for this TestingMailbox is given at the end of the post.
Update: In Akka Typed this can be achieved very elegantly with a BehaviorInterceptor. Just wrap the Behavior under test with a custom interceptor that forwards all messages and signals but lets you observe them.
The mailbox solution for untyped Akka is given below.
The actor can be configured like this:
actorUnderTest = system.actorOf(Props[MyActor]).withMailbox("testing-mailbox"))
I have to make sure the "testing-mailbox" is known by the actor system by providing a configuration:
class MyTest extends TestKit(ActorSystem("some name",
ConfigFactory.parseString("""{
testing-mailbox = {
mailbox-type = "my.package.TestingMailbox"
}
}""")))
with BeforeAndAfterAll // ... and so on
With this being set up, I can change my test like this:
myActor ! Message("A")
val nextMessage = TestingMailbox.nextMessage(actorUnderTest)
promiseFromApiCall.success(Message("B"))
Await.ready(nextMessage, 3.seconds)
myActor ! Message("C")
With a little helper method, I can even write it like this:
myActor ! Message("A")
receiveMessageAfter { promiseFromApiCall.success(Message("B")) }
myActor ! Message("C")
And this is my custom mailbox:
import akka.actor.{ActorRef, ActorSystem}
import akka.dispatch._
import com.typesafe.config.Config
import scala.concurrent.{Future, Promise}
object TestingMailbox {
val promisesByReceiver =
scala.collection.concurrent.TrieMap[ActorRef, Promise[Any]]()
class MessageQueue extends UnboundedMailbox.MessageQueue {
override def enqueue(receiver: ActorRef, handle: Envelope): Unit = {
super.enqueue(receiver, handle)
promisesByReceiver.remove(receiver).foreach(_.success(handle.message))
}
}
def nextMessage(receiver: ActorRef): Future[Any] =
promisesByReceiver.getOrElseUpdate(receiver, Promise[Any]).future
}
class TestingMailbox extends MailboxType
with ProducesMessageQueue[TestingMailbox.MessageQueue] {
import TestingMailbox._
def this(settings: ActorSystem.Settings, config: Config) = this()
final override def create(owner: Option[ActorRef],
system: Option[ActorSystem]) =
new MessageQueue()
}
If it is so important to order messages you should use ask (?) which returns Future and chain them even if you dont expect any response from an actor.
For some reason I have to use gRPC and Akka at the same time. When this actor is started as a top actor, nothing goes wrong (in this little demo). But when it becomes a child actor, it cannot receive any messages, and the following is logged:
[default-akka.actor.default-dispatcher-6] [akka://default/user/Grpc] Message [AkkaMessage.package$GlobalStart] from Actor[akka://default/user/TrackerCore#-808631363] to Actor[akka://default/user/Grpc#-1834173068] was not delivered. [1] dead letters encountered.
The example core:
class GrpcActor() extends Actor {
val ec = scala.concurrent.ExecutionContext.global
val service = grpcService.bindService(new GrpcServerImpl(), ec)
override def receive: Receive = {
case GlobalStart() => {
println("GlobalStart")
}
...
}
}
I tried to create a new ExecutionContext like:
scala.concurrent.ExecutionContext.fromExecutor(Executors.newFixedThreadPool(10))
Why is this happening, and how do can I debug a dead letters problem like this (no exception is thrown)?
Update:
Sorry I didn't list everything here. I used normal Main method to test GrpcActor as top actor, and ScalaTest to test it as child actor, which is a mistake.
class GrpcActorTest extends FlatSpec with Matchers{
implicit val system = ActorSystem()
val actor: ActorRef = system.actorOf(Props[GrpcActor])
actor ! GlobalStart()
}
It is this empty test suite that active shutdown the whole actor system. But the problem is with this line
val service = grpcService.bindService(new GrpcServerImpl(), ec)
the delivery of GlobalStart() was delayed after the shutdown.
Without that line, message can be delivered before the shutdown.
Is this a normal behavior?
(My guess: it happened that the GlobalStart() was queued after the shutdown message with that line, which did some heavy work and made the difference in time)
One way to address the problem is to make service a lazy val:
class GrpcActor extends Actor {
...
lazy val service = grpcService.bindService(new GrpcServerImpl(), ec)
...
}
A lazy val is useful for long-running operations: in this case, it defers the initialization of service until it is used for the first time. Without the lazy modifier, service is initialized when the actor is created.
An alternative approach is to add a Thread.sleep in your test to prevent the actor system from shutting down before the actor has fully initialized:
class GrpcActorTest extends FlatSpec with Matchers {
...
actor ! GlobalStart()
Thread.sleep(5000) // or whatever length of time is needed to initialize the actor
}
(As a side note, consider using the Akka Testkit for your actor tests.)
Add a supervisor strategy to its parent, add println to the actor lifecycle. There is something that kill your actor . Finally, if you provide a complete example maybe I can say more :)
The Akka documentation says that the mailbox is NOT scanned for messages. Each message is handled after another (FIFO by default) in the order of their arrival. However, when I send a message from an actor to another which is not matched be the receiving actor, it is neither moved to the deadletters actor (which would appear in the log I suppose) nor does it block handling the next message in the mailbox which arrives one second later and can be handled correctly.
What does happen to the unmatched message from the mailbox?
I am using Scala 2.10.4 and Akka 2.4-SNAPSHOT in sbt.
package main.scala
import akka.actor._
class SRActor(dest: ActorRef) extends Actor with ActorLogging {
dest ! A
dest ! B
context.stop(self)
override def receive = {
case _ => {
log.info("Finally got something")
}
}
}
class SRActorReceiver extends Actor with ActorLogging {
override def receive = {
case B =>
log.info("Finally got B")
}
}
Actor creation:
package main.scala
import akka.actor._
case object A
case object B
object ErrorApp extends App {
// SR: Send nowhere received
var system6 = ActorSystem("ErrorActorSystem")
val srActorReceiver = system6.actorOf(Props(classOf[SRActorReceiver]), "sractorreceiver")
val sractor = system6.actorOf(Props(classOf[SRActor], srActorReceiver), "sractor")
// wait until actors have finished
Thread.sleep(1000)
system6.shutdown
Copied from docs
Please note that the Akka Actor receive message loop is exhaustive,
which is different compared to Erlang and the late Scala Actors. This
means that you need to provide a pattern match for all messages that
it can accept and if you want to be able to handle unknown messages
then you need to have a default case as in the example above.
Otherwise an akka.actor.UnhandledMessage(message, sender, recipient)
will be published to the ActorSystem's EventStream.
There is also unhandled method in Actor trait that you can override. (docs)
def unhandled(message: Any): Unit
User overridable callback.
Is called when a message isn't handled by the current behavior of the actor by default it fails with either a akka.actor.DeathPactException (in case of an unhandled akka.actor.Terminated message) or publishes an akka.actor.UnhandledMessage to the actor's system's akka.event.EventStream
I read in the Akka docs that it's dangerous to close over variables from an enclosing actor.
Warning
In this case you need to carefully avoid closing over the
containing actor’s reference, i.e. do not call methods on the
enclosing actor from within the anonymous Actor class. This would
break the actor encapsulation and may introduce synchronization bugs
and race conditions because the other actor’s code will be scheduled
concurrently to the enclosing actor.
Now, I have two actors, one of which requests something from the second and does something with the result. In this example below I have put together, actor Accumulator retrieves numbers from actor NumberGenerator and adds them up, reporting the sum along the way.
This can be done in at least two different ways as this example shows with two different receive functions (A vs B). The difference between the two is that A does not close over the counter variable; instead it awaits an integer and sums it up, while B creates a Future that closes over counter and does the sum. This happens within an anonymous actor created just to process onSuccess, if I understand properly how this works.
import com.esotericsoftware.minlog.Log
import akka.actor.{Actor, Props}
import akka.pattern.{ask, pipe}
import akka.util.Timeout
import akka.util.duration._
case object Start
case object Request
object ActorTest {
var wake = 0
val accRef = Main.actorSystem.actorOf(Props[Accumulator], name = "accumulator")
val genRef = Main.actorSystem.actorOf(Props[NumberGenerator], name = "generator")
Log.info("ActorTest", "Starting !")
accRef ! Start
}
class Accumulator extends Actor {
var counter = 0
implicit val timeout = Timeout(5 seconds)
// A: WITHOUT CLOSURE
def receive = {
case Start => ask(ActorTest.genRef, Request).mapTo[Int] pipeTo self
case x: Int => counter += x; Log.info("Accumulator", "counter = " + counter); self ! Start
}
// B: WITH CLOSURE
def receive = {
case Start => ask(ActorTest.genRef, Request).mapTo[Int] onSuccess {
case x: Int => counter += x; Log.info("Accumulator", "counter = " + counter); self ! Start
}
}
}
class NumberGenerator extends Actor {
val rand = new java.util.Random()
def receive = {
case Request => sender ! rand.nextInt(11)-5
}
}
Is it absolutely evil to use closures in this case ? Of course I could use an AtomicInteger instead of an Int, or in some networking scenario using, say, netty, issue a write operation on a threadsafe channel, but this is not my point here.
To the risk of asking the ridiculous: is there a way for the Future's onSuccess to execute in this actor instead of an anonymous middle actor, without defining a case in the receive function ?
EDIT
To put it more clearly, my question is: Is there a way to force a series of Futures to run in the same thread as a given Actor ?
The problem is that the onSuccess is going to run in a different thread than the thread the actor's receive is going to run in. You could use the pipeTo approach, or use an Agent. Making counter an AtomicInteger would solve the problem, but it's not so clean - that is, it breaks the Actor model.
The easiest way of implementing such design is by using "fire-and-forget" semantic:
class Accumulator extends Actor {
private[this] var counter = 0
def receive = {
case Start => ActorTest.genRef ! Request
case x: Int => {
counter += x
Log.info("Accumulator", "counter = " + counter)
self ! Start
}
}
}
This solution is fully asynchronous, and you don't need any timeout.