actor handling multiple messages at the same time - scala

I know one of the advantages of the actor model is that by handling only one message at a time, concurrency issues are simplified. But it seems to me that my actor is handling multiple messages. In pseudo code I have
var status = 0
def receive = {
case DoSomething =>
val dest = sender()
status = 0
for {
otherActor <- resolveOtherActor("/user/OtherActor")
} yield {
for {
res <- {status = 1
otherActor ? doSomething1
}
res <- {status = 2
otherActor ? doSomething2
}
} yield {
dest ! status
}
}
case GetStatus => sender() ! status
}
If I send a DoSomething messages to this actor, and then immediately send GetStatus to this actor repeatedly, I will see status 0, 1 and 2 coming back in sequence. If the actor model only handled one message at a time, I would only ever see status 2 being returned, since I wouldn't have access to the intermediate status.
It seems that locks are still necessary with the actor pattern. What am I missing?

All bets are off when you close over an actor's mutable state and expose it to other threads, which is what your code is doing when it mutates status inside a (nested) Future. The Akka documentation clearly warns against this.
An actor does process one message at a time:
var status = 0
def receive = {
case IncrementStatus =>
status += 1
case GetStatus =>
val s = status
sender ! s
}
Sending an IncrementStatus, another IncrementStatus, then a GetStatus message from the same sender to the above actor will cause that sender to receive a 2.
However, trying to do the same thing with Futures does not guarantee the same outcome, because a Future is completed asynchronously. For example:
object NumService {
// calculates arg + 1 in the future
def addOne(arg: Int): Future[Int] = {
Future { arg + 1 }
}
}
class MyActor extends Actor {
var status = 0
def receive = {
case IncrementStatusInFuture =>
val s = status
NumService.addOne(s)
.map(UpdateStatus(_))
.pipeTo(self)
case UpdateStatus(num) =>
status = num
case GetStatus =>
val s = status
sender ! s
}
}
We map the Future to create a Future[UpdateStatus], then pipe to the actor itself the result of that Future.
If we send an IncrementStatusInFuture, another IncrementStatusInFuture, then a GetStatus message to MyActor from the same sender, we cannot guarantee that the sender will receive a 2. The actor processes those three messages in order, but one or both of the calls to NumService.addOne might not have completed by the time the actor processes the GetStatus message. This nondeterministic behavior is a characteristic of a Future; it is not a violation of the actor principle of one-message-at-a-time processing.

Related

Ask pattern which doesn't wait after result

I have an akka actor which is doing some things sequentially, it can be summarized like that :
def receive = {
case Request(...) => {
val id = generatedId()
sender() ! id
doSomeOtherStuffs()
}
}
The actor is called using some ask pattern
val id = myActor ? Request(...)
The problem here is the code calling my actor is waiting for all of the doSomeOtherStuffs() thing to complete.
What is the correct way to allow an actor to reply and then have the calling code stop waiting for that actor ?
How about this? Decouple the actual execution of other stuff from answering the Request
def receive = {
case Request(...) => {
val id = generatedId()
sender() ! id
self ! DoSomethingElse(...)
}
case DoSomethingElse(...) =>
doSomethingElse()
}

Send message to actor after restart from Supervisor

I am using BackoffSupervisor strategy to create a child actor that has to process some message. I want to implement a very simple restart strategy, in which in case of exception:
Child propagates failing message to supervisor
Supervisor restarts child and sends the failing message again.
Supervisor gives up after 3 retries
Akka persistence is not an option
So far what I have is this:
Supervisor definition:
val childProps = Props(new SenderActor())
val supervisor = BackoffSupervisor.props(
Backoff.onFailure(
childProps,
childName = cmd.hashCode.toString,
minBackoff = 1.seconds,
maxBackoff = 2.seconds,
randomFactor = 0.2
)
.withSupervisorStrategy(
OneForOneStrategy(maxNrOfRetries = 3, loggingEnabled = true) {
case msg: MessageException => {
println("caught specific message!")
SupervisorStrategy.Restart
}
case _: Exception => SupervisorStrategy.Restart
case _ ⇒ SupervisorStrategy.Escalate
})
)
val sup = context.actorOf(supervisor)
sup ! cmd
Child actor that is supposed to send the e-mail, but fails (throws some Exception) and propagates Exception back to supervisor:
class SenderActor() extends Actor {
def fakeSendMail():Unit = {
Thread.sleep(1000)
throw new Exception("surprising exception")
}
override def receive: Receive = {
case cmd: NewMail =>
println("new mail received routee")
try {
fakeSendMail()
} catch {
case t => throw MessageException(cmd, t)
}
}
}
In the above code I wrap any exception into custom class MessageException that gets propagated to SupervisorStrategy, but how to propagate it further to the new child to force reprocessing? Is this the right approach?
Edit. I attempted to resent the message to the Actor on preRestart hook, but somehow the hook is not being triggered:
class SenderActor() extends Actor {
def fakeSendMail():Unit = {
Thread.sleep(1000)
// println("mail sent!")
throw new Exception("surprising exception")
}
override def preStart(): Unit = {
println("child starting")
}
override def preRestart(reason: Throwable, message: Option[Any]): Unit = {
reason match {
case m: MessageException => {
println("aaaaa")
message.foreach(self ! _)
}
case _ => println("bbbb")
}
}
override def postStop(): Unit = {
println("child stopping")
}
override def receive: Receive = {
case cmd: NewMail =>
println("new mail received routee")
try {
fakeSendMail()
} catch {
case t => throw MessageException(cmd, t)
}
}
}
This gives me something similar to following output:
new mail received routee
caught specific message!
child stopping
[ERROR] [01/26/2018 10:15:35.690]
[example-akka.actor.default-dispatcher-2]
[akka://example/user/persistentActor-4-scala/$a/1962829645] Could not
process message sample.persistence.MessageException:
Could not process message <stacktrace>
child starting
But no logs from preRestart hook
The reason that the child's preRestart hook is not invoked is because Backoff.onFailure uses BackoffOnRestartSupervisor underneath the covers, which replaces the default restart behavior with a stop-and-delayed-start behavior that is consistent with the backoff policy. In other words, when using Backoff.onFailure, when a child is restarted, the child's preRestart method is not called because the underlying supervisor actually stops the child, then starts it again later. (Using Backoff.onStop can trigger the child's preRestart hook, but that's tangential to the present discussion.)
The BackoffSupervisor API doesn't support the automatic resending of a message when the supervisor's child restarts: you have to implement this behavior yourself. An idea for retrying messages is to let the BackoffSupervisor's supervisor handle it. For example:
val supervisor = BackoffSupervisor.props(
Backoff.onFailure(
...
).withReplyWhileStopped(ChildIsStopped)
).withSupervisorStrategy(
OneForOneStrategy(maxNrOfRetries = 3, loggingEnabled = true) {
case msg: MessageException =>
println("caught specific message!")
self ! Error(msg.cmd) // replace cmd with whatever the property name is
SupervisorStrategy.Restart
case ...
})
)
val sup = context.actorOf(supervisor)
def receive = {
case cmd: NewMail =>
sup ! cmd
case Error(cmd) =>
timers.startSingleTimer(cmd.id, Replay(cmd), 10.seconds)
// We assume that NewMail has an id field. Also, adjust the time as needed.
case Replay(cmd) =>
sup ! cmd
case ChildIsStopped =>
println("child is stopped")
}
In the above code, the NewMail message embedded in the MessageException is wrapped in a custom case class (in order to easily distinguish it from a "normal"/new NewMail message) and sent to self. In this context, self is the actor that created the BackoffSupervisor. This enclosing actor then uses a single timer to replay the original message at some point. This point in time should be far enough in the future such that the BackoffSupervisor can potentially exhaust SenderActor's restart attempts, so that the child can have ample opportunity to get in a "good" state before it receives the resent message. Obviously this example involves only one message resend regardless of the number of child restarts.
Another idea is to create a BackoffSupervisor-SenderActor pair for every NewMail message, and have the SenderActor send the NewMail message to itself in the preStart hook. One concern with this approach is the cleaning up of resources; i.e., shutting down the BackoffSupervisors (which will, in turn, shut down their respective SenderActor children) when the processing is successful or when the child restarts are exhausted. A map of NewMail ids to (ActorRef, Int) tuples (in which the ActorRef is a reference to a BackoffSupervisor actor, and the Int is the number of restart attempts) would be helpful in this case:
class Overlord extends Actor {
var state = Map[Long, (ActorRef, Int)]() // assuming the mail id is a Long
def receive = {
case cmd: NewMail =>
val childProps = Props(new SenderActor(cmd, self))
val supervisor = BackoffSupervisor.props(
Backoff.onFailure(
...
).withSupervisorStrategy(
OneForOneStrategy(maxNrOfRetries = 3, loggingEnabled = true) {
case msg: MessageException =>
println("caught specific message!")
self ! Error(msg.cmd)
SupervisorStrategy.Restart
case ...
})
)
val sup = context.actorOf(supervisor)
state += (cmd.id -> (sup, 0))
case ProcessingDone(cmdId) =>
state.get(cmdId) match {
case Some((backoffSup, _)) =>
context.stop(backoffSup)
state -= cmdId
case None =>
println(s"${cmdId} not found")
}
case Error(cmd) =>
val cmdId = cmd.id
state.get(cmdId) match {
case Some((backoffSup, numRetries)) =>
if (numRetries == 3) {
println(s"${cmdId} has already been retried 3 times. Giving up.")
context.stop(backoffSup)
state -= cmdId
} else
state += (cmdId -> (backoffSup, numRetries + 1))
case None =>
println(s"${cmdId} not found")
}
case ...
}
}
Note that SenderActor in the above example takes a NewMail and an ActorRef as constructor arguments. The latter argument allows the SenderActor to send a custom ProcessingDone message to the enclosing actor:
class SenderActor(cmd: NewMail, target: ActorRef) extends Actor {
override def preStart(): Unit = {
println(s"child starting, sending ${cmd} to self")
self ! cmd
}
def fakeSendMail(): Unit = ...
def receive = {
case cmd: NewMail => ...
}
}
Obviously the SenderActor is set up to fail every time with the current implementation of fakeSendMail. I'll leave the additional changes needed in SenderActor to implement the happy path, in which SenderActor sends a ProcessingDone message to target, to you.
In the good solution that #chunjef provides, he alert about the risk of schedule a job resend before the backoff supervisor has started the worker
This enclosing actor then uses a single timer to replay the original message at some point. This point in time should be far enough in the future such that the BackoffSupervisor can potentially exhaust SenderActor's restart attempts, so that the child can have ample opportunity to get in a "good" state before it receives the resent message.
If this happens, the scenario will be jobs going to dead letters and no further progress will be done.
I've made a simplified fiddle with this scenario.
So, the schedule delay should be larger than the maxBackoff, and this could represent an impact in job completion time.
A possible solution to avoid this scenario is making the worker actor to send a message to his father when is ready to work, like here.
The failed child actor is available as the sender in your supervisor strategy. Quoting https://doc.akka.io/docs/akka/current/fault-tolerance.html#creating-a-supervisor-strategy:
If the strategy is declared inside the supervising actor (as opposed
to within a companion object) its decider has access to all internal
state of the actor in a thread-safe fashion, including obtaining a
reference to the currently failed child (available as the sender of
the failure message).
Sending emails is a dangerous operation with some third party software in your case. Why not to apply Circuit Breaker pattern and skip the sender actor entirely? Also, you can still have an actor (with some Backoff Supervisor) and Circuit Breaker inside it (if that makes sense for you).

Akka Router increment counter on message arrival from routees

I'm trying to keep counting on each successful import. But here is a problem - Counter works if the router receives a message from its parent but if I'm trying to send a message from its children it receives it but doesn't update the global variable that is out of the scope.
I know it sounds complicated. Let me show you the code.
Here is the router
class Watcher(size: Int) extends Actor {
var router = {
val routees = Vector.fill(size) {
val w = context.actorOf(
Props[Worker]
)
context.watch(w)
ActorRefRoutee(w)
}
Router(RoundRobinRoutingLogic(), routees)
}
var sent = 0
override def supervisorStrategy(): SupervisorStrategy = OneForOneStrategy(maxNrOfRetries = 100) {
case _: DocumentNotFoundException => {
Resume
}
case _: Exception => Escalate
}
override def receive: Receive = {
case container: MessageContainer =>
router.route(container, sender)
case Success =>
sent += 1
case GetValue =>
sender ! sent
case Terminated(a) =>
router.removeRoutee(a)
val w = context.actorOf(Props[Worker])
context.watch(w)
router = router.addRoutee(w)
case undef =>
println(s"${this.getClass} received undefinable message: $undef")
}
}
Here is the worker
class Worker() extends Actor with ActorLogging {
var messages = Seq[MessageContainer]()
var received = 0
override def receive: Receive = {
case container: MessageContainer =>
try {
importMessage(container.message, container.repo)
context.parent ! Success
} catch {
case e: Exception =>
throw e
}
case e: Error =>
log.info(s"Error occurred $e")
sender ! e
case undef => println(s"${this.getClass} received undefinable message: $undef")
}
}
So on supervisor ? GetValue I get 0 but suppose to have 1000.The strangest thing is that when I debug it with the breakpoint right on the case Success => ... the value is incremented every time the new message arrives. But supervisor ? GetValue still returns 0.
Let's assume I want to count on case container: MessageContainer => ... and it will magically work; I'll get desirable number, but it doesn't show if I actually imported anything. What's going on?
Here is the test case.
#Test
def testRouter(): Unit = {
val system = ActorSystem("RouterTestSystem")
// val serv = AddressFromURIString("akka.tcp://master#host:1334")
val supervisor = system.actorOf(Props(new Watcher(20)))//.withDeploy(akka.actor.Deploy(scope = RemoteScope(serv))))
val repo = coreSession.getRepositoryName
val containers = (0 until num)
.map(_ => MessageContainer(MessageFactory.generate("/"), repo))
val watch = Stopwatch.createStarted()
(0 until num).par
.foreach( i => {
supervisor ! containers.apply(i)
})
implicit val timeout = Timeout(60 seconds)
val future = supervisor ? GetValue
val result = Await.result(future, timeout.duration).asInstanceOf[Int]
val speed = result / (watch.elapsed(TimeUnit.MILLISECONDS) / 1000.0)
println(f"Import speed: $speed%.2f")
assertEquals(num, result)
}
Can you please explained it in details. Why is it happening? Why only on message received from the children? Another approach?
Well... there can be many potential problems hidden in the parts of code that you have not shared. But, for the sake of this discussion I will assume that everything else is fine and we will just discuss problems with your shared code.
Now, let me explain a bit about Actors. To put things simply, every actor has a mailbox (where it keeps messages in the sequence they were received) and processes them one by one in the order they were received. Since the mailbox is used like a Queue we will refer to it as a Queue in this discussion.
Also... I don't know what this container.apply(i) is going to return... so I will refer to the return value of that container.apply(1) as MessageContainer__1
In your test runner you are first creating an instance of Watcher,
val supervisor = system.actorOf(Props(new Watcher(20)))
Now, lets say that you are sending these 2 messages (num = 2) to supervisor,
So supervisor's mailbox will look something like,
Queue(MessageContainer__0, MessageContainer__1)
Then you send it another message GetValue so the mailbox will look like,
Queue(MessageContainer__0, MessageContainer__1, GetValue)
Now the actor will process the first message and pass it to the workers, the mail-box will look like,
Queue(MessageContainer__1, GetValue)
Now even if your worker is ultra-fast and instantaneous in sending the reply the mailbox will look like,
Queue(MessageContainer__1, GetValue, Success)
And now since your worker super-ultra-fast and instantaneously replies with a Success, the state after passing the second MessageContainer will look like,
Queue(GetValue, Success, Success)
And... here is the root of your problem. The Supervisor sees the GetValue massage before any Success messages, no matter how fast your workers are.
And thus it will process GetValue and reply with current value of sent which is 0.

Scala counter not updating when Akka message is received

I'm trying to make a small Akka example work using the following code:
class Sender extends Actor {
#volatile var numEchoes = 0
def receive = {
case Echo => {
numEchoes += 1
println(numEchoes)
if(numEchoes < Main.NUMBER_OF_ECHOES) {
println("Heard an echo...")
}
else {
// all echoes have been received.
println("All echoes heard.")
context.system.terminate
}
}
}
}
This actor is going to receive Echo messages from several Receiver actors:
class Receiver extends Actor {
def receive = {
case Echo => sender ! Echo
}
}
However, numEchoes always prints out 1. If I was doing something wrong, I'd expect 0, but this value makes me believe something else is at play here, but I'm unable to figure out what it is.
The rest of the code is just sending this specific actor a certain number of messages (lets assume 10). I see Heard an echo... printed out 10 times, but the value of this variable does not change.
#wheaties was correct. The error was in the Main object, due to some late and careless copy pasting... When I was creating 1 Sender and 10 Receiver actors, this is the code I was using:
// Create the actor that will send out the messages
val sender = actorSystem.actorOf(Props[Sender], "sender")
var actorList = List[ActorRef]()
for(i <- 1 to NUMBER_OF_ECHOES) {
val receiver = actorSystem.actorOf(Props[Sender], "receiver"+i)
actorList ::= receiver
}
Of course, creating the receivers as Senders is completely wrong, so all I had to do was fix the typo and correctly spawn the actors:
// Create the actor that will send out the messages
val sender = actorSystem.actorOf(Props[Sender], "sender")
var actorList = List[ActorRef]()
for(i <- 1 to NUMBER_OF_ECHOES) {
val receiver = actorSystem.actorOf(Props[Receiver], "receiver"+i)
actorList ::= receiver
}

Scala Akka Play, Future doesn't return

I'm using Scala 2.10, Akka 2.1 and Play 2.1. When I send an http request to my backend, I ask one actor to compute something. The idea is to return the result of the calculation if it returns before a timeout, otherwise a different string. See code below.
val futureInt: Future[Int] = ask(testActor, Calculate(number.toInt)).mapTo[Int]
val timeoutFuture = play.api.libs.concurrent.Promise.timeout("Oops", 2.seconds)
Async {
Future.firstCompletedOf(Seq(futureInt, timeoutFuture)).map {
case i: Int => Ok("Got result " + i)
case t: String => Ok("timeout expired")
}
}
The actor is as follows:
class TestActor() extends Actor {
def receive = {
case Calculate(tonumber: Int) =>
for (a <- 1 to tonumber) {
val c: Double = scala.math.pow(a, 2)
println("a: " + a + ", c: " + c)
}
12345 // hardcoded value to return when the calculation finishes
case _ =>
println("bah")
}
}
My problem is that even if the actor finishes before the timeout, nothing is "returned" by the Future and so the timeout always expires. What am I doing wrong? Thanks a lot.
From http://doc.akka.io/docs/akka/snapshot/scala/actors.html
Using ask will send a message to the receiving Actor as with tell, and the receiving actor must reply with sender ! reply in order to complete the returned Future with a value.
and
Warning
To complete the future with an exception you need send a Failure message to the sender. This is not done automatically when an actor throws an exception while processing a message.
So instead of "returning" like you would in a usual scala function, do something along the lines of
def receive = {
case Calculate(tonumber: Int) =>
...
sender ! 12345
case _ =>
sender ! akka.actor.Status.Failure(new InvalidArgumentException)
}