I have the following unit test:
it should "return xml" in new TestScope {
val testProbe: TestProbe = TestProbe()
val someActor = system.actorOf(Props[SomeActor])
testProbe.send(someActor, MakeXmlApiCall())
testProbe.expectMsgPF() {
case Success(message) => {
assert(message == "someMessage")
}
}
}
For the actor with this receive method:
override def receive: Receive = {
case MakeXmlApiCall() => {
val status = Future {"someMessage"}
println("In SomeActor")
status onComplete {
case Success(message) => {
sender ! message
}
}
}
}
"In SomeActor" is printed, which means that control reaches the receive method.
However, I am also getting this message:
[INFO] [12/15/2016 18:42:29.463] [testSystem-akka.actor.default-dispatcher-3] [akka://testSystem/deadLetters] Message [java.lang.String] from Actor[akka://testSystem/user/$a#-1159394947] to Actor[akka://testSystem/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
So why is the message sent to the dead-letter rather than the testProbe from the unit test.
Why is this happening?
Referencing sender() (or any actor state) from a callback or operator on a future is not safe, since sender() can have changed.
You should instead use pipe the future back to the sender: status.pipeTo(sender())
In addition, your expectMsgPF() call does not match on what you're actually sending from SomeActor. You're replying with a string, not an instance of Success.
Related
i have a use case in which i have a actor hierarchy
parent -> childABC -> workerchild
Now the worker child works and send its result to its parent(childABC which is a child of parent) and that child actor(childABC) send the result back to parent actor I am using pipeTo and getting dead letters here is my code
parent actor:
final case object GetFinalValue
class MyActor extends Actor{
import context.dispatcher
import akka.pattern.pipe
val log = LoggerFactory.getLogger(this.getClass)
val myManageActor = context.actorOf(Props[ManagerMyActor],"Managemyactor")
implicit val timeout = Timeout(ReadTimeIntervalValue.getInterval(), SECONDS)
override def receive: Receive = {
case GetFinalValue=>
ask(myManageActor,GetValue).pipeTo(sender())
case message =>
log.warn(" Unhandled message received : {}", message)
unhandled(message)
}
}
childABC (acc to example I gave above)
final case object GetValue
class ManagerMyActor extends Actor{
import context.dispatcher
import akka.pattern.pipe
val log = LoggerFactory.getLogger(this.getClass)
val myTokenActor = context.actorOf(Props[TokenMyActor2],"toknMyActor2")
implicit val timeout = Timeout(ReadTimeIntervalValue.getInterval(), SECONDS)
override def receive: Receive = {
case GetValue=>
ask(myTokenActor,CalculateValue).pipeTo(sender())
case message =>
log.warn(" Unhandled message received : {}", message)
unhandled(message)
}
}
child actor:
final case object CalculateValue
class TokenMyActor2 extends Actor{
import context.dispatcher
import akka.pattern.pipe
val log = LoggerFactory.getLogger(this.getClass)
override def receive: Receive = {
case CalculateValue=>
val future = Future{ "get the string"
}
val bac = future.map{result =>
sender ! result
}//.pipeTo(sender())
case message =>
log.warn("Actor MyActor: Unhandled message received : {}", message)
unhandled(message)
}
}
def main(args: Array[String]): Unit = {
implicit val timeout = Timeout(ReadTimeIntervalValue.getInterval(), SECONDS)
val myActor = system.actorOf(Props[MyActor],"myActor")
val future = ask(myActor, GetFinalValue).mapTo[String]
future.map {str =>
log.info ("string is {}",str)
}
Here are the logs:
[INFO] [akkaDeadLetter][01/12/2021 19:17:22.000] [api-akka.actor.default-dispatcher-5] [akka://api/deadLetters] Message [java.lang.String] from Actor[akka://api/user/myActor/Managemyactor/toknMyActor2#1239397461] to Actor[akka://api/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://api/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/12/2021 19:17:41.989] [api-akka.actor.default-dispatcher-7] [akka://api/deadLetters] Message [akka.actor.Status$Failure] from Actor[akka://api/user/myActor#1829301550] to Actor[akka://api/deadLetters] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://api/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/12/2021 19:17:41.996] [api-akka.actor.default-dispatcher-7] [akka://api/deadLetters] Message [akka.actor.Status$Failure] from Actor[akka://api/user/myActor/Managemyactor#-269929265] to Actor[akka://api/deadLetters] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://api/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
Please guide me where am I mistaken, or pipeTo should not be used like this? if so what should i do to make it work
Not sure if it's intended or not but ask(myManageActor,GetValue).pipeTo(sender()) can be implemented as forward.
class MyActor extends Actor {
lazy val myManageActor: ActorRef = ???
override def receive: Receive = {
case GetFinalValue =>
myManageActor.forward(GetValue)
}
}
forward is the same as tell but it preserves the original sender of the messages.
This can be applied to MyActor and ManagerMyActor.
In the case of TokenMyActor2, you should not use
future.map{ result =>
sender ! result
}
as it it breaks akka context encapsulation, as specified in docs
When using future callbacks, such as onComplete, or map such as
thenRun, or thenApply inside actors you need to carefully avoid
closing over the containing actor’s reference, i.e. do not call
methods or access mutable state on the enclosing actor from within the
callback. This would break the actor encapsulation and may introduce
synchronization bugs and race conditions because the callback will be
scheduled concurrently to the enclosing actor. Unfortunately there is
not yet a way to detect these illegal accesses at compile time. See
also: Actors and shared mutable state
You should instead rely on Future(???).pipeTo(sender()), which is safe to use with sender().
After applying these changes, the code does work as expected
case object GetFinalValue
case object GetValue
case object CalculateValue
class MyActor extends Actor {
private val myManageActor: ActorRef =
context.actorOf(Props[ManagerMyActor], "myManageActor")
override def receive: Receive = { case GetFinalValue =>
myManageActor.forward(GetValue)
}
}
class ManagerMyActor extends Actor {
private val myTokenActor =
context.actorOf(Props[TokenMyActor2], "toknMyActor2")
override def receive: Receive = { case GetValue =>
myTokenActor.forward(CalculateValue)
}
}
class TokenMyActor2 extends Actor {
import context.dispatcher
override def receive: Receive = { case CalculateValue =>
val future = Future { "get the string" }
future.pipeTo(sender())
}
}
implicit val timeout = Timeout(3, SECONDS)
implicit val system = ActorSystem("adasd")
import system.dispatcher
val myActor = system.actorOf(Props[MyActor], "myActor")
val future = ask(myActor, GetFinalValue).mapTo[String]
future.foreach { str =>
println(s"got $str")
}
Produces got get the string.
As a final note, I'd advise not to use ask pattern within actors. The basic functionality of ask can be easily achieved with just tell and forward. Also the code is shorter and not overloaded with constant need of implicit val timeout
Just to add on top of the great post by #IvanStanislavciuc. You already noticed that you lose the reference to the sender in futures. A simple solution for that will be to keep it up front.
It means that changing in MyActor:
ask(myManageActor,GetValue).pipeTo(sender()) // Won't work
into:
val originalSender = sender()
ask(myTokenActor,CalculateValue).pipeTo(originalSender)
In ManagerMyActor, Change:
ask(myTokenActor,CalculateValue).pipeTo(sender()) // Won't work
into:
val originalSender = sender()
ask(myManageActor,GetValue).pipeTo(originalSender)
And in TokenMyActor2:
val originalSender = sender()
Future{ "get the string" }.pipeTo(originalSender)
Code run at Scastie.
case class FeatureFilter(s3Client: AmazonS3) extends Actor with ActorLogging {
override def preStart(): Unit = {
self ! Initialize
}
override def receive: Receive = {
case Initialize =>
// long running operaton
val tryfile = S3Connection(s3Client).downloadObject(...)
tryfile match {
case Success(file) =>
context.become(active(file))
case Failure(exception) =>
self ! PoisonPill
}
}
def active(file: File): Receive = {
case Query(key) =>
// do some processing and reply to sender
}
}
I am using below test for above actor:
"an actor" should {
// mocked S3 client
val client = ...
"test for presence of keys" in {
val actor = system.actorOf(Props(FeatureFilter(client)))
for (i <- 1 to 100) {
actor ! Query("test_string")
expectMsg(SomeMessage)
}
}
}
The above test fails with
java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg while waiting ...
I think this is because when the message actor ! Query("test_string") is sent to actor, it's handler is still receive, and so it doesn't respond, and hence the timeout.
But I even tried adding the handler for Query(key) in the receive method (just like in active method). Still I am getting the same error.
Could someone please point what is the issue here ?
Also when I move the S3 download task to preStart(), still the issue remains same. Isn't preStart() a blocking call ? How would the code in the test proceed until the preStart() is completed ?
akka stash sounds like the way you looking for. In case of any message that the actor support but is unhandled add on stash and unapply all if active is reached.
look at actor stash for documentation and example usage
may your code would look like
case msg => stash()
...
unstashAll()
context.become(active(file))
I am learning to use broadcast messages in Akka routers. Is there a way to receive responses for ask from all the routees to the router?
I have this sample code.
Master.scala
object Master {
case object brdcst
}
class Master extends Actor {
implicit val timeout = Timeout(5 seconds)
val router: ActorRef = context.actorOf (RoundRobinPool (3).props(Props[Worker]), "router")
override def receive: Receive = {
case brdcst => {
val future = router ? Broadcast(brdcst)
val result = Await.result(future, timeout.duration)
println("result = " + result)
}
}
}
object MasterTest extends App {
val actorSystem = ActorSystem("ActorSystem")
val actor = actorSystem.actorOf(Props[master], "root")
actor ! brdcst
}
Worker.scala
class Worker extends Actor {
val routee = context.actorOf(Props[Worker], "routee")
override def receive: Receive = {
case brdcst => sender() ! self.path.name
}
}
This code gives the following output
result = $a
[INFO] [10/16/2018 21:47:07.484] [ActorSystem-akka.actor.default-dispatcher-2] [akka://ActorSystem/deadLetters] Message [java.lang.String] from Actor[akka://ActorSystem/user/root/router/$a#340358688] to Actor[akka://ActorSystem/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior, then [Actor[akka://ActorSystem/deadLetters]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/16/2018 21:47:07.504] [ActorSystem-akka.actor.default-dispatcher-10] [akka://ActorSystem/deadLetters] Message [java.lang.String] from Actor[akka://ActorSystem/user/root/router/$b#-151225340] to Actor[akka://ActorSystem/deadLetters] was not delivered. [2] dead letters encountered. If this is not an expected behavior, then [Actor[akka://ActorSystem/deadLetters]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
So the reply is coming only from one routee. How to get responses from all the routees? (Maybe as a list like result = [$a, $b, $c])
ask (?) creates an internal actor to handle replies. This internal actor handles only a single reply and is automatically shut down: this is the reason you get only the first response, while the responses from the other two routees go to dead letters.
To get the desired behavior, use tell (!) and collect the responses from the routees. For example:
class Master extends Actor {
val numRoutees = 3
val router = context.actorOf(RoundRobinPool(numRoutees).props(Props[Worker]), "router")
def handleMessages(replies: Set[String] = Set()): Receive = {
case brdcst =>
router ! Broadcast(brdcst)
case reply: String =>
val updatedReplies = replies + reply
if (updatedReplies.size == numRoutees) {
println("result = " + updatedReplies.mkString("[", ",", "]"))
}
become(handleMessages(updatedReplies))
}
def receive = handleMessages
}
In the above example, the master encodes the routees' replies as part of its state, using become.
Also, don't use Await in actors.
I am using BackoffSupervisor strategy to create a child actor that has to process some message. I want to implement a very simple restart strategy, in which in case of exception:
Child propagates failing message to supervisor
Supervisor restarts child and sends the failing message again.
Supervisor gives up after 3 retries
Akka persistence is not an option
So far what I have is this:
Supervisor definition:
val childProps = Props(new SenderActor())
val supervisor = BackoffSupervisor.props(
Backoff.onFailure(
childProps,
childName = cmd.hashCode.toString,
minBackoff = 1.seconds,
maxBackoff = 2.seconds,
randomFactor = 0.2
)
.withSupervisorStrategy(
OneForOneStrategy(maxNrOfRetries = 3, loggingEnabled = true) {
case msg: MessageException => {
println("caught specific message!")
SupervisorStrategy.Restart
}
case _: Exception => SupervisorStrategy.Restart
case _ ⇒ SupervisorStrategy.Escalate
})
)
val sup = context.actorOf(supervisor)
sup ! cmd
Child actor that is supposed to send the e-mail, but fails (throws some Exception) and propagates Exception back to supervisor:
class SenderActor() extends Actor {
def fakeSendMail():Unit = {
Thread.sleep(1000)
throw new Exception("surprising exception")
}
override def receive: Receive = {
case cmd: NewMail =>
println("new mail received routee")
try {
fakeSendMail()
} catch {
case t => throw MessageException(cmd, t)
}
}
}
In the above code I wrap any exception into custom class MessageException that gets propagated to SupervisorStrategy, but how to propagate it further to the new child to force reprocessing? Is this the right approach?
Edit. I attempted to resent the message to the Actor on preRestart hook, but somehow the hook is not being triggered:
class SenderActor() extends Actor {
def fakeSendMail():Unit = {
Thread.sleep(1000)
// println("mail sent!")
throw new Exception("surprising exception")
}
override def preStart(): Unit = {
println("child starting")
}
override def preRestart(reason: Throwable, message: Option[Any]): Unit = {
reason match {
case m: MessageException => {
println("aaaaa")
message.foreach(self ! _)
}
case _ => println("bbbb")
}
}
override def postStop(): Unit = {
println("child stopping")
}
override def receive: Receive = {
case cmd: NewMail =>
println("new mail received routee")
try {
fakeSendMail()
} catch {
case t => throw MessageException(cmd, t)
}
}
}
This gives me something similar to following output:
new mail received routee
caught specific message!
child stopping
[ERROR] [01/26/2018 10:15:35.690]
[example-akka.actor.default-dispatcher-2]
[akka://example/user/persistentActor-4-scala/$a/1962829645] Could not
process message sample.persistence.MessageException:
Could not process message <stacktrace>
child starting
But no logs from preRestart hook
The reason that the child's preRestart hook is not invoked is because Backoff.onFailure uses BackoffOnRestartSupervisor underneath the covers, which replaces the default restart behavior with a stop-and-delayed-start behavior that is consistent with the backoff policy. In other words, when using Backoff.onFailure, when a child is restarted, the child's preRestart method is not called because the underlying supervisor actually stops the child, then starts it again later. (Using Backoff.onStop can trigger the child's preRestart hook, but that's tangential to the present discussion.)
The BackoffSupervisor API doesn't support the automatic resending of a message when the supervisor's child restarts: you have to implement this behavior yourself. An idea for retrying messages is to let the BackoffSupervisor's supervisor handle it. For example:
val supervisor = BackoffSupervisor.props(
Backoff.onFailure(
...
).withReplyWhileStopped(ChildIsStopped)
).withSupervisorStrategy(
OneForOneStrategy(maxNrOfRetries = 3, loggingEnabled = true) {
case msg: MessageException =>
println("caught specific message!")
self ! Error(msg.cmd) // replace cmd with whatever the property name is
SupervisorStrategy.Restart
case ...
})
)
val sup = context.actorOf(supervisor)
def receive = {
case cmd: NewMail =>
sup ! cmd
case Error(cmd) =>
timers.startSingleTimer(cmd.id, Replay(cmd), 10.seconds)
// We assume that NewMail has an id field. Also, adjust the time as needed.
case Replay(cmd) =>
sup ! cmd
case ChildIsStopped =>
println("child is stopped")
}
In the above code, the NewMail message embedded in the MessageException is wrapped in a custom case class (in order to easily distinguish it from a "normal"/new NewMail message) and sent to self. In this context, self is the actor that created the BackoffSupervisor. This enclosing actor then uses a single timer to replay the original message at some point. This point in time should be far enough in the future such that the BackoffSupervisor can potentially exhaust SenderActor's restart attempts, so that the child can have ample opportunity to get in a "good" state before it receives the resent message. Obviously this example involves only one message resend regardless of the number of child restarts.
Another idea is to create a BackoffSupervisor-SenderActor pair for every NewMail message, and have the SenderActor send the NewMail message to itself in the preStart hook. One concern with this approach is the cleaning up of resources; i.e., shutting down the BackoffSupervisors (which will, in turn, shut down their respective SenderActor children) when the processing is successful or when the child restarts are exhausted. A map of NewMail ids to (ActorRef, Int) tuples (in which the ActorRef is a reference to a BackoffSupervisor actor, and the Int is the number of restart attempts) would be helpful in this case:
class Overlord extends Actor {
var state = Map[Long, (ActorRef, Int)]() // assuming the mail id is a Long
def receive = {
case cmd: NewMail =>
val childProps = Props(new SenderActor(cmd, self))
val supervisor = BackoffSupervisor.props(
Backoff.onFailure(
...
).withSupervisorStrategy(
OneForOneStrategy(maxNrOfRetries = 3, loggingEnabled = true) {
case msg: MessageException =>
println("caught specific message!")
self ! Error(msg.cmd)
SupervisorStrategy.Restart
case ...
})
)
val sup = context.actorOf(supervisor)
state += (cmd.id -> (sup, 0))
case ProcessingDone(cmdId) =>
state.get(cmdId) match {
case Some((backoffSup, _)) =>
context.stop(backoffSup)
state -= cmdId
case None =>
println(s"${cmdId} not found")
}
case Error(cmd) =>
val cmdId = cmd.id
state.get(cmdId) match {
case Some((backoffSup, numRetries)) =>
if (numRetries == 3) {
println(s"${cmdId} has already been retried 3 times. Giving up.")
context.stop(backoffSup)
state -= cmdId
} else
state += (cmdId -> (backoffSup, numRetries + 1))
case None =>
println(s"${cmdId} not found")
}
case ...
}
}
Note that SenderActor in the above example takes a NewMail and an ActorRef as constructor arguments. The latter argument allows the SenderActor to send a custom ProcessingDone message to the enclosing actor:
class SenderActor(cmd: NewMail, target: ActorRef) extends Actor {
override def preStart(): Unit = {
println(s"child starting, sending ${cmd} to self")
self ! cmd
}
def fakeSendMail(): Unit = ...
def receive = {
case cmd: NewMail => ...
}
}
Obviously the SenderActor is set up to fail every time with the current implementation of fakeSendMail. I'll leave the additional changes needed in SenderActor to implement the happy path, in which SenderActor sends a ProcessingDone message to target, to you.
In the good solution that #chunjef provides, he alert about the risk of schedule a job resend before the backoff supervisor has started the worker
This enclosing actor then uses a single timer to replay the original message at some point. This point in time should be far enough in the future such that the BackoffSupervisor can potentially exhaust SenderActor's restart attempts, so that the child can have ample opportunity to get in a "good" state before it receives the resent message.
If this happens, the scenario will be jobs going to dead letters and no further progress will be done.
I've made a simplified fiddle with this scenario.
So, the schedule delay should be larger than the maxBackoff, and this could represent an impact in job completion time.
A possible solution to avoid this scenario is making the worker actor to send a message to his father when is ready to work, like here.
The failed child actor is available as the sender in your supervisor strategy. Quoting https://doc.akka.io/docs/akka/current/fault-tolerance.html#creating-a-supervisor-strategy:
If the strategy is declared inside the supervising actor (as opposed
to within a companion object) its decider has access to all internal
state of the actor in a thread-safe fashion, including obtaining a
reference to the currently failed child (available as the sender of
the failure message).
Sending emails is a dangerous operation with some third party software in your case. Why not to apply Circuit Breaker pattern and skip the sender actor entirely? Also, you can still have an actor (with some Backoff Supervisor) and Circuit Breaker inside it (if that makes sense for you).
consider the following example:
case class Payload(message: String, async: Boolean)
class EchoActor extends Actor {
override def receive: Receive = {
case Payload(message, async) =>
if (async) Future {
println(s"from: $sender")
sender ! message
} else {
println(s"from: $sender")
sender ! message
}
}
}
def main(args: Array[String]): Unit = {
val system = ActorSystem("demo")
val echo = system.actorOf(Props[EchoActor])
implicit val timeout = Timeout(2 seconds)
(echo ? Payload("Hello", async = false)).mapTo[String].foreach(println(_))
(echo ? Payload("Async Hello", async = true)).mapTo[String].foreach(println(_))
StdIn.readLine()
system.terminate()
}
console output:
from: Actor[akka://demo/temp/$a]
Hello
from: Actor[akka://demo/deadLetters]
[INFO] [04/13/2017 19:56:58.516] [demo-akka.actor.default-dispatcher-4] [akka://demo/deadLetters] Message [java.lang.String] from Actor[akka://demo/user/$a#2112869650] to Actor[akka://demo/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
i.e. the sender points to deadLetters when accessing it from another thread.
What is the reason behind that?
is it a bug?
although, we can hold a reference to the actual sender to make it work:
if (async) {
val currentSender = sender()
Future {
println(s"from: $currentSender")
currentSender ! message
}
}
but... isn't there some better approach?
This is not a bug, but a documented behavior -
http://doc.akka.io/docs/akka/2.5.0/scala/actors.html#Send_messages
Using Future means invoking an anonymous function which is not an instance of the Actor class, therefor your sender() ref is mapped to the deadLetters mailbox
The better approach is the pipe pattern.
import akka.pattern.pipe
class EchoActor extends Actor {
override def receive: Receive = {
case Payload(message, async) =>
if (async) {
Future {
message
}.pipeTo(sender)
} else {
sender ! message
}
}
}
The issue is that sender is a function and its value is only valid if called on the same thread that is processing the incoming message. When you call sender within a future, it's being called from another thread and at another point in time, notably after the actor's receive function has already returned.
pipeTo captures the value of the current sender on the same thread as the actor and before the receive function has returned. This is effectively the same as your approach with the currentSender value.