I have a supervisor below that creates child processes using the default Play! 2.2 Akka.system. When I attempt to instantaneously kill the supervisor, nothing happens and it keeps processing.
class ImportSupervisor extends Actor {
import akka.actor.AllForOneStrategy
import akka.actor.SupervisorStrategy._
import scala.concurrent.duration._
val log = Logging(context.system, this)
override val supervisorStrategy =
AllForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 hour) {
case e: Exception => Stop
}
override def preStart() {
Logger.info("supervisor starting up at "+self.toString)
}
def receive = {
case p: Props => context.actorOf(p)
}
}
Below is the code I'm using to kill the supervisor which has about 1,000 children:
Akka.system.actorSelection("akka://application/user/"+actorName) ! Kill
I can verify I am getting the ActorPath correct, but the problem is the Kill does not take effect instantaneously. I've tried the same with Stop. What am I doing wrong? Is it wrong to assume that it would shut down instantly?
Is it wrong to assume that it would shut down instantly?
Yes.
The Kill message will be enqueued in the actor's mailbox just like any other message. If there are other messages ahead of it, other the actor is busy processing another message, the Kill message will have to wait.
All of this is explained in the documentation:
http://doc.akka.io/docs/akka/2.2.4/scala/actors.html#Stopping_actors
Related
I have this situation:
ActorA sends ActorB start/stop messages every 30-40 seconds
ActorA sends ActorB strings to print (always)
ActorB must print the strings he receive, but only if ActorA sent just a start message
Now i wonder if i can do the following things:
Can ActorB read messages only under a certain condition (if a boolean is set as true) without losing the messages he receives while that boolean is set as false?
Can ActorB read a start/stop message from ActorA before the other string messages? I'd like to have this situation: ActorA sends a start message to ActorB, ActorB start printing the strings he received before the start messages (and that is still receiving) and then stop as soon as it receives a stop messages?
I don't know if I explained it well.
EDIT: Thank you, the answers are great, but I still have some doubts.
Does the become mantain the order of the messages? I mean, if i have "Start-M1-M2-Stop-M3-M4-M5-Start-M6-M7-Stop", will the printing order be "M1-M2" and then "M3-M4-M5-M6-M7" or could M6 be read before M3, M4 and M5 (if M6 is received just after the become)?
Can I give a higher priority to start/stop messages? If ActorB receives "M1-M2-M3", and then it receives a stop message while it is printing "M1", i want that ActorB saves again M2 and M3.
You can exactly solve your problem with the Stash trait and the become/unbecome functionality of Akka. The idea is the following:
When you receive a Stop message then you switch to a behaviour where you stash all messages which are not Start. When you receive a Start message, then you switch to a behaviour where you print all received messages and additionally you unstash all messages which have arrived in the meantime.
case object Start
case object Stop
case object TriggerStateChange
case object SendMessage
class ActorB extends Actor with Stash {
override def receive: Receive = {
case Start =>
context.become(printingBehavior, false)
unstashAll()
case x => stash()
}
def printingBehavior: Receive = {
case msg: String => println(msg)
case Stop => context.unbecome()
}
}
class ActorA(val actorB: ActorRef) extends Actor {
var counter = 0
var started = false
override def preStart: Unit = {
import context.dispatcher
this.context.system.scheduler.schedule(0 seconds, 5 seconds, self, TriggerStateChange)
this.context.system.scheduler.schedule(0 seconds, 1 seconds, self, SendMessage)
}
override def receive: Actor.Receive = {
case SendMessage =>
actorB ! "Message: " + counter
counter += 1
case TriggerStateChange =>
actorB ! (if (started) {
started = false
Stop
} else {
started = true
Start
})
}
}
object Akka {
def main(args: Array[String]) = {
val system = ActorSystem.create("TestActorSystem")
val actorB = system.actorOf(Props(classOf[ActorB]), "ActorB")
val actorA = system.actorOf(Props(classOf[ActorA], actorB), "ActorA")
system.awaitTermination()
}
}
Check the Become/Unbecome functionality. It lets you change the behavior of the actor.
If I understood correctly you want your ActorB to have two different states. In the first state it should cache the messages it receives. In the second state, it should print all the cached messages and start printing all the new ones.
Something like this:
case class Start
case class Stop
case class Message(val: String)
class ActorB extends Actor {
var cache = Queue()
def initial: Receive = {
case Message(msg) => cache += msg
case Start =>
for (m <- cache) self ! Message(m)
context.become(printing)
}
def printing: Receive = {
case Message(msg) => println(msg)
case Stop => context.become(initial) //or stop the actor
}
def receive = initial
}
Have Actor B alternate between two states (two different behaviours). In the initial 'pending' state, it waits for a 'start' message, while stashing any other messages.
On receipt of a 'start' message, unstash all the stored messages and become 'active', waiting on a 'stop' message and writing out the other messages received (which will include the unstashed ones). On receiveing a 'stop' message, revert to the 'pending' behaviour.
Some of my thoughts
Yes if the boolean flag is got from some system resource like db or a config file, but I don't think it should be dependent on any external message, given that the actor receive messages from multiple other actors. If ActorB is only used by ActorA, the two can be merged to one
Similar as 1, how to handle the messages from multiple actors? If there is only one actorA, the two actors can be merged. If there are multiple, the flag can be set in database, actorA change the flag in db to "Start" or "Stop". and Actor B will print or not based on the flag.
An actor should be doing something very independently on other actor's state. The start and stop is actually some state of ActorA instead of ActorB
You already have a lot of good answers, but somehow I feel compelled to offer something more brief, as what you need is not necessarily a state machine:
class ActorB extends Actor {
def receive: Receive = caching(Nil)
def caching(cached: List[String]): Receive = {
case msg: String =>
context.become(caching(msg :: cached))
case Start =>
cached.reverse.foreach(println)
context.become(caching(Nil))
}
}
I've testing the fault tolerant system of akka and so far it's been good when talking about retrying to send a msg according the maxNrOfRetries specified.
However, it does not restart the actor within the given time range, it restarts all at once, ignoring the within time range.
I tried with AllForOneStrategy and OneForOneStrategy but does not change anything.
Trying to follow this blog post: http://letitcrash.com/post/23532935686/watch-the-routees, this is the code I've been working.
class Supervisor extends Actor with ActorLogging {
var replyTo: ActorRef = _
val child = context.actorOf(
Props(new Child)
.withRouter(
RoundRobinPool(
nrOfInstances = 5,
supervisorStrategy =
AllForOneStrategy(maxNrOfRetries = 3, withinTimeRange = 10.second) {
case _: NullPointerException => Restart
case _: Exception => Escalate
})), name = "child-router")
child ! GetRoutees
def receive = {
case RouterRoutees(routees) =>
routees foreach context.watch
case "start" =>
replyTo = sender()
child ! "error"
case Terminated(actor) =>
replyTo ! -1
context.stop(self)
}
}
class Child extends Actor with ActorLogging {
override def preRestart(reason: Throwable, message: Option[Any]): Unit = {
log.info("***** RESTARTING *****")
message foreach{ self forward }
}
def receive = LoggingReceive {
case "error" =>
log.info("***** GOT ERROR *****")
throw new NullPointerException
}
}
object Boot extends App {
val system = ActorSystem()
val supervisor = system.actorOf(Props[Supervisor], "supervisor")
supervisor ! "start"
}
Am I doing anything wrong to accomplish that?
EDIT
Actually, I misunderstood the purpose of the withinTimeRange.
To schedule my retries in a time range, I'm doing the following:
override def preRestart(reason: Throwable, message: Option[Any]): Unit = {
log.info("***** RESTARTING *****")
message foreach { msg =>
context.system.scheduler.scheduleOnce(30.seconds, self, msg)
}
}
It seems to work ok.
I think you have misunderstood the purpose of the withinTimeRange arg. That value is supposed to be used in conjunction with maxNrOfRetries to provide a window in which to support the limiting of the number of retries. For example, as you have specified, the implication is that the supervisor will no longer restart an individual child if that child needs to be restarted more than 3 times in 10 seconds.
From docs:
maxNrOfRetries - the number of times a child actor is allowed to be
restarted, negative value means no limit, if the limit is exceeded the
child actor is stopped
withinTimeRange - duration of the time window
for maxNrOfRetries, Duration.Inf means no window
Your code means that when any child fails with NullPointerException more than 3 times within 10 seconds it will not be restarted again. Because of AllForOneStrategy after first Routee fails all routees are restarted. And because you've overridden preRestart to resend failed message this situation repeats again until reaches 3 failures within 10 seconds(which is achieved in less than a second).
I've a very simple example where I've an Actor (SimpleActor) that perform a periodic task by sending a message to itself. The message is scheduled in the constructor for the actor. In the normal case (i.e., without faults) everything works fine.
But what if the Actor has to deal with faults. I've another Actor (SimpleActorWithFault). This actor could have faults. In this case, I'm generating one myself by throwing an exception. When a fault happens (i.e., SimpleActorWithFault throws an exception) it is automatically restarted. However, this restart messes up the scheduler inside the Actor which no longer functions as excepted. And if the faults happens rapidly enough it generates more unexpected behavior.
My question is what's the preferred way to dealing with faults in such cases? I know I can use Try blocks to handle exceptions. But what if I'm extending another actor where I cannot put a Try in the super class or some case when I'm an excepted fault happens in the actor.
import akka.actor.{Props, ActorLogging}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
import akka.actor.Actor
case object MessageA
case object MessageToSelf
class SimpleActor extends Actor with ActorLogging {
//schedule a message to self every second
context.system.scheduler.schedule(0 seconds, 1 seconds, self, MessageToSelf)
//keeps track of some internal state
var count: Int = 0
def receive: Receive = {
case MessageA => {
log.info("[SimpleActor] Got MessageA at %d".format(count))
}
case MessageToSelf => {
//update state and tell the world about its current state
count = count + 1
log.info("[SimpleActor] Got scheduled message at %d".format(count))
}
}
}
class SimpleActorWithFault extends Actor with ActorLogging {
//schedule a message to self every second
context.system.scheduler.schedule(0 seconds, 1 seconds, self, MessageToSelf)
var count: Int = 0
def receive: Receive = {
case MessageA => {
log.info("[SimpleActorWithFault] Got MessageA at %d".format(count))
}
case MessageToSelf => {
count = count + 1
log.info("[SimpleActorWithFault] Got scheduled message at %d".format(count))
//at some point generate a fault
if (count > 5) {
log.info("[SimpleActorWithFault] Going to throw an exception now %d".format(count))
throw new Exception("Excepttttttiooooooon")
}
}
}
}
object MainApp extends App {
implicit val akkaSystem = akka.actor.ActorSystem()
//Run the Actor without any faults or exceptions
akkaSystem.actorOf(Props(classOf[SimpleActor]))
//comment the above line and uncomment the following to run the actor with faults
//akkaSystem.actorOf(Props(classOf[SimpleActorWithFault]))
}
The correct way is to push down the risky behavior into it's own actor. This pattern is called the Error Kernel pattern (see Akka Concurrency, Section 8.5):
This pattern describes a very common-sense approach to supervision
that differentiates actors from one another based on any volatile
state that they may hold.
In a nutshell, it means that actors whose state is precious should not
be allowed to fail or restart. Any actor that holds precious data is
protected such that any risky operations are relegated to a slave
actor who, if restarted, only causes good things to happen.
The error kernel pattern implies pushing levels of risk further down
the tree.
See also another tutorial here.
So in your case it would be something like this:
SimpleActor
|- ActorWithFault
Here SimpleActor acts as a supervisor for ActorWithFault. The default supervising strategy of any actor is to restart a child on Exception and escalate on anything else:
http://doc.akka.io/docs/akka/snapshot/scala/fault-tolerance.html
Escalating means that the actor itself may get restarted. Since you really don't want to restart SimpleActor you could make it always restart the ActorWithFault and never escalate by overriding the supervisor strategy:
class SimpleActor {
override def preStart(){
// our faulty actor --- we will supervise it from now on
context.actorOf(Props[ActorWithFault], "FaultyActor")
...
override val supervisorStrategy = OneForOneStrategy () {
case _: ActorKilledException => Escalate
case _: ActorInitializationException => Escalate
case _ => Restart // keep restarting faulty actor
}
}
To avoid messing up the scheduler:
class SimpleActor extends Actor with ActorLogging {
private var cancellable: Option[Cancellable] = None
override def preStart() = {
//schedule a message to self every second
cancellable = Option(context.system.scheduler.schedule(0 seconds, 1 seconds, self, MessageToSelf))
}
override def postStop() = {
cancellable.foreach(_.cancel())
cancellable = None
}
...
To correctly handle exceptions (akka.actor.Status.Failure is for correct answer to an ask in case of Ask pattern usage by sender):
...
def receive: Receive = {
case MessageA => {
try {
log.info("[SimpleActor] Got MessageA at %d".format(count))
} catch {
case e: Exception =>
sender ! akka.actor.Status.Failure(e)
log.error(e.getMessage, e)
}
}
...
I'm trying to grok Futures and ask pattern in akka.
So, I make two actors, and one asking another to send him back a message. Well, according to akka's Futures documentation, actor should ask(?) for message and it shall give him a Future instanse. Then actor should block (using Await) to get Future results.
Well, I never get my future done. Why is that?
Code is:
package head_thrash
import akka.actor._
import akka.util.Timeout
import scala.concurrent.Await
import scala.concurrent.duration._
object Main extends App {
val system = ActorSystem("actors")
val actor1 = system.actorOf(Props[MyActor], "node_1")
val actor2 = system.actorOf(Props[MyActor], "node_2")
actor2 ! "ping_other"
system.awaitTermination()
Console.println("Bye!")
}
class MyActor extends Actor with ActorLogging {
import akka.pattern.ask
implicit val timeout = Timeout(100.days)
def receive = {
case "ping_other" => {
val selection = context.actorSelection("../node_1")
log.info("Sending ping to node_1")
val result = Await.result(selection ? "ping", Duration.Inf) // <-- Blocks here forever!
log.info("Got result " + result)
}
case "ping" => {
log.info("Sending back pong!")
sender ! "pong"
}
}
}
If I change Duration.Inf to 5.seconds, then actor waits 5 seconds, tells that my future is Timeouted (by throwing TimeoutException), and then other actor finally replies with needed message. So, no async happens. Why? :-(
How should I properly implement that pattern? Thanks.
The official Akka documentation says that Await.result will cause the current thread to block and wait for the Actor to 'complete' the Future with it's reply.
It is strange that your code blocks there forever, do you have only one thread for all your application?
Anyway I guess a more "idiomatic" way to code it would be to use a callback on the future success.
def receive = {
case "ping_other" => {
val selection = context.actorSelection("../node_1")
log.info("Sending ping to node_1")
val future: Future[String] = ask(selection, "ping").mapTo[String]
future.onSuccess {
case result : String ⇒ log.info("Got result " + result)
}
}
...
Two reasons why that doesn't work.
First, "node_1" asks itself and the "ping" will not be processed because it is blocking in waiting for the ask.
Also, there is a shortcoming of actorSelection for relative paths ("../node_1"). It is processed with message passing, and since your actor is blocking it cannot process any other message. This has been improved in upcoming 2.3 version of Akka, but you should avoid blocking anyway.
I am currently trying to get started with Akka and I am facing a weird problem. I've got the following code for my Actor:
class AkkaWorkerFT extends Actor {
def receive = {
case Work(n, c) if n < 0 => throw new Exception("Negative number")
case Work(n, c) => self reply n.isProbablePrime(c);
}
}
And this is how I start my workers:
val workers = Vector.fill(nrOfWorkers)(actorOf[AkkaWorkerFT].start());
val router = Routing.loadBalancerActor(SmallestMailboxFirstIterator(workers)).start()
And this is how I shut everything down:
futures.foreach( _.await )
router ! Broadcast(PoisonPill)
router ! PoisonPill
Now what happens is if I send the workers messages with n > 0 (no exception is thrown), everything works fine and the application shuts down properly. However, as soon as I send it a single message which results in an exception, the application does not terminate because there is still an actor running, but I can't figure out where it comes from.
In case it helps, this is the stack of the thread in question:
Thread [akka:event-driven:dispatcher:event:handler-6] (Suspended)
Unsafe.park(boolean, long) line: not available [native method]
LockSupport.park(Object) line: 158
AbstractQueuedSynchronizer$ConditionObject.await() line: 1987
LinkedBlockingQueue<E>.take() line: 399
ThreadPoolExecutor.getTask() line: 947
ThreadPoolExecutor$Worker.run() line: 907
MonitorableThread(Thread).run() line: 680
MonitorableThread.run() line: 182
PS: The thread which is not terminating isn't any of the worker threads, because I've added a postStop callback, every one of them stops properly.
PPS: Actors.registry.shutdownAll workarounds the problem, but I think shutdownAll should only be used as a last resort, shouldn't it?
The proper way to handle problems inside akka actors is not to throw an exception but rather to set supervisor hierarchies
"Throwing an exception in concurrent code (let’s assume we are using
non-linked actors), will just simply blow up the thread that currently
executes the actor.
There is no way to find out that things went wrong (apart from
inspecting the stack trace).
There is nothing you can do about it."
see Fault Tolerance Through Supervisor Hierarchies (1.2)
* note * the above is true for old versions of Akka (1.2)
In newer versions (e.g. 2.2) you'd still set a supervisor hierarchy but it will trap Exceptions thrown by child processes. e.g.
class Child extends Actor {
var state = 0
def receive = {
case ex: Exception ⇒ throw ex
case x: Int ⇒ state = x
case "get" ⇒ sender ! state
}
}
and in the supervisor:
class Supervisor extends Actor {
import akka.actor.OneForOneStrategy
import akka.actor.SupervisorStrategy._
import scala.concurrent.duration._
override val supervisorStrategy =
OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 minute) {
case _: ArithmeticException ⇒ Resume
case _: NullPointerException ⇒ Restart
case _: IllegalArgumentException ⇒ Stop
case _: Exception ⇒ Escalate
}
def receive = {
case p: Props ⇒ sender ! context.actorOf(p)
}
}
see Fault Tolerance Through Supervisor Hierarchies (2.2)
Turning off the logging to make sure things terminate, as proposed by Viktor, is a bit strange. What you can do instead is:
EventHandler.shutdown()
that cleanly shuts down all the (logger) listeners that keep the world running after the exception:
def shutdown() {
foreachListener(_.stop())
EventHandlerDispatcher.shutdown()
}
Turn of the logger in the akka.conf