Looking at my log file I see different dispatcher for the same actor. that actor is created once so how can it be ? is that means that the actor was restarted due to crash ?
e.g :
[ERROR] [08/07/2017 19:20:22.618]
[my-sys-akka.actor.default-dispatcher-21]
[akka://my-sys/user/com.domain.FooActor] some_exception
[ERROR] [08/07/2017 19:20:22.619]
[my-sys-akka.actor.default-dispatcher-26]
[akka://my-sys/user/com.domain.FooActor] some_exception
[ERROR] [08/07/2017 19:20:22.619]
[my-sys-akka.actor.default-dispatcher-27]
[akka://my-sys/user/com.domain.FooActor] some_exception
I'm pretty sure that your logger is configured to log the thread from which the log was generated. Check your logger config, but I guess that the at the third position, you log the thread (thread names from the log above: my-sys-akka.actor.default-dispatcher-XX).
A dispatcher will schedule your tasks to run on a pool of threads. By definition a pool can have multiple threads, and through the logs, you see that your actor's execution is schedule on different threads belonging to a single dispatcher.
With the given logs, there's no clear indication that your actor crashed/got restarted, just indication that it (or its supervisor) called log methods (i.e., log.error()) while running on different threads. This is expected as multiple (potentially millions) actors are meant to share a few threads (in the case of the default dispatcher depends on the number of CPU, cores, ...).
You should not expect your actors to always run on the same thread.
There's no way an actor gets to run on another dispatcher (unless you explicitly configure it that way).
Related
I have started using Akka. Please clarify the following queries;
I see around 8 default-dispatcher threads are created. Where is that number defined?
I see two types of dispatchers namely default-dispatcher and internal-dispatcher are created in my setup. How is this decided?
We could see the dispatcher threads in the Debug mode of any IDE. Is there any way to visualize the Akka objects such as Actors, Routers, Receptionist, etc?
I have observed that the dispatcher threads die automatically when I leave the program running for some time. Please explain the reason for this. Does actor system auto-create and auto-delete the dispatchers?
I see a number after default-dispatcher in the logs. What does this number indicate? Does it indicate the thread number being allocated by the dispatcher for an actor?
Example: 2022-09-02 10:39:25.482 [MyApp-akka.actor.default-dispatcher-5] DEBUG
The default dispatcher configuration for Akka can be found in the reference.conf file for the akka-actor package. By default, the default dispatcher (on which user actors run if not otherwise configured), is a fork-join pool with a minimum of 8 threads and a maximum of 64 threads. The internal dispatcher (introduced in Akka 2.6 to prevent user actors from starving system actors) is also, IIRC, a fork-join pool. Other dispatcher types and configurations are possible: the comments in reference.conf go through them, though for most purposes the defaults are reasonable.
As far as I know, there are no existing tools for visualizing the actors etc. in an Akka application.
The threads are managed by the thread-pool underlying the dispatcher. The fork-join pool will terminate threads which have been idle for a certain length of time and create new threads as needed.
The threads are (by default... at the very least a custom dispatcher could override this) named dispatcher-name-threadNumber. If you log inside an actor and have your log message format include the thread name, you can see which thread the actor was scheduled onto. Note that, typically, an actor being scheduled onto one thread does not imply it will never be scheduled on another thread (in particular, things like thread-local storage are unlikely to work); it's also worth noting that as Akka dispatchers are also Scala ExecutionContexts and Java Executors, the threads can be consumed by things which aren't actors (e.g. Future callbacks in Scala).
looking at my threads (using jconsole) I see a lot of dispatchers :
akka.actor.default-dispatcher-...
however I am not creating the Actors withDispatcher. what can be the reason for multiply dispatcher ?
From the docs:
Every ActorSystem will have a default dispatcher that will be used in
case nothing else is configured for an Actor.
What you see in jconsole are the threads currently submitted to the default dispatcher. These take the name of default-dispatcher-1, default-dispatcher-2, etc. They are all run by the same unique, default-dispatcher.
The withDispatcher call allows you to specify multiple dispatchers - different from the default one.
More info here.
In my scala application I have an actor A that is the child of the guardian actor. The guardian actor has been given a one for all strategy to escalate on children shutdown.
Under various circumstances, the actor A calls context stop self that usually results in the entire application exiting, which is the desired behavior.
However, I recently observed a scenario where context stop self resulted in an error
Failed to run termination callback, due to [Futures timed out after [5000 milliseconds]]
..and the actor system did not shutdown.
The application does sometimes have hundreds of thousands of actors under Actor A, and I wonder if shutting them down caused some sort of timeout in the termination procedure.
Can anyone help me understand why the behavior described above would happen and should I be using context system shutdown instead to shutdown the system?
I'm new to akka-actor and confused with some problems:
when I create an actorSystem, and use actorOf(Props(classOf[AX], ...)) to create actor in main method, how many instances are there for my actor AX?
If the answer to Q1 was just one, does this mean whatever data-structure I created in the AX actor class's definition will only appear in one thread and I should not concern about concurrency problems?
What if one of my actor's action (one case in receive method) is a time consuming task and would take quite long time to finish? Will my single Actor instance not responding until it finish that task?
If the answer to Q3 is right, what I am supposed to do to prevent my actor from not responding? Should I start another thread and send another message back to it until finish the task? Is there a best practice there I should follow?
yes, the actor system will only create 1 actor instance for each time you call the 'actorOf' method. However, when using a Router it is possible to create 1 router which spreads the load to any number of actors. So in that case it is possible to construct multiple instances, but 'normally' using actorOf just creates 1 instance.
Yes, within an actor you do not have to worry about concurrency because Akka guarantees that any actor only processes 1 message at the time. You must take care not to somehow mutate the state of the actor from code outside the actor. So whenever exposing the actor state, always do this using an immutable class. Case classes are excellent for this. But also be ware of modifying the actor state when completing a Future from inside the actor. Since the Future runs on it's own thread you could have a concurrency issue when the Future completes and the actor is processing a next message at the same time.
The actor executes on 1 thread at the time, but this might be a different thread each time the actor executes.
Akka is a highly concurrent and distributed framework, everything is asynchronous and non-blocking and you must do the same within your application. Scala and Akka provide several solutions to do this. Whenever you have a time consuming task within an actor you might either delegate the time consuming task to another actor just for this purpose, use Futures or use Scala's 'async/await/blocking'. When using 'blocking' you give a hint to the compiler/runtime a blocking action is done and the runtime might start additional thread to prevent thread starvation. The Scala Concurrent programming book is an excellent guide to learn this stuff. Also look at the concurrent package ScalaDocs and Neophyte's Guide to Scala.
If the actor really has to wait for the time consuming task to complete, then yes, your actor can only respond when that's finished. But this is a very 'request-response' way of thinking. Try to get away from this. The actor could also respond immediately indicating the task has started and send an additional message once the task has been completed.
With time consuming tasks always be sure to use a different threadpool so the ActorSystem will not be blocked because all of it's available threads are used up by time consuming tasks. For Future's you can provide a separate ExecutionContext (do not use the ActorSystem's Dispatch context for this!), but via Akka's configuration you can also configure certain actors to run on a different thread pool.
See 3.
Success!
one instance (if you declare a router in your props then (maybe) more than one)
Yes. This is one of the advantages of actors.
Yes. An Actor will process messages sequentially.
You can use scala.concurrent.Future (do not use actor state in the future) or delegate the work to a child actor (the main actor can manage the state and can respond to messages). Future or child-actor depends on use case.
I'm using slick to store data in database, and there I use the threadLocalSession to store the sessions.
The repositories are used to do the crud, and I have an Akka service layer that access the slick repositories.
I found this link, where Adam Gent asks something near what I'm asking here: Akka and Java libraries that use ThreadLocals
My concern is about how does akka process a message, as I store the database session in a threadLocal, can I have two messages been processed at the same time in the same thread?
Let's say: Two add user messages (A and B) sent to the userservice, and message A is partially processed, and stopped, thread B start to process in the same thread that thread A has started to process, which will have the session stored in it's localSession?
Each actor processes its messages one at a time, in the order it received them*. Therefore, if you send messages A, B to the same actor, then they are never processed concurrently (of course the situation is different if you send each of the messages to different actors).
The problem with the use of ThreadLocals is that in general it is not guaranteed that an actor processes each of its messages on the same thread.
So if you send a message M1 and then a message M2 to actor A, it is guaranteed that M1 is processed before M2. What is not guaranteed that M2 is processed on the same thread as M1.
In general, you should avoid using ThreadLocals, as the whole point of actors is that they are a unit of consistency, and you are safe to modify their internal state via message passing. If you really need more control on the threads which execute the processing of messages, look into the documentation of dispatchers: http://doc.akka.io/docs/akka/2.1.0/java/dispatchers.html
*Except if you change their mailbox implementation, but that's a non-default behavior.