I have started using Akka. Please clarify the following queries;
I see around 8 default-dispatcher threads are created. Where is that number defined?
I see two types of dispatchers namely default-dispatcher and internal-dispatcher are created in my setup. How is this decided?
We could see the dispatcher threads in the Debug mode of any IDE. Is there any way to visualize the Akka objects such as Actors, Routers, Receptionist, etc?
I have observed that the dispatcher threads die automatically when I leave the program running for some time. Please explain the reason for this. Does actor system auto-create and auto-delete the dispatchers?
I see a number after default-dispatcher in the logs. What does this number indicate? Does it indicate the thread number being allocated by the dispatcher for an actor?
Example: 2022-09-02 10:39:25.482 [MyApp-akka.actor.default-dispatcher-5] DEBUG
The default dispatcher configuration for Akka can be found in the reference.conf file for the akka-actor package. By default, the default dispatcher (on which user actors run if not otherwise configured), is a fork-join pool with a minimum of 8 threads and a maximum of 64 threads. The internal dispatcher (introduced in Akka 2.6 to prevent user actors from starving system actors) is also, IIRC, a fork-join pool. Other dispatcher types and configurations are possible: the comments in reference.conf go through them, though for most purposes the defaults are reasonable.
As far as I know, there are no existing tools for visualizing the actors etc. in an Akka application.
The threads are managed by the thread-pool underlying the dispatcher. The fork-join pool will terminate threads which have been idle for a certain length of time and create new threads as needed.
The threads are (by default... at the very least a custom dispatcher could override this) named dispatcher-name-threadNumber. If you log inside an actor and have your log message format include the thread name, you can see which thread the actor was scheduled onto. Note that, typically, an actor being scheduled onto one thread does not imply it will never be scheduled on another thread (in particular, things like thread-local storage are unlikely to work); it's also worth noting that as Akka dispatchers are also Scala ExecutionContexts and Java Executors, the threads can be consumed by things which aren't actors (e.g. Future callbacks in Scala).
Related
In my Akka HTTP application, I have to initialize my actor system multiple times in some cases. Like in my services I need implicit val of executors and actor systems. So, I have initialized an actor system 4 times like this in my service layer classes:
private implicit val actorSystem = ActorSystem()
I have terminated only one actor system which is at root level. Others not terminated.
Does this impact on my application performance? Because my application is hanging after 1 to 2 days running.
Using many actor systems in your application is a bad idea.
Basically each ActorSystem comes with a default dispatcher which is backed by a fork-join pool. This pool does a really great job at balancing work and available threads. That is, if you run many actor systems on the same JVM, this will end up creating way too many (virtual) threads, killing the performance in the process. Ideally you want as many threads as you have CPU cores with optimal utilization.
Please go through this article once https://manuel.bernhardt.io/2016/08/23/akka-anti-patterns-too-many-actor-systems/
Also please do let me know if this answers your question.
Looking at my log file I see different dispatcher for the same actor. that actor is created once so how can it be ? is that means that the actor was restarted due to crash ?
e.g :
[ERROR] [08/07/2017 19:20:22.618]
[my-sys-akka.actor.default-dispatcher-21]
[akka://my-sys/user/com.domain.FooActor] some_exception
[ERROR] [08/07/2017 19:20:22.619]
[my-sys-akka.actor.default-dispatcher-26]
[akka://my-sys/user/com.domain.FooActor] some_exception
[ERROR] [08/07/2017 19:20:22.619]
[my-sys-akka.actor.default-dispatcher-27]
[akka://my-sys/user/com.domain.FooActor] some_exception
I'm pretty sure that your logger is configured to log the thread from which the log was generated. Check your logger config, but I guess that the at the third position, you log the thread (thread names from the log above: my-sys-akka.actor.default-dispatcher-XX).
A dispatcher will schedule your tasks to run on a pool of threads. By definition a pool can have multiple threads, and through the logs, you see that your actor's execution is schedule on different threads belonging to a single dispatcher.
With the given logs, there's no clear indication that your actor crashed/got restarted, just indication that it (or its supervisor) called log methods (i.e., log.error()) while running on different threads. This is expected as multiple (potentially millions) actors are meant to share a few threads (in the case of the default dispatcher depends on the number of CPU, cores, ...).
You should not expect your actors to always run on the same thread.
There's no way an actor gets to run on another dispatcher (unless you explicitly configure it that way).
Frustratingly, HawtDispatch's website describes it as "thread pooling and NIO event notification framework API."
Let's take the 'thread pooling' part first. Most of the Executors provided by Java are also basically thread pools. How is HawtDispatch different?
It is also apparently an "NIO event notification framework API." I'm assuming it is a thin layer on top NIO which takes incoming data and passes to its notion of 'thread pool,' and passes it to a consumer when the thread pool scheduler finds the time. Correct? (Any improvement over NIO is welcomed). Has anyone done any performance analysis of netty vs HD?
HawtDispatch is designed to be a single system wide fixed size thread pool. It provides implements 2 flavors of Java Executors:
Global Dispatch Queue: Submitted Runnable objects are executed concurrently (You get the same effect using a Executors.newFixedThreadPool(n) executor)
Serial Dispatch Queue: Submitted Runnable objects are executed serially (You get the same effect using a Executors.newSingleThreadExecutor() executor)
Unlike the java executor model all global and serial dispatch queues share a single fixed size thread pool. You can use thousands of serial dispatch queues without increasing your thread count. Serial dispatch queues can be used like Erlang mailboxes to drive reactive actor style applications.
Since HawtDispatch is using a fixed size thread pool for processing all global and serial queue executions, all Runnable tasks it executes must be non-blocking. In a way this is similar to the NodeJS architecture except it using multiple threads instead of just one.
In comparison to Netty, HawtDispatch is not a framework for actually processing socket data. It does not provide a framework for how encode/decode, buffer and process the socket data. All it does is execute a user configured Runnable when data can be read or written on the non-blocking socket. It's up to you application then to actually read/write the socket data.
What makes actors so lightweight?
I'm not even sure how they work. Aren't they separate threads?
When they say lightweight they mean that each actor is not mapped to a single thread.
JVM offers shared memory threads with
locks as the primary form of
concurrency abstractions. But shared
memory threads are quite heavyweight
and incur severe performance penalties
from context switching overheads. For
an actor implementation based on a
one-to-one mapping with JVM threads,
the process payload per Scala actor
will not be as lightweight that we can
spawn a million instances of an actor
for a specific computation. Hence
Scala actors have been designed as
lightweight event objects, which get
scheduled and executed on an
underlying worker thread pool that
gets automatically resized when all
threads block on long running
operations. In fact, Scala implements
a unified model of actors - thread
based and event based. Scala actors
offer two form of suspension
mechanisms - a full stack frame
suspension(implemented as receive) and
a suspension based on a continuation
closure (implemented as react). In
case of event based actors, a wait on
react is represented by a continuation
closure, i.e. a closure that captures
the rest of the actor's computation.
When the suspended actor receives a
message that matches one of the
patterns specified in the actor, the
continuation is executed by scheduling
the task to one of the worker threads
from the underlying thread pool. The
paper "Actors that Unify Threads and
Events" by Haller and Odersky
discusses the details of the
implementation.
Source
Important Reference Actors that Unify Threads and Events
I don't think we should strengthen that actor is that lightweight.
firstly thread based actors are actor per thread so not lightweight at all.
event based actors are the point where we start to feel actors are light weight. it is light weight because it does not have working thread wait and switched to another , working thread just switch from a piece of data work to another piece of data work, thus keep spinning on effective calculations.
Is there a way to monitor how many threads are actually alive and running my scala actors ?
The only way to properly do this is to inject your own executor for the actors subsystem as, by default, the actor threads do not have actor- or scala-specific names (they may just be called Thread-N or pool-N-thread-M depending on which version of Scala you are using.
Philip Haller has given instructions on using your own executor, where you can monitor thread usage if you wish, or at the very least name the threads so created. If you override thread naming you could then use the standard Java system MBeans (i.e. ThreadMXBean) to monitor the threads programmatically (or via the JConsole/JVisualVM).
Note that you can control the default mechanism using the system properties:
actors.minPoolSize
actors.maxPoolSize
actors.corePoolSize
You might try the VisualVM tool (available free from Sun). Among other things, it can monitor threads in running JVMs.