In my Akka HTTP application, I have to initialize my actor system multiple times in some cases. Like in my services I need implicit val of executors and actor systems. So, I have initialized an actor system 4 times like this in my service layer classes:
private implicit val actorSystem = ActorSystem()
I have terminated only one actor system which is at root level. Others not terminated.
Does this impact on my application performance? Because my application is hanging after 1 to 2 days running.
Using many actor systems in your application is a bad idea.
Basically each ActorSystem comes with a default dispatcher which is backed by a fork-join pool. This pool does a really great job at balancing work and available threads. That is, if you run many actor systems on the same JVM, this will end up creating way too many (virtual) threads, killing the performance in the process. Ideally you want as many threads as you have CPU cores with optimal utilization.
Please go through this article once https://manuel.bernhardt.io/2016/08/23/akka-anti-patterns-too-many-actor-systems/
Also please do let me know if this answers your question.
Related
I have started using Akka. Please clarify the following queries;
I see around 8 default-dispatcher threads are created. Where is that number defined?
I see two types of dispatchers namely default-dispatcher and internal-dispatcher are created in my setup. How is this decided?
We could see the dispatcher threads in the Debug mode of any IDE. Is there any way to visualize the Akka objects such as Actors, Routers, Receptionist, etc?
I have observed that the dispatcher threads die automatically when I leave the program running for some time. Please explain the reason for this. Does actor system auto-create and auto-delete the dispatchers?
I see a number after default-dispatcher in the logs. What does this number indicate? Does it indicate the thread number being allocated by the dispatcher for an actor?
Example: 2022-09-02 10:39:25.482 [MyApp-akka.actor.default-dispatcher-5] DEBUG
The default dispatcher configuration for Akka can be found in the reference.conf file for the akka-actor package. By default, the default dispatcher (on which user actors run if not otherwise configured), is a fork-join pool with a minimum of 8 threads and a maximum of 64 threads. The internal dispatcher (introduced in Akka 2.6 to prevent user actors from starving system actors) is also, IIRC, a fork-join pool. Other dispatcher types and configurations are possible: the comments in reference.conf go through them, though for most purposes the defaults are reasonable.
As far as I know, there are no existing tools for visualizing the actors etc. in an Akka application.
The threads are managed by the thread-pool underlying the dispatcher. The fork-join pool will terminate threads which have been idle for a certain length of time and create new threads as needed.
The threads are (by default... at the very least a custom dispatcher could override this) named dispatcher-name-threadNumber. If you log inside an actor and have your log message format include the thread name, you can see which thread the actor was scheduled onto. Note that, typically, an actor being scheduled onto one thread does not imply it will never be scheduled on another thread (in particular, things like thread-local storage are unlikely to work); it's also worth noting that as Akka dispatchers are also Scala ExecutionContexts and Java Executors, the threads can be consumed by things which aren't actors (e.g. Future callbacks in Scala).
In the Websocket chat sample provided with the play framework, it seems to me that only one actor is ever created/used; Also it uses "receive" which if I understood well, force the 1:1 mapping between an actor and a thread, effectively making this chat server mono-threaded ?
Check the code here: https://github.com/playframework/Play20/blob/master/samples/scala/websocket-chat/app/models/ChatRoom.scala
If this analysis correct? If yes, do you have pointers on how this server could be made highly scalable?
There are some details at http://www.playframework.org/documentation/2.0.1/AkkaCore on the default dispatcher configuration for websockets which are used in that example.
Each WebSocket connection state is managed by an Agent actor. A new actor is created for each WebSocket, and is killed when the socket is closed.
That web page also shows the default configuration:
websockets-dispatcher = {
fork-join-executor {
parallelism-factor = 1.0
parallelism-max = 24
}
}
By default all dispatchers will run their set of actors on a thread pool. So for each client creating a websocket, an actor will be created. How many threads are allocated depends on which executor service is used. It seems the fork-join-executor will create threads on demand up to parallelism-max.
On top of that there are also actors to process actions and promises.
There seem to be many knobs in akka to fine tune performance. See http://doc.akka.io/docs/akka/2.0.1/general/configuration.html. Making a server "highly scalable" will probably involve lots of benchmarking and some hardware.
Although the websocket connections will have an actor pool but still the ChatRoom Actor is the only one ( single actor instance ) doing the actual processing with messages, managing connects/disconnects and acting like a router for sockets which may be the bottleneck for this design as the messages are always processed sequentially for an actor. I doubt if the sample was meant for scalability but rather a simple demonstration for websockets.
Akka 2.x requires many commands to reference an ActorSystem. So, to create an instance of an actor MyActor you might say:
val system = ActorSystem()
val myActor = system.actorOf(Props[MyActor])
Because of the frequent need for an ActorSystem: many code examples omit the creation from the code and assume that the reader knows where a system variable has come from.
If your code produces actors in different places, you could duplicate this code, possibly creating additional ActorSystem instances, or you could try to share the same ActorSystem instance by referring to some global or by passing the ActorSystem around.
The Akka documentation provides a general overview of systems of actors under the heading 'Actor Systems', and there is documentation of the ActorSystem class. But neither of these help a great deal in explaining why a user of Akka can't just rely on Akka to manage this under-the-hood.
Question(s)
What are the implications of sharing the same ActorSystem object or creating a new one each time?
What are the best practices here? Passing around an ActorSystem all the time seems surprisingly heavy-handed.
Some examples give the ActorSystem a name: ActorSystem("MySystem") others just call ActorSystem(). What difference does this make, and what if you use the same name twice?
Does akka-testkit require that you share a common ActorSystem with the one you pass to the TestKit constructor?
Creating an ActorSystem is very expensive, so you want to avoid creating a new one each time you need it. Also your actors should run in the same ActorSystem, unless there is a good reason for them not to. The name of the ActorSystem is also part the the path to the actors that run in it. E.g. if you create an actor in a system named MySystem it will have a path like akka://MySystem/user/$a. If you are in an actor context, you always have a reference to the ActorSystem. In an Actor you can call context.system. I don't know what akka-testkit expects, but you could take a look at the akka tests.
So to sum it up, you should always use the same system, unless there is a good reason not to do so.
Here are some materials which might be helpful to understand "Why does document always suggest to use one ActorSystem for one logical application" :
The heaviest part of an ActorSystem is the dispatcher. Each ActorSystem has at least one. The dispatcher is the engine that makes the actors running. In order to make running, it needs threads (usually got from a thread pool). The default dispatcher uses a fork-join thread pool with at least 8 threads.
There are shared facilities, like the guardian actors, the event stream, the scheduler, etc. Some of them are in user space, some are internal. All of them need to be created and started.
One ActorSystem with one thread pool configures to the numbers of cores should give the best results in most cases.
Here document mentions logical application, I prefer to consider blocking or non-blocking application. According to dispatcher's configuration, one ActorSystem is for one configuration. If the application is for the same logics, one ActorSystem should be enough.
Here is a discussion , if you have time, you can read it. They discuss a lot, ActorSystem, local or remote, etc.
I'm new to Scala in general and Actors in particular and my problem is so basic, the online resources I have found don't cover it.
I have a CPU-intensive, easily parallelized algorithm that will be run on an n-core machine (I don't know n). How do I implement this in Actors so that all available cores address the problem?
The first way I thought of was to simple break the problem into m pieces (where m is some medium number like 10,000) and create m Actors, one for each piece, give each Actor its little piece and let 'em go.
Somehow, this struck me as inefficient. Zillions of Actors just hanging around, waiting for some CPU love, pointlessly switching contexts...
Then I thought, make some smaller number of Actors, and feed each one several pieces. The problem was, there's no reason to expect the pieces are the same size, so one core might get bogged down, with many of its tasks still queued, while other cores are idle.
I noodled around with a Supervisor that knew which Actors were busy, and eventually realized that this has to be a solved problem. There must be a standard pattern (maybe even a standard library) for dealing with this very generic issue. Any suggestions?
Take a look at the Akka library, which includes an implementaton of actors. The Dispatchers Module gives you more options for limiting actors to cpu threads (HawtDispatch-based event-driven) and/or balancing the workload (Work-stealing event-based).
Generally, there're 2 kinds of actors: those that are tied to threads (one thread per actor), and those that share 1+ thread, working behind a scheduler/dispatcher that allocates resources (= possibility to execute a task/handle incoming message against controlled thread-pool or a single thread).
I assume, you use second type of actors - event-driven actors, because you mention that you run 10k of them. No matter how many event-driven actors you have (thousands or millions), all of them will be fighting for the small thread pool to handle the message. Therefore, you will even have a worse performance dividing your task queue into that huge number of portions - scheduler will either try to handle messages sent to 10k actors against a fixed thread pool (which is slow), or will allocate new threads in the pool (if the pool is not bounded), which is dangerous (in the worst case, there will be started 10k threads to handle messages).
Event-driven actors are good for short-time (ideally, non-blocking) tasks. If you're dealing with CPU-intensive tasks I'd limit number of threads in the scheduler/dispatcher pool (when you use event-driven actors) or actors themselves (when you use thread-based actors) to the number of cores to achieve the best performance.
If you want this to be done automatically (adjust number of threads in dispatcher pool to the number of cores), you should use HawtDisaptch (or it's Akka implementation), as it was proposed earlier:
The 'HawtDispatcher' uses the
HawtDispatch threading library which
is a Java clone of libdispatch. All
actors with this type of dispatcher
are executed on a single system wide
fixed sized thread pool. The number of
of threads will match the number of
cores available on your system. The
dispatcher delivers messages to the
actors in the order that they were
producer at the sender.
You should look into Futures I think. In fact, you probably need a threadpool which simply queues threads when a max number of threads has been reached.
Here is a small example involving futures: http://blog.tackley.net/2010/01/scala-futures.html
I would also suggest that you don't pay too much attention to the context switching since you really can't do anything but rely on the underlying implementation. Of course a rule of thumb would be to keep the active threads around the number of physical cores, but as I noted above this could be handled by a threadpool with a fifo-queue.
NOTE that I don't know if Actors in general or futures are implemented with this kind of pool.
For thread pools, look at this: http://www.scala-lang.org/api/current/scala/concurrent/ThreadPoolRunner.html
and maybe this: http://www.scala-lang.org/api/current/scala/actors/scheduler/ResizableThreadPoolScheduler.html
Good luck
EDIT
Check out this piece of code using futures:
import scala.actors.Futures._
object FibFut {
def fib(i: Int): Int = if (i < 2) 1 else fib(i - 1) + fib(i - 2)
def main(args: Array[String]) {
val fibs = for (i <- 0 to 42) yield future { fib(i) }
for (future <- fibs) println(future())
}
}
It showcases a very good point about futures, namely that you decide in which order to receive the results (as opposed to the normal mailbox-system which employs a fifo-system i.e. the fastest actor sends his result first).
For any significant project, I generally have a supervisor actor, a collection of worker actors each of which can do any work necessary, and a large number of pieces of work to do. Even though I do this fairly often, I've never put it in a (personal) library because the operations end up being so different each time, and the overhead is pretty small compared to the whole coding project.
Be aware of actor starvation if you end up utilizing the general actor threadpool. I ended up simply using my own algorithm-task-owned threadpool to handle the parallelization of a long-running, concurrent task.
The upcoming Scala 2.9 is expected to include parallel data structures which should automatically handle this for some uses. While it does not use Actors, it may be something to consider for your problem.
While this feature was originally slated for 2.8, it has been postponed until the next major release.
A presentation from the last ScalaDays is here:
http://days2010.scala-lang.org/node/138/140
I have a high CPU/memory bound task that I would like my Scala program to execute in parallel. So, I'm using the Actors framework (using receive in a while(true) loop). I call the start method on the actor and send it thousands of messages to process.
During the execution of the program (takes about an hour), only 100 - 120% of the CPU is used. The machine has 8 cores. Shouldn't the actor spawn multiple threads to use up all the 8 cores, and I should see usage close to 800%?
Or am I supposed to instantiate 8 actors and send it each some of the messages (or rather get them all to read from some concurrent queue)?
Thanks.
Nope. A single actor is guaranteed to execute on exactly one thread at a time, so access to its state doesn't need to be synchronized. If you want to spread your problem across multiple cores, you need multiple actors.