I need atomic broadcast and I want to use akka cluster.
I'm not sure is there atomic broadcast in akka, so I just want to ask, to be sure.
I know, there are such things like DistributedPubSubMediator.SendToAll aswell as Cluster Aware Routers. Does any support total order broadcast ?
By atomic (total order) broadcast I mean, actor A sends broadcast bA aswell as actor B sends broadcast bB (in parallel & from another node). Eventually, (every actor receives message bA before bB) or (every actor receives message bB before bA).
The only ordering guarantee offered by Akka is that messages sent from actor A to actor B will be received in order for that pair of actors. Messages from other actors to actor B could be received before, after or interleaved with the messages from A to B.
So no, Akka does not provide the ordering guarantee you're asking for.
By default there is no such guarantee as messages are sending asynchronously. But you can just use some mediator X (any actor including DistributedPubSubMediator) which will induce such guarantee: send (bA, recipients) from A to X, send (bB, recipients) from B to X, and X should just broadcast every message to its recipients (it will be "atomic" and sequential, as done inside one actor).
If you need to do this in cluster - you could try "Cluster Singleton" to avoid single point of failure by sacrificing high availability and maybe partition-tolerance (you will have to remove nodes manually from the cluster to allow the "P")
Related
I have some actors that kill themselves when idle or other system constraints require them to. The actors that have ActorRefs to them are watching for their Terminated(ref), but there is a race condition of messages meant for the actors being sent before the termination arrives and I'm trying to figure out a clean way to handle that.
I was considering subscribing to DeadLetter and using that to signal the sender that their ref is stale and that they need to get or spawn a new target ActorRef.
However, in Akka Typed, I cannot find any way to get to dead letters other than using the untyped co-existence path, so I figure I'm likely approaching this wrong.
Is there a better pattern for dealing dead downstream refs and re-directing messages to a new downstream refs, short of requiring some kind of ack hand-shake for every message?
Consider dead letters as a debugging tool rather something to use to implement delivery guarantees with (true for both Akka typed and untyped).
If an actor needs to be certain that a message was delivered the message protocol will need to include an an ack. To do resending the actor will also need to keep a buffer for in-flight/not yet acknowledged messages to be able to resend.
We have some ideas on an abstraction for different levels of reliability for message delivery, we'll see if that fits in Akka 2.6 or happens later though, prototyped in: https://github.com/akka/akka/pull/25099
I am not quite clear about the Router, Dispatcher, and Executor in Akka system.
If I do not define a router and therefore do not give it a group actors, the actor runs in parallel or not?
If I do not define a router, but define and use a dispatcher as:
my-dispatcher {
type = Dispatcher
executor = "fork-join-executor"
fork-join-executor {
parallelism-min = 2
parallelism-factor = 2.0
parallelism-max = 10
}
throughput = 100
}
The actor runs in parallel or not?
If I do not define a router and therefore do not give it a group
actors, the actor runs in parallel or not?
The Dispatcher in an Actor system is a wrapper over an ExecutionContext which has an underlying pool of threads it executes the actors on. If all you create is a single actor and pass the messages to it, then all messages will arrive to the same actor mailbox which he will process one by one. Execution inside the actor happens synchronously.
If you want multiple actors to treat a request then that is exactly what a Router is for. If you want to have a group of actors which can treat requests the same, you allocate a router and it will manage the group of actors for you, taking care if which actor gets which amount of work.
One example is when you have a router and you define an internal pool of actors, i.e. a RoundRobinPool which simply assigns messages to actor mailboxes in a round robin fashion:
context.actorOf(
Props(new MyActor())
.withRouter(RoundRobinPool(nrOfInstances = 10)), name = "MyRouter")
This will create a Router with a RoundRobinPool which contains 10 actors.
Messages are executed serially inside an Actor
thats why it is said "one actor is no actor"
When a message is sent to the actor. It lands inside the actor mail box. Messages in the mail box are processed one after the other in a serial manner. One important thing to note here is that actor ensures that each message behaviour is executed in a single thread it could be different thread.
So routers creates groups of actors and routes the messages to group of actors so that each actor processes it simultaneously and in parallel fashion
I'm using slick to store data in database, and there I use the threadLocalSession to store the sessions.
The repositories are used to do the crud, and I have an Akka service layer that access the slick repositories.
I found this link, where Adam Gent asks something near what I'm asking here: Akka and Java libraries that use ThreadLocals
My concern is about how does akka process a message, as I store the database session in a threadLocal, can I have two messages been processed at the same time in the same thread?
Let's say: Two add user messages (A and B) sent to the userservice, and message A is partially processed, and stopped, thread B start to process in the same thread that thread A has started to process, which will have the session stored in it's localSession?
Each actor processes its messages one at a time, in the order it received them*. Therefore, if you send messages A, B to the same actor, then they are never processed concurrently (of course the situation is different if you send each of the messages to different actors).
The problem with the use of ThreadLocals is that in general it is not guaranteed that an actor processes each of its messages on the same thread.
So if you send a message M1 and then a message M2 to actor A, it is guaranteed that M1 is processed before M2. What is not guaranteed that M2 is processed on the same thread as M1.
In general, you should avoid using ThreadLocals, as the whole point of actors is that they are a unit of consistency, and you are safe to modify their internal state via message passing. If you really need more control on the threads which execute the processing of messages, look into the documentation of dispatchers: http://doc.akka.io/docs/akka/2.1.0/java/dispatchers.html
*Except if you change their mailbox implementation, but that's a non-default behavior.
My Akka FSM actor has the need to prioritize messages dependent on type. To be specific, the actor receives messages in one of these categories, in prioritized order:
Messages that triggers state transitions
Messages that query the current state
Messages that causes the actor to perform some work ("WorkMsg")
According to the Akka docs, messages can be prioritized according to the list above with a PriorityExecutorBasedEventDrivenDispatcher containing a PriorityGenerator. I've implemented the FSM actor with this dispatcher and it works well.
The problem is that this dispatcher also reorders WorkMsgs, which is not what I want.
WorkMsgs contain a timestamp and are sent to the FSM actor sorted by this timestamp. When the FSM actor processes WorkMsgs, it discards WorkMsgs that are older than the previous WorkMsg. So if these are reordered, I lose data.
Without the PriorityExecutorBasedEventDrivenDispatcher, WorkMsgs are not reordered, but then the priorities in the list above are not satisfied.
How can I maintain the priorities in the list above, while preventing reordering of messages of the same priority?
A prioritizing proxy actor can prioritize messages to be sent to your worker actor. You will have to sort and store the incoming messages as well as implement the prioritization logic. The worker will additionally have to respond to the proxy after each message to let it know that it is ready for more work.
In Java, to write a library that makes requests to a server, I usually implement some sort of dispatcher (not unlike the one found here in the Twitter4J library: http://github.com/yusuke/twitter4j/blob/master/twitter4j-core/src/main/java/twitter4j/internal/async/DispatcherImpl.java) to limit the number of connections, to perform asynchronous tasks, etc.
The idea is that N number of threads are created. A "Task" is queued and all threads are notified, and one of the threads, when it's ready, will pop an item from the queue, do the work, and then return to a waiting state. If all the threads are busy working on a Task, then the Task is just queued, and the next available thread will take it.
This keeps the max number of connections to N, and allows at most N Tasks to be operating at the same time.
I'm wondering what kind of system I can create with Actors that will accomplish the same thing? Is there a way to have N number of Actors, and when a new message is ready, pass it off to an Actor to handle it - and if all Actors are busy, just queue the message?
Akka Framework is designed to solve this kind of problems, and is exactly what you're looking for.
Look thru this docu - there're lots of highly configurable dispathers (event-based, thread-based, load-balanced, work-stealing, etc.) that manage actors mailboxes, and allow them to work in conjunction. You may also find interesting this blog post.
E.g. this code instantiates new Work Stealing Dispatcher based on the fixed thread pool, that fulfils load balancing among the actors it supervises:
val workStealingDispatcher = Dispatchers.newExecutorBasedEventDrivenWorkStealingDispatcher("pooled-dispatcher")
workStealingDispatcher
.withNewThreadPoolWithLinkedBlockingQueueWithUnboundedCapacity
.setCorePoolSize(16)
.buildThreadPool
Actor that uses the dispatcher:
class MyActor extends Actor {
messageDispatcher = workStealingDispatcher
def receive = {
case _ =>
}
}
Now, if you start 2+ instances of the actor, dispatcher will balance the load between the mailboxes (queues) of the actors (actor that has too much messages in the mailbox will "donate" some to the actors that has nothing to do).
Well, you have to see about the actors scheduler, as actors are not usually 1-to-1 with threads. The idea behind actors is that you may have many of them, but the actual number of threads will be limited to something reasonable. They are not supposed to be long running either, but rather quickly answering to messages they receive. In short, the architecture of that code seems to be wholly at odds with how one would design an actor system.
Still, each working actor may send a message to a Queue actor asking for the next task, and then loop back to react. This Queue actor would receive either queueing messages, or dequeuing messages. It could be designed like this:
val q: Queue[AnyRef] = new Queue[AnyRef]
loop {
react {
case Enqueue(d) => q enqueue d
case Dequeue(a) if q.nonEmpty => a ! (q dequeue)
}
}