Subscribing multiple actors to Dead Letters in Akka - scala

I am trying to create a simple application that has two actors:
Master actor that handles some App actions
DeadLettersListener that is supposed to handle all dead or unhandled messages
Here is the code that works perfectly:
object Hw extends App {
// creating Master actor
val masterActorSystem = ActorSystem("Master")
val master = masterActorSystem.actorOf(Props[Master], "Master")
// creating Dead Letters listener actor
val deadLettersActorSystem = ActorSystem.create("DeadLettersListener")
val listener = deadLettersActorSystem.actorOf(Props[DeadLettersListener])
// subscribe listener to Master's DeadLetters
masterActorSystem.eventStream.subscribe(listener, classOf[DeadLetter])
masterActorSystem.eventStream.subscribe(listener, classOf[UnhandledMessage])
}
According to the akka manual though, ActorSystem is a heavy object and we should create only one per application. But when I replace these lines:
val deadLettersActorSystem = ActorSystem.create("DeadLettersListener")
val listener = deadLettersActorSystem.actorOf(Props[DeadLettersListener])
with this code:
val listener = masterActorSystem.actorOf(Props[DeadLettersListener], "DeadLettersListener")
The subscription does not work any more and DeadLettersListener is not getting any Dead or Unhandled messages.
Can you please explain what am I doing wrong and give an advice how to subscribe to Dead Letters in this case?

I can't really imagine what are you doing wrong, I created a small example, and it seems to work:
object Hw extends App {
class Master extends Actor {
override def receive: Receive = {
case a => println(s"$a received in $self")
}
}
class DeadLettersListener extends Actor {
override def receive: Actor.Receive = {
case a => println(s"$a received in $self")
}
}
// creating Master actor
val masterActorSystem = ActorSystem("Master")
val master = masterActorSystem.actorOf(Props[Master], "Master")
val listener = masterActorSystem.actorOf(Props[DeadLettersListener])
// subscribe listener to Master's DeadLetters
masterActorSystem.eventStream.subscribe(listener, classOf[DeadLetter])
masterActorSystem.eventStream.subscribe(listener, classOf[UnhandledMessage])
masterActorSystem.actorSelection("/unexistingActor") ! "yo"
}
Could you try it?

Related

Akka: First message going to dead letters, from second message it is all fine

I have supervisor actor which selects child actor based on command received, whenever it creates a new child actor and sends message(ask pattern) it timesout as child actor tries to send back response but it goes to dead letters.
Here is the child actor code
class IdActor(id: String, injector: Injector) extends Actor {
override def receive: Receive = {
case cmd: GenerateIdCmd =>
val parent = sender()
...
Future(idEvent) pipeTo sender //Issue is here going to dead letters
//Future(idEvent) pipeTo parent //This also leads to same problem
//parent ! idEvent // Same issue
}
}
Here is supervisor code
class IdSupervisor(injector: Injector) extends Actor {
override def receive: Receive = {
case cmd: GenerateIdCmd =>
...
val ref = context.child(cmd.id).getOrElse {
context.actorOf(Props(classOf[IdActor], cmd.id, injector), cmd.id)
}
ask(ref, cmd) pipeTo sender
}
}
Second message is flowing back to originator, first response from all new child actors going to dead letters from there afterwards going good.
Working Solution
Issue is with Supervisor, fixed code
class IdSupervisor(injector: Injector) extends Actor {
override def receive: Receive = {
case cmd: GenerateIdCmd =>
val originator = sender
...
val ref = context.child(cmd.id).getOrElse {
context.actorOf(Props(classOf[IdActor], cmd.id, injector), cmd.id)
}
ask(ref, cmd) pipeTo originator
}
}
I suspect that the issue doesn't actually occur when a child replies to the parent as you describe, but when the parent sends a message to a child and expects a reply:
val ref = context.child(cmd.id).getOrElse {
context.actorOf(Props(classOf[IdActor], cmd.id, injector), cmd.id)
}
ask(ref, cmd) pipeTo sender
An actor is started asynchronously upon creation. The observation that, when a new child is created, the first message to that child results in dead letters, while subsequent messages to that child result in the intended behavior, suggests an actor initialization issue. What's probably happening is that the child actors receive their first GenerateIdCmd message before they have completely started.
Never ever pipe directly to sender from future inside actor, sender is def and at the moment when future is completed, it might be already different one than you expect. One of solution is to store sender before future call:
class IdActor(id: String, injector: Injector) extends Actor {
override def receive: Receive = {
case cmd: GenerateIdCmd =>
...
val originalSender = sender
Future(idEvent) pipeTo originalSender
}
}

Akka round-robin: Sending response from remote routees to sender

I am using Akka Cluster (version 2.4.10) with few nodes designated for "front-end" role and few others as "workers". The workers are on remote machines. The incoming work is distributed by the front-end actor to workers by round-robin routing. The issue is sending back the response from the "workers" back to the front-end actor. I can see that the work is getting completed by the workers. But the message sent by the workers to front-end does not reach and ends up as dead-letters. I see the below error in the log.
[Cluster-akka.actor.default-dispatcher-21] [akka://Cluster/deadLetters] Message [scala.collection.immutable.$colon$colon] from Actor[akka://Cluster/user] to Actor[akka://Cluster/deadLetters] was not delivered. [6] dead letters encountered.
I have seen this and I am following the same in my code. I have also seen this, but the solution suggested does not apply in this case, because I do not know the routees up-front. It comes through the configuration and it can change. The round-robin router configuration is as below.
akka.actor.deployment {
/frontEnd/hm = {
router = round-robin-group
nr-of-instances = 5
routees.paths = ["/user/hmWorker"]
cluster {
enabled = on
use-role = backend
allow-local-routees = on
}
}
}
The router is instantiated in front-end actor like below.
val router = context.actorOf(FromConfig.props(), name = "hm")
val controller = context.actorOf(Props(classOf[Controller], router))
The controller and the worker codes are below.
// Node 1 : Controller routes requests using round-robin
class Controller(router: ActorRef) extends Actor {
val list = List("a", "b") // Assume this is a big list
val groups = list.grouped(500)
override def receive: Actor.Receive = {
val futures = groups.map(grp => (router ? Message(grp)).mapTo[List[String]]))
val future = Future.sequence(futures).map(_.flatten)
val result = Await.result(future, 50 seconds)
println(s"Result is $result")
}
}
// Node 2
class Worker extends Actor {
override def receive: Actor.Receive = {
case Message(lst) =>
val future: Future[List[String]] = // Do Something asynchronous
future onComplete {
case Success(r) => sender.!(r)(context.parent) // This message is not delivered to Controller actor.
case Failure(th) => // Error handling
}
}
}
Please let me know what I am doing wrong here. Appreciate your help.
You shouldn't use sender() in the callback on a Future. By the time the callback is processed, the sender() is likely referring to something different than it was when you received the message.
Consider either saving the reference outside of the callback first like:
override def receive: Actor.Receive = {
case Message(lst) =>
val future: Future[List[String]] = // Do Something asynchronous
val replyTo: ActorRef = sender()
future onComplete {
case Success(r) => replyTo.!(r)(context.parent) // This message is not delivered to Controller actor.
case Failure(th) => // Error handling
}
}
Or even better, use the pipe pattern:
import akka.pattern.pipe
override def receive: Actor.Receive = {
case Message(lst) =>
val future: Future[List[String]] = // Do Something asynchronous
future.pipeTo(sender())
}

Akka-Streams ActorPublisher does not receive any Request messages

I am trying to continuously read the wikipedia IRC channel using this lib: https://github.com/implydata/wikiticker
I created a custom Akka Publisher, which will be used in my system as a Source.
Here are some of my classes:
class IrcPublisher() extends ActorPublisher[String] {
import scala.collection._
var queue: mutable.Queue[String] = mutable.Queue()
override def receive: Actor.Receive = {
case Publish(s) =>
println(s"->MSG, isActive = $isActive, totalDemand = $totalDemand")
queue.enqueue(s)
publishIfNeeded()
case Request(cnt) =>
println("Request: " + cnt)
publishIfNeeded()
case Cancel =>
println("Cancel")
context.stop(self)
case _ =>
println("Hm...")
}
def publishIfNeeded(): Unit = {
while (queue.nonEmpty && isActive && totalDemand > 0) {
println("onNext")
onNext(queue.dequeue())
}
}
}
object IrcPublisher {
case class Publish(data: String)
}
I am creating all this objects like so:
def createSource(wikipedias: Seq[String]) {
val dataPublisherRef = system.actorOf(Props[IrcPublisher])
val dataPublisher = ActorPublisher[String](dataPublisherRef)
val listener = new MessageListener {
override def process(message: Message) = {
dataPublisherRef ! Publish(Jackson.generate(message.toMap))
}
}
val ticker = new IrcTicker(
"irc.wikimedia.org",
"imply",
wikipedias map (x => s"#$x.wikipedia"),
Seq(listener)
)
ticker.start() // if I comment this...
Thread.currentThread().join() //... and this I get Request(...)
Source.fromPublisher(dataPublisher)
}
So the problem I am facing is this Source object. Although this implementation works well with other sources (for example from local file), the ActorPublisher don't receive Request() messages.
If I comment the two marked lines I can see, that my actor has received the Request(count) message from my flow. Otherwise all messages will be pushed into the queue, but not in my flow (so I can see the MSG messages printed).
I think it's something with multithreading/synchronization here.
I am not familiar enough with wikiticker to solve your problem as given. One question I would have is: why is it necessary to join to the current thread?
However, I think you have overcomplicated the usage of Source. It would be easier for you to work with the stream as a whole rather than create a custom ActorPublisher.
You can use Source.actorRef to materialize a stream into an ActorRef and work with that ActorRef. This allows you to utilize akka code to do the enqueing/dequeing onto the buffer while you can focus on the "business logic".
Say, for example, your entire stream is only to filter lines above a certain length and print them to the console. This could be accomplished with:
def dispatchIRCMessages(actorRef : ActorRef) = {
val ticker =
new IrcTicker("irc.wikimedia.org",
"imply",
wikipedias map (x => s"#$x.wikipedia"),
Seq(new MessageListener {
override def process(message: Message) =
actorRef ! Publish(Jackson.generate(message.toMap))
}))
ticker.start()
Thread.currentThread().join()
}
//these variables control the buffer behavior
val bufferSize = 1024
val overFlowStrategy = akka.stream.OverflowStrategy.dropHead
val minMessageSize = 32
//no need for a custom Publisher/Queue
val streamRef =
Source.actorRef[String](bufferSize, overFlowStrategy)
.via(Flow[String].filter(_.size > minMessageSize))
.to(Sink.foreach[String](println))
.run()
dispatchIRCMessages(streamRef)
The dispatchIRCMessages has the added benefit that it will work with any ActorRef so you aren't required to only work with streams/publishers.
Hopefully this solves your underlying problem...
I think the main problem is Thread.currentThread().join(). This line will 'hang' current thread because this thread is waiting for himself to die. Please read https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#join-long- .

Akka Actors unit testing for dummies

I'm newbie in Akka and Scala and i come from a non-concurrent world. Probably i'm doing a lot of things wrong, i will appreciate feedback even it's not related to the question.
I'm doing a simple chat application with Akka and Scala. I started (bc business requirements) by "typing feature"... it's the typical feature in whatsapp or tellegram "John is typing a message".
I have modelled it using two actors types: Talkers and Conversation, and I want to unit test my Conversation actor. My Conversation actor looks like this:
object Conversation {
def props(conversationId: UUID, talkers: List[ActorRef])(out: ActorRef) = Props(new Conversation(conversationId, talkers))
case class Typing(talkerId: TalkerId)
}
class Conversation(conversationId: UUID, talkers: List[ActorRef]) extends Actor with ActorLogging {
def receive = LoggingReceive {
case Typing(talkerId) =>
// notify all talkers that a talker is typing
// #TODO don't notify user which is typing
talkers foreach {talker: ActorRef => talker ! InterlocutorTyping(talkerId)}
}
}
I think, by now is very simple. So, before start coding in Scala and Akka I had tested this like:
I get my Conversation actor
I mock talkers
I send a message Typing to my actor
I expect that talkers should be notified
I don't really know if it's the correct approach in Scala and Akka. My test (using scalatest) looks like this:
"Conversation" should {
"Notify interlocutors when a talker is typing" in {
val talkerRef1 = system.actorOf(Props())
val talkerRef2 = system.actorOf(Props())
val talkerRef1Id = TalkerIdStub.random
val conversationId = UUID.randomUUID()
val conversationRef = system.actorOf(Props(classOf[Conversation], conversationId, List(talkerRef1, talkerRef2)))
// should I use TestActorRef ?
conversationRef ! InterlocutorTyping(talkerRef1Id)
// assert that talker2 is notified when talker1 is typing
}
}
Should I use TestActorRef? Should I use TestProbe() (I read that this is for integration tests)
How can I create Talker mocks? Is this approach correct?
It's correct to inject a List of Talkers to my conversation Actor?
I searched for documentation, but I think there are a lot too old and I'm not sure if the code examples are still functional.
Thank you for your time guys, and sorry about this noob question :=)
It's true that the testing situation in Akka is a little confusing to say the least.
In Akka generally you have two kinds of test, synchronous and asynchronous which some people term as 'unit' and 'integration' tests.
'Unit tests' are synchronous, you directly test the receive method without requiring an actor system, etc. In your case, you would want to mock the List[Talkers], call your receive method and verify that the send method is called. You can directly instantiate your actor with new Conversation(mockTalkers), it's not necessary in this case to use TestActorRef. For mocking, I recommend ScalaMock.
'Integration tests' are asynchronous, and generally test more than one actor working together. This is where you inherit TestKit, instantiate TestProbes to act as your talkers, use one to send a message to the Conversation actor, and verify that the other receives the InterlocutorTyping message.
It's up to you which kind of test you think is appropriate. My personal opinion is that unless you have complicated internal bevaviour in your actor, you should skip the synchronous tests and go straight for the asynchronous ('integration') tests as this will cover more tricky concurrency edge cases that you might otherwise miss. These are also more 'black-box' and so less sensitive to change as you evolve your design.
More details and code samples on the doc page.
Finally I did this (there are some more functionality than in the question):
object Conversation {
def props(conversationId: UUID)(out: ActorRef) = Props(new Conversation(conversationId))
case class TalkerTyping(talkerId: TalkerId)
case class TalkerStopTyping(talkerId: TalkerId)
case class Join(talker: ActorRef)
case class Leave(talker: ActorRef)
}
class Conversation(conversationId: UUID) extends Actor with ActorLogging {
var talkers : ListBuffer[ActorRef] = ListBuffer.empty
val senderFilter = { talker: ActorRef => talker != sender() }
def receive = LoggingReceive {
case Join =>
talkers += sender()
case Leave =>
talkers -= sender()
case TalkerTyping(talkerId) => // notify all talkers except sender that a talker is typing
talkers filter senderFilter foreach { talker: ActorRef => talker ! InterlocutorTyping(talkerId) }
case TalkerStopTyping(talkerId) => // notify all talkers except sender that a talker has stopped typing
talkers filter senderFilter foreach { talker: ActorRef => talker ! InterlocutorStopTyping(talkerId) }
}
}
And my test:
class ConversationSpec extends ChatUnitTestCase("ConversationSpec") {
trait ConversationTestHelper {
val talker = TestProbe()
val anotherTalker = TestProbe()
val conversationRef = TestActorRef[Conversation](Props(new Conversation(UUID.randomUUID())))
val conversationActor = conversationRef.underlyingActor
}
"Conversation" should {
"let user join it" in new ConversationTestHelper {
conversationActor.talkers should have size 0
conversationRef ! Join
conversationActor.talkers should have size 1
conversationActor.talkers should contain(testActor)
}
"let joining user leave it" in new ConversationTestHelper {
conversationActor.talkers should have size 0
conversationRef ! Join
conversationActor.talkers should have size 1
conversationActor.talkers should contain(testActor)
conversationRef ! Leave
conversationActor.talkers should have size 0
conversationActor.talkers should not contain testActor
}
"notify interlocutors when a talker is typing" in new ConversationTestHelper {
val talker1 = TestProbe()
val talker2 = TestProbe()
talker1.send(conversationRef, Join)
talker2.send(conversationRef, Join)
val talker2Id = TalkerIdStub.random
talker2.send(conversationRef, TalkerTyping(talker2Id))
talker1.expectMsgPF() {
case InterlocutorTyping(talkerIdWhoTyped) if talkerIdWhoTyped == talker2Id => true
}
talker2.expectNoMsg()
}
"notify interlocutors when a talker stop typing" in new ConversationTestHelper {
val talker1 = TestProbe()
val talker2 = TestProbe()
talker1.send(conversationRef, Join)
talker2.send(conversationRef, Join)
val talker2Id = TalkerIdStub.random
talker2.send(conversationRef, TalkerStopTyping(talker2Id))
talker1.expectMsgPF() {
case InterlocutorStopTyping(talkerIdWhoStopTyping) if talkerIdWhoStopTyping == talker2Id => true
}
talker2.expectNoMsg()
}
}
}

Scala Akka Consumer/Producer: Return Value

Problem Statement
Assume I have a file with sentences that is processed line by line. In my case, I need to extract Named Entities (Persons, Organizations, ...) from these lines. Unfortunately, the tagger is quite slow. Therefore, I decided to parallelize the computation, such that lines could be processed independent from each other and the result is collected in a central location.
Current Approach
My current approach comprises the usage of a single producer multiple consumer concept. However, I'm relative new to Akka, but I think my problem description fits well into its capabilities. Let me show you some code:
Producer
The Producer reads the file line by line and sends it to the Consumer. If it reaches the total line limit, it propagates the result back to WordCount.
class Producer(consumers: ActorRef) extends Actor with ActorLogging {
var master: Option[ActorRef] = None
var result = immutable.List[String]()
var totalLines = 0
var linesProcessed = 0
override def receive = {
case StartProcessing() => {
master = Some(sender)
Source.fromFile("sent.txt", "utf-8").getLines.foreach { line =>
consumers ! Sentence(line)
totalLines += 1
}
context.stop(self)
}
case SentenceProcessed(list) => {
linesProcessed += 1
result :::= list
//If we are done, we can propagate the result to the creator
if (linesProcessed == totalLines) {
master.map(_ ! result)
}
}
case _ => log.error("message not recognized")
}
}
Consumer
class Consumer extends Actor with ActorLogging {
def tokenize(line: String): Seq[String] = {
line.split(" ").map(_.toLowerCase)
}
override def receive = {
case Sentence(sent) => {
//Assume: This is representative for the extensive computation method
val tokens = tokenize(sent)
sender() ! SentenceProcessed(tokens.toList)
}
case _ => log.error("message not recognized")
}
}
WordCount (Master)
class WordCount extends Actor {
val consumers = context.actorOf(Props[Consumer].
withRouter(FromConfig()).
withDispatcher("consumer-dispatcher"), "consumers")
val producer = context.actorOf(Props(new Producer(consumers)), "producer")
context.watch(consumers)
context.watch(producer)
def receive = {
case Terminated(`producer`) => consumers ! Broadcast(PoisonPill)
case Terminated(`consumers`) => context.system.shutdown
}
}
object WordCount {
def getActor() = new WordCount
def getConfig(routerType: String, dispatcherType: String)(numConsumers: Int) = s"""
akka.actor.deployment {
/WordCount/consumers {
router = $routerType
nr-of-instances = $numConsumers
dispatcher = consumer-dispatcher
}
}
consumer-dispatcher {
type = $dispatcherType
executor = "fork-join-executor"
}"""
}
The WordCount actor is responsible for creating the other actors. When the Consumer is finished the Producer sends a message with all tokens. But, how to propagate the message again and also accept and wait for it? The architecture with the third WordCount actor might be wrong.
Main Routine
case class Run(name: String, actor: () => Actor, config: (Int) => String)
object Main extends App {
val run = Run("push_implementation", WordCount.getActor _, WordCount.getConfig("balancing-pool", "Dispatcher") _)
def execute(run: Run, numConsumers: Int) = {
val config = ConfigFactory.parseString(run.config(numConsumers))
val system = ActorSystem("Counting", ConfigFactory.load(config))
val startTime = System.currentTimeMillis
system.actorOf(Props(run.actor()), "WordCount")
/*
How to get the result here?!
*/
system.awaitTermination
System.currentTimeMillis - startTime
}
execute(run, 4)
}
Problem
As you see, the actual problem is to propagate the result back to the Main routine. Can you tell me how to do this in a proper way? The question is also how to wait for the result until the consumers are finished? I had a brief look into the Akka Future documentation section, but the whole system is a little bit overwhelming for beginners. Something like var future = message ? actor seems suitable. Not sure, how to do this. Also using the WordCount actor causes additional complexity. Maybe it is possible to come up with a solution that doesn't need this actor?
Consider using the Akka Aggregator Pattern. That takes care of the low-level primitives (watching actors, poison pill, etc). You can focus on managing state.
Your call to system.actorOf() returns an ActorRef, but you're not using it. You should ask that actor for results. Something like this:
implicit val timeout = Timeout(5 seconds)
val wCount = system.actorOf(Props(run.actor()), "WordCount")
val answer = Await.result(wCount ? "sent.txt", timeout.duration)
This means your WordCount class needs a receive method that accepts a String message. That section of code should aggregate the results and tell the sender(), like this:
class WordCount extends Actor {
def receive: Receive = {
case filename: String =>
// do all of your code here, using filename
sender() ! results
}
}
Also, rather than blocking on the results with Await above, you can apply some techniques for handling Futures.