akka- how to ensure all responses of dynamic number of actors are returned to parent actor? - scala

I need to create variable number of actors each time my program starts and then must ensure all responses are return after a period of time. This
link gives a good idea for fixed number of actors but what about dynamic number?
This is my code that creates actor and passes messages to them:
ruleList = ...
val childActorList: Iterable[ActorRef] = ruleList.map(ruleItem =>
context.actorOf(DbActor.props(ruleItem.parameter1, ruleItem.parameter2)))
implicit val timeout = Timeout(10.second)
childActorList.foreach(childActor =>
childActor ? (tempTableName, lastDate)
)
Updated-1
According to #Raman Mishra guides , I updated my code as bellow, this is the code in parent actor:
override val supervisorStrategy: SupervisorStrategy = {
OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 10 seconds) {
case exp: SQLException => //Resume;
throw exp
case exp:AskTimeoutException => throw exp
case other: Exception => throw other
}
}
override def receive: Receive = {
case Start(tempTableName, lastDate) => {
implicit val timeout = Timeout(10.second)
ruleList.foreach { ruleItem =>
val childActor = context.actorOf(DbActor.props(ruleItem._1, query = ruleItem._2))
ask(childActor, (tempTableName, lastDate)).mapTo[Seq[Int]]
onComplete {
lastDate)).mapTo[Seq[Int]] onComplete {
case util.Success(res) => println("done" + res + ruleItem._2)
case util.Failure(exp: AskTimeoutException) => println("Failed query:" + ruleItem._2); throw exp
case other => println(other)
}
}
And in child actor:
case (brokerTableName, lastDate) => {
Logger("Started query by actor" + self.path.name + ':' +
val repo = new Db()
val res = repo.getAggResult(query = (brokerTableName, lastDate))
val resWrapper = res match {
case elem: Future[Any] => elem
case elem:Any => Future(elem)
}
resWrapper pipeTo self
}
case res:List[Map[Any, Any]] => {
// here final result is send to parent actor
repo.insertAggresults(res, aggTableName) pipeTo context.parent
}
Now, whenever I run main app, first, parent actor starts and create child actors and send messages to them using ask method. Child actors do their tasks but the problem here is child actors response never returns back to parent actor and in every run of app, AskTimeoutException occurs. I doubt if the use of onComplete method is correct or not. Any help will be appreciated.
"Updated-2"
I found out the problem is in using context.parent instead of sender(). Also, when I pipe to sender, first part of my result, and the sender ask for second part, the problem is resolved but I don't know what happens here, why Can't I pipe to self and return the final result to parent?
This is the last code:
In parent actor:
override def receive: Receive = {
case Start(tempTableName, lastDate) => {
println("started: called by remote actor")
implicit val timeout = Timeout(5 second)
ruleList.foreach { ruleItem =>
val childActor = context.actorOf(DbActor.props(ruleItem._1, query = ruleItem._2))
ask(childActor, Broker(tempTableName, lastDate)) onComplete {
// (childActor ? Broker(tempTableName, lastDate)).mapTo[Seq[Int]] onComplete {
case util.Success(res: List[Map[Any, Any]]) => (childActor ? res) onComplete {
case util.Success(res: Seq[Any]) => println("Successfull- Num,ber of documents:" + res.length + " " + ruleItem._2)
case util.Failure(exp: AskTimeoutException) => println("Failed for writing - query:" + ruleItem._2); throw exp
}
case util.Failure(exp: AskTimeoutException) => println("Failed for reading - query :" + ruleItem._2); throw exp
case other => println(other)
}
}
}
}
In child actor:
case (brokerTableName, lastDate) => {
Logger("Started query by actor" + self.path.name + ':' +
val repo = new Db()
val res = repo.getAggResult(query = (brokerTableName, lastDate))
val resWrapper = res match {
case elem: Future[Any] => elem
case elem:Any => Future(elem)
}
resWrapper pipeTo sender()
}
case res:List[Map[Any, Any]] => {
// here final result is send to parent actor
repo.insertAggresults(res, aggTableName) pipeTo sender()
}

The reason that replying to sender() works where replying to context.parent does not is that ask creates an temporary actor to handle the response. You need to reply to this temporary actor: the sender (which is different from the parent).
Also it's not clear whether the getAggResult method is blocking. If so this will not help (see here).

Related

Processing unknown amount of actors results on single timeout

I am looking to expand the following code to work for an unknown amount of actor ask requests.
implicit val timeout = Timeout(100 millis)
val sendRequestActor = context.actorOf(Props(new SendRequest(request)), "Send_Request_".concat(getActorNumber))
val sendRequestActor2 = context.actorOf(Props(new SendRequest(request)), "Send_Request_".concat(getActorNumber))
val a1 = ask(sendRequestActor, Request).fallbackTo(Future.successful(RequestTimeout))
val a2 = ask(sendRequestActor2, Request).fallbackTo(Future.successful(RequestTimeout))
val result = for {
r1 <- a1
r2 <- a2
} yield(r1, r2)
val r = Await.result(result, 100 millis)
r match {
case (b: SuccessResponse, b2: SuccessResponse) => {
//Process Results
}
case (b: SuccessResponse, b2: RequestTimeout) => {
//Process Results
}
case (b: RequestTimeout, b2: SuccessResponse) => {
//Process Results
}
case (b: RequestTimeout, b2: RequestTimeout) => {
//Process Results
}
case _ => {}
}
I am trying to send out requests to a List of recipients(gotten from a previous database call). The number of recipients will vary each time this function is called. Recipients have a maximum of 100 milliseconds to respond before I want to time out their requests and record a RequestTimeout. The SendRequest actor will reply with SuccessResponse if the recipients respond. I am assuming I will have to change the val result for-loop to process a list, but I am unsure of how to structure everything so that I will wait the minimum amount of time(either when all actors return or when the timeout hits, whichever is lower). I do not need everything in a single return value like the example, I am fine with a list of results and matching type on each iteration.
Any help would be appreciated, please let me know if I can provide any other information.
Thank you
Edit:
Calling Class:
case object GetResponses
def main(args: Array[String]) {
val route = {
get {
complete {
//stuff
val req_list = List(req1,req2,req3)
val createRequestActor = system.actorOf(Props(new SendAll(req_list)), "Get_Response_Actor_" + getActorNumber)
val request_future = ask(createRequestActor, GetResponses).mapTo[List[Any]]
Thread.sleep(1000)
println(request_future)
//more stuff
}
}
}
Http().bindAndHandle(route, "localhost", 8080)
}
Updated Sending Class:
class SendAll(requests: List[Request]) extends Actor {
import context.{become,dispatcher}
var numProcessed = 0
var results: List[Any] = List()
requests.foreach(self ! _)
implicit val timeout = Timeout(100 millis)
def receive = {
case r: RequestMsg =>
val sendRequestActor = context.actorOf(Props(new SendRequest(r)), "Send_Request_".concat(getActorNumber))
(sendRequestActor ? Request).pipeTo(self)
case s: SuccessResponse =>
println("Got Success")
results = results :+ s
println(results.size + " == " + requests.size)
if(results.size == requests.size) {
println("Before done")
become(done)
}
case akka.actor.Status.Failure(f) =>
println("Got Failed")
results = results :+ RequestTimeout
if(results.size == requests.size) {
become(done)
}
case m =>
println("Got Other")
}
def done: Receive = {
case GetResponses =>
println("Done")
sender ! results
case _ => {
println("Done as well")
}
}
}
Output
Got Success
1 == 3
Got Success
2 == 3
Got Success
3 == 3
Before done
Future(<not completed>)
I would pass the list of requests to the actor, then pipe the responses from the child actors to self instead of using Await.result. For example:
class Handler(requests: List[RequestMsg]) extends Actor {
import context.{become, dispatcher}
var numProcessed = 0
var results: List[Any] = List()
requests.foreach(self ! _)
implicit val timeout = Timeout(100.millis)
def receive = {
case r: RequestMsg =>
val sendRequestActor = context.actorOf(Props(new SendRequest(r)), "Send_Request".concat(getActorNumber))
(sendRequestActor ? Request).pipeTo(self)
case s: SuccessResponse =>
println(s"response: $s")
results = results :+ s
if (results.size == requests.size)
become(done)
case akka.actor.Status.Failure(f) =>
println("a request failed or timed out")
results = results :+ RequestTimeout
if (results.size == requests.size)
become(done)
case m =>
println(s"Unhandled message received while processing requests: $m")
sender ! NotDone
}
def done: Receive = {
case GetResponses =>
println("sending responses")
sender ! results
}
}
You would instantiate an actor for every list of requests:
val requests1 = List(RequestMsg("one"), RequestMsg("two"), RequestMsg("three"))
val handler1 = system.actorOf(Props(new Handler(requests1)))
In this example--following the principle that an actor should have a distinct, limited sphere of responsibility--the actor simply coordinates requests and responses; it doesn't perform any processing on the collected responses. The idea is that another actor would send this actor a GetResponses messages in order to get the responses and process them (or this actor would proactively send the results to a processing actor).
The simplest solution is put all your actor refs into the List map it to List[Future] and use Future.sequence to obtain Future[List].
val route = {
get {
val listActorRefs = List(actorRef1, actorRef2, ...)
val futureListResponses = Future.sequence(listActorRefs.map(_ ? Request))
onComplete(futureListResponses) {
case Success(listResponse) => ...
complete(...)
case Failure(exception) => ...
}
}
}
A better solution is avoid a lot of actor' asks, prepare some ResponseCollector actor which will send all your message (I suggest to look at BroadcastPool) and schedule one message for itself to stop waiting and return result.

How to work with Source.Queue in Akka-Stream

I am toying around trying to use a source.queue from an Actor. I am stuck in parttern match the result of an offer operation
class MarcReaderActor(file: File, sourceQueue: SourceQueueWithComplete[Record]) extends Actor {
val inStream = file.newInputStream
val reader = new MarcStreamReader(inStream)
override def receive: Receive = {
case Process => {
if (reader.hasNext()) {
val record = reader.next()
pipe(sourceQueue.offer(record)) to self
}
}
case f:Future[QueueOfferResult] =>
}
}
}
I don't know how to check if it was Enqueued or Dropped or Failure
if i write f:Future[QueueOfferResult.Enqueued] the compile complain
Since you use pipeTo, you do no need to match on futures - the contents of the future will be sent to the actor when this future is completed, not the future itself. Do this:
override def receive: Receive = {
case Process =>
if (reader.hasNext()) {
val record = reader.next()
pipe(sourceQueue.offer(record)) to self
}
case r: QueueOfferResult =>
r match {
case QueueOfferResult.Enqueued => // element has been consumed
case QueueOfferResult.Dropped => // element has been ignored because of backpressure
case QueueOfferResult.QueueClosed => // the queue upstream has terminated
case QueueOfferResult.Failure(e) => // the queue upstream has failed with an exception
}
case Status.Failure(e) => // future has failed, e.g. because of invalid usage of `offer()`
}

Spray Dead Letter msg

I'm trying to execute the following code
trait CustomHttpService extends HttpService {
import MyJsonProtocol._
import spray.httpx.SprayJsonSupport._
implicit def executionContext = actorRefFactory.dispatcher
implicit val timeout = Timeout(5 seconds)
val offerActor = actorRefFactory.actorOf(Props[OfferActor], "offer-actor")
val defaultRoute = {
path("offer" / JavaUUID) { uuid =>
get {
respondWithMediaType(`application/json`) {
complete {
(offerActor ? Get(uuid)).mapTo[Offer]
}
}
}
}
}
}
class OfferActor extends Actor {
override def receive = {
case Get(id) =>
val future = OfferService.genericService.getById(id)
future.onComplete {
case Success(s) =>
s match {
case Some(offer) => sender ! offer
case None => println("none")
}
case Failure(f) => f
}
case id: String => println("received string id: " + id)
case _ => println("receive nothing")
}
}
Initially I was trying to return directly the future, but it was giving me an error, complaining about the promise that I was trying to cast to my Offer object.
Then I just ugly solve my future inside my actor to finally get the Offer and then return it to the sender.
Doing this I'm getting the following:
[06/09/2015 15:16:43.056]
[spray-system-akka.actor.default-dispatcher-4]
[akka://spray-system/deadLetters] Message
[com.spray.entity.Offer] from
Actor[akka://spray-system/user/spray-actor/offer-actor#-617290326] to
Actor[akka://spray-system/deadLetters] was not delivered. [2] dead
letters encountered. This logging can be turned off or adjusted with
configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.
Indeed, I'm sending a msg with an Offer that I got from the database.
Instead if I simply create an Offer like this, works perfectly.
case Get(id) => sender ! Offer(Some(id), "offer", new DateTime())
I'm believing the future.onComplete inside the actor is causing something wrong.
Any thoughts?
sender is really a function, so you could write sender() to show that it is not just accessing an immutable value. When you call sender inside the future.onComplete the value of sender isn't valid anymore.
I've run into this problem before and the way I worked around was by saving the value of sender outside of the future:
class OfferActor extends Actor {
override def receive = {
case Get(id) =>
val future = OfferService.genericService.getById(id)
val replyTo = sender
future.onComplete {
case Success(s) =>
s match {
case Some(offer) => replyTo ! offer
case None => println("none")
}
case Failure(f) => f
}
case id: String => println("received string id: " + id)
case _ => println("receive nothing")
}
}
Well, just solved it trying to block my future.
I just created a blocked version of
OfferService.genericService.getByIdBlocking(id)
Where I blocked it with
Await.result
then it worked!
So basically I had to let akka embrace my call with a future using the ask pattern but do blocking operations inside the actor.

Resolving Akka futures from ask in the event of a failure

I am calling an Actor using the ask pattern within a Spray application, and returning the result as the HTTP response. I map failures from the actor to a custom error code.
val authActor = context.actorOf(Props[AuthenticationActor])
callService((authActor ? TokenAuthenticationRequest(token)).mapTo[LoggedInUser]) { user =>
complete(StatusCodes.OK, user)
}
def callService[T](f: => Future[T])(cb: T => RequestContext => Unit) = {
onComplete(f) {
case Success(value: T) => cb(value)
case Failure(ex: ServiceException) => complete(ex.statusCode, ex.errorMessage)
case e => complete(StatusCodes.InternalServerError, "Unable to complete the request. Please try again later.")
//In reality this returns a custom error object.
}
}
This works correctly when the authActor sends a failure, but if the authActor throws an exception, nothing happens until the ask timeout completes. For example:
override def receive: Receive = {
case _ => throw new ServiceException(ErrorCodes.AuthenticationFailed, "No valid session was found for that token")
}
I know that the Akka docs say that
To complete the future with an exception you need send a Failure message to the sender. This is not done automatically when an actor throws an exception while processing a message.
But given that I use asks for a lot of the interface between the Spray routing actors and the service actors, I would rather not wrap the receive part of every child actor with a try/catch. Is there a better way to achieve automatic handling of exceptions in child actors, and immediately resolve the future in the event of an exception?
Edit: this is my current solution. However, it's quite messy to do this for every child actor.
override def receive: Receive = {
case default =>
try {
default match {
case _ => throw new ServiceException("")//Actual code would go here
}
}
catch {
case se: ServiceException =>
logger.error("Service error raised:", se)
sender ! Failure(se)
case ex: Exception =>
sender ! Failure(ex)
throw ex
}
}
That way if it's an expected error (i.e. ServiceException), it's handled by creating a failure. If it's unexpected, it returns a failure immediately so the future is resolved, but then throws the exception so it can still be handled by the SupervisorStrategy.
If you want a way to provide automatic sending of a response back to the sender in case of an unexpected exception, then something like this could work for you:
trait FailurePropatingActor extends Actor{
override def preRestart(reason:Throwable, message:Option[Any]){
super.preRestart(reason, message)
sender() ! Status.Failure(reason)
}
}
We override preRestart and propagate the failure back to the sender as a Status.Failure which will cause an upstream Future to be failed. Also, it's important to call super.preRestart here as that's where child stopping happens. Using this in an actor looks something like this:
case class GetElement(list:List[Int], index:Int)
class MySimpleActor extends FailurePropatingActor {
def receive = {
case GetElement(list, i) =>
val result = list(i)
sender() ! result
}
}
If I was to call an instance of this actor like so:
import akka.pattern.ask
import concurrent.duration._
val system = ActorSystem("test")
import system.dispatcher
implicit val timeout = Timeout(2 seconds)
val ref = system.actorOf(Props[MySimpleActor])
val fut = ref ? GetElement(List(1,2,3), 6)
fut onComplete{
case util.Success(result) =>
println(s"success: $result")
case util.Failure(ex) =>
println(s"FAIL: ${ex.getMessage}")
ex.printStackTrace()
}
Then it would properly hit my Failure block. Now, the code in that base trait works well when Futures are not involved in the actor that is extending that trait, like the simple actor here. But if you use Futures then you need to be careful as exceptions that happen in the Future don't cause restarts in the actor and also, in preRestart, the call to sender() will not return the correct ref because the actor has already moved into the next message. An actor like this shows that issue:
class MyBadFutureUsingActor extends FailurePropatingActor{
import context.dispatcher
def receive = {
case GetElement(list, i) =>
val orig = sender()
val fut = Future{
val result = list(i)
orig ! result
}
}
}
If we were to use this actor in the previous test code, we would always get a timeout in the failure situation. To mitigate that, you need to pipe the results of futures back to the sender like so:
class MyGoodFutureUsingActor extends FailurePropatingActor{
import context.dispatcher
import akka.pattern.pipe
def receive = {
case GetElement(list, i) =>
val fut = Future{
list(i)
}
fut pipeTo sender()
}
}
In this particular case, the actor itself is not restarted because it did not encounter an uncaught exception. Now, if your actor needed to do some additional processing after the future, you can pipe back to self and explicitly fail when you get a Status.Failure:
class MyGoodFutureUsingActor extends FailurePropatingActor{
import context.dispatcher
import akka.pattern.pipe
def receive = {
case GetElement(list, i) =>
val fut = Future{
list(i)
}
fut.to(self, sender())
case d:Double =>
sender() ! d * 2
case Status.Failure(ex) =>
throw ex
}
}
If that behavior becomes common, you can make it available to whatever actors need it like so:
trait StatusFailureHandling{ me:Actor =>
def failureHandling:Receive = {
case Status.Failure(ex) =>
throw ex
}
}
class MyGoodFutureUsingActor extends FailurePropatingActor with StatusFailureHandling{
import context.dispatcher
import akka.pattern.pipe
def receive = myReceive orElse failureHandling
def myReceive:Receive = {
case GetElement(list, i) =>
val fut = Future{
list(i)
}
fut.to(self, sender())
case d:Double =>
sender() ! d * 2
}
}

Scala-redis subscribes to * but receives zero messages

Integrating redis with my Scala application using Akka but for some reason it does not receive any messages. I can confirm that redis does have a ton of traffic on its side by opening the redis-cli on the command line.
After a pSubscribe it receives: subscribed to * and count = 1
My guess is that it might be related to the way Akka is set up to receive callbacks. I had to strip out Scala Actors in the scala-redis lib and replace them with Akka actors due to some conflicts.
Here's the code:
The Subscriber Actor
class Subscriber(client: RedisClient) extends Actor {
var callback: PubSubMessage => Any = { m => }
def receive: Receive = {
case Subscribe(channels) =>
client.subscribe(channels.head, channels.tail: _*)(callback)
case pSubscribe(channels) =>
client.pSubscribe(channels.head, channels.tail: _*)(callback)
case pSubscribeAll(channels) =>
Logger.info("Subscribing to all channels")
client.pSubscribe(channels.head, channels.tail: _*)(callback)
case Register(cb) =>
Logger.info("Callback is registered")
callback = cb
case Unsubscribe(channels) =>
client.unsubscribe(channels.head, channels.tail: _*)
case UnsubscribeAll =>
client.unsubscribe
}
}
Initializing the Subscriber
class RelaySub extends Actor {
// important config values
val system = ActorSystem("pubsub")
val conf = play.api.Play.current.configuration
val relayPubHost = conf.getString("relays.redis.host").get
val relayPubPort = conf.getInt("relays.redis.port").get
val rs = new RedisClient(relayPubHost, relayPubPort)
val s = system.actorOf(Props(new Subscriber(rs)))
s ! Register(callback)
s ! pSubscribeAll(Array("*"))
Logger.info("Engine Relay Subscriber has started up")
def receive: Receive = {
case Register(callback) =>
}
def callback(pubsub: PubSubMessage) = pubsub match {
case S(channel, no) => Logger.info("subscribed to " + channel + " and count = " + no)
case U(channel, no) => Logger.info("unsubscribed from " + channel + " and count = " + no)
case M(channel, msg) =>
msg match {
// exit will unsubscribe from all channels and stop subscription service
case "exit" =>
Logger.info("unsubscribe all ... no handler yet ;)")
// message "+x" will subscribe to channel x
case x if x startsWith "+" =>
Logger.info("subscribe to ... no handler yet ;)")
// message "-x" will unsubscribe from channel x
case x if x startsWith "-" =>
Logger.info("unsubscribe from ... no handler yet ;)")
// other message receive
case x =>
Logger.info("Engine: received redis message")
val channelVars = channel.split(".").toArray[String]
if(channelVars(0)!=Engine.instanceID)
channelVars(1) match {
case "relay" =>
EngineSyncLocal.constructRelay(channel, msg)
case _ =>
Logger.error("Engine: received unknown redis message")
}
}
}
}
Thanks for your help!
I found the problem. It appears to be a bug in the scala-redis client.
I added some logging in the consumer class and began receiving Engine: weird message errors which means that it doesn't recognize the incoming traffic. I'll contact the author and put in a pull request.
The code:
class Consumer(fn: PubSubMessage => Any) extends Runnable {
def start () {
val myThread = new Thread(this) ;
myThread.start() ;
}
def run {
whileTrue {
asList match {
case Some(Some(msgType) :: Some(channel) :: Some(data) :: Nil) =>
Logger.info("Engine: redis traffic")
msgType match {
case "subscribe" | "psubscribe" => fn(S(channel, data.toInt))
case "unsubscribe" if (data.toInt == 0) =>
fn(U(channel, data.toInt))
break
case "punsubscribe" if (data.toInt == 0) =>
fn(U(channel, data.toInt))
break
case "unsubscribe" | "punsubscribe" =>
fn(U(channel, data.toInt))
case "message" | "pmessage" =>
fn(M(channel, data))
case x => throw new RuntimeException("unhandled message: " + x)
}
case _ => Logger.error("Engine: weird redis message")
}
}
}
}
case x => throw new RuntimeException("unhandled message: " + x)
}
case Some(Some("pmessage")::Some(pattern)::Some(channel):: Some(message)::Nil)=>
fn(M(channel, message))
asList match is missing a case