How do I wire up a Source[String, Unit] to a streaming actor?
I think a modified version of StreamingActor from https://gist.github.com/whysoserious/96050c6b4bd5fedb6e33 will work well, but I'm having difficulty connecting the pieces.
Given source: Source[String, Unit] and ctx: RequestContext, I think the the modified StreamingActor should wire up with actorRefFactory.actorOf(fromSource(source, ctx)).
For reference, the gist above:
import akka.actor._
import akka.util.ByteString
import spray.http.HttpEntity.Empty
import spray.http.MediaTypes._
import spray.http._
import spray.routing.{HttpService, RequestContext, SimpleRoutingApp}
object StreamingActor {
// helper methods
def fromString(iterable: Iterable[String], ctx: RequestContext): Props = {
fromHttpData(iterable.map(HttpData.apply), ctx)
}
def fromStringAndCharset(iterable: Iterable[String], ctx: RequestContext, charset: HttpCharset): Props = {
fromHttpData(iterable.map(HttpData.apply), ctx)
}
def fromByteArray(iterable: Iterable[Array[Byte]], ctx: RequestContext): Props = {
fromHttpData(iterable.map(HttpData.apply), ctx)
}
def fromByteString(iterable: Iterable[ByteString], ctx: RequestContext): Props = {
fromHttpData(iterable.map(HttpData.apply), ctx)
}
def fromHttpData(iterable: Iterable[HttpData], ctx: RequestContext): Props = {
Props(new StreamingActor(iterable, ctx))
}
// initial message sent by StreamingActor to itself
private case object FirstChunk
// confirmation that given chunk was sent to client
private case object ChunkAck
}
class StreamingActor(chunks: Iterable[HttpData], ctx: RequestContext) extends Actor with HttpService with ActorLogging {
import StreamingActor._
def actorRefFactory = context
val chunkIterator: Iterator[HttpData] = chunks.iterator
self ! FirstChunk
def receive = {
// send first chunk to client
case FirstChunk if chunkIterator.hasNext =>
val responseStart = HttpResponse(entity = HttpEntity(`text/html`, chunkIterator.next()))
ctx.responder ! ChunkedResponseStart(responseStart).withAck(ChunkAck)
// data stream is empty. Respond with Content-Length: 0 and stop
case FirstChunk =>
ctx.responder ! HttpResponse(entity = Empty)
context.stop(self)
// send next chunk to client
case ChunkAck if chunkIterator.hasNext =>
val nextChunk = MessageChunk(chunkIterator.next())
ctx.responder ! nextChunk.withAck(ChunkAck)
// all chunks were sent. stop.
case ChunkAck =>
ctx.responder ! ChunkedMessageEnd
context.stop(self)
//
case x => unhandled(x)
}
}
I think your use of a StreamingActor over-complicates the underlying problem you are trying to solve. Further, the StreamingActor in the question will produce multiple HttpResponse values, 1 for each Chunk, for a single HttpRequest. This is inefficient because you can simply return 1 HttpReponse with an HttpEntity.Chunked as the Entity for your data stream source.
General Concurrency Design
Actors are for state, e.g. maintaining a running counter between connections. And even then an Agent covers a lot of ground with the additional benefit of type checking (unlike Actor.receive which turns the dead letter mailbox into your only type checker at runtime).
Concurrent computation, not state, should be handled with (in order):
Futures as a first consideration: composable, compile time type safe checking, and best choice for most cases.
akka Streams : composable, compile time type safe checking, and very useful but there is a lot of overhead resulting from the convenient back-pressure functionality. Steams also are how HttpResponse entities are formed, as demonstrated below.
Streaming CSV Files
You're underlying question is how to stream a csv file to an http client using Streams. You can begin by creating a data Source and embedding it within an HttpResponse:
def lines() = scala.io.Source.fromFile("DataFile.csv").getLines()
import akka.util.ByteString
import akka.http.model.HttpEntity
def chunkSource : Source[HttpEntity.ChunkStreamPart, Unit] =
akka.stream.scaladsl.Source(lines)
.map(ByteString.apply)
.map(HttpEntity.ChunkStreamPart.apply)
def httpFileResponse =
HttpResponse(entity = HttpEntity.Chunked(ContentTypes.`text/plain`, chunkSource))
You can then provide this response for any requests:
val fileRequestHandler = {
case HttpRequest(GET, Uri.Path("/csvFile"), _, _, _) => httpFileResponse
}
Then embed the fileRequestHandler into your server routing logic.
Related
I am trying to implement a simple one-to-many pub/sub pattern using a BroadcastHub. This fails silently for large numbers of subscribers, which makes me think I am hitting some limit on the number of streams I can run.
First, let's define some events:
sealed trait Event
case object EX extends Event
case object E1 extends Event
case object E2 extends Event
case object E3 extends Event
case object E4 extends Event
case object E5 extends Event
I have implemented the publisher using a BroadcastHub, adding a Sink.actorRefWithAck each time I want to add a new subscriber. Publishing the EX event ends the broadcast:
trait Publisher extends Actor with ActorLogging {
implicit val materializer = ActorMaterializer()
private val sourceQueue = Source.queue[Event](Publisher.bufferSize, Publisher.overflowStrategy)
private val (
queue: SourceQueueWithComplete[Event],
source: Source[Event, NotUsed]
) = {
val (q,s) = sourceQueue.toMat(BroadcastHub.sink(bufferSize = 256))(Keep.both).run()
s.runWith(Sink.ignore)
(q,s)
}
def publish(evt: Event) = {
log.debug("Publishing Event: {}", evt.getClass().toString())
queue.offer(evt)
evt match {
case EX => queue.complete()
case _ => Unit
}
}
def subscribe(actor: ActorRef, ack: ActorRef): Unit =
source.runWith(
Sink.actorRefWithAck(
actor,
onInitMessage = Publisher.StreamInit(ack),
ackMessage = Publisher.StreamAck,
onCompleteMessage = Publisher.StreamDone,
onFailureMessage = onErrorMessage))
def onErrorMessage(ex: Throwable) = Publisher.StreamFail(ex)
def publisherBehaviour: Receive = {
case Publisher.Subscribe(sub, ack) => subscribe(sub, ack.getOrElse(sender()))
case Publisher.StreamAck => Unit
}
override def receive = LoggingReceive { publisherBehaviour }
}
object Publisher {
final val bufferSize = 5
final val overflowStrategy = OverflowStrategy.backpressure
case class Subscribe(sub: ActorRef, ack: Option[ActorRef])
case object StreamAck
case class StreamInit(ack: ActorRef)
case object StreamDone
case class StreamFail(ex: Throwable)
}
Subscribers can implement the Subscriber trait to separate the logic:
trait Subscriber {
def onInit(publisher: ActorRef): Unit = ()
def onInit(publisher: ActorRef, k: KillSwitch): Unit = onInit(publisher)
def onEvent(event: Event): Unit = ()
def onDone(publisher: ActorRef, subscriber: ActorRef): Unit = ()
def onFail(e: Throwable, publisher: ActorRef, subscriber: ActorRef): Unit = ()
}
The actor logic is quite simple:
class SubscriberActor(subscriber: Subscriber) extends Actor with ActorLogging {
def subscriberBehaviour: Receive = {
case Publisher.StreamInit(ack) => {
log.debug("Stream initialized.")
subscriber.onInit(sender())
sender() ! Publisher.StreamAck
ack.forward(Publisher.StreamInit(ack))
}
case Publisher.StreamDone => {
log.debug("Stream completed.")
subscriber.onDone(sender(),self)
}
case Publisher.StreamFail(ex) => {
log.error(ex, "Stream failed!")
subscriber.onFail(ex,sender(),self)
}
case e: Event => {
log.debug("Observing Event: {}",e)
subscriber.onEvent(e)
sender() ! Publisher.StreamAck
}
}
override def receive = LoggingReceive { subscriberBehaviour }
}
One of the key points is that all subscribers must receive all messages sent by the publisher, so we have to know that all streams have materialized and all actors are ready to receive before starting the broadcast. This is why the StreamInit message is forwarded to another, user-provided actor.
To test this, I define a simple MockPublisher that just broadcasts a list of events when told to do so:
class MockPublisher(events: Event*) extends Publisher {
def receiveBehaviour: Receive = {
case MockPublish => events map publish
}
override def receive = LoggingReceive { receiveBehaviour orElse publisherBehaviour }
}
case object MockPublish
I also define a MockSubscriber who merely counts how many events it has seen:
class MockSubscriber extends Subscriber {
var count = 0
val promise = Promise[Int]()
def future = promise.future
override def onInit(publisher: ActorRef): Unit = count = 0
override def onEvent(event: Event): Unit = count += 1
override def onDone(publisher: ActorRef, subscriber: ActorRef): Unit = promise.success(count)
override def onFail(e: Throwable, publisher: ActorRef, subscriber: ActorRef): Unit = promise.failure(e)
}
And a small method for subscription:
object MockSubscriber {
def sub(publisher: ActorRef, ack: ActorRef)(implicit system: ActorSystem): Future[Int] = {
val s = new MockSubscriber()
implicit val tOut = Timeout(1.minute)
val a = system.actorOf(Props(new SubscriberActor(s)))
val f = publisher ! Publisher.Subscribe(a, Some(ack))
s.future
}
}
I put everything together in a unit test:
class SubscriberTests extends TestKit(ActorSystem("SubscriberTests")) with
WordSpecLike with Matchers with BeforeAndAfterAll with ImplicitSender {
override def beforeAll:Unit = {
system.eventStream.setLogLevel(Logging.DebugLevel)
}
override def afterAll:Unit = {
println("Shutting down...")
TestKit.shutdownActorSystem(system)
}
"The Subscriber" must {
"publish events to many observers" in {
val n = 9
val p = system.actorOf(Props(new MockPublisher(E1,E2,E3,E4,E5,EX)))
val q = scala.collection.mutable.Queue[Future[Int]]()
for (i <- 1 to n) {
q += MockSubscriber.sub(p,self)
}
for (i <- 1 to n) {
expectMsgType[Publisher.StreamInit](70.seconds)
}
p ! MockPublish
q.map { f => Await.result(f, 10.seconds) should be (6) }
}
}
}
This test succeeds for relatively small values of n, but fails for, say, val n = 90000. No caught or uncaught exception appears anywhere and neither does any out-of-memory complaint from Java (which does occur if I go even higher).
What am I missing?
Edit: Tried this on multiple computers with different specs. Debug info shows no messages reach any of the subscribers once n is high enough.
Akka Stream (and any other reactive stream, actually) provides you backpressure. If you hadn't messed up with how you create your consumers (e.g. allowing creation of 1GB JSON, which will you chop into smaller pieces only after you fetched it into memory) you should have a comfortable situation where you can consider your memory usage pretty much upper-bounded (because of how backpressure manage push-pull mechanics). Once you measure where your upper-bound lies, your can set up your JVM and container memory, so that you could let it run without fear of out of memory errors (provided that there is not other thing happening in your JVM which could cause memory usage spike).
So, from this we can see that there is some constraint on how much stream you can run in parallel - specifically you can run only as much of them as your memory allows you. CPU should not be a limitation (as you will have multiple threads), but if you will start too much of them on one machine, then CPU inevitably with have to switch between different streams making each of them slower. It might not be a technical blocker, but you might end up in a situation where processing is so slow that it doesn't fulfill its business purpose (though, I guess, you would have to run much more than few of streams at once).
In your tests you might run into some other issues as well. E.g. if you reuse the same thread pool for some blocking operations as you use for Actor System without informing the thread pool that they are blocking, you might end up with a dead lock (as a matter of the fact, you should run all IO blocking operations on a different thread pool than "computing" operations). Having 90000(!) concurrent things happening at the same time (and probably having the same small thread pool) almost guarantees running into issues (I guess you could run into issues even if instead of actors you would run the code directly on futures). Here you are using actor system in tests, which AFAIR use blocking logic only highlighting all the possible issues with small thread pools which keep blocking and non-blocking tasks in the same place.
I have an external (that is, I cannot change it) Java API which looks like this:
public interface Sender {
void send(Event e);
}
I need to implement a Sender which accepts each event, transforms it to a JSON object, collects some number of them into a single bundle and sends over HTTP to some endpoint. This all should be done asynchronously, without send() blocking the calling thread, with some fixed-size buffer and dropping new events if the buffer is full.
With akka-streams this is quite simple: I create a graph of stages (which uses akka-http to send HTTP requests), materialize it and use the materialized ActorRef to push new events to the stream:
lazy val eventPipeline = Source.actorRef[Event](Int.MaxValue, OverflowStrategy.fail)
.via(CustomBuffer(bufferSize)) // buffer all events
.groupedWithin(batchSize, flushDuration) // group events into chunks
.map(toBundle) // convert each chunk into a JSON message
.mapAsyncUnordered(1)(sendHttpRequest) // send an HTTP request
.toMat(Sink.foreach { response =>
// print HTTP response for debugging
})(Keep.both)
lazy val (eventsActor, completeFuture) = eventPipeline.run()
override def send(e: Event): Unit = {
eventsActor ! e
}
Here CustomBuffer is a custom GraphStage which is very similar to the library-provided Buffer but tailored to our specific needs; it probably does not matter for this particular question.
As you can see, interacting with the stream from non-stream code is very simple - the ! method on the ActorRef trait is asynchronous and does not need any additional machinery to be called. Each event which is sent to the actor is then processed through the entire reactive pipeline. Moreover, because of how akka-http is implemented, I even get connection pooling for free, so no more than one connection is opened to the server.
However, I cannot find a way to do the same thing with FS2 properly. Even discarding the question of buffering (I will probably need to write a custom Pipe implementation which does additional things that we need) and HTTP connection pooling, I'm still stuck with a more basic thing - that is, how to push the data to the reactive stream "from outside".
All tutorials and documentation that I can find assume that the entire program happens inside some effect context, usually IO. This is not my case - the send() method is invoked by the Java library at unspecified times. Therefore, I just cannot keep everything inside one IO action, I necessarily have to finalize the "push" action inside the send() method, and have the reactive stream as a separate entity, because I want to aggregate events and hopefully pool HTTP connections (which I believe is naturally tied to the reactive stream).
I assume that I need some additional data structure, like Queue. fs2 does indeed have some kind of fs2.concurrent.Queue, but again, all documentation shows how to use it inside a single IO context, so I assume that doing something like
val queue: Queue[IO, Event] = Queue.unbounded[IO, Event].unsafeRunSync()
and then using queue inside the stream definition and then separately inside the send() method with further unsafeRun calls:
val eventPipeline = queue.dequeue
.through(customBuffer(bufferSize))
.groupWithin(batchSize, flushDuration)
.map(toBundle)
.mapAsyncUnordered(1)(sendRequest)
.evalTap(response => ...)
.compile
.drain
eventPipeline.unsafeRunAsync(...) // or something
override def send(e: Event) {
queue.enqueue(e).unsafeRunSync()
}
is not the correct way and most likely would not even work.
So, my question is, how do I properly use fs2 to solve my problem?
Consider the following example:
import cats.implicits._
import cats.effect._
import cats.effect.implicits._
import fs2._
import fs2.concurrent.Queue
import scala.concurrent.ExecutionContext
import scala.concurrent.duration._
object Answer {
type Event = String
trait Sender {
def send(event: Event): Unit
}
def main(args: Array[String]): Unit = {
val sender: Sender = {
val ec = ExecutionContext.global
implicit val cs: ContextShift[IO] = IO.contextShift(ec)
implicit val timer: Timer[IO] = IO.timer(ec)
fs2Sender[IO](2)
}
val events = List("a", "b", "c", "d")
events.foreach { evt => new Thread(() => sender.send(evt)).start() }
Thread sleep 3000
}
def fs2Sender[F[_]: Timer : ContextShift](maxBufferedSize: Int)(implicit F: ConcurrentEffect[F]): Sender = {
// dummy impl
// this is where the actual logic for batching
// and shipping over the network would live
val consume: Pipe[F, Event, Unit] = _.evalMap { event =>
for {
_ <- F.delay { println(s"consuming [$event]...") }
_ <- Timer[F].sleep(1.seconds)
_ <- F.delay { println(s"...[$event] consumed") }
} yield ()
}
val suspended = for {
q <- Queue.bounded[F, Event](maxBufferedSize)
_ <- q.dequeue.through(consume).compile.drain.start
sender <- F.delay[Sender] { evt =>
val enqueue = for {
wasEnqueued <- q.offer1(evt)
_ <- F.delay { println(s"[$evt] enqueued? $wasEnqueued") }
} yield ()
enqueue.toIO.unsafeRunAsyncAndForget()
}
} yield sender
suspended.toIO.unsafeRunSync()
}
}
The main idea is to use a concurrent Queue from fs2. Note, that the above code demonstrates that neither the Sender interface nor the logic in main can be changed. Only an implementation of the Sender interface can be swapped out.
I don't have much experience with exactly that library but it should look somehow like that:
import cats.effect.{ExitCode, IO, IOApp}
import fs2.concurrent.Queue
case class Event(id: Int)
class JavaProducer{
new Thread(new Runnable {
override def run(): Unit = {
var id = 0
while(true){
Thread.sleep(1000)
id += 1
send(Event(id))
}
}
}).start()
def send(event: Event): Unit ={
println(s"Original producer prints $event")
}
}
class HackedProducer(queue: Queue[IO, Event]) extends JavaProducer {
override def send(event: Event): Unit = {
println(s"Hacked producer pushes $event")
queue.enqueue1(event).unsafeRunSync()
println(s"Hacked producer pushes $event - Pushed")
}
}
object Test extends IOApp{
override def run(args: List[String]): IO[ExitCode] = {
val x: IO[Unit] = for {
queue <- Queue.unbounded[IO, Event]
_ = new HackedProducer(queue)
done <- queue.dequeue.map(ev => {
println(s"Got $ev")
}).compile.drain
} yield done
x.map(_ => ExitCode.Success)
}
}
We can create a bounded queue that will consume elements from sender and make them available to fs2 stream processing.
import cats.effect.IO
import cats.effect.std.Queue
import fs2.Stream
trait Sender[T]:
def send(e: T): Unit
object Sender:
def apply[T](bufferSize: Int): IO[(Sender[T], Stream[IO, T])] =
for
q <- Queue.bounded[IO, T](bufferSize)
yield
val sender: Sender[T] = (e: T) => q.offer(e).unsafeRunSync()
def stm: Stream[IO, T] = Stream.eval(q.take) ++ stm
(sender, stm)
Then we'll have two ends - one for Java worlds, to send new elements to Sender. Another one - for stream processing in fs2.
class TestSenderQueue:
#Test def testSenderQueue: Unit =
val (sender, stream) = Sender[Int](1)
.unsafeRunSync()// we have to run it preliminary to make `sender` available to external system
val processing =
stream
.map(i => i * i)
.evalMap{ ii => IO{ println(ii)}}
sender.send(1)
processing.compile.toList.start//NB! we start processing in a separate fiber
.unsafeRunSync() // immediately right now.
sender.send(2)
Thread.sleep(100)
(0 until 100).foreach(sender.send)
println("finished")
Note that we push data in the current thread and have to run fs2 in a separate thread (.start).
I use ask mode send a request to an actor called actor-A in a function called fun-F. The actor will get an ID generated by another system in an async way, when this is completed, I will forward a message contains this ID to another actor called actor-B, the actor-B will do some DB operations and then send back the DB operation result in a message to the sender, since in my case I use forward mode, so the actor-B recognize the sender as fun-F, the akka will give the fun-F a temporary actor name, so the returned value should be delivered to the temp actor.
My question is:
If I use sync-method to get the ID from another system, then forward this message to the actor-B, after actor-B's DB operation, the result can be delivered to the value of fun-F, and the fun-F is defined as temporary actor Actor[akka://ai-feedback-service/temp/$b] by the akka framework runtime.
If I use async-method to get the ID from another system, when it completed, I will forward the message in the oncompleted {} code block in another call back thread, the DB operation in actor-B is handled successfully, but the returned value cannot be delivered to the value defined in the fun-F, and in this case the fun-F is defined as Actor[akka://ai-feedback-service/deadLetters] by the akka framwork runtime. So the actor-B lose its way and do not know how to get back or where should this message be delivered, and this will cause an Ask time out exception throws in my log.
How can I handled this issue? or how can I avoid this dead letter ask time out exception?
Below is my code:
// this is the so-called fun-F [createFeedback]
def createFeedback(query: String,
response: String,
userId: Long,
userAgent: String,
requestId: Long,
errType: Short,
memo: String): Future[java.lang.Long] = {
val ticket = Ticket(userId,
requestId,
query,
response,
errType,
userAgent,
memo)
val issueId = (jiraActor ? CreateJiraTicketSignal(ticket))
.mapTo[CreateFeedbackResponseSignal].map{ r =>
r.issueId.asInstanceOf[java.lang.Long]
}
issueId
}
//this is the so-called actor-A [jiraActor]
//receive method are run in its parent actor for some authorization
//in this actor only override the handleActorMsg method to deal msg
override def handleActorMsg(msg: ActorMsgSignal): Unit = {
msg match {
case s:CreateJiraTicketSignal =>
val issueId = createIssue(cookieCache.cookieContext.flag,
cookieCache.cookieContext.cookie,
s.ticket)
println(s">> ${sender()} before map $issueId")
issueId.map{
case(id:Long) =>
println(s">> again++issueId = $id ${id.getClass}")
println(s">>> $self / ${sender()}")
println("again ++ jira action finished")
dbActor.forward(CreateFeedbackSignal(id,s.ticket))
case(message:String) if(!s.retry) =>
self ! CreateJiraTicketSignal(s.ticket,true)
case(message:String) if(s.retry) =>
log.error("cannot create ticket :" + message)
}
println(s">> after map $issueId")
}
//this is the so-called actor-B [dbActor]
override def receive: Receive = {
case CreateFeedbackSignal(issueId:Long, ticket:Ticket) =>
val timestampTicks = System.currentTimeMillis()
val description: String = Json.obj("question" -> ticket.query,
"answer" -> ticket.response)
.toString()
dao.createFeedback(issueId,
ticket.usrId.toString,
description,
FeedbackStatus.Open.getValue
.asInstanceOf[Byte],
new Timestamp(timestampTicks),
new Timestamp(timestampTicks),
ticket.usrAgent,
ticket.errType,
ticket.memo)
println(s">> sender = ${sender()}")
sender() ! (CreateFeedbackResponseSignal(issueId))
println("db issue id is " + issueId)
println("db action finished")
}
To avoid the dead letters issue, do the following:
For every request, use an identifier (probably the requestId) that you can associate with the ultimate target for the request. That is, tie the requestId that you're passing to the createFeedback method to the caller (ActorRef) of that method, then pass this id through your messaging chain. You can use a map to hold these associations.
Change CreateFeedbackResponseSignal(issueId) to include the requestId from the Ticket class: CreateFeedbackResponseSignal(requestId, issueId).
When dealing with the asynchronous result of a Future from inside an actor, pipe the result of the Future to self instead of using a callback.
With this approach, the result of createIssue will be sent to jiraActor when the result is available. jiraActor then sends that result to dbActor.
jiraActor will be the sender in dbActor. When jiraActor receives the result from dbActor, jiraActor can look up the reference to the target in its internal map.
Below is a simple example that mimics your use case and is runnable in ScalaFiddle:
import akka.actor._
import akka.pattern.{ask, pipe}
import akka.util.Timeout
import language.postfixOps
import scala.concurrent._
import scala.concurrent.duration._
case class Signal(requestId: Long)
case class ResponseSignal(requestId: Long, issueId: Long)
object ActorA {
def props(actorB: ActorRef) = Props(new ActorA(actorB))
}
class ActorA(dbActor: ActorRef) extends Actor {
import context.dispatcher
var targets: Map[Long, ActorRef] = Map.empty
def receive = {
case Signal(requestId) =>
val s = sender
targets = targets + (requestId -> s)
createIssue(requestId).mapTo[Tuple2[Long, Long]].pipeTo(self) // <-- use pipeTo
case ids: Tuple2[Long, Long] =>
println(s"Sending $ids to dbActor")
dbActor ! ids
case r: ResponseSignal =>
println(s"Received from dbActor: $r")
val target = targets.get(r.requestId)
println(s"In actorA, sending to: $target")
target.foreach(_ ! r)
targets = targets - r.requestId
}
}
class DbActor extends Actor {
def receive = {
case (requestId: Long, issueId: Long) =>
val response = ResponseSignal(requestId, issueId)
println(s"In dbActor, sending $response to $sender")
sender ! response
}
}
val system = ActorSystem("jiratest")
implicit val ec = system.dispatcher
val dbActor = system.actorOf(Props[DbActor])
val jiraActor = system.actorOf(Props(new ActorA(dbActor)))
val requestId = 2L
def createIssue(requestId: Long): Future[(Long, Long)] = {
println(s"Creating an issue ID for requestId[$requestId]")
Future((requestId, 99L))
}
def createFeedback(): Future[Long] = {
implicit val timeout = Timeout(5.seconds)
val res = (jiraActor ? Signal(requestId)).mapTo[ResponseSignal]
res.map(_.issueId)
}
createFeedback().onComplete { x =>
println(s"Done: $x")
}
Running the above code in ScalaFiddle results in the following output:
Creating an issue ID for requestId[2]
Sending (2,99) to dbActor
In dbActor, sending ResponseSignal(2,99) to Actor[akka://jiratest/user/$b#-710097339]
Received from dbActor: ResponseSignal(2,99)
In actorA, sending to: Some(Actor[akka://jiratest/temp/$a])
Done: Success(99)
I've built an Akka actor that queries an API at regular intervals, like this:
val cancellable =
system.scheduler.schedule(0 milliseconds,
5 seconds,
actor,
QueryController(1))
The Actor, in essence is:
object UpdateStatistics {
/**
* Query the controller for the given switch Id
*
* #param dpId Switch's Id
*/
case class QueryController(dpId: Int)
case object Stop
def props: Props = Props[UpdateStatistics]
}
class UpdateStatistics extends Actor with akka.actor.ActorLogging {
import UpdateStatistics._
def receive = {
case QueryController(id) =>
import context.dispatcher
log.info(s"Receiving request to query controller")
Future { FlowCollector.getSwitchFlows(1) } onComplete {
f => self ! f.get
}
case Stop =>
log.info(s"Shuting down")
context stop self
case json: JValue =>
log.info("Getting json response, computing features...")
val features = FeatureExtractor.getFeatures(json)
log.debug(s"Features: $features")
sender ! features
case x =>
log.warning("Received unknown message: {}", x)
}
}
What I am trying to do is get the json:Jvalue message out of UpdateStatistics actor. Reading the Akka docs I thought this may be useful:
implicit val i = inbox()
i.select() {
case x => println(s"Valor Devuelto $x")
}
println(i receive(2.second))
But I do not know how to modify UpdateStatistics actor in order to send the result to the inbox above.
Another option I've read in the docs are event streams.
But I do not think this is the correct way.
Is there a way of achieving what I want to do? Or do I need to use a second Actor to which I send the JSON response?
You probably are looking for the ask pattern in AKKA. This will allow you to return a value to the sender.
import akka.pattern.ask
import akka.util.duration._
implicit val timeout = Timeout(5 seconds)
val future = actor ? QueryController(1)
val result = Await.result(future, timeout.duration).asInstanceOf[JValue]
println(result)
To make this work, you need to send the response to the original sender, rather than self. Also, you should beware the dangers of closing over sender in a future when handling messages.
I am trying to figure out how I can setup a Master Actor that calls the appropriate children, in support of some spray routes where I am trying to emulate db calls. I am new to akka / spray, so just trying to gain a better understanding of how you would properly setup spray -> actors -> db calls (etc.). I can get the response back from the top level actor, but when I try to get it back from one actor level below the parent I can't seem to get anything to work.
When looking at the paths of the actors, it appears that from the way I am making the call from my spray route that I am passing from a temp actor. Below is what I have so far for stubbing this out. This has to be just user error / ignorance, just not sure how to proceed. Any suggestions would be appreciated.
The Demo Spray Service and Redis Actor code snippets below show where I am calling the actor from my route and the multiple actors where I am having the issue (want my route to get response from SummaryActor). Thanks!
Boot:
object Boot extends App {
// we need an ActorSystem to host our application in
implicit val system = ActorSystem("on-spray-can")
// create and start our service actor
val service = system.actorOf(Props[DemoServiceActor], "demo-service")
implicit val timeout = Timeout(5.seconds)
// start a new HTTP server on port 8080 with our service actor as the handler
IO(Http) ? Http.Bind(service, interface = "localhost", port = 8080)
}
Demo Service Actor (For Spray)
class DemoServiceActor extends Actor with Api {
// the HttpService trait defines only one abstract member, which
// connects the services environment to the enclosing actor or test
def actorRefFactory = context
// this actor only runs our route, but you could add
// other things here, like request stream processing
// or timeout handling
def receive = handleTimeouts orElse runRoute(route)
//Used to watch for request timeouts
//http://spray.io/documentation/1.1.2/spray-routing/key-concepts/timeout-handling/
def handleTimeouts: Receive = {
case Timedout(x: HttpRequest) =>
sender ! HttpResponse(StatusCodes.InternalServerError, "Too late")
}
}
//Master trait for handling large APIs
//http://stackoverflow.com/questions/14653526/can-spray-io-routes-be-split-into-multiple-controllers
trait Api extends DemoService {
val route = {
messageApiRouting
}
}
Demo Spray Service (Route):
trait DemoService extends HttpService with Actor {
implicit val timeout = Timeout(5 seconds) // needed for `?` below
val redisActor = context.actorOf(Props[RedisActor], "redisactor")
val messageApiRouting =
path("summary" / Segment / Segment) { (dataset, timeslice) =>
onComplete(getSummary(redisActor, dataset, timeslice)) {
case Success(value) => complete(s"The result was $value")
case Failure(ex) => complete(s"An error occurred: ${ex.getMessage}")
}
}
def getSummary(redisActor: ActorRef, dataset: String, timeslice: String): Future[String] = Future {
val dbMessage = DbMessage("summary", dataset + timeslice)
val future = redisActor ? dbMessage
val result = Await.result(future, timeout.duration).asInstanceOf[String]
result
}
}
Redis Actor (Mock no actual redis client yet)
class RedisActor extends Actor with ActorLogging {
// val pool = REDIS
implicit val timeout = Timeout(5 seconds) // needed for `?` below
val summaryActor = context.actorOf(Props[SummaryActor], "summaryactor")
def receive = {
case msg: DbMessage => {
msg.query match {
case "summary" => {
log.debug("Summary Query Request")
log.debug(sender.path.toString)
summaryActor ! msg
}
}
}
//If not match log an error
case _ => log.error("Received unknown message: {} ")
}
}
class SummaryActor extends Actor with ActorLogging{
def receive = {
case msg: DbMessage =>{
log.debug("Summary Actor Received Message")
//Send back to Spray Route
}
}
}
The first problem with your code is that you need to forward from the master actor to the child so that the sender is properly propagated and available for the child to respond to. So change this (in RedisActor):
summaryActor ! msg
To:
summaryActor forward msg
That's the primary issue. Fix that and your code should start working. There is something else that needs attention though. Your getSummary method is currently defined as:
def getSummary(redisActor: ActorRef, dataset: String, timeslice: String): Future[String] =
Future {
val dbMessage = DbMessage("summary", dataset + timeslice)
val future = redisActor ? dbMessage
val result = Await.result(future, timeout.duration).asInstanceOf[String]
result
}
The issue here is that the ask operation (?) already returns a Future, so there and you are blocking on it to get the result, wrapping that in another Future so that you can return a Future for onComplete to work with. You should be able to simplify things by using the Future returned from ask directly like so:
def getSummary(redisActor: ActorRef, dataset: String, timeslice: String): Future[String] = {
val dbMessage = DbMessage("summary", dataset + timeslice)
(redisActor ? dbMessage).mapTo[String]
}
Just an important comment on the above approaches.
Since the getSummary(...) function returns a Future[String] object and you call it in onComplete(...) function you need to import:
import ExecutionContext.Implicits.global
That way you will have ExecutionContext in scope by letting Future
declare an implicit ExecutionContext parameter.
** If you don't, you will end up getting a mismatching error
since onComplete(...) expects an onComplete Future
magnet Object but you gave a Future[String] Object.