I try to implement a sample slightly different from the tutorial:
object Test extends App {
class A extends Actor {
override def receive: Receive = {
case 10 => context.system.terminate()
case x =>
println(s"Received: $x")
sender() ! x
}
}
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
implicit val dispatcher = system.dispatcher
implicit val askTimeout = Timeout(1, TimeUnit.SECONDS)
val a = system.actorOf(Props[A])
val graph = RunnableGraph.fromGraph(GraphDSL.create() {
{
implicit builder: GraphDSL.Builder[NotUsed] => {
import GraphDSL.Implicits._
val source: Source[Int, NotUsed] = Source(1 to 10)
source ~> Flow[Int].ask(2)(a) ~> Sink.foreach[String](println)
ClosedShape
}
}
})
val x = graph.run()
}
In result i get this:
Received: 1
Received: 2
But i expect this:
Received: 1
1
Received: 2
2
...
Received: 9
9
If i send messages to actor via Source.actorRef then all is ok and actor receive all messages. It seems like materialized actor wait a response from referenced actor.
How to fix it?
If you replace the Sink.foreach[String](println) with Sink.onComplete(println)
you will see that the flow terminates with an error:
Received: 1
Received: 2
Failure(java.lang.ClassCastException: Cannot cast java.lang.Integer to scala.runtime.Nothing$)
The problem is that Source.ask needs to know what class to map the actor's reply to see API docs. Otherwise it will try to cast the reply to Nothing and internally throw an exception.
To fix your original code replace ask with ask[Int]:
val graph = RunnableGraph.fromGraph(GraphDSL.create() {
{
implicit builder: GraphDSL.Builder[NotUsed] => {
import GraphDSL.Implicits._
val source: Source[Int, NotUsed] = Source(1 to 10)
source ~> Flow[Int].ask[Int](2)(a) ~> Sink.foreach(println)
ClosedShape
}
}
})
Related
I'm trying to test my actor logic with AkkaTestKit. The problem is that my actor uses ask pattern. So I need to answer somehow. It look as this:
case class AskExecution(id: Long)
override def receive: Receive = {
case id : Long =>
implicit val dispatcher = context.dispatcher
implicit val timeout = Timeout(10 seconds)
val executor = sender
//How to answer this?
val f = executor ? AskExecution(id) map(v => v.asInstanceOf[Option[Long]])
f.onComplete{
case Success(k) =>
case Failure(_) =>
}
}
In test I use it as follows:
val ca = TestActorRef(new TheActor())
ca ! 0L //I send 0, Now I want to answer the ask
//How to do so?
To make your code easier to test, give your actor a reference to the executor actor (the actor that handles the AskExecution message).
import akka.pattern.pipe
class TheActor(executor: ActorRef) extends Actor {
def receive = {
case id: Long =>
val s = sender
implicit val dispatcher = context.dispatcher
implicit val timeout = 10.seconds
(executor ? AskExecution(id)).mapTo[Option[Long]].pipeTo(s)
}
class Executor extends Actor {
def receive = {
case AskExecution(id) =>
// do something to get a result
val result: Option[Long] = ???
sender ! result
}
}
To test, assuming your test class extends TestKit and mixes in the ImplicitSender trait:
val executor = system.actorOf(Props[Executor])
val theActor = system.actorOf(Props(classOf[TheActor], executor))
within(10.seconds) {
theActor ! 0L
expectMsgClass(classOf[Option[Long]])
}
// test the executor directly
within(10.seconds) {
executor ! AskExecution(3L)
expectMsgClass(classOf[Option[Long]])
}
I have an app that generate reports, with akka-http + akka-actors + akka-camel + akka-streams. When a post request arrives , the ActiveMqProducerActor enqueue the request into ActiveMq Broker. Then the ActiveMqConsumerActor consumes the message and start the task using akka-streams(in this actor i need the materializer) .
The main class create the ActorSystem and the ActorMaterializer, but i dont know how is the correct way to "inject" the materializer into the akka-actor
object ReportGeneratorApplication extends App {
implicit val system: ActorSystem = ActorSystem()
implicit val executor = system.dispatcher
implicit val materializer = ActorMaterializer()
val camelExtension: Camel = CamelExtension(system);
val amqc: ActiveMQComponent = ActiveMQComponent.activeMQComponent(env.getString("jms.url"))
amqc.setUsePooledConnection(true)
amqc.setAsyncConsumer(true)
amqc.setTrustAllPackages(true)
amqc.setConcurrentConsumers(1)
camelExtension.context.addComponent("jms", amqc);
val jmsProducer: ActorRef = system.actorOf(Props[ActiveMQProducerActor])
//Is this the correct way to pass the materializer?
val jmsConsumer: ActorRef = system.actorOf(Props(new ActiveMQConsumerActor()(materializer)), name = "jmsConsumer")
val endpoint: ReportEndpoint = new ReportEndpoint(jmsProducer);
Http().bindAndHandle(endpoint.routes, "localhost", 8881)
}
The ReportEndPoint class, that have the jmsProducerActor . Mongo is a trait with CRUD methods. JsonSupport(==SprayJsonSupport)
class ReportEndpoint(jmsProducer: ActorRef)
(implicit val system:ActorSystem,
implicit val executor: ExecutionContext,
implicit val materializer : ActorMaterializer)
extends JsonSupport with Mongo {
val routes =
pathPrefix("reports"){
post {
path("generate"){
entity(as[DataRequest]) { request =>
val id = java.util.UUID.randomUUID.toString
// **Enqueue the request into ActiveMq**
jmsProducer ! request
val future: Future[Seq[Completed]] = insertReport(request)
complete {
future.map[ToResponseMarshallable](r => r.head match {
case r : Completed => println(r); s"Reporte Generado con id $id"
case _ => HttpResponse(StatusCodes.InternalServerError, entity = "Error al generar reporte")
})
}
}
}
} ....
The idea of ActiveMqConsumerActor, is send the messages, with streams and backpressure, one by one,because ReportBuilderActor makes many mongo operations (and the datacenter it`s not very good).
//Is this the correct way to pass the materializer??
class ActiveMQConsumerActor (implicit materializer : ActorMaterializer) extends Consumer with Base {
override def endpointUri: String = env.getString("jms.queue")
val log = Logging(context.system, this)
val reportActor: ActorRef = context.actorOf(Props(new ReportBuilderActor()(materializer)), name = "reportActor")
override def receive: Receive = {
case msg: CamelMessage => msg.body match {
case data: DataRequest => {
//I need only one task running
Source.single(data).buffer(1, OverflowStrategy.backpressure).to(Sink.foreach(d => reportActor ! d)).run()
}
case _ => log.info("Invalid")
}
case _ => UnhandledMessage
}
}
Is a good idea have implicit values in companion objects?
Thanks!!
How do you throttle Flow in the latest Akka (2.4.6) ? I'd like to throttle Http client flow to limit number of requests to 3 requests per second. I found following example online but it's for old Akka and akka-streams API changed so much that I can't figure out how to rewrite it.
def throttled[T](rate: FiniteDuration): Flow[T, T] = {
val tickSource: Source[Unit] = TickSource(rate, rate, () => ())
val zip = Zip[T, Unit]
val in = UndefinedSource[T]
val out = UndefinedSink[T]
PartialFlowGraph { implicit builder =>
import FlowGraphImplicits._
in ~> zip.left ~> Flow[(T, Unit)].map { case (t, _) => t } ~> out
tickSource ~> zip.right
}.toFlow(in, out)
}
Here is my best attempt so far
def throttleFlow[T](rate: FiniteDuration) = Flow.fromGraph(GraphDSL.create() { implicit builder =>
import GraphDSL.Implicits._
val ticker = Source.tick(rate, rate, Unit)
val zip = builder.add(Zip[T, Unit.type])
val map = Flow[(T, Unit.type)].map { case (value, _) => value }
val messageExtractor = builder.add(map)
val in = Inlet[T]("Req.in")
val out = Outlet[T]("Req.out")
out ~> zip.in0
ticker ~> zip.in1
zip.out ~> messageExtractor.in
FlowShape.of(in, messageExtractor.out)
})
it throws exception in my main flow though :)
private val queueHttp = Source.queue[(HttpRequest, (Any, Promise[(Try[HttpResponse], Any)]))](1000, OverflowStrategy.backpressure)
.via(throttleFlow(rate))
.via(poolClientFlow)
.mapAsync(4) {
case (util.Success(resp), any) =>
val strictFut = resp.entity.toStrict(5 seconds)
strictFut.map(ent => (util.Success(resp.copy(entity = ent)), any))
case other =>
Future.successful(other)
}
.toMat(Sink.foreach({
case (triedResp, (value: Any, p: Promise[(Try[HttpResponse], Any)])) =>
p.success(triedResp -> value)
case _ =>
throw new RuntimeException()
}))(Keep.left)
.run
where poolClientFlow is Http()(system).cachedHostConnectionPool[Any](baseDomain)
Exception is:
Caused by: java.lang.IllegalArgumentException: requirement failed: The output port [Req.out] is not part of the underlying graph.
at scala.Predef$.require(Predef.scala:219)
at akka.stream.impl.StreamLayout$Module$class.wire(StreamLayout.scala:204)
Here is an attempt that uses the throttle method as mentioned by #Qingwei. The key is to not use bindAndHandle(), but to use bind() and throttle the flow of incoming connections before handling them. The code is taken from the implementation of bindAndHandle(), but leaves out some error handling for simplicity. Please don't do that in production.
implicit val system = ActorSystem("test")
implicit val mat = ActorMaterializer()
import system.dispatcher
val maxConcurrentConnections = 4
val handler: Flow[HttpRequest, HttpResponse, NotUsed] = complete(LocalDateTime.now().toString)
def handleOneConnection(incomingConnection: IncomingConnection): Future[Done] =
incomingConnection.flow
.watchTermination()(Keep.right)
.joinMat(handler)(Keep.left)
.run()
Http().bind("127.0.0.1", 8080)
.throttle(3, 1.second, 1, ThrottleMode.Shaping)
.mapAsyncUnordered(maxConcurrentConnections)(handleOneConnection)
.to(Sink.ignore)
.run()
I need to create an akka.stream.scaladsl.Source[T, Unit] from a collection of Future[T].
E.g., having a collection of futures returning integers,
val f1: Future[Int] = ???
val f2: Future[Int] = ???
val fN: Future[Int] = ???
val futures = List(f1, f2, fN)
how to create a
val source: Source[Int, Unit] = ???
from it.
I cannot use Future.sequence combinator, since then I would wait for each future to complete before getting anything from the source. I want to get results in any order as soon as any future completes.
I understand that Source is a purely functional API and it should not run anything before somehow materializing it. So, my idea is to use an Iterator (which is lazy) to create a source:
Source { () =>
new Iterator[Future[Int]] {
override def hasNext: Boolean = ???
override def next(): Future[Int] = ???
}
}
But that would be a source of futures, not of actual values. I could also block on next using Await.result(future) but I'm not sure which tread pool's thread will be blocked. Also this will call futures sequentially, while I need parallel execution.
UPDATE 2: it turned out there was a much easier way to do it (thanks to Viktor Klang):
Source(futures).mapAsync(1)(identity)
UPDATE: here is what I've got based on #sschaef answer:
def futuresToSource[T](futures: Iterable[Future[T]])(implicit ec: ExecutionContext): Source[T, Unit] = {
def run(actor: ActorRef): Unit = {
futures.foreach { future =>
future.onComplete {
case Success(value) =>
actor ! value
case Failure(NonFatal(t)) =>
actor ! Status.Failure(t) // to signal error
}
}
Future.sequence(futures).onSuccess { case _ =>
actor ! Status.Success(()) // to signal stream's end
}
}
Source.actorRef[T](futures.size, OverflowStrategy.fail).mapMaterializedValue(run)
}
// ScalaTest tests follow
import scala.concurrent.ExecutionContext.Implicits.global
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
"futuresToSource" should "convert futures collection to akka-stream source" in {
val f1 = Future(1)
val f2 = Future(2)
val f3 = Future(3)
whenReady {
futuresToSource(List(f1, f2, f3)).runFold(Seq.empty[Int])(_ :+ _)
} { results =>
results should contain theSameElementsAs Seq(1, 2, 3)
}
}
it should "fail on future failure" in {
val f1 = Future(1)
val f2 = Future(2)
val f3 = Future.failed(new RuntimeException("future failed"))
whenReady {
futuresToSource(List(f1, f2, f3)).runWith(Sink.ignore).failed
} { t =>
t shouldBe a [RuntimeException]
t should have message "future failed"
}
}
Creating a source of Futures and then "flatten" it via mapAsync:
scala> Source(List(f1,f2,fN)).mapAsync(1)(identity)
res0: akka.stream.scaladsl.Source[Int,Unit] = akka.stream.scaladsl.Source#3e10d804
One of the easiest ways to feed a Source is through an Actor:
import scala.concurrent.Future
import akka.actor._
import akka.stream._
import akka.stream.scaladsl._
implicit val system = ActorSystem("MySystem")
def run(actor: ActorRef): Unit = {
import system.dispatcher
Future { Thread.sleep(100); actor ! 1 }
Future { Thread.sleep(200); actor ! 2 }
Future { Thread.sleep(300); actor ! 3 }
}
val source = Source
.actorRef[Int](0, OverflowStrategy.fail)
.mapMaterializedValue(ref ⇒ run(ref))
implicit val m = ActorMaterializer()
source runForeach { int ⇒
println(s"received: $int")
}
The Actor is created through the Source.actorRef method and made available through the mapMaterializedValue method. run simply takes the Actor and sends all the completed values to it, which can then be accessed through source. In the example above, the values are sent directly in the Future, but this can of course be done everywhere (for example in the onComplete call on the Future).
I am looking for a SinkSource that provides a Sink and a Source. If an element flows into that Sink it should be provided at the corresponding Source. The following code shows what I mean:
object SinkSource {
def apply[T] = new {
def sink: Sink[T] = ???
def source: Source[T] = ???
}
}
val flowgraph = FlowGraph { implicit fgb =>
import FlowGraphImplicits._
val sinksource = SinkSource[Int]
Source(1 to 5) ~> sinksource.sink
sinksource.source ~> Sink.foreach(print)
}
implicit val actorSystem = ActorSystem(name = "System")
implicit val flowMaterializer = FlowMaterializer()
val materializedMap = flowgraph.run()
If executed this should print: 12345
So, does a SinkSource exist (haven't seen it in the API) or does anyone know how to implement it?
I should mention that I need distinct access to Sink and Source so that Flow isn't a solution in this particular form:
Source(1 to 5) ~> Flow[Int] ~> Sink.foreach(println)
As so often, ideas come to mind if question was already asked: It turned out, I don't need a Sink and a Source, JunctionInPort and JunctionOutPort are sufficient.
So here it goes:
object SinkSource {
def apply[T](implicit fgb: FlowGraphBuilder) = new SinkSource[T]
}
class SinkSource[T](implicit fgb: FlowGraphBuilder) {
import FlowGraphImplicits._
private val merge = Merge[T]
private val bcast = Broadcast[T]
Source.empty ~> merge
merge ~> bcast
bcast ~> Sink.ignore
def in: JunctionInPort[T] = merge
def out: JunctionOutPort[T] = bcast
}
val flowgraph = FlowGraph { implicit fgb =>
import FlowGraphImplicits._
val source = Source(1 to 5)
val sink = Sink.foreach(println)
val sinkSource = SinkSource[Int]
source ~> sinkSource.in
sinkSource.out ~> sink
}
implicit val actorSystem = ActorSystem(name = "System")
implicit val flowMaterializer = FlowMaterializer()
val materializedMap = flowgraph.run()