Shut down akka system with spray from non akka system - scala

Im very new to akka and futures in scala. Im using spray to get the output of a URL and return a Future[String] to another object. Here is the object that is making the HTTP request.
object ActionsService {
private implicit val formats = DefaultFormats
implicit val system = ActorSystem()
import system.dispatcher
val pipeline = sendReceive ~> unmarshal[String]
def getActions: Future[String] ={
val out = getOutput("http://www.google.com")
out
}
def getOutput(url: String): Future[String] ={
val response = pipeline (Get (url) )
response
}
def shutdown(code: Int): Unit = {
IO(Http).ask(Http.CloseAll)(1.second).await
system.shutdown()
}
}
And here is the main method in the other object where I am trying to shutdown the actor system.
import ExecutionContext.Implicits.global
def main(args: Array[String]) {
val test = ActionsService.getActions
test.onComplete {
case Success(x) => println(x)
case Failure(y) => println(y)
}
ActionsService.system.shutdown()
For some reason the akka system wont shutdown. Ive also tried to shut down the akka system using my shutdown method but I get an Exception in thread "main" akka.pattern.AskTimeoutException: Timed out error (which also occurs in the main method when the shutdown method is called).
akka version is 2.2.3

You didn't specify version of Akka you're using but in last version you need to call
system.terminate()
instead and wait for termination like this (ass an example)
Await.ready(system.whenTerminated, Duration.Inf)

The answer above did not end up working for me so I had to block the thread by adding a sleep period for the main method before the shutdown method is called.
def main(args: Array[String]) {
val test = ActionsService.getActions
Thread.sleep(700)
test.onComplete {
case Success(x) => println(x)
case Failure(y) => println(y)
}
CorporateActionsService.system.shutdown()
}

Related

Scala main class with futures not terminating

I have a scala main class
object Job extends App {
def myProcedure() = {
sqlu"""CALL `dbName`.`update_history();"""
}
implicit val system: ActorSystem = ActorSystem()
implicit val mat: ActorMaterializer = ActorMaterializer()
implicit val ec = system.dispatcher
implicit val session: SlickSession = SlickSession.forConfig("my-mysql")
val proc = session.db.run(myProcedure))
val terminatedF = proc.flatMap { rec =>
println("value of the procedure ::" + rec)
session.close()
system.terminate()
}
Await.result(terminatedF, Duration.Inf)
println("terminated :::")
}
I see that the value of rec gets printed, and also terminated ::: i.e. last line also gets printed. However the program does not end. Am I missing anything here ?
This was a problem with user defined threads.
I used Await.result on the last Future and then did sys.exit(0).
Also in build.sbt
I used
fork in run := true
This perfectly works

How to react to an exception (upstream failure) in an akka flow

I have an akka Flow[I, O] that is out of my control because its coming in from some third-party code. I need to react to whenever an input element does not produce an output element (for example, because an exception was thrown in some part of the flow). For that, I need the input element that produced the failure. I do not find any API on the flow or similar that allows me to register a handler or react to it in any way. How can I do that?
You want to Resume rather than Stop when the akka streams flow throws an exception. After collecting all successful elements, you can Seq#diff to tell what elements are dropped due to exception thrown.
import scala.concurrent.ExecutionContext
import scala.util.{Failure, Success}
object Exception {
case class MyException(n: Int) extends RuntimeException
def main(args: Array[String]): Unit = {
implicit val system: ActorSystem = ActorSystem("Exception")
implicit val ec: ExecutionContext = system.dispatcher
val decider: Supervision.Decider = {
case _: MyException => Supervision.Resume
case _ => Supervision.Stop
}
val flow = Flow[Int]
.map(n =>
if (n % 2 == 1) throw MyException(n)
else n
)
val in = 1 to 10
val outFuture = Source(in)
.via(flow)
.withAttributes(ActorAttributes.supervisionStrategy(decider))
.runWith(Sink.seq)
outFuture.onComplete {
case Success(out) =>
println("dropped elements are " + (in.diff(out)))
case Failure(_) =>
println("unknown failure")
}
}
}
The console outputs are:
dropped elements are Vector(1, 3, 5, 7, 9)
Reference: How to get object that caused failure in Akka Streams?

Do i need to create ActorMaterializer multiple times?

I have an app that generate reports, with akka-http + akka-actors + akka-camel + akka-streams. When a post request arrives , the ActiveMqProducerActor enqueue the request into ActiveMq Broker. Then the ActiveMqConsumerActor consumes the message and start the task using akka-streams(in this actor i need the materializer) .
The main class create the ActorSystem and the ActorMaterializer, but i dont know how is the correct way to "inject" the materializer into the akka-actor
object ReportGeneratorApplication extends App {
implicit val system: ActorSystem = ActorSystem()
implicit val executor = system.dispatcher
implicit val materializer = ActorMaterializer()
val camelExtension: Camel = CamelExtension(system);
val amqc: ActiveMQComponent = ActiveMQComponent.activeMQComponent(env.getString("jms.url"))
amqc.setUsePooledConnection(true)
amqc.setAsyncConsumer(true)
amqc.setTrustAllPackages(true)
amqc.setConcurrentConsumers(1)
camelExtension.context.addComponent("jms", amqc);
val jmsProducer: ActorRef = system.actorOf(Props[ActiveMQProducerActor])
//Is this the correct way to pass the materializer?
val jmsConsumer: ActorRef = system.actorOf(Props(new ActiveMQConsumerActor()(materializer)), name = "jmsConsumer")
val endpoint: ReportEndpoint = new ReportEndpoint(jmsProducer);
Http().bindAndHandle(endpoint.routes, "localhost", 8881)
}
The ReportEndPoint class, that have the jmsProducerActor . Mongo is a trait with CRUD methods. JsonSupport(==SprayJsonSupport)
class ReportEndpoint(jmsProducer: ActorRef)
(implicit val system:ActorSystem,
implicit val executor: ExecutionContext,
implicit val materializer : ActorMaterializer)
extends JsonSupport with Mongo {
val routes =
pathPrefix("reports"){
post {
path("generate"){
entity(as[DataRequest]) { request =>
val id = java.util.UUID.randomUUID.toString
// **Enqueue the request into ActiveMq**
jmsProducer ! request
val future: Future[Seq[Completed]] = insertReport(request)
complete {
future.map[ToResponseMarshallable](r => r.head match {
case r : Completed => println(r); s"Reporte Generado con id $id"
case _ => HttpResponse(StatusCodes.InternalServerError, entity = "Error al generar reporte")
})
}
}
}
} ....
The idea of ActiveMqConsumerActor, is send the messages, with streams and backpressure, one by one,because ReportBuilderActor makes many mongo operations (and the datacenter it`s not very good).
//Is this the correct way to pass the materializer??
class ActiveMQConsumerActor (implicit materializer : ActorMaterializer) extends Consumer with Base {
override def endpointUri: String = env.getString("jms.queue")
val log = Logging(context.system, this)
val reportActor: ActorRef = context.actorOf(Props(new ReportBuilderActor()(materializer)), name = "reportActor")
override def receive: Receive = {
case msg: CamelMessage => msg.body match {
case data: DataRequest => {
//I need only one task running
Source.single(data).buffer(1, OverflowStrategy.backpressure).to(Sink.foreach(d => reportActor ! d)).run()
}
case _ => log.info("Invalid")
}
case _ => UnhandledMessage
}
}
Is a good idea have implicit values in companion objects?
Thanks!!

In akka-stream how to create a unordered Source from a futures collection

I need to create an akka.stream.scaladsl.Source[T, Unit] from a collection of Future[T].
E.g., having a collection of futures returning integers,
val f1: Future[Int] = ???
val f2: Future[Int] = ???
val fN: Future[Int] = ???
val futures = List(f1, f2, fN)
how to create a
val source: Source[Int, Unit] = ???
from it.
I cannot use Future.sequence combinator, since then I would wait for each future to complete before getting anything from the source. I want to get results in any order as soon as any future completes.
I understand that Source is a purely functional API and it should not run anything before somehow materializing it. So, my idea is to use an Iterator (which is lazy) to create a source:
Source { () =>
new Iterator[Future[Int]] {
override def hasNext: Boolean = ???
override def next(): Future[Int] = ???
}
}
But that would be a source of futures, not of actual values. I could also block on next using Await.result(future) but I'm not sure which tread pool's thread will be blocked. Also this will call futures sequentially, while I need parallel execution.
UPDATE 2: it turned out there was a much easier way to do it (thanks to Viktor Klang):
Source(futures).mapAsync(1)(identity)
UPDATE: here is what I've got based on #sschaef answer:
def futuresToSource[T](futures: Iterable[Future[T]])(implicit ec: ExecutionContext): Source[T, Unit] = {
def run(actor: ActorRef): Unit = {
futures.foreach { future =>
future.onComplete {
case Success(value) =>
actor ! value
case Failure(NonFatal(t)) =>
actor ! Status.Failure(t) // to signal error
}
}
Future.sequence(futures).onSuccess { case _ =>
actor ! Status.Success(()) // to signal stream's end
}
}
Source.actorRef[T](futures.size, OverflowStrategy.fail).mapMaterializedValue(run)
}
// ScalaTest tests follow
import scala.concurrent.ExecutionContext.Implicits.global
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
"futuresToSource" should "convert futures collection to akka-stream source" in {
val f1 = Future(1)
val f2 = Future(2)
val f3 = Future(3)
whenReady {
futuresToSource(List(f1, f2, f3)).runFold(Seq.empty[Int])(_ :+ _)
} { results =>
results should contain theSameElementsAs Seq(1, 2, 3)
}
}
it should "fail on future failure" in {
val f1 = Future(1)
val f2 = Future(2)
val f3 = Future.failed(new RuntimeException("future failed"))
whenReady {
futuresToSource(List(f1, f2, f3)).runWith(Sink.ignore).failed
} { t =>
t shouldBe a [RuntimeException]
t should have message "future failed"
}
}
Creating a source of Futures and then "flatten" it via mapAsync:
scala> Source(List(f1,f2,fN)).mapAsync(1)(identity)
res0: akka.stream.scaladsl.Source[Int,Unit] = akka.stream.scaladsl.Source#3e10d804
One of the easiest ways to feed a Source is through an Actor:
import scala.concurrent.Future
import akka.actor._
import akka.stream._
import akka.stream.scaladsl._
implicit val system = ActorSystem("MySystem")
def run(actor: ActorRef): Unit = {
import system.dispatcher
Future { Thread.sleep(100); actor ! 1 }
Future { Thread.sleep(200); actor ! 2 }
Future { Thread.sleep(300); actor ! 3 }
}
val source = Source
.actorRef[Int](0, OverflowStrategy.fail)
.mapMaterializedValue(ref ⇒ run(ref))
implicit val m = ActorMaterializer()
source runForeach { int ⇒
println(s"received: $int")
}
The Actor is created through the Source.actorRef method and made available through the mapMaterializedValue method. run simply takes the Actor and sends all the completed values to it, which can then be accessed through source. In the example above, the values are sent directly in the Future, but this can of course be done everywhere (for example in the onComplete call on the Future).

Return Future Message from an Actor

I have an existing API which returns a Future. Now introducing an Actor for one of the use cases and trying to continue using same service API from it. From below you can see MyService.saveValues return future.
object MyActor {
implicit val ec = scala.concurrent.ExecutionContext.Implicits.global
implicit val timeout = Timeout(1 second)
case class SampleMessage(list: List[String])
val actorSystem = //get actorSystem that was created at the startup
def saveMessage(list: List[String]) ={
val res = (actorSystem.actorOf(Props[MyActor]) ? SaveMyMessage(list) ).mapTo[Future[\/[Throwable,String]]
//res map{ r=>
//}
}
}
class MyActor extends Actor {
import MyActor._
def receive = {
case SaveMyMessage(list) =>
val originalSender = sender
val res : Future[\/[Throwable,String] ] = MyService.saveValues(list)
originalSender ! res
}
}
As you can see in def saveMessage I am using ask to wait for a result from an actor. However ask also creates its own future so result (val res) in saveMessage becomes Future[Future[T]] which looks annoying. What's the best way to deal with this scenario?
pipeTo forwards the result of a Future to an ActorRef.
import akka.pattern.pipe
val originalSender = sender
val res : Future[\/[Throwable,String] ] = MyService.saveValues(list)
res pipeTo originalSender
If saveValues throws, your future never complete and will end up timing something out though.
If you ever end up with Future[Future[A]], but want Future[A] as a result of sequencing/traversing or something, you can always "flatMap that shit."
import ExecutionContext.Implicits.global
val foo: Future[Future[Int]] = ???
val bar: Future[Int] = foo.flatMap(identity)
If you already depend on scalaz (and scalaz.contrib if on scalaz 7.0.x), you can use join defined in monad syntax.
import scalaz.syntax.monad._
import scalaz.contrib.std.scalaFuture._
val bar2: Future[Int] = foo.join