Given the following code:
import spray.http._
import spray.client.pipelining._
import scala.concurrent.Future
implicit val system = ActorSystem()
import system.dispatcher // execution context for futures
val pipeline: HttpRequest => Future[HttpResponse] = sendReceive
val response: Future[HttpResponse] = pipeline(Get("http://spray.io/"))
The following pseudo-code function waits 10 seconds, returning "GOOD" if the HttpResponse returned, or "BAD" on an Await#result exception (see docs.
import scala.concurrent.Await
import scala.concurrent.duration._
def f(fut: Future[HttpResponse]): String = {
try {
val result = Await.result(fut, 10.seconds)
"GOOD"
}
catch e # (_: InterruptedException | _: IllegalArgumentException
| _: TimeoutException ) => "BAD"
}
In my catch, is it only necessary to catch exception thrown by Await#result? In other words, am I not catching any possible exceptions here?
The Await.result itself can throw the exceptions you caught, however, if the future it awaits does not complete successfully, it forwards the exception contained by the future. You might want to read the Blocking section from here: Futures and Promises.
So yes, there may be exceptions you aren't catching, anything that can result from the failed computation of a HttpResponse.
Blocking in real code is usually bad and should be done only for testing purposes, but if you really need to, I would recommend to wrap the Await in a scala.util.Try so you could manipulate it elegantly later and also keep the information of when and why it failed.
Related
I'm trying to use completionTimeout in an akka streams flow. I've provided a contrived example where the flow takes 10 seconds but I've added a completionTimeout with a timeout of 1 second. I would expect this flow to timeout after 1 second. However, in the example the flow completes in 10 seconds without any errors.
Why doesn't the flow timeout? Is there a better way to timeout a flow?
import akka.NotUsed
import akka.actor.ActorSystem
import akka.stream.scaladsl.{Flow, Sink, Source}
import org.scalatest.{FlatSpec, Matchers}
import scala.concurrent.duration._
import scala.concurrent.{Await, Future}
class Test extends FlatSpec with Matchers {
implicit val system = ActorSystem("test")
"This Test" should "fail but passes and I don't know why" in {
//This takes 10 seconds to complete
val flow: Flow[String, String, NotUsed] = Flow[String]
.map(str => {
println(s"Processing ${str}")
Thread.sleep(10000)
})
.map(_ => {"Done!"})
val future: Future[String] =
Source.single("Input")
.via(flow)
.completionTimeout(1 second) // Set a timeout of 1 second
.runWith(Sink.last)
val result = Await.result(future, 15 seconds)
result should be("Done!")
}
}
In executing a given stream, Akka Stream leverages operator fusion to fuse stream operators by a single underlying actor for optimal performance. For your main thread to catch the timeout, you could introduce asynchrony by means of .async:
val future: Future[String] =
Source.single("Input")
.via(flow)
.async // <--- asynchronous boundary
.completionTimeout(1 second)
.runWith(Sink.last)
future.onComplete(println)
// Processing Input
// Failure(java.util.concurrent.TimeoutException: The stream has not been completed in 1 second.)
An alternative to introduce asynchrony is to use the mapAsync flow stage:
val flow: Flow[String, String, NotUsed] = Flow[String]
.map(str => {
println(s"Processing ${str}")
Thread.sleep(10000)
})
.mapAsync(1)(_ => Future("Done!")) // <--- asynchronous flow stage
Despite getting the same timeout error, you may notice it'll take ~10s to see result when using mapAsync, whereas only ~1s using async. That's because while mapAsync introduces an asynchronous flow stage, it's not an asynchronous boundary (like what async does) and is still subject to operator fusion.
I am new to scala and I have to make an async call ( Elastic indexing using elastic4s ) while iterating through a for each loop. What is the best way to do that in scala.
val data = List("1","2","3","4")
data.foreach(element=>{
asyncCall(element)
})
How to do some action after all the asyncCall is completed. I don'nt want to do anything with the response. Just print success if everything success and print fail if any of the call fails. asyncCall returns Future[T].
You should use Future.sequence. It changes List[Future[T]] to Future[List[T]].
So in your example:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}
val f = Future.sequence(data.map(asyncCall)) // Future[List[String]]
//you can now do something when future finishes:
f.onComplete{
case Success(s) => println(s"Evethings fine: $s")
case Failure(e) => println(s"Something went wrong: $e")
}
Suppose I am writing a function to send a few concurrent HTTP GET requests and wait for all the responses with a time-out. If at least one response does not have status 200 or does not come within the time-out my function should return failure.
I am writing this function tryGets like this:
import java.net.URL
import scala.concurrent.duration._
import scala.concurrent.{Await, ExecutionContext, Future}
import scala.util.Try
def unsafeGet(url: URL): String = {
val in = url.openStream()
scala.io.Source.fromInputStream(in).mkString
}
def futureGet(url: URL)
(implicit ec: ExecutionContext): Future[String] = Future {
unsafeGet(url)
}
def tryGets(urls: Seq[URL], timeOut: Duration)
(implicit ec: ExecutionContext): Try[Seq[String]] = Try {
val fut = Future.sequence(urls.map(futureGet))
Await.result(fut, timeOut)
}
Does it make sense ?
Does not it leak future instances in case of time-out ?
If one of the Future's time out, then the rest of the Future's will continue to execute because the future's are eager and will continue to run on the Execution Context. What you could do is fold over the Urls but this will execute them in serial.
urls.foldleft(Future.sucessful(Seq.empty)) { (future, url) =>
future.flatMap(accum => futureGet(url).map(accum :+ _))
}
I have an Rest API provided by akka-http. In some cases I need to get data from an external database (Apache HBase), and I would like the query to fail if the database takes too long to deliver the data.
One naïve way is to wrap the call inside a Future and then block it with an Await.result with the needed duration.
import scala.concurrent.duration._
import scala.concurrent.{Await, Future}
object AsyncTest1 extends App {
val future = Future {
getMyDataFromDB()
}
val myData = Await.result(future, 100.millis)
}
The seems to be inefficient as this implementation needs two threads. Is There an efficient way to do this ?
I have another use case where I want to send multiple queries in parallel and then aggregates the results, with the same delay limitation.
val future1 = Future {
getMyDataFromDB1()
}
val future2 = Future {
getMyDataFromDB2()
}
val foldedFuture = Future.fold(
Seq(future1, future2))(MyAggregatedData)(myAggregateFunction)
)
val myData = Await.result(foldedFuture, 100.millis)
Same question here, what is the most efficient way to implement this ?
Thanks for your help
One solution would be to use Akka's after function which will let you pass a duration, after which the future throws an exception or whatever you want.
Take a look here. It demonstrates how to implement this.
EDIT:
I guess I'll post the code here in case the link gets broken in future:
import scala.concurrent._
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
import scala.util.{Failure, Success}
import akka.actor.ActorSystem
import akka.pattern.after
val system = ActorSystem("theSystem")
lazy val f = future { Thread.sleep(2000); true }
lazy val t = after(duration = 1 second, using = system.scheduler)(Future.failed(new TimeoutException("Future timed out!")))
val fWithTimeout = Future firstCompletedOf Seq(f, t)
fWithTimeout.onComplete {
case Success(x) => println(x)
case Failure(error) => println(error)
}
I expect that the following code will generate value 5 as I have written recover to handle the exception and return 5. But I always see the exception in IntelliJ. What is the mistake here?
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success, Try}
import scala.concurrent.duration._
import scala.language.postfixOps
object ConcurrencyExample extends App {
val time = System.currentTimeMillis()
val futures1: Future[Int] = Future {
println("in Future.. sleeping")
Thread.sleep(10)
println("awakein future")
throw new Exception("future failed")
1
}
val result = futures1.map(x=>x+3)
result recover {
case e:Exception => 5
}
result onComplete{
case Success(v)=>println("result is "+v)
case Failure(e) =>println("failed result:"+e)
}
Await.result(result,100 millis)
}
result
in Future.. sleeping
awakein future
Exception in thread "main" LegacyException: future failed
It is not working because the future that would do the recovering is not the one you are calling Await.result() on.
From the Future's scaladoc, we learn that recover creates a new future that will handle any matching throwable that this future might contain. If there is no match, or if this future contains a valid result then the new future will contain the same.
Thus, it is this new future you are now interested in.
Try this:
val result = futures1.map(x=>x+3)
.recover {
case e:Exception => 5
}
Await.result(result,100 millis)