When I run the code below it terminates and nothing happens, is there a way to catch the exception in future2.map ?
object TestFutures1 {
val future1 = Future {
throw new Exception("error")
}
}
object TestFutures2 extends App {
val future2 = TestFutures1.future1
future2.map { result => println(result) }
Thread.sleep(5000)
}
Generally speaking, there are two options (with some sub-options) to handle future exceptions.
You can add a callback that will be called when the future completes with an exception:
future2
.map { result => println(result) }
.onFailure { exc =>
exc.printStackTrace
}
1a. .onFailure is only used for side effects, you cannot use, for example, to handle the exception. If you want to handle in and rescue the future, you can use .recover or .recoverWith (the only different between the two is that the latter takes a function that returns a Future, while the former deals with the actual result, same idea as map vs. flatMap:
future2
.map(println)
.recover { case e =>
e.printStackTrace
"recovered"
}.map(println)
The other option is to wait for the future to complete, and access it's result:
try {
// Instead of Thread.sleep - you don't need that:
Await.result(future2, 2 seconds)
} catch { case exc =>
exc.printStackTrace
}
Except #Dima's answer, also can use onComplete to catch failure, like:
val re = future2.map { result => println(result) }
re onComplete {
case Success(s) => ...
case Failure(e) => e.printStackTrace()
}
Related
Considering a sequence of futures each returning Either[Status, Resp].
How would you propagate error status codes through a for comprehension which is using Future and not Either?
The code bellow does not work, since the parsing exception is not caught by .recover of the last future
The use case is Scala Play ActionRefiners which returns Future[Either[Status, TRequest[A]]].
def parseId(id: String):Future[Int] = {
Future.successful(Integer.parseInt(id))
}
def getItem(id: Int)(implicit ec: ExecutionContext): Future[Either[Status, String]] =
Future(Some("dummy res from db " + id)).transformWith {
case Success(opt) => opt match {
case Some(item) => Future.successful(Right(item))
case _ => Future.successful(Left(NotFound))
}
case Failure(_) => Future.successful(Left(InternalServerError))
}
(for {
id <- parseId("bad request")
resp <- getItem(id)
} yield resp).recover {
case _:NumberFormatException => Left(BadRequest)
}
I could move the .recover to parseId, but this makes the for comprehension very ugly - having to treat the Either[Status, id] in the middle
def parseId(id: String):Future[Either[Status, Int]] = {
Future.successful(Right(Integer.parseInt(id))).recover {
case _:NumberFormatException => Left(BadRequest)
}
}
Your exception is not caught because you are not throwing it inside the Future: Future.successful is immediately satisfied with the result of the expression you give it, if it throws an exception, it is executed on the current thread.
Try removing the .successful: Future(id.toInt) will do what you want.
Also, I would recommend to get rid of all the Eithers: these are highly overrated/overused, especially in the context of Future (that already wrap their result into Try anyhow), and just make the code more complicated and less readable without offering much benefit.
case class FailureReason(status: Status)
extends Exception(status.toString)
def notFound() = throw FailureReason(NotFound)
def internalError() = throw FailureReason(InternalError)
def badRequest() = throw FailureReason(BadRequest)
def parseId(id: String):Future[Int] = Future(id.toInt)
def getItem(id: Int): Future[String] = Future(Some("dummy"))
.map { _.getOrElse(notFound) }
.recover { _ => internalError }
// this is the same as your for-comprehension, just looking less ugly imo :)
parseId("foo").flatMap(getItem).recover {
case _: NumberFormatException => badRequest()
}
// if you still want `Either` in the end for some reason:
.map(Right.apply[Status, String])
.recover {
case _: NumberFormatException => Left(BadRequest) // no need for the first recover above if you do this
case FailureReason(status) => Left(status)
}
I have a method that may return Future - successful or failed or can even throw an exception. I can avoid this by putting try catch block on entire method and return Future all the time but i would like to avoid it for now. I have few problems with calling such method:
1) In caller code, if I use map I expect execution of a method and expect a Future or an exception which I tried to handle in following manner:
object ETLCoordinator {
private def getBusinessListFromModules(modulePaths: Iterable[File]) : Future[String] = {
implicit val ec = ExecutionContext.global
println("Inside getBusinessListFromModules..")
throw new java.lang.RuntimeException("failed to get businesses") //Exception is thrown before future was constructed
Future("ok")
}
def main(args: Array[String]) {
println("Inside Future Test..")
implicit val ec = ExecutionContext.global
val modulePaths = Iterable(new File("mdrqaint/MDR/QAINT/QAINTX"))
val fut1 = getBusinessListFromModules(modulePaths) //This is outside of try and which should be okay
try {
fut1.map { res =>
println("things after Successful fut1")
}.recover{
case t: Throwable => println("Failed future in fut1: "+ t.getMessage)
}
} catch {
case t: Throwable => println("Exception in fut1: "+ t.getMessage)
}
}
}
Output: ( No execution of recover or catch block above)
Inside Future Test..
Inside getBusinessListFromModules..
Exception in thread "main" java.lang.RuntimeException: failed to get businesses
But if I put val fut1 = getBusinessListFromModules(modulePaths) inside Try block then Exception is get caught in Catch block and I get output:
Inside Future Test..
Inside getBusinessListFromModules..
Exception in fut1: failed to get businesses
Why is this? I though Future execution happens upon calling some of its methods like map, flatmap, onSuccess, onComplete etc. In this case call to map is already inside Try block.
2) What is the better way to define and call such methods? Try/catch block in a caller or try/catch in method itself? or any other way. I tried wrapping the calling method in Future so I get Future[Future[String]] in caller. I was able to avoid all try-catch.
val fut1 = Future(getBusinessListFromModules(modulePaths))
//try {
fut1.map { res =>
res.map{ str =>
println("things after Successful fut1")
}.recover{
case t: Throwable => println("Failed in future of fut1: "+ t.getMessage)
}
println("things after Successful fut1 wrapper")
}.recover{
case t: Throwable => println("Failed to create future in fut1: "+ t.getMessage)
}
3) If there is another method inbetween which does delegation to getBusinessListFromModules but it itself is non-future method.
object ETLController {
private def getBusinessListFromModules(modulePaths: Iterable[File]) : Future[String] = {
implicit val ec = ExecutionContext.global
println("Inside getBusinessListFromModules..")
//throw new java.lang.RuntimeException("failed to get businesses")
Future("ok")
}
private def callGetBusList(modulePaths: Iterable[File]) : String = {
implicit val ec = ExecutionContext.global
val etlF = getBusinessListFromModules(modulePaths)
etlF onComplete {
case Success(itr) => {
println("Future getBusinessListFromModules success: "+ itr)
throw new java.lang.RuntimeException("RTE from callGetBusList")
}
case Failure(t) => {
println("Future getBusinessListFromModules throws an error")
}
}
"callGetBusList was a success"
}
def main(args: Array[String]) {
println("Inside Future Test..")
implicit val ec = ExecutionContext.global
val modulePaths = Iterable(new File("mdrqaint/MDR/QAINT/QAINTX"))
try {
val fut = Future(callGetBusList(modulePaths))
fut.map { res =>
println("successful future!")
}.recover{
case t: Throwable => println("Failed future: "+ t.getMessage)
}
} catch {
case t: Throwable => println("callGetBusList failed:" + t.getMessage)
}
}
}
Output: (no recover or catch block execution!)
Inside Future Test..
Inside getBusinessListFromModules..
Future getBusinessListFromModules success: ok
java.lang.RuntimeException: RTE from callGetBusList
at ..
successful future!
I even try double wrapping Future calls :
val fut = Future(Future(callGetBusList(modulePaths)))
fut.map { res =>
res.map { str =>
println("successful inner future! "+ str)
}.recover{
case t: Throwable => println("Failed inner future: "+ t.getMessage)
}
println("successful outer future!")
}.recover{
case t: Throwable => println("Failed outer future: "+ t.getMessage)
}
Output:
Future getBusinessListFromModules success: ok
java.lang.RuntimeException: RTE from callGetBusList
at
successful inner future! callGetBusList was a success
successful outer future!
I get "callGetBusList was a success" which seems like RuntimeException inside onComplete method got lost! How do I catch it in a final caller? What are the better practice to handle such future dependencies?
UPDATE:
based on #dk14 explanation, Opted to convert middle method to return Future and basically all methods to return some kind of Future and not a plain Exception.
object ETLController {
private def getBusinessListFromModules(modulePaths: Iterable[File]) : Future[String] = {
implicit val ec = ExecutionContext.global
println("Inside getBusinessListFromModules..")
Future {
Thread.sleep(2000)
throw new java.lang.RuntimeException("failed to get businesses")
"ok"
}
}
private def callGetBusList(modulePaths: Iterable[File]) : Future[String] = {
implicit val ec = ExecutionContext.global
val etlF = getBusinessListFromModules(modulePaths)
etlF map { itr =>
println("Future getBusinessListFromModules success: "+ itr)
throw new java.lang.RuntimeException("RTE from callGetBusList")
} recover {
case t: Throwable => {
println("Future callGetBusList throws an error: " + t.getMessage)
throw t
}
}
}
def main(args: Array[String]) {
println("Inside Future Test..")
implicit val ec = ExecutionContext.global
val modulePaths = Iterable(new File("mdrqaint/MDR/QAINT/QAINTX"))
val fut = callGetBusList(modulePaths)
fut.map { str =>
println("successful future! "+ str)
}.recover{
case t: Throwable => println("Failed future: "+ t.getMessage)
}
println("Active threads: " +Thread.activeCount())
sys.allThreads().foreach(t => t.join())
}
}
1) Futures are firing eagerly and they aren't referentially transparent.
Answers to the referenced question also contain some insights about Future's internal behavior, so I'd like to skip it here.
In order to manage side-effects concerning execution pools/queues/threads in a more predictable way, you could consider scalaz/monix/fs2 Task or iteratee/scalaz/cats Eval (more abstract lazy evaluation, and intended for sync stuff) + Cont (continuations are abstracting over subscriptions) as alternative. All are referentially transparent and start execution lazily "on-demand".
2) The best way is the one you don't like: to not throw exception outisde of Future context.
You might also consider flatMap to avoid Future[Future[T]]
3) Double wrapping Futures directly a-la Future(Future(...)) doesn't change anything. Your method is executed on val etlF = g... (in the same thread) no matter what it returns. Future("ok")'s content (lambda) is executed eagerly (with "small" unpredictable delay) on a different thread but [execution task is being submitted to the pool] still inside getBusinessListFromModules.
One workaround (not really recommended) is val etlF = Future(getBusinessListFromModules(...)).flatMap(identity) which would return you a future wrapping any exception coming directly from getBusinessListFromModules and indirectly from getBusinessListFromModules's internal Future.
It's better to refactor getBusinessListFromModules itself however, also introduce different exception types for different kinds of trouble (validation, sync vs async, so on) your method might get into.
P.S. There are ways to mix async and sync exception handling, but in practice it's hard to analyze and predict such mixed behavior (which you probably noticed already). And code gets ugly.
In the following example, what is the expected behaviour?
import scala.concurrent.ExecutionContext.Implicits.global
val someFuture = Future { 1+1 }.map(elem => {
println("mapping immediately")
elem.toString
})
someFuture.onComplete {
case scala.util.Success(value) => println(s"Got the callback = $value")
case Failure(e) => e.printStackTrace()
}
Is the following output always guaranteed?
mapping immediately
Got the callback = 2
It depends.
val someFuture = Future { 1+1 }.map(elem => {
println("mapping immediately")
elem.toString
})
someFuture will be executed eagerly and the println will be executed.
someFuture.onComplete {
case scala.util.Success(value) => println(s"Got the callback = $value")
case Failure(e) => e.printStackTrace()
}
onComplete is an event listener for a Future. This is called only when this future is completed, either through an exception, or a value, to apply the provided function.
Therefore, it will always execute after the execution of the someFuture.
Assuming that the someFuture is not going to fail, the order
`mapping immediately
Got the callback = 2`
is guaranteed.
For what ever reason, if there's an exception thrown, then the result would change. In that case Got the callback = 2 will not be printed and mapping immediately may or may not be printed.
Hope this helps.
I was hoping code like follows would wait for both futures, but it does not.
object Fiddle {
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2) // in the general case, this would be a dynamically sized list
val seq = Future.sequence(lf)
seq.onComplete {
_ => lf.foreach(f => println(f.isCompleted))
}
}
val a = FuturesSequence
I assumed seq.onComplete would wait for them all to complete before completing itself, but not so; it results in:
true
false
.sequence was a bit hard to follow in the source of scala.concurrent.Future, I wonder how I would implement a parallel that waits for all original futures of a (dynamically sized) sequence, or what might be the problem here.
Edit: A related question: https://worldbuilding.stackexchange.com/questions/12348/how-do-you-prove-youre-from-the-future :)
One common approach to waiting for all results (failed or not) is to "lift" failures into a new representation inside the future, so that all futures complete with some result (although they may complete with a result that represents failure). One natural way to get that is lifting to a Try.
Twitter's implementation of futures provides a liftToTry method that makes this trivial, but you can do something similar with the standard library's implementation:
import scala.util.{ Failure, Success, Try }
val lifted: List[Future[Try[Int]]] = List(f1, f2).map(
_.map(Success(_)).recover { case t => Failure(t) }
)
Now Future.sequence(lifted) will be completed when every future is completed, and will represent successes and failures using Try.
And so, a generic solution for waiting on all original futures of a sequence of futures may look as follows, assuming an execution context is of course implicitly available.
import scala.util.{ Failure, Success, Try }
private def lift[T](futures: Seq[Future[T]]) =
futures.map(_.map { Success(_) }.recover { case t => Failure(t) })
def waitAll[T](futures: Seq[Future[T]]) =
Future.sequence(lift(futures)) // having neutralized exception completions through the lifting, .sequence can now be used
waitAll(SeqOfFutures).map {
// do whatever with the completed futures
}
A Future produced by Future.sequence completes when either:
all the futures have completed successfully, or
one of the futures has failed
The second point is what's happening in your case, and it makes sense to complete as soon as one of the wrapped Future has failed, because the wrapping Future can only hold a single Throwable in the failure case. There's no point in waiting for the other futures because the result will be the same failure.
This is an example that supports the previous answer. There is an easy way to do this using just the standard Scala APIs.
In the example, I am creating 3 futures. These will complete at 5, 7, and 9 seconds respectively. The call to Await.result will block until all futures have resolved. Once all 3 futures have completed, a will be set to List(5,7,9) and execution will continue.
Additionally, if an exception is thrown in any of the futures, Await.result will immediately unblock and throw the exception. Uncomment the Exception(...) line to see this in action.
try {
val a = Await.result(Future.sequence(Seq(
Future({
blocking {
Thread.sleep(5000)
}
System.err.println("A")
5
}),
Future({
blocking {
Thread.sleep(7000)
}
System.err.println("B")
7
//throw new Exception("Ha!")
}),
Future({
blocking {
Thread.sleep(9000)
}
System.err.println("C")
9
}))),
Duration("100 sec"))
System.err.println(a)
} catch {
case e: Exception ⇒
e.printStackTrace()
}
Even though it is quite old question But this is how I got it running in recent time.
object Fiddle {
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2) // in the general case, this would be a dynamically sized list
val seq = Future.sequence(lf)
import scala.concurrent.duration._
Await.result(seq, Duration.Inf)
}
This won't get completed and will wait till all the future gets completed. You can change the waiting time as per your use case. I have kept it to infinite and that was required in my case.
We can enrich Seq[Future[T]] with its own onComplete method through an implicit class:
def lift[T](f: Future[T])(implicit ec: ExecutionContext): Future[Try[T]] =
f map { Success(_) } recover { case e => Failure(e) }
def lift[T](fs: Seq[Future[T]])(implicit ec: ExecutionContext): Seq[Future[Try[T]]] =
fs map { lift(_) }
implicit class RichSeqFuture[+T](val fs: Seq[Future[T]]) extends AnyVal {
def onComplete[U](f: Seq[Try[T]] => U)(implicit ec: ExecutionContext) = {
Future.sequence(lift(fs)) onComplete {
case Success(s) => f(s)
case Failure(e) => throw e // will never happen, because of the Try lifting
}
}
}
Then, in your particular MWE, you can do:
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2)
lf onComplete { _ map {
case Success(v) => ???
case Failure(e) => ???
}}
This solution has the advantage of allowing you to call an onComplete on a sequence of futures as you would on a single future.
Create the Future with a Try to avoid extra hoops.
implicit val ec = ExecutionContext.global
val f1 = Future {
Try {
throw new Throwable("kaboom")
}
}
val f2 = Future {
Try {
Thread.sleep(1000L)
2
}
}
Await.result(
Future.sequence(Seq(f1, f2)), Duration("2 sec")
) foreach {
case Success(res) => println(s"Success. $res")
case Failure(e) => println(s"Failure. ${e.getMessage}")
}
I need to generate various code snippets based on user posted data, so those snippet generation can be parallel, I am trying to use futures for each and snippet generation and combine them and send AsyncResult, here is the code
val p = for {
pr <- Future { ProviderGenerator.generate(content) }
c <- Future { ModelGenerator.generate(content) }
} yield(pr, c)
Async {
if (p.isCompleted)
println("Futures completed")
p.onSuccess({
case (pr: ActionResult, c: ActionResult) => {
response.add(pr)
response.add(c)
Ok(Json.toJson(response))
}
})
p onFailure {
case _ => println("It failed")
}
p.map {
case (pr: ActionResult, c: ActionResult) => {
println("I am in case")
response.add(pr)
response.add(c)
Ok(Json.toJson(response))
}
case _ => {
println("I am in else case")
Ok(Json.toJson(response))
}
}
}
This is never sending result to user, where as if I use single future in it, I am getting response. How do I handle such multiple futures?
I don't see an obvious problem - try adding a p onFailure { case e: Throwable => Logger.error("failed", e) } perhaps and see if it shows one future is not completing.