I've been working up my answer to Is there a standard Scala function for running a block with a timeout?, and have run into a problem if an exception is thrown in a Future.
def runWithTimeout[T](timeoutMs: Long)(f: => T) : Option[T] = {
awaitAll(timeoutMs, future(f)).head.asInstanceOf[Option[T]]
}
So that
runWithTimeout(50) { "result" } should equal (Some("result"))
runWithTimeout(50) { Thread.sleep(100); "result" } should equal (None)
But if I throw an exception in my block it doesn't leak, but is swallowed - so that the following fails with "..no exception was thrown"
intercept[Exception] {
runWithTimeout(50) { throw new Exception("deliberate") }
}.getMessage should equal("deliberate")
Syserr has a stack trace with the message
<function0>: caught java.lang.Exception: deliberate
but I can't find where in the Scala runtime that is printed.
Apart from wrapping f in another block which catches exceptions and propagates them if thrown, is there any way to persuade awaitAll and/or Future to throw?
Short answer: no.
Exceptions don't do what you want when you're working in a threaded context, because you want to know about the exception in the caller, and the exception happens in the future's thread.
Instead, if you want to know what the exception was, you should return an Either[Exception,WhatYouWant]--of course, you have to catch that exception within the future and package it up.
scala> scala.actors.Futures.future{
try { Right("fail".toInt) } catch { case e: Exception => Left(e) }
}
res0: scala.actors.Future[Product with Serializable with Either[Exception,Int]] = <function0>
scala> res0() // Apply the future
res1: Product with Serializable with Either[Exception,Int] =
Left(java.lang.NumberFormatException: For input string: "fail")
Disclaimer: I work for Typesafe
Or.... you could use Akka and it would give you what you want without you having to go through hoops for it.
val f: Future[Int] = actor !!! message
Then
f.get
Will throw the exception that happened in the actor
f.await.exception
will give you an Option[Throwable]
Working my way through #Rex Kerr's suggestion, I've created
object Timeout {
val timeoutException = new TimeoutException
def runWithTimeout[T](timeoutMs: Long)(f: => T) : Either[Throwable, T] = {
runWithTimeoutIgnoreExceptions(timeoutMs)(exceptionOrResult(f)) match {
case Some(x) => x
case None => Left(timeoutException)
}
}
def runWithTimeout[T](timeoutMs: Long, default: T)(f: => T) : Either[Throwable, T] = {
val defaultAsEither: Either[Throwable, T] = Right(default)
runWithTimeoutIgnoreExceptions(timeoutMs, defaultAsEither)(exceptionOrResult(f))
}
def runWithTimeoutIgnoreExceptions[T](timeoutMs: Long)(f: => T) : Option[T] = {
awaitAll(timeoutMs, future(f)).head.asInstanceOf[Option[T]]
}
def runWithTimeoutIgnoreExceptions[T](timeoutMs: Long, default: T)(f: => T) : T = {
runWithTimeoutIgnoreExceptions(timeoutMs)(f).getOrElse(default)
}
private def exceptionOrResult[T](f: => T): Either[Throwable, T] =
try {
Right(f)
} catch {
case x => Left(x)
}
}
so that
#Test def test_exception {
runWithTimeout(50) { "result" }.right.get should be ("result")
runWithTimeout(50) { throw new Exception("deliberate") }.left.get.getMessage should be ("deliberate")
runWithTimeout(50) { Thread.sleep(100); "result" }.left.get should be (Timeout.timeoutException)
runWithTimeout(50, "no result") { "result" }.right.get should be ("result")
runWithTimeout(50, "no result") { throw new Exception("deliberate") }.left.get.getMessage should be ("deliberate")
runWithTimeout(50, "no result") { Thread.sleep(100); "result" }.right.get should be ("no result")
}
Again, I'm a bit of a Scala novice, so would welcome feedback.
scala.concurrent.ops.future includes exception handling.
So, instead of importing scala.actors.Futures.future, import scala.concurrent.ops.future instead.
That simple change in which import is there will cause the caller's call to .get to rethrow the exception. It works great!
Or use Future.liftTryTry, turns it from Future[Object] to Future[Try[Object]], and you can match on the Try[Object] and check for an exception case Throw(e) and log / exit gracefully
You need to override the method exceptionHandler in order to catch exceptions. So your option is to define your own future method so it creates a MyFutureActor with exceptionHandler.
EDIT: FutureActor is private, so subclassing it isn't possible.
Another option is to use links to know when exceptions happened.
However, I think Rex Kerr's approach is better - just wrap the function in something that will catch the Exception. Too bad future doesn't already do that.
Related
I am a newbie for Scala and now am trying to complete an exercise. How can I return an InvalidCartException while the function return type is Try[Price]
//Success: return the calculated price
//Failure: InvalidCartException
def calculateCartPrice(cart:Cart): Try[Price] = {
if(isCartValid(cart)) {
//Calculations happen here
return Try(Price(totalPrice));
}
}
def isCartValid(cart: Cart): Boolean = {
//THIS WORKS FINE
}
Thank you for the help
If you mean "how to make the Try contain an exception", then use the Failure() like below:
def calculateCartPrice(cart:Cart): Try[Price] = {
if(isCartValid(cart)) {
//Calculations happen here
Success(Price(totalPrice));
} else {
Failure(new InvalidCartException())
}
}
Then, given a Try you can use getOrElse to get the value of success or throw the exception.
Try will catch the exception for you, so put the code that can throw the exception in there. For example
def divideOneBy(x: Int): Try[Int] = Try { 1 / x}
divideOneBy(0) // Failure(java.lang.ArithmeticException: / by zero)
If what you have is a Try and you want to throw the exception when you have a Failure, then you can use pattern matching to do that:
val result = divideByOne(0)
result match {
case Failure(exception) => throw exception
case Success(_) => // What happens here?
}
The Neophyte's Guide to Scala has lots of useful information for people new to Scala (I found it invaluable when I was learning).
The Scala Try construct together with its flatMap does not work as I would expect or want it to. The TL;DR is that I want to do a series of operations that can fail in two ways: either by raising an exception, which should be promoted and caught higher up in the call stack, or by returning Failure, as the failure must logically be handled in different parts of the program.
I would expect something like this to do the trick:
def firstStepSucceeds(): Try[Int] = Try {
1
}
def secondStepThrows(input: Int) = {
throw new Exception("Exception thrown in second step")
}
// I expect this to propagate the exception thrown in secondStepThrows
firstStepSucceeds() flatMap (secondStepThrows _)
(Full Scastie with example)
However, in this case, the flatMap() call actually implicitly catches the uncaught exception thrown by secondStepThrows, which is not what I want (which is why I left out the Try block). Is there a way to get the same behaviour without the implicit exception-catching?
What happens in a Try should stay in a Try. Most Scala programmers would be very surprised if a function returning a Try also sometimes threw an exception.
The typical pattern if you want to handle exceptions in different places is to differentiate by the type of the exception. So
val partiallyRecoveredTry = originalTry.recover{
case _: SecondStepException => "second step had an exception"
}
// Further up the call stack
partiallyRecoveredTry.getOrElse("first step had an exception")
Try.flatMap() did not caught exceptions implicitely, it is the essence of Try.
When you use it, it is very explicit, and that's the goal.
I don't really understand what you want, but is something like that is possible for you ?
try {
val first = firstStepSucceeds()
val second = first.map(secondStepThrows).get
val third = secondStepFails(second)
// ...
}
catch {
case e: Exception => ???
}
I did some further experimentation, and what I ended up with was this reimplementation of Try as (the now right-biased and hence monadic) Either:
object CatchAll {
def apply[SomeType](block: => SomeType) = try { Right(block) }
catch { case e: Throwable => Left(e) }
}
def firstStepSucceeds() = CatchAll {
1
}
def firstStepFails() = CatchAll {
throw new Exception("First step failed")
}
def secondStepSucceeds(input: Int) = CatchAll {
input + 1
}
def secondStepFails(input: Int) = CatchAll {
throw new Exception("Second step failed in try block!")
}
def secondStepThrows(input: Int) = {
throw new Exception("Second step failed unexpectedly!")
}
firstStepSucceeds() flatMap (secondStepSucceeds _)
firstStepFails() flatMap (secondStepSucceeds _)
firstStepSucceeds() flatMap (secondStepFails _)
// This now throws an exception as expected
//firstStepSucceeds() flatMap (secondStepThrows _)
I have a method that may return Future - successful or failed or can even throw an exception. I can avoid this by putting try catch block on entire method and return Future all the time but i would like to avoid it for now. I have few problems with calling such method:
1) In caller code, if I use map I expect execution of a method and expect a Future or an exception which I tried to handle in following manner:
object ETLCoordinator {
private def getBusinessListFromModules(modulePaths: Iterable[File]) : Future[String] = {
implicit val ec = ExecutionContext.global
println("Inside getBusinessListFromModules..")
throw new java.lang.RuntimeException("failed to get businesses") //Exception is thrown before future was constructed
Future("ok")
}
def main(args: Array[String]) {
println("Inside Future Test..")
implicit val ec = ExecutionContext.global
val modulePaths = Iterable(new File("mdrqaint/MDR/QAINT/QAINTX"))
val fut1 = getBusinessListFromModules(modulePaths) //This is outside of try and which should be okay
try {
fut1.map { res =>
println("things after Successful fut1")
}.recover{
case t: Throwable => println("Failed future in fut1: "+ t.getMessage)
}
} catch {
case t: Throwable => println("Exception in fut1: "+ t.getMessage)
}
}
}
Output: ( No execution of recover or catch block above)
Inside Future Test..
Inside getBusinessListFromModules..
Exception in thread "main" java.lang.RuntimeException: failed to get businesses
But if I put val fut1 = getBusinessListFromModules(modulePaths) inside Try block then Exception is get caught in Catch block and I get output:
Inside Future Test..
Inside getBusinessListFromModules..
Exception in fut1: failed to get businesses
Why is this? I though Future execution happens upon calling some of its methods like map, flatmap, onSuccess, onComplete etc. In this case call to map is already inside Try block.
2) What is the better way to define and call such methods? Try/catch block in a caller or try/catch in method itself? or any other way. I tried wrapping the calling method in Future so I get Future[Future[String]] in caller. I was able to avoid all try-catch.
val fut1 = Future(getBusinessListFromModules(modulePaths))
//try {
fut1.map { res =>
res.map{ str =>
println("things after Successful fut1")
}.recover{
case t: Throwable => println("Failed in future of fut1: "+ t.getMessage)
}
println("things after Successful fut1 wrapper")
}.recover{
case t: Throwable => println("Failed to create future in fut1: "+ t.getMessage)
}
3) If there is another method inbetween which does delegation to getBusinessListFromModules but it itself is non-future method.
object ETLController {
private def getBusinessListFromModules(modulePaths: Iterable[File]) : Future[String] = {
implicit val ec = ExecutionContext.global
println("Inside getBusinessListFromModules..")
//throw new java.lang.RuntimeException("failed to get businesses")
Future("ok")
}
private def callGetBusList(modulePaths: Iterable[File]) : String = {
implicit val ec = ExecutionContext.global
val etlF = getBusinessListFromModules(modulePaths)
etlF onComplete {
case Success(itr) => {
println("Future getBusinessListFromModules success: "+ itr)
throw new java.lang.RuntimeException("RTE from callGetBusList")
}
case Failure(t) => {
println("Future getBusinessListFromModules throws an error")
}
}
"callGetBusList was a success"
}
def main(args: Array[String]) {
println("Inside Future Test..")
implicit val ec = ExecutionContext.global
val modulePaths = Iterable(new File("mdrqaint/MDR/QAINT/QAINTX"))
try {
val fut = Future(callGetBusList(modulePaths))
fut.map { res =>
println("successful future!")
}.recover{
case t: Throwable => println("Failed future: "+ t.getMessage)
}
} catch {
case t: Throwable => println("callGetBusList failed:" + t.getMessage)
}
}
}
Output: (no recover or catch block execution!)
Inside Future Test..
Inside getBusinessListFromModules..
Future getBusinessListFromModules success: ok
java.lang.RuntimeException: RTE from callGetBusList
at ..
successful future!
I even try double wrapping Future calls :
val fut = Future(Future(callGetBusList(modulePaths)))
fut.map { res =>
res.map { str =>
println("successful inner future! "+ str)
}.recover{
case t: Throwable => println("Failed inner future: "+ t.getMessage)
}
println("successful outer future!")
}.recover{
case t: Throwable => println("Failed outer future: "+ t.getMessage)
}
Output:
Future getBusinessListFromModules success: ok
java.lang.RuntimeException: RTE from callGetBusList
at
successful inner future! callGetBusList was a success
successful outer future!
I get "callGetBusList was a success" which seems like RuntimeException inside onComplete method got lost! How do I catch it in a final caller? What are the better practice to handle such future dependencies?
UPDATE:
based on #dk14 explanation, Opted to convert middle method to return Future and basically all methods to return some kind of Future and not a plain Exception.
object ETLController {
private def getBusinessListFromModules(modulePaths: Iterable[File]) : Future[String] = {
implicit val ec = ExecutionContext.global
println("Inside getBusinessListFromModules..")
Future {
Thread.sleep(2000)
throw new java.lang.RuntimeException("failed to get businesses")
"ok"
}
}
private def callGetBusList(modulePaths: Iterable[File]) : Future[String] = {
implicit val ec = ExecutionContext.global
val etlF = getBusinessListFromModules(modulePaths)
etlF map { itr =>
println("Future getBusinessListFromModules success: "+ itr)
throw new java.lang.RuntimeException("RTE from callGetBusList")
} recover {
case t: Throwable => {
println("Future callGetBusList throws an error: " + t.getMessage)
throw t
}
}
}
def main(args: Array[String]) {
println("Inside Future Test..")
implicit val ec = ExecutionContext.global
val modulePaths = Iterable(new File("mdrqaint/MDR/QAINT/QAINTX"))
val fut = callGetBusList(modulePaths)
fut.map { str =>
println("successful future! "+ str)
}.recover{
case t: Throwable => println("Failed future: "+ t.getMessage)
}
println("Active threads: " +Thread.activeCount())
sys.allThreads().foreach(t => t.join())
}
}
1) Futures are firing eagerly and they aren't referentially transparent.
Answers to the referenced question also contain some insights about Future's internal behavior, so I'd like to skip it here.
In order to manage side-effects concerning execution pools/queues/threads in a more predictable way, you could consider scalaz/monix/fs2 Task or iteratee/scalaz/cats Eval (more abstract lazy evaluation, and intended for sync stuff) + Cont (continuations are abstracting over subscriptions) as alternative. All are referentially transparent and start execution lazily "on-demand".
2) The best way is the one you don't like: to not throw exception outisde of Future context.
You might also consider flatMap to avoid Future[Future[T]]
3) Double wrapping Futures directly a-la Future(Future(...)) doesn't change anything. Your method is executed on val etlF = g... (in the same thread) no matter what it returns. Future("ok")'s content (lambda) is executed eagerly (with "small" unpredictable delay) on a different thread but [execution task is being submitted to the pool] still inside getBusinessListFromModules.
One workaround (not really recommended) is val etlF = Future(getBusinessListFromModules(...)).flatMap(identity) which would return you a future wrapping any exception coming directly from getBusinessListFromModules and indirectly from getBusinessListFromModules's internal Future.
It's better to refactor getBusinessListFromModules itself however, also introduce different exception types for different kinds of trouble (validation, sync vs async, so on) your method might get into.
P.S. There are ways to mix async and sync exception handling, but in practice it's hard to analyze and predict such mixed behavior (which you probably noticed already). And code gets ugly.
Is it possible to asynchronously catch all errors given a list of futures?
I was thinking of something like:
Future { throw Exception } zip Future { throw Exception } recover ...
But I only receive a Throwable in the recover part. Any idea?
zip's docs clearly say it prefers errors from this over those from that (my emphasis):
def zip[U](that: Future[U]): Future[(T, U)] Zips the values of this
and that future, and creates a new future holding the tuple of their
results.
If this future fails, the resulting future is failed with the
throwable stored in this. Otherwise, if that future fails, the
resulting future is failed with the throwable stored in that.
You could implement your own function to combine two futures into Future[(Try[T], Try[U])], like:
def zipToTry[T,U](a: Future[T], b: Future[U])
(implicit executor: ExecutionContext)
: Future[(Try[T], Try[U])] = {
val t: Try[T] = a.map { Success(_) }.recover { Failure(_) }
val u: Try[U] = b.map { Success(_) }.recover { Failure(_) }
t.zip(u)
}
(i haven't tested this, but you get the idea)
I am using some client library and had some code that ignored a specific exception using scala.util.control.Exception.ignoring:
ignoring(classOf[ReallyNotThatExceptionalException]) {
stuff()
}
Now that library changed to wrap all exceptionn in another exception class, which forced me to change my code to this:
try { stuff() }
catch {
case e:WrapperException if e.getCause != null && e.getCause.isInstanceOf[ReallyNotThatExceptionalException] => { }
}
So what I'm looking for a more readable way to express "catch exceptions that are caused by".
0__'s answer is good, but it would be better if you were not forced to write a specific object (CausedByFoo) for each potential exception.
As it happens, there is not much to change to end up with a generic CausedBy helper object:
class Foo(message: String) extends Exception(message)
class Bar(cause: Throwable) extends Exception(cause)
object CausedBy {
def unapply(e: Throwable): Option[Throwable] = Option(e.getCause)
}
def test(block: => Unit): String =
try {
block
"ok"
} catch {
case CausedBy(ex: Foo) => "not ok: " + ex.getMessage
}
test(println("hello"))
test(println("hello".toInt)) // uncaught exception
test(throw new Bar(new Foo("Ooops, foo error!"))) // caught
As should be obvious, you can use CausedBy with any exception (by example by doing case CausedBy(ex: Baz).
You can even nest it to handle an exception caused by an exception caused by an exception (by doing something like case CausedBy(CausedBy(ex: Foo))
The catch block can handle any regular pattern match, so
class Foo extends Exception
class Bar(cause: Exception) extends Exception(cause)
object CausedByFoo {
def unapply(e: Exception): Boolean = e.getCause match {
case _: Foo => true
case _ => false
}
}
def test(block: => Unit): String =
try {
block
"ok"
} catch {
case CausedByFoo() => "not ok"
}
test(println("hello"))
test(println("hello".toInt)) // uncaught exception
test(throw new Bar(new Foo)) // caught
I tweaked RĂ©gis' answer to allow matching against both outer and inner exception type.
trait CausedBy[T <: Throwable] {
def unapply(e: T): Option[Throwable] = Option(e.getCause)
}
object IOExceptionCausedBy extends CausedBy[IOException]
Then you can match like this:
try {
// do some DB thing
} catch {
case IOExceptionCausedBy(exception: PSQLException) =>
// handle PSQL exception
}