Why does my Scala future not propagate an Error? - scala

This code works as expected:
it("should propagate exceptions") {
intercept[RuntimeException] {
val future = Future { Thread.sleep(10); sys.error("whoops"); 22 }
Await.result(future, Duration.Inf)
}.getMessage should equal ("whoops")
}
But this doesn't:
it("should propagate errors") {
intercept[StackOverflowError] {
val future = Future { Thread.sleep(10); throw new StackOverflowError("dang"); 22 }
Await.result(future, Duration.Inf)
}.getMessage should equal ("dang")
}
The future in this second test never returns. Why doesn't an Error subclass (as opposed to an Exception subclass) terminate my future? How should I handle Errors?
EDIT: This is possibly related, but not identical, to Why does Scala Try not catching java.lang.StackOverflowError?. I'm not using Try here. The core issue is that the Future never returns at all; I can't catch any error from it because it just hangs.

The reporter facility is for catastrophes, which just hooks into the thread's UncaughtExceptionHandler, but it looks like it works out of the box with just the default thread factory:
scala 2.13.0-M5> import concurrent._,java.util.concurrent.Executors
import concurrent._
import java.util.concurrent.Executors
scala 2.13.0-M5> val ec = ExecutionContext.fromExecutor(null, e => println(s"Handle: $e"))
ec: scala.concurrent.ExecutionContextExecutor = scala.concurrent.impl.ExecutionContextImpl$$anon$3#5e7c141d[Running, parallelism = 4, size = 0, active = 0, running = 0, steals = 0, tasks = 0, submissions = 0]
scala 2.13.0-M5> val f = Future[Int](throw new NullPointerException)(ec)
f: scala.concurrent.Future[Int] = Future(<not completed>)
scala 2.13.0-M5> f
res0: scala.concurrent.Future[Int] = Future(Failure(java.lang.NullPointerException))
scala 2.13.0-M5> val f = Future[Int](throw new StackOverflowError)(ec)
Handle: java.lang.StackOverflowError
f: scala.concurrent.Future[Int] = Future(<not completed>)
whereas
scala 2.13.0-M5> val ec = ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor, e => println(s"Handle: $e"))
ec: scala.concurrent.ExecutionContextExecutor = scala.concurrent.impl.ExecutionContextImpl#317a118b
scala 2.13.0-M5> val f = Future[Int](throw new StackOverflowError)(ec)
f: scala.concurrent.Future[Int] = Future(<not completed>)
Exception in thread "pool-1-thread-1" java.lang.StackOverflowError
at $line14.$read$$iw$$iw$$iw$$iw$.$anonfun$f$1(<console>:1)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:261)
at scala.util.Success.map(Try.scala:209)
at scala.concurrent.impl.Promise$Transformation.doMap(Promise.scala:420)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:402)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
You could construct a rig that registers a future when it runs, and a safe await that knows when threads have blown up. Maybe you want to retry an algorithm with a lower max recursion depth, for example.

As pointed out in the comments, this is a duplicate of Why does Scala Try not catching java.lang.StackOverflowError?
According to Scala documentation.
Note: only non-fatal exceptions are caught by the combinators on Try (see >scala.util.control.NonFatal). Serious system errors, on the other hand, will be >thrown.
No Throwable -> Errors are catched by Try
Also to answer your question about how error handling is usually done.
In Scala you can use try / catch for code that can cause exceptions (very similar to Java):
try {
// ... Your dangerous code in here
} catch {
case ioe: IOException => ... //
case e: Exception => ...
}
And you should always have the more specific exceptions first.
The code you provided would look something like this:
https://scastie.scala-lang.org/2DJXJ6ESS9ySJZSwSodmZg
Also I tried out your code and it definitely produces the StackOverFlowerror.
But it can't catch it properly like the above mentioned link explains.

Related

ZIO Fiber orElse generate exception messages

I want to use the combinator orElse on ZIO Fibers.
From docs:
If the first fiber succeeds, the composed fiber will succeed with its result; otherwise, the composed fiber will complete with the exit value of the second fiber (whether success or failure).
import zio._
import zio.console._
object MyApp extends App {
def f1 :Task[Int] = IO.fail(new Exception("f1 fail"))
def f2 :Task[Int] = IO.succeed(2)
val myAppLogic =
for {
f1f <- f1.fork
f2f <- f2.fork
ff = f1f.orElse(f2f)
r <- ff.join
_ <- putStrLn(s"Result is [$r]")
} yield ()
def run(args: List[String]) =
myAppLogic.fold(_ => 1, _ => 0)
}
I run it with sbt in console. And output:
[info] Running MyApp
Fiber failed.
A checked error was not handled.
java.lang.Exception: f1 fail
at MyApp$.f1(MyApp.scala:6)
at MyApp$.<init>(MyApp.scala:11)
at MyApp$.<clinit>(MyApp.scala)
at MyApp.main(MyApp.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Result is [2]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sbt.Run.invokeMain(Run.scala:93)
at sbt.Run.run0(Run.scala:87)
at sbt.Run.execute$1(Run.scala:65)
at sbt.Run.$anonfun$run$4(Run.scala:77)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at sbt.util.InterfaceUtil$$anon$1.get(InterfaceUtil.scala:10)
at sbt.TrapExit$App.run(TrapExit.scala:252)
at java.lang.Thread.run(Thread.java:748)
Fiber:Id(1574829590403,2) was supposed to continue to: <empty trace>
Fiber:Id(1574829590403,2) ZIO Execution trace: <empty trace>
Fiber:Id(1574829590403,2) was spawned by:
Fiber:Id(1574829590397,1) was supposed to continue to:
a future continuation at MyApp$.myAppLogic(MyApp.scala:12)
a future continuation at MyApp$.run(MyApp.scala:19)
Fiber:Id(1574829590397,1) ZIO Execution trace: <empty trace>
Fiber:Id(1574829590397,1) was spawned by:
Fiber:Id(1574829590379,0) was supposed to continue to:
a future continuation at zio.App.main(App.scala:57)
a future continuation at zio.App.main(App.scala:56)
[Fiber:Id(1574829590379,0) ZIO Execution trace: <empty trace>
I see the result of seconds Fiber, is Result is [2]
But why it output these unnecessary exception/warning messages?
By default a fiber failure warning is generated when a fiber that is not joined back fails so that errors do not get lost. But as you correctly note in some cases this is not necessary as the error is handled internally by the program logic, in this case by the orElse combinator. We have been working through a couple of other cases of spurious warnings being generated and I just opened a ticket for this one here. I expect we will have this resolved in the next release.
This happens because the default instance of Platform being created by zio.App has a default which reports uninterrupted, failed fibers to the console:
def reportFailure(cause: Cause[_]): Unit =
if (!cause.interrupted)
System.err.println(cause.prettyPrint)
To avoid this, you can provide your own Platform instance which doesn't do so:
import zio._
import zio.console._
import zio.internal.{Platform, PlatformLive}
object MyApp extends App {
override val Platform: Platform = PlatformLive.Default.withReportFailure(_ => ())
def f1: Task[Int] = IO.fail(new Exception("f1 fail"))
def f2: Task[Int] = IO.succeed(2)
val myAppLogic =
for {
f1f <- f1.fork
f2f <- f2.fork
ff = f1f.orElse(f2f)
r <- ff.join
_ <- putStrLn(s"Result is [$r]")
} yield ()
def run(args: List[String]) =
myAppLogic.fold(_ => 1, _ => 0)
}
Which yields:
Result is [2]
As #Adam Fraser noted, this will probably get fixed in a nearby release.
Edit:
Should be fixed after https://github.com/zio/zio/pull/2339 was merged

Exceptions Thrown by Await#result

Given the following code:
import spray.http._
import spray.client.pipelining._
import scala.concurrent.Future
implicit val system = ActorSystem()
import system.dispatcher // execution context for futures
val pipeline: HttpRequest => Future[HttpResponse] = sendReceive
val response: Future[HttpResponse] = pipeline(Get("http://spray.io/"))
The following pseudo-code function waits 10 seconds, returning "GOOD" if the HttpResponse returned, or "BAD" on an Await#result exception (see docs.
import scala.concurrent.Await
import scala.concurrent.duration._
def f(fut: Future[HttpResponse]): String = {
try {
val result = Await.result(fut, 10.seconds)
"GOOD"
}
catch e # (_: InterruptedException | _: IllegalArgumentException
| _: TimeoutException ) => "BAD"
}
In my catch, is it only necessary to catch exception thrown by Await#result? In other words, am I not catching any possible exceptions here?
The Await.result itself can throw the exceptions you caught, however, if the future it awaits does not complete successfully, it forwards the exception contained by the future. You might want to read the Blocking section from here: Futures and Promises.
So yes, there may be exceptions you aren't catching, anything that can result from the failed computation of a HttpResponse.
Blocking in real code is usually bad and should be done only for testing purposes, but if you really need to, I would recommend to wrap the Await in a scala.util.Try so you could manipulate it elegantly later and also keep the information of when and why it failed.

Slick 3.0-RC3 fails with java.util.concurrent.RejectedExecutionException

I'm trying to get familiar with Slick 3.0 and Futures (using Scala 2.11.6). I use simple code based on Slick's Multi-DB Cake Pattern example. Why does the following code terminate with an exception and how to fix it?
import scala.concurrent.Await
import scala.concurrent.duration._
import slick.jdbc.JdbcBackend.Database
import scala.concurrent.ExecutionContext.Implicits.global
class Dispatcher(db: Database, dal: DAL) {
import dal.driver.api._
def init() = {
db.run(dal.create)
try db.run(dal.stuffTable += Stuff(23,"hi"))
finally db.close
val x = {
try db.run(dal.stuffTable.filter(_.serial === 23).result)
finally db.close
}
// This crashes:
val result = Await.result(x, 2 seconds)
}
}
Execution fails with:
java.util.concurrent.RejectedExecutionException: Task slick.backend.DatabaseComponent$DatabaseDef$$anon$2#5c73f637 rejected from java.util.concurrent.ThreadPoolExecutor#4129c44c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
at scala.concurrent.impl.ExecutionContextImpl$$anon$1.execute(ExecutionContextImpl.scala:136)
at slick.backend.DatabaseComponent$DatabaseDef$class.runSynchronousDatabaseAction(DatabaseComponent.scala:224)
at slick.jdbc.JdbcBackend$DatabaseDef.runSynchronousDatabaseAction(JdbcBackend.scala:38)
at slick.backend.DatabaseComponent$DatabaseDef$class.runInContext(DatabaseComponent.scala:201)
at slick.jdbc.JdbcBackend$DatabaseDef.runInContext(JdbcBackend.scala:38)
at slick.backend.DatabaseComponent$DatabaseDef$class.runInternal(DatabaseComponent.scala:75)
at slick.jdbc.JdbcBackend$DatabaseDef.runInternal(JdbcBackend.scala:38)
at slick.backend.DatabaseComponent$DatabaseDef$class.run(DatabaseComponent.scala:72)
at slick.jdbc.JdbcBackend$DatabaseDef.run(JdbcBackend.scala:38)
at Dispatcher.init(Dispatcher.scala:15)
at SlickDemo$.main(SlickDemo.scala:16)
at SlickDemo.main(SlickDemo.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
I think that something is not correct in what you are trying to do: Slick's run method doesn't return Unit and doesn't fail with an exception - as it used to in previous versions. run now returns a Future, so if you want to run actions in sequence you need to flatMap the steps, or use a for-comprehension:
def init() = {
val = results for {
_ <- db.run(dal.create)
_ <- db.run(dal.stuffTable += Stuff(23, "hi"))
r <- db.run(dal.stuffTable.filter(_.serial === 23).result)
} yield r
}
I am not sure that you really need to use db.close that way: that is actually what may be causing the error (i.e. the db is closed in concurrence with the future that runs the actual queries so the execution can't happen).
If you want to handle errors use Future's capabilities, e.g.:
result.onFailure { case NonFatal(ex) => // do something with the exception }

Why is scala Await.result timing out in repl when passed the same future twice?

I found this a bit confusing. I thought futures in scala are inmutable containers that once set, always return the same value.
So I have a future:
val y = future {Thread.sleep(1000); 1};
Now when i immediately (before the future resolves) pass it to Await.result block twice:
Await.result(for (r1 <- y; r2 <- y) yield (r1, r2), 60 seconds)
I get a TimetoutException.
However if i do it once the future is resolved, everything works fine and returns (1,1) as expected.
What is the reason of this behavior?
Edit:
I'm using implicit ExecutionContext.Implicits.global and scala.concurrent # scala 2.10.3
Edit2:
If I create another future instance doing the same thing and do Await.result on them both it doesn't block.
This appears to be an artifact of executing it in the REPL.
You can even reproduce it using 2 separate future instances, without any call to Thread.sleep,
and using only pre-fulfilled futures (which means that there is not even any future thread involed).
Yes, seriously:
import scala.concurrent._
import duration._
import ExecutionContext.Implicits.global
val x = Future.successful(1)
val y = Future.successful(2)
Await.result(x.flatMap{_ => y.map{ _ => 0 } }, Duration(10, SECONDS)) // triggers a timeout
Interestingly this does not trigger any timeout if you change the last line to this:
Await.result(x.flatMap{_ => Future.successful(2).map{ _ => 0 } }, Duration(10, SECONDS))
It seems that the culprint is that your whole code snippet, when evaled in the REPL, is actually wrapped in an object.
This means that x and y here are actually members of the object, rather than local variables
More importantly, the call to Await is now part of the constructor of this wrapper object.
For some reason that I have yet to investigate, it seems to be the fact that the call to Await is done in a constructor that triggers the blocking
(you can verify it easily by wrapping this call in a dummy class and instantiating it).
The reason you don't want to start threads from class initializers is that cycles in classloading will block. A Scala object is just an instance created in a static block. The closure's reference to y is a cycle. In 2.11, use -Yrepl-class-based to help it work.
Lets examine the stack trace for the hang when the closure class needs X.z:
apm#mara:~$ goof
Welcome to Scala version 2.11.0-RC3 (OpenJDK 64-Bit Server VM, Java 1.7.0_25).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import scala.concurrent._
import scala.concurrent._
scala> import duration._
import duration._
scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global
scala> object X { val y = Future { 9 } ; val z = Future { 7 } ; val r = Await.result(for (a <- y; b <- z) yield (a+b), 5.seconds) }
defined object X
scala> X.r
java.lang.NoClassDefFoundError: Could not initialize class $line15.$read$$iw$$iw$$iw$$iw$$iw$$iw$X$
at $line15.$read$$iw$$iw$$iw$$iw$$iw$$iw$X$$anonfun$3.apply(<console>:14)
at $line15.$read$$iw$$iw$$iw$$iw$$iw$$iw$X$$anonfun$3.apply(<console>:14)
at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:251)
at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:249)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
java.util.concurrent.TimeoutException: Futures timed out after [5 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:111)
at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.result(package.scala:111)
... 34 elided
And just to show it working:
apm#mara:~$ goof -Yrepl-class-based
Welcome to Scala version 2.11.0-RC3 (OpenJDK 64-Bit Server VM, Java 1.7.0_25).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import scala.concurrent._
import scala.concurrent._
scala> import duration._
import duration._
scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global
scala> object X { val y = Future { 9 } ; val z = Future { 7 } ; val r = Await.result(for (a <- y; b <- z) yield (a+b), 5.seconds) }
defined object X
scala> X.r
res0: Int = 16

Future/Promise without waiting for result possible? Return Unit

I have a question about scala's Future.
Currently I have a program that runs through a directory and checks if there is a document.
If there is a file the program should convert these file into a ".pdf"
My Code looks like this (It's pseudocode):
for(file <- directory) {
if(timestamp > filetimestamp) {
Future {
// do a convert job that returns UNIT
}
}
}
Is this valid code or do I need to wait for the return value?
Are there any other alternative's that are as lightweight as Futures?
To convert inside a Future, simply use map and flatMap. The actual operations are performed asynchronously when the callbacks are complete, but they are type safe.
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
for(file <- directory) {
if(timestamp > filetimestamp) {
val future = Future {
// do a convert job that returns UNIT
} map {
file => // whatever you want.
}
}
Warning!
If any Future throws a "NonFatal" error, it will be swallowed. This is a serious gotcha when using Future[Unit]: if no code ever evaluates the future, errors can disappear down a black hole. (It affects any Future[_], but if you are returning a value, you normally do something with it, so the error is discovered.)
scala> import scala.concurrent.ExecutionContext.Implicits.global
scala.concurrent.Future { throw new IllegalArgumentException("foo") }
scala> res16: scala.concurrent.Future[Nothing] = scala.concurrent.impl.Promise$DefaultPromise#64dd3f78
scala> import scala.concurrent.ExecutionContext.Implicits.global
scala.concurrent.Future { throw new IllegalArgumentException("foo"); 42 }
scala> res11: scala.concurrent.Future[Int] = scala.concurrent.impl.Promise$DefaultPromise#65c8295b
An alternative which accomplishes the same thing, but does not hide the error:
scala> val context = scala.concurrent.ExecutionContext.Implicits.global
context.execute(new Runnable {
def run() = throw new IllegalArgumentException("foo")
})
context: scala.concurrent.ExecutionContextExecutor = scala.concurrent.impl.ExecutionContextImpl#1fff4cac
scala> | | java.lang.IllegalArgumentException: foo
at $line48.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$anon$1.run(<console>:34)
at $line48.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$anon$1.run(<console>:33)
at scala.concurrent.impl.ExecutionContextImpl$$anon$3.exec(ExecutionContextImpl.scala:107)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Yes, that is valid. All the Futures you create will be run, and their return values will be discarded and garbage collected.