I'm converting Future code to IO. I have code similar to this
def doSomething: Future[Foo] = {
Future {
//some code the result of which we don't care about
}
Future {
//Foo
}
}
And then at the end of the program, I doSomething.unsafeRunSync. How do I convert these Futures to IOs while maintaining the fire-and-forget functionality of the first Future? In using IO's async API, I am worried about accidentally blocking the thread when I later call unsafeRunSync on doSomething.
A solution that uses only cats-effect could use IO.start. This, combined with the fact that you will then never join the resulting Fiber, will look something like this:
import cats.effect._
import cats.implicits._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
object ExampleApp extends App{
val fireAndForget =
IO(println("Side effect pre-sleep")) *>
IO.sleep(2.seconds) *>
IO(println("Side effect post-sleep"))
val toBeUsed = IO{
println("Inside second one")
42
}
val result = for {
fiber <- IO.shift *> fireAndForget.start
res <- toBeUsed.handleErrorWith { error =>
// This is just in case you 'toBeUsed' can actually fail,
// and you might want to cancel the original side-effecting IO
fiber.cancel *> IO.raiseError(error) }
} yield res
println(result.unsafeRunSync())
println("Waiting 3 seconds...")
IO.sleep(3.seconds).unsafeRunSync()
println("Done")
}
This will print (most of the times) something similar to:
Side effect pre-sleep
Inside second one
42 // Up until here, will be printed right away
Waiting 3 seconds... // It will then be waiting a while
Side effect post-sleep // ...at which point the side effecting code terminates
Done
Finally, here are more details about Fiber and IO.shift
I believe that you need to wrap the first Future in such a way that it completes immediately. We ignore exeptions, or catch them however, but they are contained within its own thread. The parameter cb is the promise that needs to complete; so we short-circuit the completion by providing a value immediately.
def firstFuture(implicit ec: ExecutionContext): IO[Unit] = {
IO.async[Unit] { cb =>
ec.execute(() => {
try {
//some code the result of which we don't care about
} catch {
}
})
cb(Right(()))
}
}
In the for-comprehension, the firstFuture will complete immediately even though its thread will have a long-running task active on it.
def doSomething(implicit ec: ExecutionContext): IO[Foo] = {
for {
_ <- firstFuture
IO.async[Foo] { fb =>
// Foo
}
}
}
I'm implementing an algorithm that can be easily parallelized but can't figure out how to create a proper number of futures and how to abort early. At the moment the outline of the code is along these lines
def solve: Boolean = {
var result = false
while(!result && i < iterations) {
val futures = (1 to threads) map { _ => solveIter(geInitialValues()) }
val loopResult = Future.fold(futures)(false)((acc, r) => acc || r )
result = Await.result(loopResult, Duration.Inf)
i+=1
}
}
def solveIter(initialValues: Values): Future[Boolean] = Future {
/* Takes a lot of time */
}
The obvious problem is explicitly set level of parallelism that may or may not be suitable for current execution context. If all futures are created at once how to make Future.fold to abort early?
You can't cancel a Future, because Futures are read-only. But you can use a Promise which is "the write part from a Future".
Example code:
this code timeouts after 5 seconds because the futures are not done (solveIter never completes)
to complete a promise, remove comment which adds a completed promise to the 'promises' - the other futures will be cancel
remove 'promises.foreach(_.trySuccess(false))' and you get the timeout again, because the other futures doesn't getting cancel
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
import scala.concurrent.{Await, Future, Promise}
import scala.util.Try
// create a bunch of promises
val promises = ((1 to 10) map { _ =>
val p = Promise[Boolean]()
p.completeWith(solveIter())
p
}) // :+ Promise().success(true)
// ^^ REMOVE THIS COMMENT TO ADD A PROMISE WHICH COMPLETES
// get the futures from the promises
val futures = promises.map(_.future)
// loop over all futures
futures.foreach(oneFuture =>
// register callback when future is done
oneFuture.foreach{
case true =>
println("future with 'true' result found")
// stop others
promises.foreach(_.trySuccess(false))
case _ => // future completes with false
})
// wait at most 5 seconds till all futures are done
Try(Await.ready(Future.sequence(futures), 5.seconds)).recover { case _ =>
println("TIMEOUT")
}
def solveIter(): Future[Boolean] = Future {
/* Takes a VERY VERY VERY .... lot of time */
Try(Await.ready(Promise().future, Duration.Inf))
false
}
I'm confused about how Scala Future works. Can someone explain what's wrong with code:
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
val numbers = List(1,2,3,4,5,6,7,8,9,10)
def showNumbers() = {
numbers
}
val futueNumber = future {
showNumbers.filter((a: Int) => a % 2 == 0)
}
futueNumber onSuccess {
case resultList => resultList
}
futueNumber
I expect to receive a result:
List(1,2,3,4,5,6,7,8,9,10)
But REPL outputs:
res1: scala.concurrent.Future[List[Int]] = scala.concurrent.impl.Promise$DefaultPromise#3644febc
Please, give me a short hint, how to fix this
The simple answer is that you need to await the future for you example to work:
val result = Await.result(futureNumber, 1.second)
However, the purpose of future's is not to block on asynchronous work, as Await does, but instead to use the result of the future when becomes available.
The key to using future is understanding that it does not actually hold the value of your computation, but rather is promise that at some point in the future, it will provide that value.
The usual use case is a request coming into a server that requires some work, which may take some time and itself wait on IO, like this:
def receiveMsg(msg: String, caller: Caller) = {
doSomeAsyncWork(msg).map(response => caller.send(response)
}
I.e. you are called with an object that has a send method that you want to call when doSomeAsyncWork is done, but you don't want to block the current thread until that future completes. Instead you map the future, which means that response => caller.send(response) will be called for you when doSomeAsyncWork is done.
futureNumber is of type Future[List[Int]], since you defined it as such in
val futureNumber = future {
showNumbers.filter((a: Int) => a % 2 == 0)
}
Therefore, when you print futureNumber, that is what gets printed. The variable does not get reassigned to the future value! If you want to print the value of the future, you need to do it inside the onSucess callback:
futureNumber onSuccess {
case resultList => println(resultList)
}
Alternatively, you can block waiting for the future with
import scala.concurrent.ExecutionContext.Implicits.global
import duration._
val theList = scala.concurrent.Await.result(futureNumber, 10 seconds)
println(theList)
Although this might not work if done in the REPL.
The whole point of the Future type is that it acts as a marker that alerts you to the presence of concurrency and should help you handle it better.
there is an aspect of futures that I do not exactly understand from the official tutorial ref. http://docs.scala-lang.org/overviews/core/futures.html
Do futures in scala have a built in time-out mechanism of some kind? Let's say the example below was a 5 gigabyte text file... does the implied scope of "Implicits.global" eventually cause onFailure to fire in a non-blocking way or can that be defined? And without a default time-out of some kind, wouldn't that imply it's possible neither success nor failure would ever fire?
import scala.concurrent._
import ExecutionContext.Implicits.global
val firstOccurence: Future[Int] = future {
val source = scala.io.Source.fromFile("myText.txt")
source.toSeq.indexOfSlice("myKeyword")
}
firstOccurence onSuccess {
case idx => println("The keyword first appears at position: " + idx)
}
firstOccurence onFailure {
case t => println("Could not process file: " + t.getMessage)
}
You only get timeout behavior when you use blocking to get the results of the Future. If you want to use the non-blocking callbacks onComplete, onSuccess or onFailure, then you would have to roll your own timeout handling. Akka has built in timeout handling for request/response (?) messaging between actors, but not sure if you want to start using Akka. FWIW, in Akka, for timeout handling, they compose two Futures together via Future.firstCompletedOf, one which represents the actual async task and one that represents the timeout. If the timeout timer (via a HashedWheelTimer) pops first, you get a failure on the async callback.
A very simplified example of rolling your own might go something like this. First, an object for scheduling timeouts:
import org.jboss.netty.util.{HashedWheelTimer, TimerTask, Timeout}
import java.util.concurrent.TimeUnit
import scala.concurrent.duration.Duration
import scala.concurrent.Promise
import java.util.concurrent.TimeoutException
object TimeoutScheduler{
val timer = new HashedWheelTimer(10, TimeUnit.MILLISECONDS)
def scheduleTimeout(promise:Promise[_], after:Duration) = {
timer.newTimeout(new TimerTask{
def run(timeout:Timeout){
promise.failure(new TimeoutException("Operation timed out after " + after.toMillis + " millis"))
}
}, after.toNanos, TimeUnit.NANOSECONDS)
}
}
Then a function to take a Future and add timeout behavior to it:
import scala.concurrent.{Future, ExecutionContext, Promise}
import scala.concurrent.duration.Duration
def withTimeout[T](fut:Future[T])(implicit ec:ExecutionContext, after:Duration) = {
val prom = Promise[T]()
val timeout = TimeoutScheduler.scheduleTimeout(prom, after)
val combinedFut = Future.firstCompletedOf(List(fut, prom.future))
fut onComplete{case result => timeout.cancel()}
combinedFut
}
Note that the HashedWheelTimer I am using here is from Netty.
All of these answers require additional dependencies. I decided to write a version using java.util.Timer which is an efficient way to run a function in the future, in this case to trigger a timeout.
Blog post with more details here
Using this with Scala's Promise, we can make a Future with timeout as follows:
package justinhj.concurrency
import java.util.concurrent.TimeoutException
import java.util.{Timer, TimerTask}
import scala.concurrent.duration.FiniteDuration
import scala.concurrent.{ExecutionContext, Future, Promise}
import scala.language.postfixOps
object FutureUtil {
// All Future's that use futureWithTimeout will use the same Timer object
// it is thread safe and scales to thousands of active timers
// The true parameter ensures that timeout timers are daemon threads and do not stop
// the program from shutting down
val timer: Timer = new Timer(true)
/**
* Returns the result of the provided future within the given time or a timeout exception, whichever is first
* This uses Java Timer which runs a single thread to handle all futureWithTimeouts and does not block like a
* Thread.sleep would
* #param future Caller passes a future to execute
* #param timeout Time before we return a Timeout exception instead of future's outcome
* #return Future[T]
*/
def futureWithTimeout[T](future : Future[T], timeout : FiniteDuration)(implicit ec: ExecutionContext): Future[T] = {
// Promise will be fulfilled with either the callers Future or the timer task if it times out
val p = Promise[T]
// and a Timer task to handle timing out
val timerTask = new TimerTask() {
def run() : Unit = {
p.tryFailure(new TimeoutException())
}
}
// Set the timeout to check in the future
timer.schedule(timerTask, timeout.toMillis)
future.map {
a =>
if(p.trySuccess(a)) {
timerTask.cancel()
}
}
.recover {
case e: Exception =>
if(p.tryFailure(e)) {
timerTask.cancel()
}
}
p.future
}
}
I've just created a TimeoutFuture class for a coworker:
TimeoutFuture
package model
import scala.concurrent._
import scala.concurrent.duration._
import play.libs.Akka
import play.api.libs.concurrent.Execution.Implicits._
object TimeoutFuture {
def apply[A](timeout: FiniteDuration)(block: => A): Future[A] = {
val prom = promise[A]
// timeout logic
Akka.system.scheduler.scheduleOnce(timeout) {
prom tryFailure new java.util.concurrent.TimeoutException
}
// business logic
Future {
prom success block
}
prom.future
}
}
Usage
val future = TimeoutFuture(10 seconds) {
// do stuff here
}
future onComplete {
case Success(stuff) => // use "stuff"
case Failure(exception) => // catch exception (either TimeoutException or an exception inside the given block)
}
Notes:
Assumes Play! framework (but it's easy enough to adapt)
Every piece of code runs in the same ExecutionContext which may not be ideal.
Play framework contains Promise.timeout so you can write code like following
private def get(): Future[Option[Boolean]] = {
val timeoutFuture = Promise.timeout(None, Duration("1s"))
val mayBeHaveData = Future{
// do something
Some(true)
}
// if timeout occurred then None will be result of method
Future.firstCompletedOf(List(mayBeHaveData, timeoutFuture))
}
I'm quite surprise this is not standard in Scala. My versions is short and has no dependencies
import scala.concurrent.Future
sealed class TimeoutException extends RuntimeException
object FutureTimeout {
import scala.concurrent.ExecutionContext.Implicits.global
implicit class FutureTimeoutLike[T](f: Future[T]) {
def withTimeout(ms: Long): Future[T] = Future.firstCompletedOf(List(f, Future {
Thread.sleep(ms)
throw new TimeoutException
}))
lazy val withTimeout: Future[T] = withTimeout(2000) // default 2s timeout
}
}
Usage example
import FutureTimeout._
Future { /* do smth */ } withTimeout
If you want the writer (promise holder) to be the one who controls the timeout logic, use akka.pattern.after, in the following way:
val timeout = akka.pattern.after(10 seconds, system.scheduler)(Future.failed(new TimeoutException(s"timed out during...")))
Future.firstCompletedOf(Seq(promiseRef.future, timeout))
This way, if your promise completion logic never takes place, your caller's future will still be completed at some point with a failure.
You can specify the timeout when you wait on the future:
For scala.concurrent.Future, the result method lets you specify a timeout.
For scala.actors.Future, Futures.awaitAll lets you specify a timeout.
I do not think there is a timeout built-in the execution of a Future.
Nobody's mentioned akka-streams, yet. The flows have an easy completionTimeout method, and applying that on a single-source stream works like a Future.
But, akka-streams also does cancellation so it can actually end the source from running, i.e. it signals the timeout to the source.
Monix Task has timeout support
import monix.execution.Scheduler.Implicits.global
import monix.eval._
import scala.concurrent.duration._
import scala.concurrent.TimeoutException
val source = Task("Hello!").delayExecution(10.seconds)
// Triggers error if the source does not complete in 3 seconds after runOnComplete
val timedOut = source.timeout(3.seconds)
timedOut.runOnComplete(r => println(r))
//=> Failure(TimeoutException)
This version works without using any external timer (just Await.result)
import scala.concurrent._
import scala.concurrent.duration.FiniteDuration
object TimeoutFuture {
def apply[A](
timeout: FiniteDuration
)(block: => A)(implicit executor: ExecutionContext): Future[A] =
try {
Future { Await.result(Future { block }, timeout) }
} catch {
case _: TimeoutException => Future.failed(new TimeoutException(s"Timed out after ${timeout.toString}"))
}
}
I'm using this version (based on Play example above) which uses Akka system dispatcher:
object TimeoutFuture {
def apply[A](system: ActorSystem, timeout: FiniteDuration)(block: => A): Future[A] = {
implicit val executionContext = system.dispatcher
val prom = Promise[A]
// timeout logic
system.scheduler.scheduleOnce(timeout) {
prom tryFailure new java.util.concurrent.TimeoutException
}
// business logic
Future {
try {
prom success block
} catch {
case t: Throwable => prom tryFailure t
}
}
prom.future
}
}
The simplest way to specify timeout on Future IMO is scala's built in mechanism using scala.concurrent.Await.ready This will throw a TimeoutException if the Future takes longer than the specified timeout. Otherwise, it will return the Future itself.
Here is a simple contrived example
import scala.concurrent.ExecutionContext.Implicits._
import scala.concurrent.duration._
val f1: Future[Int] = Future {
Thread.sleep(1100)
5
}
val fDoesntTimeout: Future[Int] = Await.ready(f1, 2000 milliseconds)
val f: Future[Int] = Future {
Thread.sleep(1100)
5
}
val fTimesOut: Future[Int] = Await.ready(f, 100 milliseconds)
You can wait for a future to finish by making use of Await.
import scala.concurrent.duration._
import scala.concurrent.{Await, Future}
val meaningOfLife: Int = Await.result(Future(42), 1.nano)
println (meaningOfLife)
The above prints 42
You may need an implicit ExecutionContext available in which case, just add:
import scala.concurrent.ExecutionContext.Implicits.global
Another way to do it is to use Coeval from monix. This method does not work in all situations, and you can read all about it here.
The basic idea is that sometimes a future does not really take any time and is returning the result of a synchronous function call or value, so this future can be evaluated on the current thread. This is also useful for testing and mocking futures. Also you don't have to specify a timeout which is expected, but still nice to not have to worry about that.
You start by transforming the future into a Task and wrap that task in a Coeval then cross your fingers as you wait to see what you get. This is a very simple example to show how it works:
You need an implicit Scheduler to be able to use it:
import monix.execution.Scheduler.Implicits.global
Coeval(Task.fromFuture(Future (42)).runSyncStep).value() match {
case Right(v) => println(v)
case Left(task) => println("Task did not finish")
}
The above completes and prints 42 to the console.
Coeval(Task.fromFuture(Future {
scala.concurrent.blocking {
42
}
}).runSyncStep).value() match {
case Right(v) => println(v)
case Left(task) => println("Task did not finish")
}
This example prints Task did not finish:
You can simply run the future to completion without giving any timeout interval by setting the timeout to infinite as below:
**import scala.concurrent.duration._
Await.result(run(executionContext), Duration.Inf)**
run function can be as below :
def run(implicit ec: ExecutionContext) = {
val list = Seq(
Future { println("start 1"); Thread.sleep(1000); println("stop 1")},
Future { println("start 2"); Thread.sleep(2000); println("stop 2")},
Future { println("start 3"); Thread.sleep(3000); println("stop 3")},
Future { println("start 4"); Thread.sleep(4000); println("stop 4")},
Future { println("start 5"); Thread.sleep(5000); println("stop 5")}
)
Future.sequence(list)
}
In Scala 2.10, along with the new Future/Promise API, they introduced a Duration and Deadline utilities (as described here). I looked around but couldn't find anything that comes with the scala standard library, to do something like:
val deadline = 5 seconds fromNow
After(deadline){
//do stuff
}
//or
val deadlineFuture: Future[Nothing] = (5 seconds fromNow).asFuture
deadlineFuture onComplete {
//do stuff
}
Is there anything like that available that I've missed, or will I have to implement this kind of behavior myself?
Not quite built in, but they provide just enough rope.
The gist is to wait on an empty promise that must disappoint (i.e., time out).
import scala.concurrent._
import scala.concurrent.duration._
import scala.util._
import ExecutionContext.Implicits.global
object Test extends App {
val v = new SyncVar[Boolean]()
val deadline = 5 seconds fromNow
future(Await.ready(Promise().future, deadline.timeLeft)) onComplete { _ =>
println("Bye, now.")
v.put(true)
}
v.take()
// or
val w = new SyncVar[Boolean]()
val dropdeadline = 5 seconds fromNow
val p = Promise[Boolean]()
p.future onComplete {_ =>
println("Bye, now.")
w.put(true)
}
Try(Await.ready(Promise().future, dropdeadline.timeLeft))
p trySuccess true
w.take()
// rolling it
implicit class Expiry(val d: Deadline) extends AnyVal {
def expiring(f: =>Unit) {
future(Await.ready(Promise().future, d.timeLeft)) onComplete { _ =>
f
}
}
}
val x = new SyncVar[Boolean]()
5 seconds fromNow expiring {
println("That's all, folks.")
x.put(true)
}
x.take() // wait for it
}
Its just a timestamp holder. For example you need to distribute execution of N sequential tasks, in T hours. When you have finished with the first one, you check a deadline and schedule next task depending on (time left)/(tasks left) interval. At some point of time isOverdue() occurs, and you just execute tasks left, in parallel.
Or you could check isOverdue(), and if still false, use timeLeft() for setting timeout on executing the next task, for example.
It's much better than manipulating with Date and Calendar to determine time left. Also Duration was used in Akka for timing.