I want to create a function similar to the following. Basically the function, say F will create a Future say fut1. When fut1 resolves, then another Future say fut2 should get created inside fut1. The fut2 should return the final value of the function F. The code has to be non-blocking all the way. I have written something like this but the return type is not Future[Int] but Future[Future[Int]]. I understand why this is the case (because map creates a Future) but I am unable to figure out how to return Future[Int] from this code.
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
def fut:Future[Int] = {
val f1 = Future{ 1 } //create a Future
f1.map (x => { //when f1 finishes, create another future
println(x)
val f2 = Future{ 2 }
f2.map(x=> x) //this creates another Future and thus the return is Future[Future[Int]]
})
}
You can achieve this using flat map or for comprehension.
FlatMap-
def futureFunctionWithFlatMap: Future[Int] = {
val f1 = Future {
1
}
f1.flatMap(x => {
println(x)
val f2 = Future {
2
}
f2.map(x => x)
})
}
For Comprehension
def futureFunctionWithForComprehension: Future[Int] = {
for {
f1 <- Future { 1 }
f2 <- {
println(f1)
Future { 2 }
}
} yield f2
}
Use flatMap
val f1 = Future{ 1 } //create a Future
val f2: Future[Int] = f1.flatMap(x => {
//will be triggered only after x is ready
Future{2}
})
In the code below I have to return the result of a future that is invoked after another future. I'm getting the following error in the future2.map line:
type mismatch; found : scala.concurrent.Future[play.api.mvc.Result]
required: play.api.mvc.Result
How to make this work?
def method1 = Action.async { request =>
val future1 = f1
future1.map { result1 =>
val future2 = f2
future2.map { result2 =>
Ok(result1+result2+"")
}
}
}
def f1 = Future { 1 }
def f2 = Future { 2 }
You could do this in many ways. But first, you need to understand how map and flatMap work with Future:
def map[S](f: (T) ⇒ S): Future[S]
def map[S](f: (T) ⇒ Future[S]): Future[Future[S]]
def flatMap[S](f: (T) ⇒ Future[S]): Future[S]
Note that,in above signatures, you are calling map and flatMap with a value that already is a future i.e Future[<some-value>].map(...) or Future[<some-value>].flatMap(...).
Approach 1:
def method1 = Action.async { request =>
val future1 = f1
future1.flatMap { result1 => //replaced map with flatMap
val future2 = f2
future2.map { result2 =>
Ok(result1+result2+"")
}
}
}
def f1 = Future { 1 }
def f2 = Future { 2 }
Approach 2:
def method1 = Action.async { request =>
val future1 = f1
future1.flatMap { result1 => //replaced map with flatMap
val future2 = f2
future2.flatMap { result2 => //replaced map with flatMap
Future.successful{Ok(result1+result2+"")} // used Future.successful{} to generate a Future of Result
}
}
}
def f1 = Future { 1 }
def f2 = Future { 2 }
Changing future1.map to future1.flatMap should do the trick. Mapping over a Future returns another Future and changes the value inside. In this case, you're returning a Future that contains another Future that contains a Result. By using flatMap, it essentially flattens the nested Future[Future[Result]] into a Future[Result].
Here is the problem, I have a library which has a blocking method return Try[T]. But since it's a blocking one, I would like to make it non-blocking using Future[T]. In the future block, I also would like to compute something that's depend on the origin blocking method's return value.
But if I use something like below, then my nonBlocking will return Future[Try[T]] which is less convince since Future[T] could represent Failure[U] already, I would rather prefer propagate the exception to Future[T] is self.
def blockMethod(x: Int): Try[Int] = Try {
// Some long operation to get an Int from network or IO
throw new Exception("Network Exception") }
}
def nonBlocking(x: Int): Future[Try[Int]] = future {
blockMethod(x).map(_ * 2)
}
Here is what I tried, I just use .get method in future {} block, but I'm not sure if this is the best way to do that.
def blockMethod(x: Int): Try[Int] = Try {
// Some long operation to get an Int from network or IO
throw new Exception("Network Exception") }
}
def nonBlocking(x: Int): Future[Int] = future {
blockMethod(x).get * 2
}
Is this correct way to do that? Or there is a more scala idiomatic way to convert t Try[T] to Future[T]?
Here's an example that doesn't block, note that you probably want to use your own execution context and not scala's global context:
import scala.util._
import scala.concurrent._
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
object Main extends App {
def blockMethod(x: Int): Try[Int] = Try {
// Some long operation to get an Int from network or IO
Thread.sleep(10000)
100
}
def tryToFuture[A](t: => Try[A]): Future[A] = {
future {
t
}.flatMap {
case Success(s) => Future.successful(s)
case Failure(fail) => Future.failed(fail)
}
}
// Initiate long operation
val f = tryToFuture(blockMethod(1))
println("Waiting... 10 seconds to complete")
// Should return before 20 seconds...
val res = Await.result(f, 20 seconds)
println(res) // prints 100
}
In my opinion: Try & Future is a monadic construction and idiomatic way to is monadic composition (for-comprehension):
That you need to define monad transformer for Future[Try[_]] (code for your library):
case class FutureT[R](run : Future[Try[R]])(implicit e: ExecutionContext) {
def map[B](f : R => B): FutureT[B] = FutureT(run map { _ map f })
def flatMap[B](f : R => FutureT[B]): FutureT[B] = {
val p = Promise[Try[B]]()
run onComplete {
case Failure(e) => p failure e
case Success(Failure(e)) => p failure e
case Success(Success(v)) => f(v).run onComplete {
case Failure(e) => p failure e
case Success(s) => p success s
}
}
FutureT(p.future)
}
}
object FutureT {
def futureTry[R](run : => Try[R])(implicit e: ExecutionContext) =
new FutureT(future { run })
implicit def toFutureT[R](run : Future[Try[R]]) = FutureT(run)
implicit def fromFutureT[R](futureT : FutureT[R]) = futureT.run
}
and usage example:
def blockMethod(x: Int): Try[Int] = Try {
Thread.sleep(5000)
if(x < 10) throw new IllegalArgumentException
else x + 1
}
import FutureT._
// idiomatic way :)
val async = for {
x <- futureTry { blockMethod(15) }
y <- futureTry { blockMethod(25) }
} yield (x + y) * 2 // possible due to using modan transformer
println("Waiting... 10 seconds to complete")
val res = Await.result(async, 20 seconds)
println(res)
// example with Exception
val asyncWithError = for {
x <- futureTry { blockMethod(5) }
y <- futureTry { blockMethod(25) }
} yield (x + y) * 2 // possible due to using modan transformer
// Can't use Await because will get exception
// when extract value from FutureT(Failure(java.lang.IllegalArgumentException))
// no difference between Failure produced by Future or Try
asyncWithError onComplete {
case Failure(e) => println(s"Got exception: $e.msg")
case Success(res) => println(res)
}
// Output:
// Got exception: java.lang.IllegalArgumentException.msg
Suppose I have several futures and need to wait until either any of them fails or all of them succeed.
For example: Let there are 3 futures: f1, f2, f3.
If f1 succeeds and f2 fails I do not wait for f3 (and return failure to the client).
If f2 fails while f1 and f3 are still running I do not wait for them (and return failure)
If f1 succeeds and then f2 succeeds I continue waiting for f3.
How would you implement it?
You could use a for-comprehension as follows instead:
val fut1 = Future{...}
val fut2 = Future{...}
val fut3 = Future{...}
val aggFut = for{
f1Result <- fut1
f2Result <- fut2
f3Result <- fut3
} yield (f1Result, f2Result, f3Result)
In this example, futures 1, 2 and 3 are kicked off in parallel. Then, in the for comprehension, we wait until the results 1 and then 2 and then 3 are available. If either 1 or 2 fails, we will not wait for 3 anymore. If all 3 succeed, then the aggFut val will hold a tuple with 3 slots, corresponding to the results of the 3 futures.
Now if you need the behavior where you want to stop waiting if say fut2 fails first, things get a little trickier. In the above example, you would have to wait for fut1 to complete before realizing fut2 failed. To solve that, you could try something like this:
val fut1 = Future{Thread.sleep(3000);1}
val fut2 = Promise.failed(new RuntimeException("boo")).future
val fut3 = Future{Thread.sleep(1000);3}
def processFutures(futures:Map[Int,Future[Int]], values:List[Any], prom:Promise[List[Any]]):Future[List[Any]] = {
val fut = if (futures.size == 1) futures.head._2
else Future.firstCompletedOf(futures.values)
fut onComplete{
case Success(value) if (futures.size == 1)=>
prom.success(value :: values)
case Success(value) =>
processFutures(futures - value, value :: values, prom)
case Failure(ex) => prom.failure(ex)
}
prom.future
}
val aggFut = processFutures(Map(1 -> fut1, 2 -> fut2, 3 -> fut3), List(), Promise[List[Any]]())
aggFut onComplete{
case value => println(value)
}
Now this works correctly, but the issue comes from knowing which Future to remove from the Map when one has been successfully completed. As long as you have some way to properly correlate a result with the Future that spawned that result, then something like this works. It just recursively keeps removing completed Futures from the Map and then calling Future.firstCompletedOf on the remaining Futures until there are none left, collecting the results along the way. It's not pretty, but if you really need the behavior you are talking about, then this, or something similar could work.
You can use a promise, and send to it either the first failure, or the final completed aggregated success:
def sequenceOrBailOut[A, M[_] <: TraversableOnce[_]](in: M[Future[A]] with TraversableOnce[Future[A]])(implicit cbf: CanBuildFrom[M[Future[A]], A, M[A]], executor: ExecutionContext): Future[M[A]] = {
val p = Promise[M[A]]()
// the first Future to fail completes the promise
in.foreach(_.onFailure{case i => p.tryFailure(i)})
// if the whole sequence succeeds (i.e. no failures)
// then the promise is completed with the aggregated success
Future.sequence(in).foreach(p trySuccess _)
p.future
}
Then you can Await on that resulting Future if you want to block, or just map it into something else.
The difference with for comprehension is that here you get the error of the first to fail, whereas with for comprehension you get the first error in traversal order of the input collection (even if another one failed first). For example:
val f1 = Future { Thread.sleep(1000) ; 5 / 0 }
val f2 = Future { 5 }
val f3 = Future { None.get }
Future.sequence(List(f1,f2,f3)).onFailure{case i => println(i)}
// this waits one second, then prints "java.lang.ArithmeticException: / by zero"
// the first to fail in traversal order
And:
val f1 = Future { Thread.sleep(1000) ; 5 / 0 }
val f2 = Future { 5 }
val f3 = Future { None.get }
sequenceOrBailOut(List(f1,f2,f3)).onFailure{case i => println(i)}
// this immediately prints "java.util.NoSuchElementException: None.get"
// the 'actual' first to fail (usually...)
// and it returns early (it does not wait 1 sec)
Here is a solution without using actors.
import scala.util._
import scala.concurrent._
import java.util.concurrent.atomic.AtomicInteger
// Nondeterministic.
// If any failure, return it immediately, else return the final success.
def allSucceed[T](fs: Future[T]*): Future[T] = {
val remaining = new AtomicInteger(fs.length)
val p = promise[T]
fs foreach {
_ onComplete {
case s # Success(_) => {
if (remaining.decrementAndGet() == 0) {
// Arbitrarily return the final success
p tryComplete s
}
}
case f # Failure(_) => {
p tryComplete f
}
}
}
p.future
}
You can do this with futures alone. Here's one implementation. Note that it won't terminate execution early! In that case you need to do something more sophisticated (and probably implement the interruption yourself). But if you just don't want to keep waiting for something that isn't going to work, the key is to keep waiting for the first thing to finish, and stop when either nothing is left or you hit an exception:
import scala.annotation.tailrec
import scala.util.{Try, Success, Failure}
import scala.concurrent._
import scala.concurrent.duration.Duration
import ExecutionContext.Implicits.global
#tailrec def awaitSuccess[A](fs: Seq[Future[A]], done: Seq[A] = Seq()):
Either[Throwable, Seq[A]] = {
val first = Future.firstCompletedOf(fs)
Await.ready(first, Duration.Inf).value match {
case None => awaitSuccess(fs, done) // Shouldn't happen!
case Some(Failure(e)) => Left(e)
case Some(Success(_)) =>
val (complete, running) = fs.partition(_.isCompleted)
val answers = complete.flatMap(_.value)
answers.find(_.isFailure) match {
case Some(Failure(e)) => Left(e)
case _ =>
if (running.length > 0) awaitSuccess(running, answers.map(_.get) ++: done)
else Right( answers.map(_.get) ++: done )
}
}
}
Here's an example of it in action when everything works okay:
scala> awaitSuccess(Seq(Future{ println("Hi!") },
Future{ Thread.sleep(1000); println("Fancy meeting you here!") },
Future{ Thread.sleep(2000); println("Bye!") }
))
Hi!
Fancy meeting you here!
Bye!
res1: Either[Throwable,Seq[Unit]] = Right(List((), (), ()))
But when something goes wrong:
scala> awaitSuccess(Seq(Future{ println("Hi!") },
Future{ Thread.sleep(1000); throw new Exception("boo"); () },
Future{ Thread.sleep(2000); println("Bye!") }
))
Hi!
res2: Either[Throwable,Seq[Unit]] = Left(java.lang.Exception: boo)
scala> Bye!
For this purpose I would use an Akka actor. Unlike the for-comprehension, it fails as soon as any of the futures fail, so it's a bit more efficient in that sense.
class ResultCombiner(futs: Future[_]*) extends Actor {
var origSender: ActorRef = null
var futsRemaining: Set[Future[_]] = futs.toSet
override def receive = {
case () =>
origSender = sender
for(f <- futs)
f.onComplete(result => self ! if(result.isSuccess) f else false)
case false =>
origSender ! SomethingFailed
case f: Future[_] =>
futsRemaining -= f
if(futsRemaining.isEmpty) origSender ! EverythingSucceeded
}
}
sealed trait Result
case object SomethingFailed extends Result
case object EverythingSucceeded extends Result
Then, create the actor, send a message to it (so that it will know where to send its reply to) and wait for a reply.
val actor = actorSystem.actorOf(Props(new ResultCombiner(f1, f2, f3)))
try {
val f4: Future[Result] = actor ? ()
implicit val timeout = new Timeout(30 seconds) // or whatever
Await.result(f4, timeout.duration).asInstanceOf[Result] match {
case SomethingFailed => println("Oh noes!")
case EverythingSucceeded => println("It all worked!")
}
} finally {
// Avoid memory leaks: destroy the actor
actor ! PoisonPill
}
This question has been answered but I am posting my value class solution (value classes were added in 2.10) since there isn't one here. Please feel free to criticize.
implicit class Sugar_PimpMyFuture[T](val self: Future[T]) extends AnyVal {
def concurrently = ConcurrentFuture(self)
}
case class ConcurrentFuture[A](future: Future[A]) extends AnyVal {
def map[B](f: Future[A] => Future[B]) : ConcurrentFuture[B] = ConcurrentFuture(f(future))
def flatMap[B](f: Future[A] => ConcurrentFuture[B]) : ConcurrentFuture[B] = concurrentFutureFlatMap(this, f) // work around no nested class in value class
}
def concurrentFutureFlatMap[A,B](outer: ConcurrentFuture[A], f: Future[A] => ConcurrentFuture[B]) : ConcurrentFuture[B] = {
val p = Promise[B]()
val inner = f(outer.future)
inner.future onFailure { case t => p.tryFailure(t) }
outer.future onFailure { case t => p.tryFailure(t) }
inner.future onSuccess { case b => p.trySuccess(b) }
ConcurrentFuture(p.future)
}
ConcurrentFuture is a no overhead Future wrapper that changes the default Future map/flatMap from do-this-then-that to combine-all-and-fail-if-any-fail. Usage:
def func1 : Future[Int] = Future { println("f1!");throw new RuntimeException; 1 }
def func2 : Future[String] = Future { Thread.sleep(2000);println("f2!");"f2" }
def func3 : Future[Double] = Future { Thread.sleep(2000);println("f3!");42.0 }
val f : Future[(Int,String,Double)] = {
for {
f1 <- func1.concurrently
f2 <- func2.concurrently
f3 <- func3.concurrently
} yield for {
v1 <- f1
v2 <- f2
v3 <- f3
} yield (v1,v2,v3)
}.future
f.onFailure { case t => println("future failed $t") }
In the example above, f1,f2 and f3 will run concurrently and if any fail in any order the future of the tuple will fail immediately.
You might want to checkout Twitter's Future API. Notably the Future.collect method. It does exactly what you want: https://twitter.github.io/scala_school/finagle.html
The source code Future.scala is available here:
https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/util/Future.scala
You can use this:
val l = List(1, 6, 8)
val f = l.map{
i => future {
println("future " +i)
Thread.sleep(i* 1000)
if (i == 12)
throw new Exception("6 is not legal.")
i
}
}
val f1 = Future.sequence(f)
f1 onSuccess{
case l => {
logInfo("onSuccess")
l.foreach(i => {
logInfo("h : " + i)
})
}
}
f1 onFailure{
case l => {
logInfo("onFailure")
}
I'm new to scala and I try to combine several Futures in scala 2.10RC3. The Futures should be executed in sequential order. In the document Scala SIP14 the method andThen is defined in order to execute Futures in sequential order. I used this method to combine several Futures (see example below). My expectation was that it prints 6 but actually the result is 0. What am I doing wrong here? I have two questions:
First, why is the result 0. Second, how can I combine several Futures, so that the execution of the second Future does not start before the first Future has been finished.
val intList = List(1, 2, 3)
val sumOfIntFuture = intList.foldLeft(Future { 0 }) {
case (future, i) => future andThen {
case Success(result) => result + i
case Failure(e) => println(e)
}
}
sumOfIntFuture onSuccess { case x => println(x) }
andThen is for side-effects. It allows you to specify some actions to do after future is completed and before it used for something else.
Use map:
scala> List(1, 2, 3).foldLeft(Future { 0 }) {
| case (future, i) => future map { _ + i }
| } onSuccess { case x => println(x) }
6
I like this generic approach:
trait FutureImplicits {
class SeriallyPimp[T, V](futures: Seq[T]) {
def serially(f: T => Future[V])(implicit ec: ExecutionContext): Future[Seq[V]] = {
val buf = ListBuffer.empty[V]
buf.sizeHint(futures.size)
futures.foldLeft(Future.successful(buf)) { (previousFuture, next) =>
for {
previousResults <- previousFuture
nextResult <- f(next)
} yield previousResults += nextResult
}
}
}
implicit def toSeriallyPimp[T, V](xs: Seq[T]): SeriallyPimp[T, V] =
new SeriallyPimp(xs)
}
Then mix-in the above trait and use it like this:
val elems: Seq[Elem] = ???
val save: Elem => Future[Result] = ???
val f: Future[Seq[Result]] = elems serially save
This code could be improved to preserve the input collection type. See this article for example.