Because Task.parSequenceUnordered "is like parSequence, except that you don’t get ordering for results or effects.", I would expect the following to return List(2, 1):
import monix.eval.Task
import monix.execution.Scheduler.Implicits.global
object TaskSequence extends App {
val taskOne = Task {
Thread.sleep(3000)
println("First executed")
1
}
val taskTwo = Task {
Thread.sleep(2000)
println("Second executed")
2
}
val parallelUnordered = Task.parSequenceUnordered(List(taskOne, taskTwo)).runSyncUnsafe()
println(s"Return vals for Task.parSequenceUnordered( ) were $parallelUnordered")
}
However, I get List(1, 2) as a result. Using .runAsync( ) doesn't make a difference either:
Task.parSequenceUnordered(List(taskOne, taskTwo)).runAsync( result => result match {
case Right(value) => println(value)
case Left(value) => value
})
Thread.sleep(5000) // don't let program exit
The second Task finishes first, so I should get the 2 back first in the return List, right? Can any Monix experts weigh in? Thanks
It's because of the implementation of parSequenceUnordered, which builds the result list by using O(1) prepend operations.
There's a
private sealed abstract class State[+A] {
def isActive: Boolean
def enqueue[B >: A](value: B): State[B]
}
to represent the internal state machine for the task, and a couple of the State subclasses look like
final case class Active[+A](list: List[A], remaining: Int) extends State[A] {
def isActive = true
def enqueue[B >: A](value: B): Active[B] =
Active(value :: list, remaining - 1)
}
So because it uses value :: list to accumulate the results, they are built in reverse in terms of which result comes in first.
An experiment with more Tasks:
val taskOne = Task {
Thread.sleep(4000)
println("First executed")
1
}
val taskTwo = Task {
Thread.sleep(2000)
println("Second executed")
2
}
val taskThree = Task {
Thread.sleep(1000)
println("Third executed")
3
}
val taskFour = Task {
Thread.sleep(5000)
println("Fourth executed")
4
}
Task.parSequenceUnordered(List(taskOne, taskTwo, taskThree, taskFour)).runAsync( result => result match {
case Right(value) => println(value)
case Left(value) => value
}
Resulted in:
Second executed
Third executed
First executed
Fourth executed
List(4, 1, 3, 2)
This shows that the results are returned in opposite order when using Task.parSequenceUnordered( ), from last completed to first completed.
Related
Suppose I have the following code:
val futureInt1 = getIntAsync1();
val futureInt2 = getIntAsync2();
val futureSum = for {
int1 <- futureInt1
int2 <- futureInt2
} yield (int1 + int2)
val sum = Await.result(futureSum, 60 seconds)
Now suppose one of getIntAsync1 or getIntAsync2 takes a very long time, and it leads to Await.result throwing exception:
Caused by: java.util.concurrent.TimeoutException: Futures timed out after [60 seconds]
How am I supposed to know which one of getIntAsync1 or getIntAsync2 was still pending and actually lead to the timeout?
Note that here I'm merging 2 futures with zip, and this is a simple example for the question, but in my app I have this kind of code at different level (ie getIntAsync1 itself can use Future.zip or Future.sequence, map/flatMap/applicative)
Somehow what I'd like is to be able to log the pending concurrent operation stacktraces when a timeout occur on my main thread, so that I can know where are the bottlenexts on my system.
I have an existing legacy API backend which is not fully reactive yet and won't be so soon. I'm trying to increase response times by using concurrency. But since using this kind of code, It's become way more painful to understand why something takes a lot of time in my app. I would appreciate any tip you can provide to help me debugging such issues.
The key is realizing is that a the Future doesn't time out in your example—it's your calling thread which pauses for at most X time.
So, if you want to model time in your Futures you should use zipWith on each branch and zip with a Future which will contain a value within a certain amount of time. If you use Akka then you can use akka.pattern.after for this, together with Future.firstCompletedOf.
Now, even if you do, how do you figure out why any of your futures weren't completed in time, perhaps they depended on other futures which didn't complete.
The question boils down to: Are you trying to do some root-cause analysis on throughput? Then you should monitor your ExecutionContext, not your Futures. Futures are only values.
The proposed solution wraps each future from for block into TimelyFuture which requires timeout and name. Internally it uses Await to detect individual timeouts.
Please bear in mind this style of using futures is not intended for production code as it uses blocking. It is for diagnostics only to find out which futures take time to complete.
package xxx
import java.util.concurrent.TimeoutException
import scala.concurrent.{Future, _}
import scala.concurrent.duration.Duration
import scala.util._
import scala.concurrent.duration._
class TimelyFuture[T](f: Future[T], name: String, duration: Duration) extends Future[T] {
override def onComplete[U](ff: (Try[T]) => U)(implicit executor: ExecutionContext): Unit = f.onComplete(x => ff(x))
override def isCompleted: Boolean = f.isCompleted
override def value: Option[Try[T]] = f.value
#scala.throws[InterruptedException](classOf[InterruptedException])
#scala.throws[TimeoutException](classOf[TimeoutException])
override def ready(atMost: Duration)(implicit permit: CanAwait): TimelyFuture.this.type = {
Try(f.ready(atMost)(permit)) match {
case Success(v) => this
case Failure(e) => this
}
}
#scala.throws[Exception](classOf[Exception])
override def result(atMost: Duration)(implicit permit: CanAwait): T = {
f.result(atMost)
}
override def transform[S](ff: (Try[T]) => Try[S])(implicit executor: ExecutionContext): Future[S] = {
val p = Promise[S]()
Try(Await.result(f, duration)) match {
case s#Success(_) => ff(s) match {
case Success(v) => p.success(v)
case Failure(e) => p.failure(e)
}
case Failure(e) => e match {
case e: TimeoutException => p.failure(new RuntimeException(s"future ${name} has timed out after ${duration}"))
case _ => p.failure(e)
}
}
p.future
}
override def transformWith[S](ff: (Try[T]) => Future[S])(implicit executor: ExecutionContext): Future[S] = {
val p = Promise[S]()
Try(Await.result(f, duration)) match {
case s#Success(_) => ff(s).onComplete({
case Success(v) => p.success(v)
case Failure(e) => p.failure(e)
})
case Failure(e) => e match {
case e: TimeoutException => p.failure(new RuntimeException(s"future ${name} has timed out after ${duration}"))
case _ => p.failure(e)
}
}
p.future
}
}
object Main {
import scala.concurrent.ExecutionContext.Implicits.global
def main(args: Array[String]): Unit = {
val f = Future {
Thread.sleep(5);
1
}
val g = Future {
Thread.sleep(2000);
2
}
val result: Future[(Int, Int)] = for {
v1 <- new TimelyFuture(f, "f", 10 milliseconds)
v2 <- new TimelyFuture(g, "g", 10 milliseconds)
} yield (v1, v2)
val sum = Await.result(result, 1 seconds) // as expected, this throws exception : "RuntimeException: future g has timed out after 10 milliseconds"
}
}
If you are merely looking for informational metrics on which individual future was taking a long time (or in combination with others), your best bet is to use a wrapper when creating the futures to log the metrics:
object InstrumentedFuture {
def now() = System.currentTimeMillis()
def apply[T](name: String)(code: => T): Future[T] = {
val start = now()
val f = Future {
code
}
f.onComplete {
case _ => println(s"Future ${name} took ${now() - start} ms")
}
f
}
}
val future1 = InstrumentedFuture("Calculator") { /*...code...*/ }
val future2 = InstrumentedFuture("Differentiator") { /*...code...*/ }
You can check if a future has completed by calling its isComplete method
if (futureInt1.isComplete) { /*futureInt2 must be the culprit */ }
if (futureInt2.isComplete) { /*futureInt1 must be the culprit */ }
As a first approach i would suggest to lift your Future[Int] into Future[Try[Int]]. Something like that:
object impl {
def checkException[T](in: Future[T]): Future[Try[T]] =
in.map(Success(_)).recover {
case e: Throwable => {
Failure(new Exception("Error in future: " + in))
}
}
implicit class FutureCheck(s: Future[Int]) {
def check = checkException(s)
}
}
Then a small function to combine the results, something like that:
object test {
import impl._
val futureInt1 = Future{ 1 }
val futureInt2 = Future{ 2 }
def combine(a: Try[Int], b: Try[Int])(f: (Int, Int) => (Int)) : Try[Int] = {
if(a.isSuccess && b.isSuccess) {
Success(f(a.get, b.get))
}
else
Failure(new Exception("Error adding results"))
}
val futureSum = for {
int1 <- futureInt1.check
int2 <- futureInt2.check
} yield combine(int1, int2)(_ + _)
}
In futureSum you will have a Try[Int] with the integer or a Failure with the exception corresponding with the possible error.
Maybe this can be useful
val futureInt1 = getIntAsync1();
val futureInt2 = getIntAsync2();
val futureSum = for {
int1 <- futureInt1
int2 <- futureInt2
} yield (int1 + int2)
Try(Await.result(futureSum, 60 seconds)) match {
case Success(sum) => println(sum)
case Failure(e) => println("we got timeout. the unfinished futures are: " + List(futureInt1, futureInt2).filter(!_.isCompleted)
}
I want to do something like this.
def foo(s : String): Future[Boolean] = Future {
val a = someLongRunningMethod
if(!a)
return false or throw some exception // what should i return
//do some more thing
val b= someMoreLongRunningMethod
if(b)
return true
return false
}
but not able to use return with boolean. I got type mismatch error.
Error:(32, 12) type mismatch;
found : Boolean(false)
required: scala.concurrent.Future[Boolean]
return false
I am new to Scala. I am using foo method as this. I am not sure if it's the best way to use it. Please suggest how should i achieve it ?
val r = foo("Nishant")
r.onComplete {
case Success(result) => {
//Do something with my list
println("success: " + result)
}
case Failure(exception) => {
//Do something with my error
println("failure")
}
}
val re = Await.result(r, 10 second)
In Scala last expression in the block is the return value of the codeblock or function. Keyword return is optional in scala.
Notice that we run second task only if one task returns true. If first task returns false then we are done. That means first task is really important for our computation as its the decision maker.
Your version modified:
def longRunning1: Boolean = ???
def longRunning2: Boolean = ???
import scala.concurrent.ExecutionContext.Implicits.global
def foo(s : String): Future[Boolean] = Future {
val a: Boolean = longRunning1
if(a) {
val b: Boolean = longRunning2
b
} else false
}
Version 1:
Run futures (computations or long running methods) simultaneously and choose the results later. Here we discard the result of the second computation if we consider or want the result of the first computation.
import scala.concurrent.ExecutionContext.Implicits.global
def foo(s: String): Future[Boolean] = {
val f1 = Future {
Thread.sleep(20000) //Just to simulate long running task
Random.nextBoolean()
}
val f2 = Future {
Thread.sleep(1000) //Just to simulate long running task
Random.nextBoolean()
}
(f1 zip f2) map {
case (false, _) => false
case (true, f2Result) => f2Result
case _ => false
}
}
Version 2:
Run first method and then based on the result of the first method try to run the second method one after other. Computation is chained using map.
import scala.concurrent.ExecutionContext.Implicits.global
def foo(s: String): Future[Boolean] = {
val f1 = Future {
Thread.sleep(20000) //Just to simulate long running task
Random.nextBoolean()
}
f1.map { result =>
if (result) result
else {
Thread.sleep(1000) //Just to simulate long running task
Random.nextBoolean()
}
}
}
I've been trying to simplify the way I do futures in Scala. I got at one point a Future[Option[Future[Option[Boolean]] but I've simplified it further below. Is there a better way to simplify this?
Passing a future of "failed" doesn't seem like the best way to do this. i.e. in the sequential world I simply returned "FAIL!!" any time it failed rather than continuing to the end. Are there other ways?
val doSimpleWork = Future {
//Do any arbitrary work (can be a different function)
true //or false
}
val doComplexWork = Future {
//Do any arbitrary work (can be a different function)
Some("result") //or false
}
val failed = Future {
//Do no work at all!!! Just return
false
}
val fut1 = doSimpleWork
val fut2 = doSimpleWork
val fut3 = (fut1 zip fut2).map({
case (true, true) => true
case _ => false
})
val fut4 = fut3.flatMap({
case true =>
doComplexWork.flatMap({
case Some("result") =>
doSimpleWork
case None =>
failed
})
case false =>
failed
})
fut4.map({
case true =>
"SUCCESS!!!"
case _ =>
"FAIL!!"
})
Note that in your example, because you're eagerly instantiating the Futures to a val, all of them will start executing as soon as you declare them (val x = Future {...}). Using methods instead will make the Futures execute only when they're requested by the chain of execution.
One way to avoid the further computation would be to throw an exception, then handle it with onFailure:
def one = future { println("one") ; Some(1) }
def two = future { println("two") ; throw new Exception("no!"); 2 }
def three = future { println("three") ; 3 }
val f = one flatMap {
result1 => two flatMap {
result2 => three
}
}
f onFailure {
case e: Exception =>
println("failed somewhere in the chain")
}
You can see here that "three" isn't supposed to be printed out, because we fail on two. This is the case:
one
two
failed somewhere in the chain
a "Monad transformer" is a construct which lets you combine the "effects" of two monads, the scalaz project provides several different monad transformers. My suggestion is that you can use the OptionT monad transformer to simplify your code if you also make use of the fact that Option[Unit] is isomorphic to Boolean (Some(()) == true and None == false). Here's a complete example:
import scalaz._
import Scalaz._
import scala.concurrent._
import ExecutionContext.Implicits.global
import scala.concurrent.duration._
object Foo {
// We need a Monad instance for Future, here is a valid one, or you can use the implementation
// in the scalaz-contrib project, see http://typelevel.org
implicit def futureMonad(implicit executor: ExecutionContext): Monad[Future] = new Monad[Future] {
override def bind[A, B](fa: Future[A])(f: A ⇒ Future[B]) = fa flatMap f
override def point[A](a: ⇒ A) = Future(a)
override def map[A, B](fa: Future[A])(f: A ⇒ B) = fa map f
}
// OptionT allows you to combine the effects of the Future and Option monads
// to more easily work with a Future[Option[A]]
val doSimpleWork : OptionT[Future,Unit] = OptionT(Future {
// Option[Unit] is isomorphic to Boolean
Some(()) //or None
})
val simpleFail : OptionT[Future,Unit] = OptionT(Future {
None
})
val doComplexWork: OptionT[Future,String] = OptionT(Future {
Some("result") //or None
})
val f1 = doSimpleWork
val f2 = doSimpleWork
val f3 = doComplexWork
val f4 = doSimpleWork
def main(argv: Array[String]) {
val result = for {
_ <- f1
// we don't get here unless both the future succeeded and the result was Some
_ <- f2
_ <- f3
r <- f4
} yield(r)
result.fold((_ => println("SUCCESS!!")),println("FAIL!!"))
// "run" will get you to the Future inside the OptionT
Await.result(result.run, 1 second)
}
}
You could try something like this, using for comprehensions to clean up the code a bit:
def doSimpleWork = Future{
//do some simple work
true
}
def doComplexWork = Future{
//do something complex here
Some("result")
}
val fut1 = doSimpleWork
val fut2 = doSimpleWork
val fut = for{
f1Result <- fut1
f2Result <- fut2
if (f1Result && f2Result)
f3Result <- doComplexWork
if (f3Result.isDefined)
f4Result <- doSimpleWork
} yield "success"
fut onComplete{
case Success(value) => println("I succeeded")
case Failure(ex) => println("I failed: " + ex.getMessage)
}
And if you really just wanted to print out "success" or "failed" at the end, you could change that last piece of code to:
fut.recover{case ex => "failed"} onSuccess{
case value => println(value)
}
Now, to explain what's going on. For starters, we've defined two functions that return Futures that are doing some async work. The doSimpleWork function will do some simple work and return a boolean success/fail indicator. The doComplexWork function will do something more complex (and time consuming) and return an Option[String] representing a result. We then kick off two parallel invocations of doSimpleWork before entering the for comprehension. In for for comp, we get the results of fut1 and fut2 (in that order) before checking if they were both successful. If not, we would stop here, and the fut val would be failed with a NoSuchElementException which is what you get when a condition like this fails in a for comp. If both were successful, we would continue on and invoke the doComplexWork function and wait for its result. We would then check its result and if it was Some, we would kick off one last invocation of doSimpleWork. If that succeeds, we would yield the string "success". If you check the type of the fut val, its of type Future[String].
From there, we use one of the async callback functions to check if the whole sequence of calls either made it all the way through (the Success case), or failed somewhere in the process (the Failure case), printing out something related to which case it hit. In the alternate final code block, we recover from any possible failure by returning the String "failed" "and then use just the onSuccess callback which will print either "success" or "failed" depending on what happened.
Suppose I have several futures and need to wait until either any of them fails or all of them succeed.
For example: Let there are 3 futures: f1, f2, f3.
If f1 succeeds and f2 fails I do not wait for f3 (and return failure to the client).
If f2 fails while f1 and f3 are still running I do not wait for them (and return failure)
If f1 succeeds and then f2 succeeds I continue waiting for f3.
How would you implement it?
You could use a for-comprehension as follows instead:
val fut1 = Future{...}
val fut2 = Future{...}
val fut3 = Future{...}
val aggFut = for{
f1Result <- fut1
f2Result <- fut2
f3Result <- fut3
} yield (f1Result, f2Result, f3Result)
In this example, futures 1, 2 and 3 are kicked off in parallel. Then, in the for comprehension, we wait until the results 1 and then 2 and then 3 are available. If either 1 or 2 fails, we will not wait for 3 anymore. If all 3 succeed, then the aggFut val will hold a tuple with 3 slots, corresponding to the results of the 3 futures.
Now if you need the behavior where you want to stop waiting if say fut2 fails first, things get a little trickier. In the above example, you would have to wait for fut1 to complete before realizing fut2 failed. To solve that, you could try something like this:
val fut1 = Future{Thread.sleep(3000);1}
val fut2 = Promise.failed(new RuntimeException("boo")).future
val fut3 = Future{Thread.sleep(1000);3}
def processFutures(futures:Map[Int,Future[Int]], values:List[Any], prom:Promise[List[Any]]):Future[List[Any]] = {
val fut = if (futures.size == 1) futures.head._2
else Future.firstCompletedOf(futures.values)
fut onComplete{
case Success(value) if (futures.size == 1)=>
prom.success(value :: values)
case Success(value) =>
processFutures(futures - value, value :: values, prom)
case Failure(ex) => prom.failure(ex)
}
prom.future
}
val aggFut = processFutures(Map(1 -> fut1, 2 -> fut2, 3 -> fut3), List(), Promise[List[Any]]())
aggFut onComplete{
case value => println(value)
}
Now this works correctly, but the issue comes from knowing which Future to remove from the Map when one has been successfully completed. As long as you have some way to properly correlate a result with the Future that spawned that result, then something like this works. It just recursively keeps removing completed Futures from the Map and then calling Future.firstCompletedOf on the remaining Futures until there are none left, collecting the results along the way. It's not pretty, but if you really need the behavior you are talking about, then this, or something similar could work.
You can use a promise, and send to it either the first failure, or the final completed aggregated success:
def sequenceOrBailOut[A, M[_] <: TraversableOnce[_]](in: M[Future[A]] with TraversableOnce[Future[A]])(implicit cbf: CanBuildFrom[M[Future[A]], A, M[A]], executor: ExecutionContext): Future[M[A]] = {
val p = Promise[M[A]]()
// the first Future to fail completes the promise
in.foreach(_.onFailure{case i => p.tryFailure(i)})
// if the whole sequence succeeds (i.e. no failures)
// then the promise is completed with the aggregated success
Future.sequence(in).foreach(p trySuccess _)
p.future
}
Then you can Await on that resulting Future if you want to block, or just map it into something else.
The difference with for comprehension is that here you get the error of the first to fail, whereas with for comprehension you get the first error in traversal order of the input collection (even if another one failed first). For example:
val f1 = Future { Thread.sleep(1000) ; 5 / 0 }
val f2 = Future { 5 }
val f3 = Future { None.get }
Future.sequence(List(f1,f2,f3)).onFailure{case i => println(i)}
// this waits one second, then prints "java.lang.ArithmeticException: / by zero"
// the first to fail in traversal order
And:
val f1 = Future { Thread.sleep(1000) ; 5 / 0 }
val f2 = Future { 5 }
val f3 = Future { None.get }
sequenceOrBailOut(List(f1,f2,f3)).onFailure{case i => println(i)}
// this immediately prints "java.util.NoSuchElementException: None.get"
// the 'actual' first to fail (usually...)
// and it returns early (it does not wait 1 sec)
Here is a solution without using actors.
import scala.util._
import scala.concurrent._
import java.util.concurrent.atomic.AtomicInteger
// Nondeterministic.
// If any failure, return it immediately, else return the final success.
def allSucceed[T](fs: Future[T]*): Future[T] = {
val remaining = new AtomicInteger(fs.length)
val p = promise[T]
fs foreach {
_ onComplete {
case s # Success(_) => {
if (remaining.decrementAndGet() == 0) {
// Arbitrarily return the final success
p tryComplete s
}
}
case f # Failure(_) => {
p tryComplete f
}
}
}
p.future
}
You can do this with futures alone. Here's one implementation. Note that it won't terminate execution early! In that case you need to do something more sophisticated (and probably implement the interruption yourself). But if you just don't want to keep waiting for something that isn't going to work, the key is to keep waiting for the first thing to finish, and stop when either nothing is left or you hit an exception:
import scala.annotation.tailrec
import scala.util.{Try, Success, Failure}
import scala.concurrent._
import scala.concurrent.duration.Duration
import ExecutionContext.Implicits.global
#tailrec def awaitSuccess[A](fs: Seq[Future[A]], done: Seq[A] = Seq()):
Either[Throwable, Seq[A]] = {
val first = Future.firstCompletedOf(fs)
Await.ready(first, Duration.Inf).value match {
case None => awaitSuccess(fs, done) // Shouldn't happen!
case Some(Failure(e)) => Left(e)
case Some(Success(_)) =>
val (complete, running) = fs.partition(_.isCompleted)
val answers = complete.flatMap(_.value)
answers.find(_.isFailure) match {
case Some(Failure(e)) => Left(e)
case _ =>
if (running.length > 0) awaitSuccess(running, answers.map(_.get) ++: done)
else Right( answers.map(_.get) ++: done )
}
}
}
Here's an example of it in action when everything works okay:
scala> awaitSuccess(Seq(Future{ println("Hi!") },
Future{ Thread.sleep(1000); println("Fancy meeting you here!") },
Future{ Thread.sleep(2000); println("Bye!") }
))
Hi!
Fancy meeting you here!
Bye!
res1: Either[Throwable,Seq[Unit]] = Right(List((), (), ()))
But when something goes wrong:
scala> awaitSuccess(Seq(Future{ println("Hi!") },
Future{ Thread.sleep(1000); throw new Exception("boo"); () },
Future{ Thread.sleep(2000); println("Bye!") }
))
Hi!
res2: Either[Throwable,Seq[Unit]] = Left(java.lang.Exception: boo)
scala> Bye!
For this purpose I would use an Akka actor. Unlike the for-comprehension, it fails as soon as any of the futures fail, so it's a bit more efficient in that sense.
class ResultCombiner(futs: Future[_]*) extends Actor {
var origSender: ActorRef = null
var futsRemaining: Set[Future[_]] = futs.toSet
override def receive = {
case () =>
origSender = sender
for(f <- futs)
f.onComplete(result => self ! if(result.isSuccess) f else false)
case false =>
origSender ! SomethingFailed
case f: Future[_] =>
futsRemaining -= f
if(futsRemaining.isEmpty) origSender ! EverythingSucceeded
}
}
sealed trait Result
case object SomethingFailed extends Result
case object EverythingSucceeded extends Result
Then, create the actor, send a message to it (so that it will know where to send its reply to) and wait for a reply.
val actor = actorSystem.actorOf(Props(new ResultCombiner(f1, f2, f3)))
try {
val f4: Future[Result] = actor ? ()
implicit val timeout = new Timeout(30 seconds) // or whatever
Await.result(f4, timeout.duration).asInstanceOf[Result] match {
case SomethingFailed => println("Oh noes!")
case EverythingSucceeded => println("It all worked!")
}
} finally {
// Avoid memory leaks: destroy the actor
actor ! PoisonPill
}
This question has been answered but I am posting my value class solution (value classes were added in 2.10) since there isn't one here. Please feel free to criticize.
implicit class Sugar_PimpMyFuture[T](val self: Future[T]) extends AnyVal {
def concurrently = ConcurrentFuture(self)
}
case class ConcurrentFuture[A](future: Future[A]) extends AnyVal {
def map[B](f: Future[A] => Future[B]) : ConcurrentFuture[B] = ConcurrentFuture(f(future))
def flatMap[B](f: Future[A] => ConcurrentFuture[B]) : ConcurrentFuture[B] = concurrentFutureFlatMap(this, f) // work around no nested class in value class
}
def concurrentFutureFlatMap[A,B](outer: ConcurrentFuture[A], f: Future[A] => ConcurrentFuture[B]) : ConcurrentFuture[B] = {
val p = Promise[B]()
val inner = f(outer.future)
inner.future onFailure { case t => p.tryFailure(t) }
outer.future onFailure { case t => p.tryFailure(t) }
inner.future onSuccess { case b => p.trySuccess(b) }
ConcurrentFuture(p.future)
}
ConcurrentFuture is a no overhead Future wrapper that changes the default Future map/flatMap from do-this-then-that to combine-all-and-fail-if-any-fail. Usage:
def func1 : Future[Int] = Future { println("f1!");throw new RuntimeException; 1 }
def func2 : Future[String] = Future { Thread.sleep(2000);println("f2!");"f2" }
def func3 : Future[Double] = Future { Thread.sleep(2000);println("f3!");42.0 }
val f : Future[(Int,String,Double)] = {
for {
f1 <- func1.concurrently
f2 <- func2.concurrently
f3 <- func3.concurrently
} yield for {
v1 <- f1
v2 <- f2
v3 <- f3
} yield (v1,v2,v3)
}.future
f.onFailure { case t => println("future failed $t") }
In the example above, f1,f2 and f3 will run concurrently and if any fail in any order the future of the tuple will fail immediately.
You might want to checkout Twitter's Future API. Notably the Future.collect method. It does exactly what you want: https://twitter.github.io/scala_school/finagle.html
The source code Future.scala is available here:
https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/util/Future.scala
You can use this:
val l = List(1, 6, 8)
val f = l.map{
i => future {
println("future " +i)
Thread.sleep(i* 1000)
if (i == 12)
throw new Exception("6 is not legal.")
i
}
}
val f1 = Future.sequence(f)
f1 onSuccess{
case l => {
logInfo("onSuccess")
l.foreach(i => {
logInfo("h : " + i)
})
}
}
f1 onFailure{
case l => {
logInfo("onFailure")
}
I'm new to scala and I try to combine several Futures in scala 2.10RC3. The Futures should be executed in sequential order. In the document Scala SIP14 the method andThen is defined in order to execute Futures in sequential order. I used this method to combine several Futures (see example below). My expectation was that it prints 6 but actually the result is 0. What am I doing wrong here? I have two questions:
First, why is the result 0. Second, how can I combine several Futures, so that the execution of the second Future does not start before the first Future has been finished.
val intList = List(1, 2, 3)
val sumOfIntFuture = intList.foldLeft(Future { 0 }) {
case (future, i) => future andThen {
case Success(result) => result + i
case Failure(e) => println(e)
}
}
sumOfIntFuture onSuccess { case x => println(x) }
andThen is for side-effects. It allows you to specify some actions to do after future is completed and before it used for something else.
Use map:
scala> List(1, 2, 3).foldLeft(Future { 0 }) {
| case (future, i) => future map { _ + i }
| } onSuccess { case x => println(x) }
6
I like this generic approach:
trait FutureImplicits {
class SeriallyPimp[T, V](futures: Seq[T]) {
def serially(f: T => Future[V])(implicit ec: ExecutionContext): Future[Seq[V]] = {
val buf = ListBuffer.empty[V]
buf.sizeHint(futures.size)
futures.foldLeft(Future.successful(buf)) { (previousFuture, next) =>
for {
previousResults <- previousFuture
nextResult <- f(next)
} yield previousResults += nextResult
}
}
}
implicit def toSeriallyPimp[T, V](xs: Seq[T]): SeriallyPimp[T, V] =
new SeriallyPimp(xs)
}
Then mix-in the above trait and use it like this:
val elems: Seq[Elem] = ???
val save: Elem => Future[Result] = ???
val f: Future[Seq[Result]] = elems serially save
This code could be improved to preserve the input collection type. See this article for example.