There are some suggestions on StackOverflow on how to handle a list of Futures but I want to try my own approach. But I am unable to compile the following code
I have a list of Futures.
I want to count how many of them passed or failed. I should get (2,1)
I store this in a tuple
The approach I want to take is go through each element of the list. The element of the list is Future[Int]. for each element, I call flatMap which calls the next cycle of recursion (I am assuming that if flatMap gets called then that particular future would have been successful so I increment pass count). Similarly, I want to call next cycle of recursion in recover and increment fail count but I am getting compilation error in it.
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success, Try}
import scala.concurrent.duration._
import scala.language.postfixOps
object ConcurrencyExample extends App {
type pass = Int
type fail = Int
val time = System.currentTimeMillis()
//use recursion to process each Future in the list
def segregate(l:List[Future[Int]]):Future[Tuple2[pass,fail]] = {
def go(l:List[Future[Int]],t:Tuple2[pass,fail]):Future[Tuple2[pass,fail]] = {
l match {
case Nil => Future{t}
//l is List of Future[Int]. flatMap each successful Future
//recover each failed Future
case l::ls => {
l flatMap (x => go(ls, (t._1 + 1, t._2)))
**l.recover({ case e => go(ls, (t._1 + 1, t._2))})**//I get error here
}
}
}
go(l,(0,0))
}
//hardcoded future
val futures2: List[Future[Int]] = List(Future {
1
}, Future {
2
}, Future {
throw new Exception("error")
})
val result = segregate(futures2)
result onComplete {
case Success(v) => println("pp:" + v)
case Failure(v) => println("fp:" + v)
}
Await.result(result,1000 millis)
}
#evan058 is correct about the signature of recover. But you can fix your program by changing recover to recoverWith.
recoverWith is to recover as flatMap is to map.
Here's the complete solution (with minor stylistic improvements):
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success, Try}
import scala.concurrent.duration._
import scala.language.postfixOps
object ConcurrencyExample extends App {
type pass = Int
type fail = Int
val time = System.currentTimeMillis()
//use recursion to process each Future in the list
def segregate[T](fs:List[Future[T]]):Future[(pass,fail)] = {
def go(fs:List[Future[T]],r:Future[(pass,fail)]):Future[(pass,fail)] = fs match {
case Nil => r
case l::ls =>
val fx = l.transform({_ => (1, 0)}, identity).recoverWith[(pass,fail)]({case _: Exception => Future(0, 1) })
for (x <- fx; t <- r; g <- go(ls, Future(t._1+x._1,t._2+x._2))) yield g
}
go(fs,Future((0,0)))
}
//hardcoded future
val futures2 = List(Future(1), Future(2), Future(throw new Exception("error")))
val result = segregate(futures2)
result onComplete {
case Success(v) => println(s"successes: ${v._1}, failures: ${v._2}")
case Failure(v) => v.printStackTrace()
}
Await.result(result,1000 millis)
}
If you look at the docs, the signature of recover is:
def recover[U >: T](pf: PartialFunction[Throwable, U])(implicit executor: ExecutionContext): Future[U]
You are calling recover on l which is a Future[Int] so recover is expecting a U >: Int.
However you are calling go again which has the return type Future[(pass, fail)] which is not >: Int.
Related
I need to process sequence of transformations into one transformation, in order to if I have some failed Future it should be just ignored(I am trying to do it without recover or recoverWith, but smth going wrong, code fails when meet any failure)
type Transformation[T] = T => Future[T]
//in - Seq(trans1, trans2, trans3)
in.reduce[Transformation[T]](
(acc, el) =>
acc.andThen[Future[T]](
ft =>
ft.flatMap(el)
.transformWith[T](
t =>
t match {
case Failure(exception) => ft //this line is the most suspicious for me
case Success(value) => Future.successful(value)
}
)
)
)
The transformWith doesn't seem to provide you the possibility to recover with the original input value from the previous step, because it has to work with a Try. What is it supposed to do if the Try turns out to be a failure? By that time, it does not have the original input to fall back to, it only has a Throwable, not a T. So, transformWith seems insufficient.
If the pipeline is just a few transformations long, you can try foldLeft with fallbackTo:
import scala.concurrent._
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
type Transformation[T] = T => Future[T]
def chainTrafos[T](
trafos: List[Transformation[T]],
value: T
): Future[T] = {
trafos.foldLeft(Future { value })(
(f, t) => f.flatMap(x => t(x).fallbackTo(Future { x }))
)
}
val ta: Transformation[Int] = x => Future { x * x }
val tb: Transformation[Int] = x => Future.failed(new Error("oops"))
val tc: Transformation[Int] = x => Future { x - 58 }
println(Await.result(chainTrafos(List(ta, tb, tc), 10), 10.seconds))
prints
42
With Futures there is an easy way to transform Seq[Future] to a Future[Seq]:
Future.sequence(seqOfFutures)
I could not find an analog thing with Try.
It works with foldLeft but what I really like would have something like Try.sequence(seqOfTry).
Is there a reason that such a function is not provided?
How is this done properly?
Semantics:
A List of the values on Success: Success(Seq(1,2,3,4))
For Failure there are 2 possibilities:
Fails on the fist Failure and returns it. This is handled by this question: listtryt-to-trylistt-in-scala
Gathers all Failures and returns a 'compound' Failure.
Is there also a solution for the 'compound' Failure?
As per Luis' suggestion Validated is designed for error accumulation so consider traverse like so
la.traverse(_.toEither.toValidatedNec)
lb.traverse(_.toEither.toValidatedNec)
which outputs
res2: cats.data.ValidatedNec[Throwable,List[Int]] = Invalid(Chain(java.lang.RuntimeException: boom, java.lang.RuntimeException: crash))
res3: cats.data.ValidatedNec[Throwable,List[Int]] = Valid(List(1, 2, 3))
where
import cats.syntax.traverse._
import cats.instances.list._
import cats.syntax.either._
import scala.util.{Failure, Success, Try}
val la: List[Try[Int]] = List(Success(1), Success(2), Failure(new RuntimeException("boom")), Success(3), Failure(new RuntimeException("crash")))
val lb: List[Try[Int]] = List(Success(1), Success(2), Success(3))
Without error accumulation we could just sequence like so
import cats.implicits._
la.sequence
which outputs
res0: scala.util.Try[List[Int]] = Failure(java.lang.RuntimeException: boom)
This is a solution to the second question.
case class CompoundError(errs: List[Throwable]) extends Throwable
def toTry[T](list: List[Try[T]]): Try[List[T]] =
list.partition(_.isSuccess) match {
case (res, Nil) =>
Success(res.map(_.get))
case (_, errs) =>
Failure(CompoundError(errs.collect { case Failure(e) => e }))
}
The partition operation separates the successes and failures, and the match returns the appropriate value depending on whether there are any failures or not.
Previous solution:
case class CompoundError(errs: List[Throwable]) extends Throwable
def toTry[T](list: List[Try[T]]): Try[List[T]] = {
val (res, errs) = list.foldLeft((List.empty[T], List.empty[Throwable])) {
case ((res, errs), item) =>
item match {
case Success(t) => (t :: res, errs)
case Failure(e) => (res, e :: errs)
}
}
errs match {
case Nil => Success(res.reverse)
case _ => Failure(CompoundError(errs.reverse))
}
}
Suppose I have the following code:
val futureInt1 = getIntAsync1();
val futureInt2 = getIntAsync2();
val futureSum = for {
int1 <- futureInt1
int2 <- futureInt2
} yield (int1 + int2)
val sum = Await.result(futureSum, 60 seconds)
Now suppose one of getIntAsync1 or getIntAsync2 takes a very long time, and it leads to Await.result throwing exception:
Caused by: java.util.concurrent.TimeoutException: Futures timed out after [60 seconds]
How am I supposed to know which one of getIntAsync1 or getIntAsync2 was still pending and actually lead to the timeout?
Note that here I'm merging 2 futures with zip, and this is a simple example for the question, but in my app I have this kind of code at different level (ie getIntAsync1 itself can use Future.zip or Future.sequence, map/flatMap/applicative)
Somehow what I'd like is to be able to log the pending concurrent operation stacktraces when a timeout occur on my main thread, so that I can know where are the bottlenexts on my system.
I have an existing legacy API backend which is not fully reactive yet and won't be so soon. I'm trying to increase response times by using concurrency. But since using this kind of code, It's become way more painful to understand why something takes a lot of time in my app. I would appreciate any tip you can provide to help me debugging such issues.
The key is realizing is that a the Future doesn't time out in your example—it's your calling thread which pauses for at most X time.
So, if you want to model time in your Futures you should use zipWith on each branch and zip with a Future which will contain a value within a certain amount of time. If you use Akka then you can use akka.pattern.after for this, together with Future.firstCompletedOf.
Now, even if you do, how do you figure out why any of your futures weren't completed in time, perhaps they depended on other futures which didn't complete.
The question boils down to: Are you trying to do some root-cause analysis on throughput? Then you should monitor your ExecutionContext, not your Futures. Futures are only values.
The proposed solution wraps each future from for block into TimelyFuture which requires timeout and name. Internally it uses Await to detect individual timeouts.
Please bear in mind this style of using futures is not intended for production code as it uses blocking. It is for diagnostics only to find out which futures take time to complete.
package xxx
import java.util.concurrent.TimeoutException
import scala.concurrent.{Future, _}
import scala.concurrent.duration.Duration
import scala.util._
import scala.concurrent.duration._
class TimelyFuture[T](f: Future[T], name: String, duration: Duration) extends Future[T] {
override def onComplete[U](ff: (Try[T]) => U)(implicit executor: ExecutionContext): Unit = f.onComplete(x => ff(x))
override def isCompleted: Boolean = f.isCompleted
override def value: Option[Try[T]] = f.value
#scala.throws[InterruptedException](classOf[InterruptedException])
#scala.throws[TimeoutException](classOf[TimeoutException])
override def ready(atMost: Duration)(implicit permit: CanAwait): TimelyFuture.this.type = {
Try(f.ready(atMost)(permit)) match {
case Success(v) => this
case Failure(e) => this
}
}
#scala.throws[Exception](classOf[Exception])
override def result(atMost: Duration)(implicit permit: CanAwait): T = {
f.result(atMost)
}
override def transform[S](ff: (Try[T]) => Try[S])(implicit executor: ExecutionContext): Future[S] = {
val p = Promise[S]()
Try(Await.result(f, duration)) match {
case s#Success(_) => ff(s) match {
case Success(v) => p.success(v)
case Failure(e) => p.failure(e)
}
case Failure(e) => e match {
case e: TimeoutException => p.failure(new RuntimeException(s"future ${name} has timed out after ${duration}"))
case _ => p.failure(e)
}
}
p.future
}
override def transformWith[S](ff: (Try[T]) => Future[S])(implicit executor: ExecutionContext): Future[S] = {
val p = Promise[S]()
Try(Await.result(f, duration)) match {
case s#Success(_) => ff(s).onComplete({
case Success(v) => p.success(v)
case Failure(e) => p.failure(e)
})
case Failure(e) => e match {
case e: TimeoutException => p.failure(new RuntimeException(s"future ${name} has timed out after ${duration}"))
case _ => p.failure(e)
}
}
p.future
}
}
object Main {
import scala.concurrent.ExecutionContext.Implicits.global
def main(args: Array[String]): Unit = {
val f = Future {
Thread.sleep(5);
1
}
val g = Future {
Thread.sleep(2000);
2
}
val result: Future[(Int, Int)] = for {
v1 <- new TimelyFuture(f, "f", 10 milliseconds)
v2 <- new TimelyFuture(g, "g", 10 milliseconds)
} yield (v1, v2)
val sum = Await.result(result, 1 seconds) // as expected, this throws exception : "RuntimeException: future g has timed out after 10 milliseconds"
}
}
If you are merely looking for informational metrics on which individual future was taking a long time (or in combination with others), your best bet is to use a wrapper when creating the futures to log the metrics:
object InstrumentedFuture {
def now() = System.currentTimeMillis()
def apply[T](name: String)(code: => T): Future[T] = {
val start = now()
val f = Future {
code
}
f.onComplete {
case _ => println(s"Future ${name} took ${now() - start} ms")
}
f
}
}
val future1 = InstrumentedFuture("Calculator") { /*...code...*/ }
val future2 = InstrumentedFuture("Differentiator") { /*...code...*/ }
You can check if a future has completed by calling its isComplete method
if (futureInt1.isComplete) { /*futureInt2 must be the culprit */ }
if (futureInt2.isComplete) { /*futureInt1 must be the culprit */ }
As a first approach i would suggest to lift your Future[Int] into Future[Try[Int]]. Something like that:
object impl {
def checkException[T](in: Future[T]): Future[Try[T]] =
in.map(Success(_)).recover {
case e: Throwable => {
Failure(new Exception("Error in future: " + in))
}
}
implicit class FutureCheck(s: Future[Int]) {
def check = checkException(s)
}
}
Then a small function to combine the results, something like that:
object test {
import impl._
val futureInt1 = Future{ 1 }
val futureInt2 = Future{ 2 }
def combine(a: Try[Int], b: Try[Int])(f: (Int, Int) => (Int)) : Try[Int] = {
if(a.isSuccess && b.isSuccess) {
Success(f(a.get, b.get))
}
else
Failure(new Exception("Error adding results"))
}
val futureSum = for {
int1 <- futureInt1.check
int2 <- futureInt2.check
} yield combine(int1, int2)(_ + _)
}
In futureSum you will have a Try[Int] with the integer or a Failure with the exception corresponding with the possible error.
Maybe this can be useful
val futureInt1 = getIntAsync1();
val futureInt2 = getIntAsync2();
val futureSum = for {
int1 <- futureInt1
int2 <- futureInt2
} yield (int1 + int2)
Try(Await.result(futureSum, 60 seconds)) match {
case Success(sum) => println(sum)
case Failure(e) => println("we got timeout. the unfinished futures are: " + List(futureInt1, futureInt2).filter(!_.isCompleted)
}
I'd like to await a scala future that may have failed. If I use Await.result the exception will be thrown. Instead, if I have f: Future[String] I would like a method Await.resultOpt(f): Option[String] or Await.resultEither(f): Either[String].
I could get this by using scala.util.control.Exception.catching or I could f map (Right(_)) recover { case t: Throwable => Left(t) }, but there must be a more straightforward way.
You could use Await.ready which simply blocks until the Future has either succeeded or failed, then returns a reference back to that Future.
From there, you would probably want to get the Future's value, which is an Option[Try[T]]. Due to the Await.ready call, it should be safe to assume that the value is a Some. Then it's just a matter of mapping between a Try[T] and an Either[Throwable, T].
The short version:
val f: Future[T] = ...
val result: Try[T] = Await.ready(f, Duration.Inf).value.get
val resultEither = result match {
case Success(t) => Right(t)
case Failure(e) => Left(e)
}
The shorter version, just to promote the API:
scala> val f = Future(7)
f: scala.concurrent.Future[Int] = scala.concurrent.impl.Promise$DefaultPromise#13965637
scala> f.value.get
res0: scala.util.Try[Int] = Success(7)
scala> import scala.util._
import scala.util._
scala> Either.cond(res0.isSuccess, res0.get, res0.failed.get)
res2: scala.util.Either[Throwable,Int] = Right(7)
scala> val f = Future[Int](???)
f: scala.concurrent.Future[Int] = scala.concurrent.impl.Promise$DefaultPromise#64c4c1
scala> val v = f.value.get
v: scala.util.Try[Int] = Failure(java.util.concurrent.ExecutionException: Boxed Error)
scala> Either.cond(v.isSuccess, v.get, v.failed.get)
res4: scala.util.Either[Throwable,Int] = Left(java.util.concurrent.ExecutionException: Boxed Error)
It has a slight advantage in being a one-liner.
But of course, after adding a .toEither extension method, you don't care how many lines it took.
You could start to make your own type utils and do something like so
trait RichTypes {
import scala.util.{Try, Success, Failure}
import scala.concurrent.{Await, Future}
import scala.concurrent.duration.Duration
implicit class RichFuture[T](f: Future[T]) {
def awaitResult(d: Duration): Either[Throwable, T] = {
Try(Await.result(f, d)).toEither
}
}
implicit class RichTry[T](tri: Try[T]) {
def toEither(): Either[Throwable, T] = {
tri.fold[Either[Throwable, T]](Left(_), Right(_))
}
}
}
object Example
extends App
with RichTypes {
import scala.concurrent.Future
import scala.concurrent.duration._
val succ = Future.successful("hi").awaitResult(5.seconds)
val fail = Future.failed(new Exception("x")).awaitResult(5.seconds)
println(succ) // Right(hi)
println(fail) // Left(Exception(x))
}
I separated it out for a Try to also have a .fold :).
Suppose I have several futures and need to wait until either any of them fails or all of them succeed.
For example: Let there are 3 futures: f1, f2, f3.
If f1 succeeds and f2 fails I do not wait for f3 (and return failure to the client).
If f2 fails while f1 and f3 are still running I do not wait for them (and return failure)
If f1 succeeds and then f2 succeeds I continue waiting for f3.
How would you implement it?
You could use a for-comprehension as follows instead:
val fut1 = Future{...}
val fut2 = Future{...}
val fut3 = Future{...}
val aggFut = for{
f1Result <- fut1
f2Result <- fut2
f3Result <- fut3
} yield (f1Result, f2Result, f3Result)
In this example, futures 1, 2 and 3 are kicked off in parallel. Then, in the for comprehension, we wait until the results 1 and then 2 and then 3 are available. If either 1 or 2 fails, we will not wait for 3 anymore. If all 3 succeed, then the aggFut val will hold a tuple with 3 slots, corresponding to the results of the 3 futures.
Now if you need the behavior where you want to stop waiting if say fut2 fails first, things get a little trickier. In the above example, you would have to wait for fut1 to complete before realizing fut2 failed. To solve that, you could try something like this:
val fut1 = Future{Thread.sleep(3000);1}
val fut2 = Promise.failed(new RuntimeException("boo")).future
val fut3 = Future{Thread.sleep(1000);3}
def processFutures(futures:Map[Int,Future[Int]], values:List[Any], prom:Promise[List[Any]]):Future[List[Any]] = {
val fut = if (futures.size == 1) futures.head._2
else Future.firstCompletedOf(futures.values)
fut onComplete{
case Success(value) if (futures.size == 1)=>
prom.success(value :: values)
case Success(value) =>
processFutures(futures - value, value :: values, prom)
case Failure(ex) => prom.failure(ex)
}
prom.future
}
val aggFut = processFutures(Map(1 -> fut1, 2 -> fut2, 3 -> fut3), List(), Promise[List[Any]]())
aggFut onComplete{
case value => println(value)
}
Now this works correctly, but the issue comes from knowing which Future to remove from the Map when one has been successfully completed. As long as you have some way to properly correlate a result with the Future that spawned that result, then something like this works. It just recursively keeps removing completed Futures from the Map and then calling Future.firstCompletedOf on the remaining Futures until there are none left, collecting the results along the way. It's not pretty, but if you really need the behavior you are talking about, then this, or something similar could work.
You can use a promise, and send to it either the first failure, or the final completed aggregated success:
def sequenceOrBailOut[A, M[_] <: TraversableOnce[_]](in: M[Future[A]] with TraversableOnce[Future[A]])(implicit cbf: CanBuildFrom[M[Future[A]], A, M[A]], executor: ExecutionContext): Future[M[A]] = {
val p = Promise[M[A]]()
// the first Future to fail completes the promise
in.foreach(_.onFailure{case i => p.tryFailure(i)})
// if the whole sequence succeeds (i.e. no failures)
// then the promise is completed with the aggregated success
Future.sequence(in).foreach(p trySuccess _)
p.future
}
Then you can Await on that resulting Future if you want to block, or just map it into something else.
The difference with for comprehension is that here you get the error of the first to fail, whereas with for comprehension you get the first error in traversal order of the input collection (even if another one failed first). For example:
val f1 = Future { Thread.sleep(1000) ; 5 / 0 }
val f2 = Future { 5 }
val f3 = Future { None.get }
Future.sequence(List(f1,f2,f3)).onFailure{case i => println(i)}
// this waits one second, then prints "java.lang.ArithmeticException: / by zero"
// the first to fail in traversal order
And:
val f1 = Future { Thread.sleep(1000) ; 5 / 0 }
val f2 = Future { 5 }
val f3 = Future { None.get }
sequenceOrBailOut(List(f1,f2,f3)).onFailure{case i => println(i)}
// this immediately prints "java.util.NoSuchElementException: None.get"
// the 'actual' first to fail (usually...)
// and it returns early (it does not wait 1 sec)
Here is a solution without using actors.
import scala.util._
import scala.concurrent._
import java.util.concurrent.atomic.AtomicInteger
// Nondeterministic.
// If any failure, return it immediately, else return the final success.
def allSucceed[T](fs: Future[T]*): Future[T] = {
val remaining = new AtomicInteger(fs.length)
val p = promise[T]
fs foreach {
_ onComplete {
case s # Success(_) => {
if (remaining.decrementAndGet() == 0) {
// Arbitrarily return the final success
p tryComplete s
}
}
case f # Failure(_) => {
p tryComplete f
}
}
}
p.future
}
You can do this with futures alone. Here's one implementation. Note that it won't terminate execution early! In that case you need to do something more sophisticated (and probably implement the interruption yourself). But if you just don't want to keep waiting for something that isn't going to work, the key is to keep waiting for the first thing to finish, and stop when either nothing is left or you hit an exception:
import scala.annotation.tailrec
import scala.util.{Try, Success, Failure}
import scala.concurrent._
import scala.concurrent.duration.Duration
import ExecutionContext.Implicits.global
#tailrec def awaitSuccess[A](fs: Seq[Future[A]], done: Seq[A] = Seq()):
Either[Throwable, Seq[A]] = {
val first = Future.firstCompletedOf(fs)
Await.ready(first, Duration.Inf).value match {
case None => awaitSuccess(fs, done) // Shouldn't happen!
case Some(Failure(e)) => Left(e)
case Some(Success(_)) =>
val (complete, running) = fs.partition(_.isCompleted)
val answers = complete.flatMap(_.value)
answers.find(_.isFailure) match {
case Some(Failure(e)) => Left(e)
case _ =>
if (running.length > 0) awaitSuccess(running, answers.map(_.get) ++: done)
else Right( answers.map(_.get) ++: done )
}
}
}
Here's an example of it in action when everything works okay:
scala> awaitSuccess(Seq(Future{ println("Hi!") },
Future{ Thread.sleep(1000); println("Fancy meeting you here!") },
Future{ Thread.sleep(2000); println("Bye!") }
))
Hi!
Fancy meeting you here!
Bye!
res1: Either[Throwable,Seq[Unit]] = Right(List((), (), ()))
But when something goes wrong:
scala> awaitSuccess(Seq(Future{ println("Hi!") },
Future{ Thread.sleep(1000); throw new Exception("boo"); () },
Future{ Thread.sleep(2000); println("Bye!") }
))
Hi!
res2: Either[Throwable,Seq[Unit]] = Left(java.lang.Exception: boo)
scala> Bye!
For this purpose I would use an Akka actor. Unlike the for-comprehension, it fails as soon as any of the futures fail, so it's a bit more efficient in that sense.
class ResultCombiner(futs: Future[_]*) extends Actor {
var origSender: ActorRef = null
var futsRemaining: Set[Future[_]] = futs.toSet
override def receive = {
case () =>
origSender = sender
for(f <- futs)
f.onComplete(result => self ! if(result.isSuccess) f else false)
case false =>
origSender ! SomethingFailed
case f: Future[_] =>
futsRemaining -= f
if(futsRemaining.isEmpty) origSender ! EverythingSucceeded
}
}
sealed trait Result
case object SomethingFailed extends Result
case object EverythingSucceeded extends Result
Then, create the actor, send a message to it (so that it will know where to send its reply to) and wait for a reply.
val actor = actorSystem.actorOf(Props(new ResultCombiner(f1, f2, f3)))
try {
val f4: Future[Result] = actor ? ()
implicit val timeout = new Timeout(30 seconds) // or whatever
Await.result(f4, timeout.duration).asInstanceOf[Result] match {
case SomethingFailed => println("Oh noes!")
case EverythingSucceeded => println("It all worked!")
}
} finally {
// Avoid memory leaks: destroy the actor
actor ! PoisonPill
}
This question has been answered but I am posting my value class solution (value classes were added in 2.10) since there isn't one here. Please feel free to criticize.
implicit class Sugar_PimpMyFuture[T](val self: Future[T]) extends AnyVal {
def concurrently = ConcurrentFuture(self)
}
case class ConcurrentFuture[A](future: Future[A]) extends AnyVal {
def map[B](f: Future[A] => Future[B]) : ConcurrentFuture[B] = ConcurrentFuture(f(future))
def flatMap[B](f: Future[A] => ConcurrentFuture[B]) : ConcurrentFuture[B] = concurrentFutureFlatMap(this, f) // work around no nested class in value class
}
def concurrentFutureFlatMap[A,B](outer: ConcurrentFuture[A], f: Future[A] => ConcurrentFuture[B]) : ConcurrentFuture[B] = {
val p = Promise[B]()
val inner = f(outer.future)
inner.future onFailure { case t => p.tryFailure(t) }
outer.future onFailure { case t => p.tryFailure(t) }
inner.future onSuccess { case b => p.trySuccess(b) }
ConcurrentFuture(p.future)
}
ConcurrentFuture is a no overhead Future wrapper that changes the default Future map/flatMap from do-this-then-that to combine-all-and-fail-if-any-fail. Usage:
def func1 : Future[Int] = Future { println("f1!");throw new RuntimeException; 1 }
def func2 : Future[String] = Future { Thread.sleep(2000);println("f2!");"f2" }
def func3 : Future[Double] = Future { Thread.sleep(2000);println("f3!");42.0 }
val f : Future[(Int,String,Double)] = {
for {
f1 <- func1.concurrently
f2 <- func2.concurrently
f3 <- func3.concurrently
} yield for {
v1 <- f1
v2 <- f2
v3 <- f3
} yield (v1,v2,v3)
}.future
f.onFailure { case t => println("future failed $t") }
In the example above, f1,f2 and f3 will run concurrently and if any fail in any order the future of the tuple will fail immediately.
You might want to checkout Twitter's Future API. Notably the Future.collect method. It does exactly what you want: https://twitter.github.io/scala_school/finagle.html
The source code Future.scala is available here:
https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/util/Future.scala
You can use this:
val l = List(1, 6, 8)
val f = l.map{
i => future {
println("future " +i)
Thread.sleep(i* 1000)
if (i == 12)
throw new Exception("6 is not legal.")
i
}
}
val f1 = Future.sequence(f)
f1 onSuccess{
case l => {
logInfo("onSuccess")
l.foreach(i => {
logInfo("h : " + i)
})
}
}
f1 onFailure{
case l => {
logInfo("onFailure")
}