How can I have 3 Futures run asynchronously and display each answer? - scala

I'm learning scala and I want to run 3 futures asynchronously and get an answer for each one.
But I can't get it to work.
import scala.concurrent._
import scala.concurrent.duration._
import scala.concurrent.Future
import scala.util.{Success, Failure}
import scala.concurrent.ExecutionContext.Implicits.global
def f_Fail(): Future[String] = Future {
Thread.sleep(1000)
throw new RuntimeException("わざと失敗")
}
def f_success(): Future[String] = Future {
Thread.sleep(1000)
"future!"
}
val f1 = f_success()
val f2 = f_success()
val f3 = f_Fail()
val future =for {
n <- f1
m <- f2
p <- f3
} yield n+"\n"+m+"\n"+p
Await.ready(future,Duration.Inf).value.get match {
case Success(v) => println(v)
case Failure(e) => println("not found")
}
Expected results
"future!"
"future!"
"not found"
Actual result
"not found"

A for-comprehension is so-called "syntactic sugar" for nested flatMap calls and finally a map. In your case, the for-comprehension can be effectively re-written as:
val future = f1.flatMap(n => f2.flatMap(m => f3.map(p => n+"\n"+m+"\n"+p)))
In the context of Futures, the flatMap method requires the Future upon which its called to be completed before the result is available for the following step. Simplifying for clarity, consider the following:
f1.flatMap(n => f2.map(m => m + n))
In order for the lamdba passed to the flatMap method to be invoked, the result of the calculation must be known.
Effectively, flatMap calls represent the concept of a direct dependency on a previous computation being executed successfully. In the same way, its "sugared" counterpart, the for-comprehension, represents the same.
This means that if any step of the chain fails, the overall result will be failed, which is what you are experiencing in your example.
What you can do is the following:
val f1 = f_success().recover(_ => "not found")
val f2 = f_success().recover(_ => "not found")
val f3 = f_Fail().recover(_ => "not found")
for {
n <- f1
m <- f2
p <- f3
} {
println(n)
println(m)
println(p)
}
Notice that:
Futures are "eager", i.e. the computation is started immediately when the Future itself is instantiated. This means that the first three lines effectively start each computation independently and (given enough threads and resources) concurrently or even in parallel.
each Future defines how to be recovered by returning the "not found" string in case of error (which is the case for f3).
the side effect (println) is performed as part of the for-comprehension itself, allowing you to avoid blocking on the synchronous context -- in this case, the for-comprehension without yield is equivalent to executing the final step inside a foreach instead of a map, which is better suited to express side effects.
Now, let's say that you want to start your Futures lazily by simply wrapping their definition in a function. You'll notice that suddenly the execution doesn't necessarily take advantage of multiple threads and cores and takes 3 seconds instead of one, as in the following example:
def f1 = f_success().recover(_ => "not found")
def f2 = f_success().recover(_ => "not found")
def f3 = f_Fail().recover(_ => "not found")
for {
n <- f1
m <- f2
p <- f3
} {
println(n)
println(m)
println(p)
}
This is because of the semantics of flatMap. You can solve this problem by using a construct that doesn't imply some form of direct dependency between the steps of the calculation, like zip:
def f1 = f_success().recover(_ => "not found")
def f2 = f_success().recover(_ => "not found")
def f3 = f_Fail().recover(_ => "not found")
for (((n, m), p) <- f1.zip(f2).zip(f3)) {
println(n)
println(m)
println(p)
}
This runs again in ~1 second as you might expect.
Alternatively, if you want to still return the result as a Future, you can of course use yield as follows:
val future = for (((n, m), p) <- f1.zip(f2).zip(f3)) yield s"$n\n$m\n$p"
You can read more about for-comprehensions here on the Scala book.

Related

How to discard other Futures if the critical Future is finished in Scala?

Let's say I have three remote calls in order to construct my page. One of them (X) is critical for the page and the other two (A, B) just used to enhance the experience.
Because criticalFutureX is too important to be effected by futureA and futureB, so I want the overall latency of of all remote calls to be Not more than X.
That means, in case of criticalFutureX finishes, I want to discard futureA and futureB.
val criticalFutureX = ...
val futureA = ...
val futureB = ...
// the overall latency of this for-comprehension depends on the longest among X, A and B
for {
x <- criticalFutureX
a <- futureA
b <- futureB
} ...
In the above example, even though they are executed in parallel, the overall latency depends on the longest among X, A and B, which is not what I want.
Latencies:
X: |----------|
A: |---------------|
B: |---|
O: |---------------| (overall latency)
There is firstCompletedOf but it can not be used to explicit say "in case of completed of criticalFutureX".
Is there something like the following?
val criticalFutureX = ...
val futureA = ...
val futureB = ...
for {
x <- criticalFutureX
a <- futureA // discard when criticalFutureX finished
b <- futureB // discard when criticalFutureX finished
} ...
X: |----------|
A: |-----------... discarded
B: |---|
O: |----------| (overall latency)
You can achieve this with a promise
def completeOnMain[A, B](main: Future[A], secondary: Future[B]) = {
val promise = Promise[Option[B]]()
main.onComplete {
case Failure(_) =>
case Success(_) => promise.trySuccess(None)
}
secondary.onComplete {
case Failure(exception) => promise.tryFailure(exception)
case Success(value) => promise.trySuccess(Option(value))
}
promise.future
}
Some testing code
private def runFor(first: Int, second: Int) = {
def run(millis: Int) = Future {
Thread.sleep(millis);
millis
}
val start = System.currentTimeMillis()
val combined = for {
_ <- Future.unit
f1 = run(first)
f2 = completeOnMain(f1, run(second))
r1 <- f1
r2 <- f2
} yield (r1, r2)
val result = Await.result(combined, 10.seconds)
println(s"It took: ${System.currentTimeMillis() - start}: $result")
}
runFor(3000, 4000)
runFor(3000, 1000)
Produces
It took: 3131: (3000,None)
It took: 3001: (3000,Some(1000))
This kind of task is very hard to achieve efficiently, reliably and safely with Scala standard library Futures. There is no way to interrupt a Future that hasn't completed yet, meaning that even if you choose to ignore its result, it will still keep running and waste memory and CPU time. And even if there was a method to interrupt a running Future, there is no way to ensure that resources that were allocated (network connections, open files etc.) will be properly released.
I would like to point out that the implementation given by Ivan Stanislavciuc has a bug: if the main Future fails, then the promise will never be completed, which is unlikely to be what you want.
I would therefore strongly suggest looking into modern concurrent effect systems like ZIO or cats-effect. These are not only safer and faster, but also much easier. Here's an implementation with ZIO that doesn't have this bug:
import zio.{Exit, Task}
import Function.tupled
def completeOnMain[A, B](
main: Task[A], secondary: Task[B]): Task[(A, Exit[Throwable, B])] =
(main.forkManaged zip secondary.forkManaged).use {
tupled(_.join zip _.interrupt)
}
Exit is a type that describes how the secondary task ended, i. e. by successfully returning a B or because of an error (of type Throwable) or due to interruption.
Note that this function can be given a much more sophisticated signature that tells you a lot more about what's going on, but I wanted to keep it simple here.

.zip three futures in Scala [duplicate]

This question already has answers here:
Return Future[(Int,Int)] instead of (Future[Int],Future[Int])
(2 answers)
Closed 5 years ago.
I need the result variable below to contain Future[(String,String,String)] with the result of futures f1, f2 and f3, but instead I'm getting Future[((String, String), String)]. I need the three futures to run in parallel. How to make this work?
def futureA = Future { "A" }
def futureB = Future { "B" }
def futureC = Future { "C" }
def futureFunc = {
val cond1 = 1
val cond2 = 0
val f1 = if (cond1 > 0)
futureA
else
Future {""}
val f2 = if (cond2 > 0)
futureB
else
Future {""}
val f3 = futureC
val fx = f1.zip(f2)
val result = fx.zip(f3)
}
If you create your futures beforehand, you can combine them in a for comprehension and they will run in parallel:
for {
a <- f1
b <- f2
c <- f3
} yield (a, b, c)
res0: scala.concurrent.Future[(String, String, String)]
I tried to create more solutions and here is result:
def futureFunc = {
val cond1 = 1
val cond2 = 0
val f1 = if (cond1 > 0)
futureA
else
Future {""}
val f2 = if (cond2 > 0)
futureB
else
Future {""}
val f3 = futureC
//#1
Future.sequence(List(f1, f2, f3)).map {
case List(a, b, c) => (a, b, c)
}
//#2
for{
f11 <- f1
f22 <- f2
f33 <- f3
} yield (f11, f22, f33)
//#3
f1.zip(f2).zip(f3).map{
case ((f11,f22),f33) => (f11,f22,f33)
}
}
First one uses Future sequence, for creating Future[List[]] and then mapping this list for tuple (because of type safety we don't have method for tupling list).
Second is usage of for comprehension as described by Sascha, as you may know it is syntactic sugar for maps and flatmaps which is preferred to work with futures.
Last one is using zips, as you wanted, but you still need to map last future to obtain tuple which you want.
All operations are non blocking, but for all operations you need to know exactly futures which you will be using. You can use additional libraries for tupling lists, and then use first solution for not well known amount for futures. For readability i think for comprehension is best.

How to remove a fold from within a for-comprehension?

I have the following snippet in Scala using Scalaz and Task/EitherT:
def myMethod(request: MyRequest) : EitherT[Task, Failure, Success] =
EitherT[Task, Failure, Success](
for {
aamOpt <- async.findUser(request)
aamEith <- Task.now(aamOpt.\/>(NoUserFound()))
result <- aamEith.fold(err => Task.now(-\/(err)), aam => process(aam)).run)
} yield result)
Where async.findUser returns a Task[Option[MyUser]] and process(aam) returns EitherT[Task, Failure, Success]. These return types are needed as these methods interact (down the line) with external services.
Is there any way to simplify the last line of the for comprehension into something nicer? I mean this one:
result <- aamEith.fold(err => Task.now(-\/(err)), aam => process(aam)).run)
I would probably immediately lift the result from async.findUser into the relevant EitherT monad and then just flatMap:
def myMethod(request: MyRequest): EitherT[Task, Failure, Success] =
EitherT.eitherT[Task, Failure, MyUser](
async.findUser(request).map(_.\/>(NoUserFound()))
).flatMap(process)
This is untested but something like it should more or less work.

Scala Futures - flatMap and onFailure

If I have some computation that takes a while I might place it in a scala.concurrent.Future:
val f = Future { someLongRunningFunction() }
and let's say I want to do something else asynchronously once that computation is completed:
f.flatMap{ _ => anotherLongRunningFunction() }
In the event that f's initial block fails, how do I "idiomatically" handle this when using flatMap or other combinators? Is it merely a case of using recover or onFailure before the flatMap?
I like the elegance and simplicity of using a flatMap but it seems failure scenarios get in the way of it.
Edit: the second future is reliant on the first, hence the flatMap. I'm looking for a solution that'll elegantly let me chain like I would with flatMap but also handle failures of the first.
To quote the scaladoc for flatMap:
Creates a new future by applying a function to the successful result
of this future, and returns the result of the function as the new
future. If this future is completed with an exception then the new
future will also contain this exception.
Notice the bold, meaning that whatever you pass to flatMap will only be executed if the initial future completes successfully.
So you should only handle the result of the entire execution:
val result = future1.flatMap {
result => functionReturningFuture2(result)
}
and then:
result.onFailure // or
result.onSuccess // or
result.recover
If you have several futures you can put them in a for comprehension.
val futureResult = for {
result1 <- future1
result2 <- future2
...
} yield {
//computation with results
}
You can add a recover at the end in case you want to process any exception you may find:
futureResult.recover{
case exceptionResult: Throwable => // Process exception
}
I think this is more clean that using flatMap.
I'm only starting with Future, but here are some ideas.
If you really want to use flatMap, you have to turn the failure into a success.
for{ a <- f recover r
b <- another(a)
} yield b
This works if the return type of r is :> the result type of f.
Or you can pass the problem of what to do with the failure on to the next process
for{ a <- f map (x => Success(x)) recover (ex => Failure(ex))
b <- another(a)
} yield b
Here the argument type of another would be Try[T] where the type of f is Future[T].

Scala - Execute arbitrary number of Futures sequentially but dependently [duplicate]

This question already has answers here:
Is there sequential Future.find?
(3 answers)
Closed 8 years ago.
I'm trying to figure out the neatest way to execute a series of Futures in sequence, where one Future's execution depends on the previous. I'm trying to do this for an arbitrary number of futures.
User case:
I have retrieved a number of Ids from my database.
I now need to retrieve some related data on a web service.
I want to stop once I've found a valid result.
I only care about the result that succeeded.
Executing these all in parallel and then parsing the collection of results returned isn't an option. I have to do one request at a time, and only execute the next request if the previous request returned no results.
The current solution is along these lines. Using foldLeft to execute the requests and then only evaluating the next future if the previous future meets some condition.
def dblFuture(i: Int) = { i * 2 }
val list = List(1,2,3,4,5)
val future = list.foldLeft(Future(0)) {
(previousFuture, next) => {
for {
previousResult <- previousFuture
nextFuture <- { if (previousResult <= 4) dblFuture(next) else previousFuture }
} yield (nextFuture)
}
}
The big downside of this is a) I keep processing all items even once i've got a result i'm happy with and b) once I've found the result I'm after, I keep evaluating the predicate. In this case it's a simple if, but in reality it could be more complicated.
I feel like I'm missing a far more elegant solution to this.
Looking at your example, it seems as though the previous result has no bearing on subsequent results, and instead what only matters is that the previous result satisfies some condition to prevent the next result from being computed. If that is the case, here is a recursive solution using filter and recoverWith.
def untilFirstSuccess[A, B](f: A => Future[B])(condition: B => Boolean)(list: List[A]): Future[B] = {
list match {
case head :: tail => f(head).filter(condition).recoverWith { case _: Throwable => untilFirstSuccess(f)(condition)(tail) }
case Nil => Future.failed(new Exception("All failed.."))
}
}
filter will only be called when the Future has completed, and recoverWith will only be called if the Future has failed.
def dblFuture(i: Int): Future[Int] = Future {
println("Executing.. " + i)
i * 2
}
val list = List(1, 2, 3, 4, 5)
scala> untilFirstSuccess(dblFuture)(_ > 6)(list)
Executing.. 1
Executing.. 2
Executing.. 3
Executing.. 4
res1: scala.concurrent.Future[Int] = scala.concurrent.impl.Promise$DefaultPromise#514f4e98
scala> res1.value
res2: Option[scala.util.Try[Int]] = Some(Success(8))
Neatest way, and "true functional programming" is scalaz-stream ;) However you'll need to switch to scalaz.concurrent.Task from scala Future for abstraction for "future result". It's a bit different. Task is pure, and Future is "running computation", but they have a lot in common.
import scalaz.concurrent.Task
import scalaz.stream.Process
def dblTask(i: Int) = Task {
println(s"Executing task $i")
i * 2
}
val list = Seq(1,2,3,4,5)
val p: Process[Task, Int] = Process.emitAll(list)
val result: Task[Option[Int]] =
p.flatMap(i => Process.eval(dblTask(i))).takeWhile(_ < 10).runLast
println(s"result = ${result.run}")
Result:
Executing task 1
Executing task 2
Executing task 3
Executing task 4
Executing task 5
result = Some(8)
if your computation is already scala Future, you can transform it to Task
implicit class Transformer[+T](fut: => SFuture[T]) {
def toTask(implicit ec: scala.concurrent.ExecutionContext): Task[T] = {
import scala.util.{Failure, Success}
import scalaz.syntax.either._
Task.async {
register =>
fut.onComplete {
case Success(v) => register(v.right)
case Failure(ex) => register(ex.left)
}
}
}
}