Running futures sequentially - scala

The objective of the code below is to execute Future f3 or f4 depending on a condition. Note that the condition depends on the result of Future f1 or f2, so it has to wait. This seems to work, however since f1 and f2 are futures this code shouldn't run sequentially. Is this code correct?
object TestFutures extends App {
val f1 = Future { 1 }
val f2 = Future { 2 }
val f3 = Future { 3 }
val f4 = Future { 4 }
val y = 1
for {
condition <- if (y>0) f1 else f2
_ <- if (condition==1) f3.map {a => println("333")} else f4.map {b => println("444")}
} yield ()
Thread.sleep(5000)
}

No it is not correct. When you create a Future like you do it, it starts the computations immediately. Before reaching for comprehension, all of your 4 futures are running already. You need to create them later, depending on the conditions.
val y = 1
for {
condition <- if (y > 0) Future { 1 } else Future { 2 }
_ <- if (condition == 1)
Future { 3 }.map(a => println("333"))
else
Future { 4 }.map(b => println("444"))
} yield ()
It is probably good to extract creating each of those to a method, that you will just call, for sake of readability.
It should be obvious they start running when they are created because you can just say
Future(1).map(x => println(x))
and it works without any sort of triggering. Anyway try to run the following code
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
def printWhenCompleted[A](f: Future[A]): Future[A] = f.map { x =>
println(x)
x
}
val f1 = printWhenCompleted(Future { 1 })
val f2 = printWhenCompleted(Future { 2 })
val f3 = printWhenCompleted(Future { 3 })
for {
r3 <- f3
r2 <- f2
r1 <- f1
} yield r1 + r2 + r3
it should give you those numbers in random order, instead of sequential 3, 2, 1
Edit
Here is implementation of the first code (without println) using flatMap
val futureCondition = if (y > 0) Future(1) else Future(2)
futureCondition.flatMap(condition => if (condition == 1) Future(3) else Future(4))

Related

How can I have 3 Futures run asynchronously and display each answer?

I'm learning scala and I want to run 3 futures asynchronously and get an answer for each one.
But I can't get it to work.
import scala.concurrent._
import scala.concurrent.duration._
import scala.concurrent.Future
import scala.util.{Success, Failure}
import scala.concurrent.ExecutionContext.Implicits.global
def f_Fail(): Future[String] = Future {
Thread.sleep(1000)
throw new RuntimeException("わざと失敗")
}
def f_success(): Future[String] = Future {
Thread.sleep(1000)
"future!"
}
val f1 = f_success()
val f2 = f_success()
val f3 = f_Fail()
val future =for {
n <- f1
m <- f2
p <- f3
} yield n+"\n"+m+"\n"+p
Await.ready(future,Duration.Inf).value.get match {
case Success(v) => println(v)
case Failure(e) => println("not found")
}
Expected results
"future!"
"future!"
"not found"
Actual result
"not found"
A for-comprehension is so-called "syntactic sugar" for nested flatMap calls and finally a map. In your case, the for-comprehension can be effectively re-written as:
val future = f1.flatMap(n => f2.flatMap(m => f3.map(p => n+"\n"+m+"\n"+p)))
In the context of Futures, the flatMap method requires the Future upon which its called to be completed before the result is available for the following step. Simplifying for clarity, consider the following:
f1.flatMap(n => f2.map(m => m + n))
In order for the lamdba passed to the flatMap method to be invoked, the result of the calculation must be known.
Effectively, flatMap calls represent the concept of a direct dependency on a previous computation being executed successfully. In the same way, its "sugared" counterpart, the for-comprehension, represents the same.
This means that if any step of the chain fails, the overall result will be failed, which is what you are experiencing in your example.
What you can do is the following:
val f1 = f_success().recover(_ => "not found")
val f2 = f_success().recover(_ => "not found")
val f3 = f_Fail().recover(_ => "not found")
for {
n <- f1
m <- f2
p <- f3
} {
println(n)
println(m)
println(p)
}
Notice that:
Futures are "eager", i.e. the computation is started immediately when the Future itself is instantiated. This means that the first three lines effectively start each computation independently and (given enough threads and resources) concurrently or even in parallel.
each Future defines how to be recovered by returning the "not found" string in case of error (which is the case for f3).
the side effect (println) is performed as part of the for-comprehension itself, allowing you to avoid blocking on the synchronous context -- in this case, the for-comprehension without yield is equivalent to executing the final step inside a foreach instead of a map, which is better suited to express side effects.
Now, let's say that you want to start your Futures lazily by simply wrapping their definition in a function. You'll notice that suddenly the execution doesn't necessarily take advantage of multiple threads and cores and takes 3 seconds instead of one, as in the following example:
def f1 = f_success().recover(_ => "not found")
def f2 = f_success().recover(_ => "not found")
def f3 = f_Fail().recover(_ => "not found")
for {
n <- f1
m <- f2
p <- f3
} {
println(n)
println(m)
println(p)
}
This is because of the semantics of flatMap. You can solve this problem by using a construct that doesn't imply some form of direct dependency between the steps of the calculation, like zip:
def f1 = f_success().recover(_ => "not found")
def f2 = f_success().recover(_ => "not found")
def f3 = f_Fail().recover(_ => "not found")
for (((n, m), p) <- f1.zip(f2).zip(f3)) {
println(n)
println(m)
println(p)
}
This runs again in ~1 second as you might expect.
Alternatively, if you want to still return the result as a Future, you can of course use yield as follows:
val future = for (((n, m), p) <- f1.zip(f2).zip(f3)) yield s"$n\n$m\n$p"
You can read more about for-comprehensions here on the Scala book.

Difference between { zip map } and { flatMap map } in Future of Scala

I'm reading 《hands on scala》, and one of its exercise is parallelizing merge sort.
I want to know why for-comprehension, which can be translated into flatMap and map, takes more time than zip and map.
my code:
def mergeSortParallel0[T: Ordering](items: IndexedSeq[T]): Future[IndexedSeq[T]] = {
if (items.length <= 16) Future.successful(mergeSortSequential(items))
else {
val (left, right) = items.splitAt(items.length / 2)
for (
l <- mergeSortParallel0(left);
r <- mergeSortParallel0(right)
) yield merge(l, r)
}
}
the standard answer provided by book:
def mergeSortParallel0[T: Ordering](items: IndexedSeq[T]): Future[IndexedSeq[T]] = {
if (items.length <= 16) Future.successful(mergeSortSequential(items))
else {
val (left, right) = items.splitAt(items.length / 2)
mergeSortParallel0(left).zip(mergeSortParallel0(right)).map{
case (sortedLeft, sortedRight) => merge(sortedLeft, sortedRight)
}
}
}
flatMap or map are sequential operations on Scala Future and on their own have nothing to do with running things in parallel. They can be viewed as simple callbacks executed when a Future completes. Or in other words, provided code inside map(...) or flatMap(...) will start to execute only when the previous Future is finished.
zip on the other hand will run your Futures in parallel and return the result as a Tuple when both of them are complete. Similarly, you could use zipWith which takes a function to transform the results of two Futures (combines zip and map operations):
mergeSortParallel0(left).zipWith(mergeSortParallel0(right)){
case (sortedLeft, sortedRight) => merge(sortedLeft, sortedRight)
}
Another way to achieve parallelism is to declare Futures outside for-comprehension. This works as Futures in Scala are 'eager' and they start as soon as you declare them (assign to val):
def mergeSortParallel0[T: Ordering](items: IndexedSeq[T]): Future[IndexedSeq[T]] = {
if (items.length <= 16) Future.successful(mergeSortSequential(items))
else {
val (left, right) = items.splitAt(items.length / 2)
val leftF = mergeSortParallel0(left)
val rightF = mergeSortParallel0(right)
for {
sortedLeft <- leftF
sortedRight <- rightF
} yield {
merge(sortedLeft, sortedRight)
}
}
}

.zip three futures in Scala [duplicate]

This question already has answers here:
Return Future[(Int,Int)] instead of (Future[Int],Future[Int])
(2 answers)
Closed 5 years ago.
I need the result variable below to contain Future[(String,String,String)] with the result of futures f1, f2 and f3, but instead I'm getting Future[((String, String), String)]. I need the three futures to run in parallel. How to make this work?
def futureA = Future { "A" }
def futureB = Future { "B" }
def futureC = Future { "C" }
def futureFunc = {
val cond1 = 1
val cond2 = 0
val f1 = if (cond1 > 0)
futureA
else
Future {""}
val f2 = if (cond2 > 0)
futureB
else
Future {""}
val f3 = futureC
val fx = f1.zip(f2)
val result = fx.zip(f3)
}
If you create your futures beforehand, you can combine them in a for comprehension and they will run in parallel:
for {
a <- f1
b <- f2
c <- f3
} yield (a, b, c)
res0: scala.concurrent.Future[(String, String, String)]
I tried to create more solutions and here is result:
def futureFunc = {
val cond1 = 1
val cond2 = 0
val f1 = if (cond1 > 0)
futureA
else
Future {""}
val f2 = if (cond2 > 0)
futureB
else
Future {""}
val f3 = futureC
//#1
Future.sequence(List(f1, f2, f3)).map {
case List(a, b, c) => (a, b, c)
}
//#2
for{
f11 <- f1
f22 <- f2
f33 <- f3
} yield (f11, f22, f33)
//#3
f1.zip(f2).zip(f3).map{
case ((f11,f22),f33) => (f11,f22,f33)
}
}
First one uses Future sequence, for creating Future[List[]] and then mapping this list for tuple (because of type safety we don't have method for tupling list).
Second is usage of for comprehension as described by Sascha, as you may know it is syntactic sugar for maps and flatmaps which is preferred to work with futures.
Last one is using zips, as you wanted, but you still need to map last future to obtain tuple which you want.
All operations are non blocking, but for all operations you need to know exactly futures which you will be using. You can use additional libraries for tupling lists, and then use first solution for not well known amount for futures. For readability i think for comprehension is best.

Unexpected Future.map() execution order

I have a following Scala program:
object FutureMapTest extends App {
println("start")
val f: Future[Long] = Future {
Thread.sleep(2000)
val x = 1
println(s"started with ${x}")
x
}
f.map { i =>
println(s"mapped to ${i*2}")
}
f.map {
val nothing = "nothing"
println(s"mapped to ${nothing}")
_ * 2
}
Thread.sleep(3000)
println("end")
}
What I'd expect it to print on the console is
start
started with 1
followed by (in any order):
mapped to 2
mapped to nothing
followed by
end
What it actually prints is:
start
mapped to nothing
started with 1
mapped to 2
end
So, it seems like the second "map" block gets executed immediately, without waiting for the original future to complete. How is that possible?
You can even remove Thread.sleep() from the original future block, the result would still be the same.
There are a couple sources of confusion here.
This:
f.map {
val nothing = "nothing"
println(s"mapped to ${nothing}")
_ * 2
}
Expands to:
f.map {
val nothing = "nothing"
println(s"mapped to ${nothing}")
i => i * 2
}
What does this mean? Future#map expects a function argument of a A => B for some Future[A]. The expression:
val nothing = "nothing"
println(s"mapped to ${nothing}")
i => i * 2
..Evaluates to Long => Long, but the val assignment and println are evaluated first because they are part of the expression that returns the function. i => i * 2 isn't executed until f completes. This is similar to (Scala puzzler 001):
scala> List(1, 2, 3) map {
| val a = 1 // this only happens once, not three times
| i => a + i + 1
| }
res0: List[Int] = List(3, 4, 5)
Changing it to this will exhibit the behavior you expect (now that val assignment and println are part of the function body):
f.map { i =>
val nothing = "nothing"
println(s"mapped to ${nothing}")
i * 2
}
Here's another way to look at it:
f.map {
println("evaluated immediately")
i => { println("evaluated after f"); i * 2 }
}

Can I use Action.async with multiple Futures?

In a previous SO question, I got advice on using Scala Futures with PlayFramework, thank you. Now things have gotten a bit more complicated. Let's say that before I just had to map where fruit could be found:
def getMapData(coll: MongoCollection[Document], s: String): Future[Seq[Document]] = ...
def mapFruit(collection: MongoCollection[Document]) = Action.async {
val fut = getMapData(collection, "fruit")
fut.map { docs: Seq[Document] =>
Ok(docs.toJson)
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
}
It turns out that people care more about Apples than Bananas or Cherries, so if no more than 100 items should appear on the map, people want Apples to have priority over Bananas and Cherries, but not more than some percentage of items on a map should be Apples. Some function pickDocs determines the proper mix. I thought something like this might just work, but no:
def mapApplesBananasCherries(collection: MongoCollection[Document]) = Action.async {
val futA = getMapData(collection, "apples")
val futB = getMapData(collection, "bananas")
val futC = getMapData(collection, "cherries")
futA.map { docsA: Seq[Document] =>
futB.map { docsB: Seq[Document] =>
futC.map { docsC: Seq[Document] =>
val docsPicked = pickDocs(100, docsA, docsB, docsC)
Ok(docsPicked.toJson)
}
}
// won't compile without something here, e.g. Ok("whatever")
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
}
Life was simple when I just had one Future, but now I have three. What can I do to make this to (1) work and (2) again be simple? I can't really construct a web response until all three Futures have values.
Basically, you should use flatMap
futA.flatMap { docsA: Seq[String] =>
futB.flatMap { docsB: Seq[String] =>
futC.map { docsC: Seq[String] =>
docsPicked = pickDocs(100, docsA, docsB, docsC)
Ok(docsPicked.toJson)
}
}
}
Also, you can use for comprehension:
val res = for {
docsA <- futA
docsB <- futB
docsC <- futC
} yield Ok(pickDocs(100, docsA, docsB, docsC).toJson)
res.recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
If my understanding is that you want to execute apples, cherries and bananas in that priority, I would code it similar to this
import scala.concurrent.{Await, Future}
import scala.util.Random
import scala.concurrent.duration._
object WaitingFutures extends App {
implicit val ec = scala.concurrent.ExecutionContext.Implicits.global
val apples = Future {50 + Random.nextInt(100)}
val cherries = Future {50 + Random.nextInt(100)}
val bananas = Future {50 + Random.nextInt(100)}
val mix = for {
app <- apples
cher <- if (app < 100) cherries else Future {0}
ban <- if (app + cher < 100) bananas else Future {0}
} yield (app,cher,ban)
mix.onComplete {m =>
println(s"mix ${m.get}")
}
Await.result(mix, 3 seconds)
}
if apples returns more than 100 when the future completes, it doesn't wait until cherries or bananas are done, but returns a dummy future with 0. If it's not enough it will wait until cherries are executed and so on.
NB I didn't put much effort on how to signal the if, so I'm using the dummy future which might not be the best approach.
This doesn't compile because your nested future block is returning a Future[Future[Future[Response]]]. If you instead use flatMap on the futures, Your futures will not be nested.
If you want this to be a little less repetitive, you can use Future.sequence instead to kick off futures simultaneously. You can either use pattern matching to re-extract the lists:
val futureCollections = List("apples", "bananas", "cherries").map{ getMapData(collection, _) }
Future.sequence(futureCollections) map { case docsA :: docsB :: docsC :: Nil =>
Ok(pickDocs(100, docsA, docsB, docsC).toJson)
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
or you could just hand the pickDocs function a list of lists (sorted by priority) for it to pick from.
Future.sequence(futureCollections) map { docLists =>
Ok(pickDocs(docLists, 100, 0.75f).toJson)
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
This pickDocs implementation will take a percentage of the head of the list, unless there aren't enough documents in the full list, in which it takes more, then recursively apply the same percentage on the remaining slots lists.
def pickDocs[T](lists: List[List[T]], max: Int, dampPercentage: Float): List[T] = {
lists match {
case Nil => Nil
case head :: tail =>
val remainingLength = tail.flatten.length
val x = max - remainingLength
val y = math.ceil(max * dampPercentage).toInt
val fromHere = head.take(x max y)
fromHere ++ pickDocs(tail, max - fromHere.length, dampPercentage)
}
}
This is a very common pattern for Futures and similar classes that "contain values" (e.g. Option, List)
To combine the results you want to use the flatMap method and the resulting code is
def mapApplesBananasCherries(collection: MongoCollection[Document]) = Action.async {
val futA = getMapData(collection, "apples")
val futB = getMapData(collection, "bananas")
val futC = getMapData(collection, "cherries")
futA.flatMap { docsA =>
futB.flatMap { docsB =>
futC.map { docsC =>
val docsPicked = pickDocs(100, docsA, docsB, docsC)
Ok(docsPicked.toJson)
}
}
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
}
In fact it's so common that a special syntax exists to make it more readable, called for-comprehension: the following code is equivalent to the previous snippet
def mapApplesBananasCherries(collection: MongoCollection[Document]) = Action.async {
val futA = getMapData(collection, "apples")
val futB = getMapData(collection, "bananas")
val futC = getMapData(collection, "cherries")
for {
apples <- futA
bananas <- futB
cherries <- futC
} yield {
val docsPicked = pickDocs(100, apples, bananas, cherries)
Ok(docsPicked.toJson)
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
}