Convert into a For Comprehension in Scala - scala

Question

Testing this I can see that it works:
def twoHtmlFutures = Action { request =>
val async1 = as1.index(embed = true)(request) // Future[Result]
val async2 = as2.index(embed = true)(request) // Future[Result]
val async1Html = async1.flatMap(x => Pagelet.readBody(x)) // Future[Html]
val async2Html = async2.flatMap(x => Pagelet.readBody(x)) // Future[Html]
val source1 = Source.fromFuture(async1Html) // Source[Html, NotUsed]
val source2 = Source.fromFuture(async2Html) // Source[Html, NotUsed]
val merged = source1.merge(source2) // Source[Html, NotUsed]
Ok.chunked(merged)
}
But trying to put it into a For Comprehension is not working for me. This is what I tried:
def twoHtmlFutures2 = Action.async { request =>
val async1 = as1.index(embed = true)(request)
val async2 = as2.index(embed = true)(request)
for {
async1Res <- async1 // from Future[Result] to Result
async2Res <- async2 // from Future[Result] to Result
async1Html <- Pagelet.readBody(async1Res) // from Result to Html
async2Html <- Pagelet.readBody(async2Res) // from Result to Html
} yield {
val source1 = single(async1Html) // from Html to Source[Html, NotUsed]
val source2 = single(async2Html) // from Html to Source[Html, NotUsed]
val merged = source1.merge(source2) // Source[Html, NotUsed]
Ok.chunked(merged)
}
}
But this just jumps on-screen at the same time rather than at different times (streamed) as the first example does. Any helpers out there to widen my eyelids?Thanks

Answer 1

Monads are a sequencing shape and Futures models this as causal dependence (first-this-future-completes-then-that-future-completes):
val x = Future(something).map(_ => somethingElse) or Future(something).flatMap(_ => Future(somethingElse)
However, there's a little trick one can do in for comprehensions:
def twoHtmlFutures = Action { request =>
Ok.chunked(
Source.fromFutureSource(
for { _ <- Future.unit // For Scala version <= 2.11 use Future.successful(())
async1 = as1.index(embed = true)(request) // Future[Result]
async2 = as2.index(embed = true)(request) // Future[Result]
async1Html = async1.flatMap(x => Pagelet.readBody(x)) // Future[Html]
async2Html = async2.flatMap(x => Pagelet.readBody(x)) // Future[Html]
source1 = Source.fromFuture(async1Html) // Source[Html, NotUsed]
source2 = Source.fromFuture(async2Html) // Source[Html, NotUsed]
} yield source1.merge(source2) // Source[Html, NotUsed]
)
)
I describe this technique in greater detail in this blogpost.
An alternate solution to your problem could be:
def twoHtmlFutures = Action { request =>
Ok.chunked(
Source.fromFuture(as1.index(embed = true)(request)).merge(Source.fromFuture(as2.index(embed = true)(request))).mapAsyncUnordered(2)(b => Pagelet.readBody(b))
)
}

Answer 2

For comprehension and flatMap (which is its desugared version) are used to sequence things.
In the context of Future, this means that in a for comprehension, each of the statement is started only once the previous one has successfully ended.
In your case, you want two Futures run in parallel. This is not what flatMap (or for comprehension) is.
What your code do is the following:
do the first index call
when that's over, do the second index call
when that's over, do the first readBody
when that's over, do the second readBody
when that's over create two (synchronous) sources with the values from the two previous steps, merge them, and start returning the merged source as chunked response.
What your previous code did was
do the first index call
when that's over, do the first readBody
in the meantime, do the same for the second index and readBody
in the meantime, create a source that will output an element when the first readBody yields a result
in the meantime, do the same for the second
merge these two sources, and start all at once to give the merged output as a chunked response.
So, in this case, you start your chunked response just after receiving the request (but with nothing inside yet, waiting for the Futures to be resolved), while in the former case, you wait at each computation for the previous one to be over, even if you don't need its result to go on.
What you should remember, is that you should use flatMap on Future, only if you need the result from a previous computation or if you wish for another computation to be over before doing something else. The same goes for for comprehension, which is just a nice-looking way of chaining flatMaps.

Answer 3

Here's what your proposed for comprehension looks like after it's been desugared (courtesy of IntelliJ's "Desugar Scala code ..." menu option):
async1.flatMap((async1Res: Nothing) =>
async2.flatMap((async2Res: Nothing) =>
Pagelet.readBody(async1Res).flatMap((async1Html: Nothing) =>
Pagelet.readBody(async2Res).map((async2Html: Nothing) =>
Ok.chunked(merged)))))
As you can see, the nesting, and the concluding flatMap/map pair, are very different from your original code plan.
As a general rule, every <- in a single for comprehension is turned into a flatMap() except for the final one, which is a map(), and each is nested inside the previous.