Dependent futures - scala

Starting playing with Scala futures, I get stuck with dependent futures.
Let's get a example. I search for places and get a Future[Seq[Place]]. For each of theses places, I search for the closest subway stations (the service resurns a Future[List[Station]]).
I would write this:
Place.get()
.map { places =>
places.map { place =>
Station.closestFrom(place).map { stations =>
SearchResult(place, stations)
}
}
}
That thing will make me get a Future[Seq[Future[SearchResult]]]... which is... not what I would have expected.
What did I miss to get a Future[Seq[SearchResult]] ?
Thanks for all,
Alban

You are missing two Future concepts in your solution: flatMap and Future.sequence
To explain each:
flatMap is like map except instead of giving it a function from future.map(A => B) you give it a function from future.flatMap(A => Future[B]). This way you can chain Futures together.
Future.sequence is a helper function that combines a list of futures to a future of a list: Seq[Future[A]] => Future[Seq[A]]
Using these two features of the Future API we can change your answer to be:
Place.get().flatMap { places =>
Future.sequence(places.map { place =>
Station.closestFrom(place).map { stations =>
SearchResult(place, stations)
}
})
}

Short version
Working with futures is generaly easier using for-comprehension rather than directly map/flatMap. In your situation it should look like this:
for {places <- Place.get()
searchResults <- Future.traverse(places)(place => for (stations <- Station.closestFrom(place))
yield SearchResult(place,stations)
)
} yield searchResults
Detailed Version
Future being a monad, it offers you several ways to chain your operations.
If you want to apply an 'regular' function f : A => B to what's inside the box myfuture : Future[A], indeed map is the way to get a Future[B]. But in the present situation Station.closestFrom a does not give you a List[Stattion] but a Future[List[Station]].
If you want to apply a monadic operation h : A => Future[B] or chain several of them (here Places.get and Station.closestFrom), flatMap is the way to go. Apply h to a Future[A] gives you a Future[B].
If you want to apply a monadic operation h : A => Future[B] to a collection like a places : Seq[A], you should use Future.traverse : Seq[A] => (A => Future[B]) => Future[Seq[B]].
Furthermore, Scala's for-compresention is just syntactic sugar for flatMap/map so instead of writing complex code using those directly you can use a clean and clear for loop. The loop:
for { variable1 <- f1
variable2 <- f2
} yield expression
is equivalent to (without optimisations) :
f1.flatMap( variable1 => f2.map(variable2 => expression))
Don't hesitate to use for-comprehension, it really helps.

Related

Scala: How do I convert this flatMap/map into a for-comprehension?

I am having trouble with understanding how to convert this into a for-comprehension. Could someone please explain how this can be done?
(parseCommand(x)
.flatMap { c =>
calculate(c).map(r => renderResult(c, r))
})
.left
.map(e => e.value)
.merge
```
You haven't provided enough information to answer your question.
What type does parseCommand() return?
What type does calculate() return?
Why translate to a for comprehension? What's your goal?
Assuming that parseCommand() and calculate() return the same or compatible types, then the 1st map() and flatMap() can be translated like so:
(for {
c <- parseCommand(x)
r <- calculate(c)
} yield renderResult(c, r)
).left
.map(e => e.value)
.merge
...
The 2nd map() can't be folded in to this for because there can be only one map() per for comprehension. You could, however, turn it into its own, nested, for comprehension, but remember that it makes no difference to the compiler, so the only real reason to use for is to enhance code readability, and nested fors seldom achieve that.
I managed to convert it like this:
(for {
c <- parseCommand(x)
r <- calculate(c)
} yield renderResult(c, r)).left.map(x => x.value).merge
}

Chaining together operations on an Option to a Future, then back to an Option?

I'm writing an authentication client that takes an Option[Credentials] as a parameter. This Credentials object has a .token method on it which I will then use to construct an HTTP request to post to an endpoint. This returns a Future[HttpResponse], which I then need to validate, unmarshal, and then convert back to my return type, which is an Option[String].
My first thought was to use a for comprehension like this:
val resp = for {
c <- creds
req <- buildRequest(c.token)
resp <- Http().singleRequest(req)
} yield resp
but then I found out that monads cannot be composed like that. My next thought is to do something like this:
val respFut = Http().singleRequest(buildRequest(token))
respFut.onComplete {
case Success(resp) => Some("john.doe")//do stuff
case Failure(_) => None
}
Unfortunately onComplete returns a unit, and map leaves me with a Future[Option[String]], and the only way I currently know to strip off the future wrapper is using the pipeTo methods in the akka framework. How can I convert this back to just an option string?
Once you've got a Future[T], it's usually good practice to not try to unbox it until you absolutely have to. Can you change your method to return a Future[Option[String]]? How far up the call stack can you deal with futures? Ideally it's all the way.
Something like this will give you a Future[Option[String]] as a result:
val futureResult = creds map {
case Some(c) => {
val req = buildRequest(c.token)
val futureResponse = Http().singleRequest(req)
futureResponse.map(res => Some(convertResponseToString(res)))
}
case None => Future(None)
}
If you really need to block and wait on the result, you can do Await.result as described here.
And if you want to do it in a more monadic style (in a for-comprehension, like you tried), cats has an OptionT type that will help with that, and I think scalaz does as well. But whether you want to get into either of those libraries is up to you.
It's easy to "upgrade" an Option to a Future[Option[...]], so use Future as your main monad. And deal with the simpler case first:
val f: Future[Option[String]] =
// no credential? just wrap a `None` in a successful future
credsOpt.fold(Future.successful(Option.empty[String])) {creds =>
Http()
.singleRequest(buildRequest(creds.token))
.map(convertResponseToString)
.recover {case _ => Option.empty[String]}
}
The only way to turn that future into Option[String] is to wait for it with Await.result(...)... but it's better if that future can be passed along to the next caller (no blocking).
I'm not 100% certain about what all your types are, but it seems like you want a for comprehension that mixes option and futures. I've often been in that situation and I find I can just chain my for comprehensions as a way to make the code look a bit better.
val resp = for {
c <- creds
req <- buildRequest(c.token)
} yield for {
resp <- Http().singleRequest(req)
} yield resp
resp becomes an Option[Future[HttpResponse]] which you can match / partial func around with None meaning the code never got to execute because it failed its conditions. This is a dumb little trick I use to make comprehensions look better and I hope it gives you a hint towards your solution.

Scala: Most idiomatic way to conditionally execute one Future depending on another?

What is the most idiomatic and concise way to perform the following: One future that may or may not execute depending or a result of a previous future execution, like so:
def getItFromHere : Future[Option[Something]] = ...
def getItFromThere : Future[Option[Something]] = ...
def getIt : Future[Option[Something]] = {
for {
maybeSomething <- getItFromHere
probablySomething <- maybeSomething.getOrElse(getItFromThere) // Obviously can't be done!!!
}
yield probablySomething
}
A specific use-case example:
Get an item from cache and only if it wasn't found in cache, get it from the database.
I'm taking an assumption here that getItFromHere and getItFromThere will not fail with an exception. They will either return Some[Something] or None. Feel free to take this assumption into account, or giving a better without it.
Note: I understand the internal mechanics of for-comprehension which is actually translated to map/flatMap/filter internally.
You could fail the future if no item is found in the cache and then recover it with the retrieval.
// assuming getItFromHere() fails if no item is found
getItFromHere() recoverWith { case _ => getItFromThere() }
Or with the Option you could do it like this:
getItFromHere() flatMap {
case Some(x) => Future.successful(Some(x))
case None => getItFromThere()
}
Actually your example is almost there. You only need to match the Future type expected by the monadic composition:
for {
fut <- someFuture
something <- fut.map(x=>someFuture).getOrElse(someBetterFuture)
} yield something
For the sake of simplicity, here someFuture is a val. If you use a def, avoid re-calculating the operation by packaging your result back into a future. Folding that into the question's code:
for {
maybeSomething <- getItFromHere
probablySomething <- maybeSomething.map(x=> Future.succcessful(maybeSomething)).getOrElse(getItFromThere)
} yield probablySomething
if you use the OptionT transformer from scalaz:
def getItFromHere : OptionT[Future,Something] = OptionT(...)
def getItFromThere : OptionT[Future,Something] = OptionT(...)
def getIt : Future[Option[Something]] = (getItFromHere orElse getItFromThere).run

Mapping many Eithers to one Either with many

Say I have a monadic function in called processOne defined like this:
def processOne(input: Input): Either[ErrorType, Output] = ...
Given a list of "Inputs", I would like to return a corresponding list of "Outputs" wrapped in an Either:
def processMany(inputs: Seq[Input]): Either[ErrorType, Seq[Output]] = ...
processMany will call processOne for each input it has, however, I would like it to terminate the first time (if any) that processOne returns a Left, and return that Left, otherwise return a Right with a list of the outputs.
My question: what is the best way to implement processMany? Is it possible to accomplish this behavior using a for expression, or is it going to be necessary for me to iterate the list myself recursively?
With Scalaz 7:
def processMany(inputs: Seq[Input]): Either[ErrorType, Seq[Output]] =
inputs.toStream traverseU processOne
Converting inputs to a Stream[Input] takes advantage of the non-strict traverse implementation for Stream, i.e. gives you the short-circuiting behaviour you want.
By the way, you tagged this "monads", but traversal requires only an applicative functor (which, as it happens, is probably defined in terms of the monad for Either). For further reference, see the paper The Essence of the Iterator Pattern, or, for a Scala-based interpretation, Eric Torreborre's blog post on the subject.
The easiest with standard Scala, which doesn't evaluate more than is necessary, would probably be
def processMany(inputs: Seq[Input]): Either[ErrorType, Seq[Output]] = {
Right(inputs.map{ x =>
processOne(x) match {
case Right(r) => r
case Left(l) => return Left(l)
}
})
}
A fold would be more compact, but wouldn't short-circuit when it hit a left (it'd just keep carrying it along while you iterated through the entire input).
For now, I've decided to just solve this using recursion, as I am reluctant to add a dependency to a library (Scalaz).
(Types and names in my application have been changed here in order to appear more generic)
def processMany(inputs: Seq[Input]): Either[ErrorType, Seq[Output]] = {
import scala.annotation.tailrec
#tailrec
def traverse(acc: Vector[Output], inputs: List[Input]): Either[ErrorType, Seq[Output]] = {
inputs match {
case Nil => Right(acc)
case input :: more =>
processOne(input) match {
case Right(output) => traverse(acc :+ output, more)
case Left(e) => Left(e)
}
}
}
traverse(Vector[Output](), inputs.toList)
}

Idiomatic way to find a matching line in Scala

I have an Iterable[String] representing the lines in a file and I'd like to find the first line in that sequence that matches a regular expression and return a numerical value extracted by the regex. The file is big enough that it wouldn't make sense to load the whole thing into memory and then call toString() or something, so I'll need to go through it a line at a time.
Here's what I have (it works):
val RateRegex : Regex = ".....".r
def getRate(source : Source) : Option[Double] = {
import java.lang.Double._
for(line <- source.getLines() ) {
line match {
case RateRegex(rawRate) => return Some(parseDouble(rawRate))
case None => ()
}
}
return None
}
This seems ugly to me. It feels very imperative and case None => () might as well be replaced with a comment that says "you're doing it wrong."
I think I want something like def findFirstWhereNonNone(p : Function[A,Option[B]]) => Option[B] where the collection's elements are of type A.
Are there built-in methods that would let me do this in a more functional way? Should I just write that method?
P.S. While I'm at it, is there an alternative to using java.lang.Double.parseDouble? Scala's Double class doesn't expose it.
P.P.S I've seen a lot of posts on SO suggesting that the Source API shouldn't be used in production, but they're all from 2008 and 2009. Is that still the case? If so, what should I use for IO?
Update
I now have:
import util.matching.Regex.Groups
for{line <- source.getLines()
Groups(rawRate) <- RateRegex.findFirstMatchIn(line)} {
return Some(parseDouble(rawRate))
}
return None
which feels a lot better to me.
EDIT: This third alternative ia quite neat:
source
.getLines()
.collectFirst{ case RateRegex(x) => x.toDouble}
Not sure if it's more functional, but you can use the behaviour of foreach/for-comprehensions on Options
def getRate(source : Source) : Option[Double] = {
for {line <- source.getLines()
rawRate <- RateRegex.findFirstIn(line)}
return Some(rawRate toDouble)
return None
}
This works too (quite similar to EasyAngel's answer):
source
.getLines()
.map{RateRegex.findFirstMatchIn(_)}
.filter{_.isDefined}
.map{_.get.group(0).toDouble}
.head
.toList
.headOption
The last three are a little ugly. The take(1) is to ensure we only evaluate up to the first match. The toList is to force the evaluation, and the headOption to extract the first value as Some() or None if there is none. Is there a more idiomatic way of doing this?
Here is one of the possible solutions:
def getRates(source : Source) = source.getLines.map {
case RateRegex(rate) => Some(rate toDouble)
case _ => None
} filter (_ isDefined) toList
Please note, that this function now returns now List[Option[Double]] of all found rates. It's also important, that Iterator remains lazy until I call toList
Update
As was asked in comments, here is solution, that returns only first occurrence:
def getRate(source : Source): Option[Double] = source.getLines.map {
case RateRegex(rate) => Some(rate toDouble)
case _ => None
} find (_ isDefined) getOrElse None