Future[Option[Boolean]] in a for comprehension.. Simple right? - scala

Suppose I have:
val res:Future[Option[Boolean]] = Future(Some(true))
and I can do:
res.map(opt => opt.map(r => print(!r)))
I guess the for comprehension for this would be:
for {
opt <- res
r <- opt
} yield (print(!r))
but this does not work! I get an error ie:
error: type mismatch;
found : Option[Unit]
required: scala.concurrent.Future[?]
r <- opt
How do I use a Future[Option[Boolean]] in a for comprehension to extract or convert the Boolean?
Note: this is a simplification of the problem I have at the moment with many Future[Option[Boolean]] variables that I would like to use together in a for comprehension.

A for-comprehension really makes this look like it should all work, doesn't it? But let's think about what you're asking for.
First, note that for un-nests:
for {xs <- List(List(5)); x <- xs} yield x
produces
List(5)
Now, without even getting into the type signatures or desugaring, we can think about replacing List with some arbitrary type T, and we'll call the contained type A:
for { xs <- T(T(a: A)); x <- xs } yield x
and we should get a
T[A]
back (presumably the one we put in, but the types don't actually promise us that).
Okay, but what about
for { xs <- T(U(a: A)); x <- xs } yield x
? This is strictly more general than the case where the two things have the same nesting. Well, if T and U both have a common supertype S, then we can just view the whole thing as S(S(a: A)), so we'll at least get an S back. But what about in the general case?
The bottom line is that it depends. For example, let's consider the case where T=Future, U=Option. We have the following possibilities:
Success(Some(a))
Success(None)
Failure(t: Throwable)
Now, can we come up with any coherent policy for unwrapping? If we unwrap into a Future, then what A do you use for the Success(None) case? You don't have one available to return. Likewise, if you try to vanquish the outer Future, how do you know, without explicitly stating it somehow to the compiler, that Failure should be mapped to None (if indeed it should--maybe it should go to a default!).
So the bottom line is that you just can't do this correctly in general without specifying what is supposed to happen for every pair T[U[_]]. (I encourage the interested reader to peruse tutorials on monads and monad transformers.)
There is a way out, though: if you can explicitly turn your U into a T, or your T into your U, you can take advantage of the unwrapping capability. It's pretty easy to turn an Option into a Future , so the easiest solution is
for { opt <- res; r <- Future(opt.get) } yield r
(where just let the exception get thrown on none.get). Alternatively, you can turn the Future into an Option with the slightly ugly
for { opt <- res.value.flatMap(_.toOption); r <- opt } yield r

Equivalent code for
for {
opt <- res
r <- opt
} yield (print(!r))
is not
res.map(opt => opt.map(r => print(!r)))
but
res.flatMap(opt => opt.map(r => print(!r)))
And it makes no sense in this case.
For the chain of maps you could use nested for-comprehensions
for { opt <- res }
for { r <- opt }
print(!r)
But map looks better.

Well, for comprehensions are deceiving in the way they look. Your for-comprehension expands to:
res.flatMap(opt => opt.map(r => print(!r))
Which is obviously wrong as flatMap expects a return type of Future[T] and you are providing Option[Unit]
Though sometimes, for code tidiness you would want to have a single for loop with many such expressions. In those cases you could:
scala> implicit def toFuture[T](t: => T):Future[T] = {
| val p = Promise[T]()
| p.tryComplete(Try(t))
| p.future
| }
scala> for {
| opt <- res
| r <- opt
| } yield {print(!r)}
false
The above produces:
res.flatMap[Option[Unit]](((opt: Option[Boolean]) =>
toFuture[Option[Unit]](opt.map[Unit](((r: Boolean) => print(!r))))))
Edit: Though you need to take all the pain if you are using yield. If you do not wish to use for comprehension as an expression, then you can do as you desired:
scala> val i = Future(Some(true))
i: scala.concurrent.Future[Some[Boolean]] = scala.concurrent.impl.Promise$DefaultPromise#6b24a494
scala> val j = Option(1)
j: Option[Int] = Some(1)
scala> val k = Right(1).right
k: scala.util.Either.RightProjection[Nothing,Int] = RightProjection(Right(1))
scala>
| for{
| x <- i
| y <- j
| z <- k
| }{
| println(i,j,k)
| }
(scala.concurrent.impl.Promise$DefaultPromise#6b24a494,Some(1),RightProjection(Right(1)))
This way no implicit is required. As compiler uses foreach at every junction. -Xprint:typer gives:
i.foreach[Unit](((x: Option[Boolean]) =>
j.foreach[Any](((y: Int) =>
k.foreach[Unit](((z: Int) => println(scala.this.Tuple3.apply[scala.concurrent.Future[Option[Boolean]], Option[Int], Either.RightProjection[Nothing,Int]](i, j, k))))))))
}

Related

Combining List, Future and Option in for-comprehension - scalaz

I've got following problem:
val sth: Future[Seq[T, S]] = for {
x <- whatever: Future[List[T]]
y <- x: List[T]
z <- f(y): Future[Option[S]]
n <- z: Option[S]
} yield y: T -> n: S
I would want to make this code to work(I guess everyone understands the idea as I've added types).
By "to work" I mean, that I would want to stay with the for-comprehension structure and fulfil expected types in the end. I know there are "ugly" ways to do it, but I want to learn how to do it pure :)
As I read the internet I've reached the conclusion that my problem may be solved by the monad transformers & scalaz. Unfortunately, I couldn't find an example to help understand better how should I proceed.
At the moment I've tried scalaz and Eff monad libs, but I guess I still don't understand how it works because I couldn't solve my problem.
I will be grateful for any help.
EDIT: It supposed to be future of sequence, also regarding the "whatever" I get it as a parameter of the function, sorry for misleading you
You could do something like what you need using the scalaz ListT monad transformer
object Test {
import scalaz._
import ListT._
type T = String
type S = Int
val whatever: Future[List[T]] = ??? // you get this somewhere
def f(y: T): Future[Option[S]] = ??? // function that returns future of option
val sth: Future[List[(T, S)]] = (for {
y <- listT(whatever)
// you cannot mix list and option, but you can convert the option to a list of 1 item
n <- listT(f(y).map(_.toList))
} yield y -> n).run
}
N.B.: Since you start with a future, you cannot return a Seq[(T,S)], you can only have a future. You have to call Await.result if you want to block and get the result.
The problem with for comprehension is that it's not some kind of magic monadic "unwrapper", it's just a sequence of map, flatMap and filter.
As you may know map and flatMap operate only on "inner" type, leaving "outer" type of monad unchanged. This means you can't do this:
for {
x <- whatever: Future[List[T]]
y <- x: List[T]
} yield y
inside single for. Instead, you can do something like this:
for (x <- whatever: Future[List[T]])
yield for (y <- x: List[T]) yield y
Which looks kinda ugly.
Back to your case, I's easier to write whole transformation explicitly using map and flatMap, as it gives you greater visibility and control:
whatever.flatMap {
x: List[T] =>
Future.sequence(x.map {
y: T => f(y).map(y -> _)
}).map(_.collect {
case (y, Some(n)) => y -> n
})
}
Also, #trustnoone mentioned, you can't get rid of the Future without explicitly calling Await.

Scala Either map Right or return Left

Is it possible to handle Either in similar way to Option? In Option, I have a getOrElse function, in Either I want to return Left or process Right. I'm looking for the fastest way of doing this without any boilerplate like:
val myEither:Either[String, Object] = Right(new Object())
myEither match {
case Left(leftValue) => value
case Right(righValue) =>
"Success"
}
In Scala 2.12,
Either is right-biased, which means that Right is assumed to be the default case to operate on. If it is Left, operations like map, flatMap, ... return the Left value unchanged
so you can do
myEither.map(_ => "Success").merge
if you find it more readable than fold.
You can use .fold:
scala> val r: Either[Int, String] = Right("hello")
r: Either[Int,String] = Right(hello)
scala> r.fold(_ => "got a left", _ => "Success")
res7: String = Success
scala> val l: Either[Int, String] = Left(1)
l: Either[Int,String] = Left(1)
scala> l.fold(_ => "got a left", _ => "Success")
res8: String = got a left
Edit:
Re-reading your question it's unclear to me whether you want to return the value in the Left or another one (defined elsewhere)
If it is the former, you can pass identity to .fold, however this might change the return type to Any:
scala> r.fold(identity, _ => "Success")
res9: Any = Success
Both cchantep's and Marth's are good solutions to your immediate problem. But more broadly, it's difficult to treat Either as something fully analogous to Option, particularly in letting you express sequences of potentially failable computations for comprehensions. Either has a projection API (used in cchantep's solution), but it is a bit broken. (Either's projections break in for comprehensions with guards, pattern matching, or variable assignment.)
FWIW, I've written a library to solve this problem. It augments Either with this API. You define a "bias" for your Eithers. "Right bias" means that ordinary flow (map, get, etc) is represented by a Right object while Left objects represent some kind of problem. (Right bias is conventional, although you can also define a left bias if you prefer.) Then you can treat the Either like an Option; it offers a fully analogous API.
import com.mchange.leftright.BiasedEither
import BiasedEither.RightBias._
val myEither:Either[String, Object] = ...
val o = myEither.getOrElse( "Substitute" )
More usefully, you can now treat Either like a true scala monad, i.e. use flatMap, map, filter, and for comprehensions:
val myEither : Either[String, Point] = ???
val nextEither = myEither.map( _.x ) // Either[String,Int]
or
val myEither : Either[String, Point] = ???
def findGalaxyAtPoint( p : Point ) : Either[String,Galaxy] = ???
val locPopPair : Either[String, (Point, Long)] = {
for {
p <- myEither
g <- findGalaxyAtPoint( p )
} yield {
(p, g.population)
}
}
If all processing steps succeeded, locPopPair will be a Right[Long]. If anything went wrong, it will be the first Left[String] encountered.
It's slightly more complex, but a good idea to define an empty token. Let's look at a slight variation on the for comprehension above:
val locPopPair : Either[String, (Point, Long)] = {
for {
p <- myEither
g <- findGalaxyAtPoint( p ) if p.x > 1000
} yield {
(p, g.population)
}
}
What would happen if the test p.x > 1000 failed? We'd want to return some Left that signifies "empty", but there is no universal appropriate value (not all Left's are Left[String]. As of now, what would happen is the code would throw a NoSuchElementException. But we can specify an empty token ourselves, as below:
import com.mchange.leftright.BiasedEither
val RightBias = BiasedEither.RightBias.withEmptyToken[String]("EMPTY")
import RightBias._
val myEither : Either[String, Point] = ???
def findGalaxyAtPoint( p : Point ) : Either[String,Galaxy] = ???
val locPopPair : Either[String, (Point, Long)] = {
for {
p <- myEither
g <- findGalaxyAtPoint( p ) if p.x > 1000
} yield {
(p, g.population)
}
}
Now, if the p.x > 1000 test fails, there will be no Exception, locPopPair will just be Left("EMPTY").
I guess you can do as follows.
def foo(myEither: Either[String, Object]) =
myEither.right.map(rightValue => "Success")
In scala 2.13, you can use myEither.getOrElse
Right(12).getOrElse(17) // 12
Left(12).getOrElse(17) // 17

Performance difference in flatMap compared to for-comprehension

I am still new to Scala, and one of the things I read about is that for-comprehension is equivalent to a flatMap to a certain extent. However, in my code (below), flatMap is taking at least as twice as long to compute. What could be the reason for this?
This is the slow one:
facts.flatMap(f => factActionsMap(f)).filter(_.isValid(facts))
This is the fast equivalent one:
for {
f <- facts
a <- factActionsMap(f)
if a.isValid(facts)
} yield a
factActionsMap is a map between Fact and a Set[Action].
facts is just a Set[Fact].
Let's check the translation:
scala> trait Fact
defined trait Fact
scala> trait Action { def isValid(s: Set[Fact]): Boolean }
defined trait Action
scala> def facts: Set[Fact] = ???
facts: Set[Fact]
scala> def factActionsMap: Map[Fact, Set[Action]] = ???
factActionsMap: Map[Fact,Set[Action]]
scala> import scala.reflect.runtime.{universe => u}
import scala.reflect.runtime.{universe=>u}
scala> val expr = u reify {
| for {
| f <- facts
| a <- factActionsMap(f)
| if a.isValid(facts)
| } yield a
| }
expr: reflect.runtime.universe.Expr[scala.collection.immutable.Set[Action]] = Expr[scala.collection.immutable.Set[Action]]($read.f
acts.flatMap(((f) => $read.factActionsMap.apply(f).withFilter(((a) => a.isValid($read.facts))).map(((a) => a))(Set.canBuildFrom)))
(Set.canBuildFrom))
scala> u show expr.tree
res0: String = $read.facts.flatMap(((f) => $read.factActionsMap.apply(f).withFilter(((a) => a.isValid($read.facts))).map(((a) => a
))(Set.canBuildFrom)))(Set.canBuildFrom)
So, removing REPL stuff (everything starting with $) and the implicit parameters, plus reformatting, we get:
facts.flatMap(f => factActionsMap(f).withFilter(a => a.isValid(facts)).map(a => a))
There are two main differences between this and what you came up with. First, withFilter is applied to fastActionsMap(f) result, whereas you apply to facts.flatMap result. This means flatMap will work over all results, instead of just the ones accepted.
Second, it uses withFilter instead of filter, which avoids creating an extra collection.
Unless I'm mistaken, a more precise equivalent to the second expression would be
facts.flatMap(f => factActionsMap(f).withFilter(_.isValid(facts)))
withFilter is like filter, except that is is lazy, so if you're not doing anything with the contents of the Set, you haven't actually applied the filter. You could check this hypothesis by, for example, printing all elements of the set before calling System.currentTimeMillis for the second time.

for..else for Option types in Scala?

Suppose I have two Options and, if both are Some, execute one code path, and if note, execute another. I'd like to do something like
for (x <- xMaybe; y <- yMaybe) {
// do something
}
else {
// either x or y were None, handle this
}
Outside of if statements or pattern matching (which might not scale if I had more than two options), is there a better way of handling this?
Very close to your syntax proposal by using yield to wrap the for output in an Option:
val result = {
for (x <- xMaybe; y <- yMaybe) yield {
// do something
}
} getOrElse {
// either x or y were None, handle this
}
The getOrElse block is executed only if one or both options are None.
You could pattern match both Options at the same time:
(xMaybe, yMaybe) match {
case (Some(x), Some(y)) => "x and y are there"
case _ => "x and/or y were None"
}
The traverse function in Scalaz generalises your problem here. It takes two arguments:
T[F[A]]
A => F[B]
and returns F[T[B]]. The T is any traversable data structure such as List and the F is any applicative functor such as Option. Therefore, to specialise, your desired function has this type:
List[Option[A]] => (A => Option[B]) => Option[List[B]]
So put all your Option values in a List
val z = List(xMaybe, yMaybe)
Construct the function got however you want to collection the results:
val f: X => Option[Y] = ...
and call traverse
val r = z traverse f
This programming patterns occurs very often. It has a paper that talks all about it, The Essence of the Iterator Pattern.
note: I just wanted to fix the URL but the CLEVER edit help tells me I need to change at least 6 characters so I include this useful link too (scala examples):
http://etorreborre.blogspot.com/2011/06/essence-of-iterator-pattern.html
Why would something like this not work?
val opts = List[Option[Int]](Some(1), None, Some(2))
if (opts contains None) {
// Has a None
} else {
// Launch the missiles
val values = opts.map(_.get) // We know that there is no None in the list so get will not throw
}
If you don't know the number of values you are dealing with, then Tony's answer is the best. If you do know the number of values you are dealing with then I would suggest using an applicative functor.
((xMaybe |#| yMaybe) { (x, y) => /* do something */ }).getOrElse(/* something else */)
You said you want the solution to be scalable:
val optional = List(Some(4), Some(3), None)
if(optional forall {_.isDefined}) {
//All defined
} else {
//At least one not defined
}
EDIT: Just saw that Emil Ivanov's solution is a bit more elegant.
Starting Scala 2.13, we can alternatively use Option#zip which concatenates two options to Some tuple of their values if both options are defined or else None:
opt1 zip opt2 match {
case Some((x, y)) => "x and y are there"
case None => "x and/or y were None"
}
Or with Option#fold:
(opt1 zip opt2).fold("x and/or y were None"){ case (x, y) => "x and y are there" }
For scaling to many options, try something along these lines:
def runIfAllSome[A](func:(A)=>Unit, opts:Option[A]*) = {
if(opts.find((o)=>o==None) == None) for(opt<-opts) func(opt.get)
}
With this, you can do:
scala> def fun(i:Int) = println(i)
fun: (i: Int)Unit
scala> runIfAllSome(fun, Some(1), Some(2))
1
2
scala> runIfAllSome(fun, None, Some(1))
scala>
I think the key point here is to think in term of types as what you want to do. As I understand it you want to iterate over a list of Option pairs and then do something based on a certain condition. So the interesting bit of your question would be , what would the return type look like you would except? I think it would look something like this: Either[List[Option], List [Option,Option]] . on the error side (left) you would accumulate the option which was paired with a None (and was left alone so to speak) . On the right side you sum the non empty options which represent your successful values. So we would just need a function which does exactly that. Validate each pair and accumulate it according to it's result( success - failure) . I hope this helps , if not please explain in more detail your usecase. Some links to implement what I described : http://applicative-errors-scala.googlecode.com/svn/artifacts/0.6/pdf/index.pdf and : http://blog.tmorris.net/automated-validation-with-applicatives-and-semigroups-for-sanjiv/

Issues with maps and their entries in Scala

I have a recursive function that takes a Map as single parameter. It then adds new entries to that Map and calls itself with this larger Map. Please ignore the return values for now. The function isn't finished yet. Here's the code:
def breadthFirstHelper( found: Map[AIS_State,(Option[AIS_State], Int)] ): List[AIS_State] = {
val extension =
for(
(s, v) <- found;
next <- this.expand(s) if (! (found contains next) )
) yield (next -> (Some(s), 0))
if ( extension.exists( (s -> (p,c)) => this.isGoal( s ) ) )
List(this.getStart)
else
breadthFirstHelper( found ++ extension )
}
In extension are the new entries that shall get added to the map. Note that the for-statement generates an iterable, not a map. But those entries shall later get added to the original map for the recursive call. In the break condition, I need to test whether a certain value has been generated inside extension. I try to do this by using the exists method on extension. But the syntax for extracting values from the map entries (the stuff following the yield) doesn't work.
Questions:
How do I get my break condition (the boolean statement to the if) to work?
Is it a good idea to do recursive work on a immutable Map like this? Is this good functional style?
When using a pattern-match (e.g. against a Tuple2) in a function, you need to use braces {} and the case statement.
if (extension.exists { case (s,_) => isGoal(s) } )
The above also uses the fact that it is more clear when matching to use the wildcard _ for any allowable value (which you subsequently do not care about). The case xyz gets compiled into a PartialFunction which in turn extends from Function1 and hence can be used as an argument to the exists method.
As for the style, I am not functional programming expert but this seems like it will be compiled into a iterative form (i.e. it's tail-recursive) by scalac. There's nothing which says "recursion with Maps is bad" so why not?
Note that -> is a method on Any (via implicit conversion) which creates a Tuple2 - it is not a case class like :: or ! and hence cannot be used in a case pattern match statement. This is because:
val l: List[String] = Nil
l match {
case x :: xs =>
}
Is really shorthand/sugar for
case ::(x, xs) =>
Similarly a ! b is equivalent to !(a, b). Of course, you may have written your own case class ->...
Note2: as Daniel says below, you cannot in any case use a pattern-match in a function definition; so while the above partial function is valid, the following function is not:
(x :: xs) =>
This is a bit convoluted for me to follow, whatever Oxbow Lakes might think.
I'd like first to clarify one point: there is no break condition in for-comprehensions. They are not loops like C's (or Java's) for.
What an if in a for-comprehension means is a guard. For instance, let's say I do this:
for {i <- 1 to 10
j <- 1 to 10
if i != j
} yield (i, j)
The loop isn't "stopped" when the condition is false. It simply skips the iterations for which that condition is false, and proceed with the true ones. Here is another example:
for {i <- 1 to 10
j <- 1 to 10
if i % 2 != 0
} yield (i, j)
You said you don't have side-effects, so I can skip a whole chapter about side effects and guards on for-comprehensions. On the other hand, reading a blog post I made recently on Strict Ranges is not a bad idea.
So... give up on break conditions. They can be made to work, but they are not functional. Try to rephrase the problem in a more functional way, and the need for a break condition will be replaced by something else.
Next, Oxbow is correct in that (s -> (p,c) => isn't allowed because there is no extractor defined on an object called ->, but, alas, even (a :: b) => would not be allowed, because there is no pattern matching going on in functional literal parameter declaration. You must simply state the parameters on the left side of =>, without doing any kind of decomposition. You may, however, do this:
if ( extension.exists( t => val (s, (p,c)) = t; this.isGoal( s ) ) )
Note that I replaced -> with ,. This works because a -> b is a syntactic sugar for (a, b), which is, itself, a syntactic sugar for Tuple2(a, b). As you don't use neither p nor c, this works too:
if ( extension.exists( t => val (s, _) = t; this.isGoal( s ) ) )
Finally, your recursive code is perfectly fine, though probably not optimized for tail-recursion. For that, you either make your method final, or you make the recursive function private to the method. Like this:
final def breadthFirstHelper
or
def breadthFirstHelper(...) {
def myRecursiveBreadthFirstHelper(...) { ... }
myRecursiveBreadthFirstHelper(...)
}
On Scala 2.8 there is an annotation called #TailRec which will tell you if the function can be made tail recursive or not. And, in fact, it seems there will be a flag to display warnings about functions that could be made tail-recursive if slightly changed, such as above.
EDIT
Regarding Oxbow's solution using case, that's a function or partial function literal. It's type will depend on what the inference requires. In that case, because that's that exists takes, a function. However, one must be careful to ensure that there will always be a match, otherwise you get an exception. For example:
scala> List(1, 'c') exists { case _: Int => true }
res0: Boolean = true
scala> List(1, 'c') exists { case _: String => true }
scala.MatchError: 1
at $anonfun$1.apply(<console>:5)
... (stack trace elided)
scala> List(1, 'c') exists { case _: String => true; case _ => false }
res3: Boolean = false
scala> ({ case _: Int => true } : PartialFunction[AnyRef,Boolean])
res5: PartialFunction[AnyRef,Boolean] = <function1>
scala> ({ case _: Int => true } : Function1[Int, Boolean])
res6: (Int) => Boolean = <function1>
EDIT 2
The solution Oxbow proposes does use pattern matching, because it is based on function literals using case statements, which do use pattern matching. When I said it was not possible, I was speaking of the syntax x => s.