I have two functions which return Futures. I'm trying to feed a modified result from first function into the other using a for-yield comprehension.
This approach works:
val schoolFuture = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- schoolStore.getSchool(sid.get) if sid.isDefined
} yield s
However I'm not happy with having the "if" in there, it seems that I should be able to use a map instead.
But when I try with a map:
val schoolFuture: Future[Option[School]] = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- sid.map(schoolStore.getSchool(_))
} yield s
I get a compile error:
[error] found : Option[scala.concurrent.Future[Option[School]]]
[error] required: scala.concurrent.Future[Option[School]]
[error] s <- sid.map(schoolStore.getSchool(_))
I've played around with a few variations, but haven't found anything attractive that works. Can anyone suggest a nicer comprehension and/or explain what's wrong with my 2nd example?
Here is a minimal but complete runnable example with Scala 2.10:
import concurrent.{Future, Promise}
case class User(userId: Int)
case class UserDetails(userId: Int, schoolId: Option[Int])
case class School(schoolId: Int, name: String)
trait Error
class UserStore {
def getUserDetails(userId: Int): Future[Either[Error, UserDetails]] = Promise.successful(Right(UserDetails(1, Some(1)))).future
}
class SchoolStore {
def getSchool(schoolId: Int): Future[Option[School]] = Promise.successful(Option(School(1, "Big School"))).future
}
object Demo {
import concurrent.ExecutionContext.Implicits.global
val userStore = new UserStore
val schoolStore = new SchoolStore
val user = User(1)
val schoolFuture: Future[Option[School]] = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- sid.map(schoolStore.getSchool(_))
} yield s
}
(Edited to give a correct answer!)
The key here is that Future and Option don't compose inside for because there aren't the correct flatMap signatures. As a reminder, for desugars like so:
for ( x0 <- c0; w1 = d1; x1 <- c1 if p1; ... ; xN <- cN) yield f
c0.flatMap{ x0 =>
val w1 = d1
c1.filter(x1 => p1).flatMap{ x1 =>
... cN.map(xN => f) ...
}
}
(where any if statement throws a filter into the chain--I've given just one example--and the equals statements just set variables before the next part of the chain). Since you can only flatMap other Futures, every statement c0, c1, ... except the last had better produce a Future.
Now, getUserDetails and getSchool both produce Futures, but sid is an Option, so we can't put it on the right-hand side of a <-. Unfortunately, there's no clean out-of-the-box way to do this. If o is an option, we can
o.map(Future.successful).getOrElse(Future.failed(new Exception))
to turn an Option into an already-completed Future. So
for {
ud <- userStore.getUserDetails(user.userId) // RHS is a Future[Either[...]]
sid = ud.right.toOption.flatMap(_.schoolId) // RHS is an Option[Int]
fid <- sid.map(Future.successful).getOrElse(Future.failed(new Exception)) // RHS is Future[Int]
s <- schoolStore.getSchool(fid)
} yield s
will do the trick. Is that better than what you've got? Doubtful. But if you
implicit class OptionIsFuture[A](val option: Option[A]) extends AnyVal {
def future = option.map(Future.successful).getOrElse(Future.failed(new Exception))
}
then suddenly the for-comprehension looks reasonable again:
for {
ud <- userStore.getUserDetails(user.userId)
sid <- ud.right.toOption.flatMap(_.schoolId).future
s <- schoolStore.getSchool(sid)
} yield s
Is this the best way to write this code? Probably not; it relies upon converting a None into an exception simply because you don't know what else to do at that point. This is hard to work around because of the design decisions of Future; I'd suggest that your original code (which invokes a filter) is at least as good of a way to do it.
This answer to a similar question about Promise[Option[A]] might help. Just substitute Future for Promise.
I'm inferring the following types for getUserDetails and getSchool from your question:
getUserDetails: UserID => Future[Either[??, UserDetails]]
getSchool: SchoolID => Future[Option[School]]
Since you ignore the failure value from the Either, transforming it to an Option instead, you effectively have two values of type A => Future[Option[B]].
Once you've got a Monad instance for Future (there may be one in scalaz, or you could write your own as in the answer I linked), applying the OptionT transformer to your problem would look something like this:
for {
ud <- optionT(getUserDetails(user.userID) map (_.right.toOption))
sid <- optionT(Future.successful(ud.schoolID))
s <- optionT(getSchool(sid))
} yield s
Note that, to keep the types compatible, ud.schoolID is wrapped in an (already completed) Future.
The result of this for-comprehension would have type OptionT[Future, SchoolID]. You can extract a value of type Future[Option[SchoolID]] with the transformer's run method.
What behavior would you like to occur in the case that the Option[School] is None? Would you like the Future to fail? With what kind of exception? Would you like it to never complete? (That sounds like a bad idea).
Anyways, the if clause in a for-expression desugars to a call to the filter method. The contract on Future#filteris thus:
If the current future contains a value which satisfies the predicate,
the new future will also hold that value. Otherwise, the resulting
future will fail with a NoSuchElementException.
But wait:
scala> None.get
java.util.NoSuchElementException: None.get
As you can see, None.get returns the exact same thing.
Thus, getting rid of the if sid.isDefined should work, and this should return a reasonable result:
val schoolFuture = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- schoolStore.getSchool(sid.get)
} yield s
Keep in mind that the result of schoolFuture can be in instance of scala.util.Failure[NoSuchElementException]. But you haven't described what other behavior you'd like.
We've made small wrapper on Future[Option[T]] which acts like one monad (nobody even checked none of monad laws, but there is map, flatMap, foreach, filter and so on) - MaybeLater. It behaves much more than an async option.
There are a lot of smelly code there, but maybe it will be usefull at least as an example.
BTW: there are a lot of open questions(here for ex.)
It's easier to use https://github.com/qifun/stateless-future or https://github.com/scala/async to do A-Normal-Form transform.
Related
How can I remove the option so it is just Try[Int] and not Try[Option[Int]]?
val m = Map("a" -> "1a", "b" -> "2")
Try(m.get("a").map(_.trim.toInt))
>>es17: scala.util.Try[Option[Int]] = Failure(java.lang.NumberFormatException: For input string: "1a")
Map#get returns an Option[String], but you can use Map#apply instead, which will return String, in this case.
scala> Try(m("a").trim.toInt)
res3: scala.util.Try[Int] = Failure(java.lang.NumberFormatException: For input string: "1a")
scala> Try(m("b").trim.toInt)
res4: scala.util.Try[Int] = Success(2)
apply throws an exception if the key you're looking for doesn't exist, but Try will catch it, anyway.
This answer goes in more detail about the comment:
I was wondering if there was a way to use flapmap? Your solution works for me, just want to learn of other alternatives.
As you've probably heard, Option and Try are monad instances and while monads are handy to represent sequence of computations, they don't compose with other monads. In other words, we can't compose Option and Try. We need to find a common ground.
The difference in semantics between Option and Try is that Try contains information about the case when a result is absent.
We can go from Try to Option using Try#toOption effectively loosing any failure information we may have.
If we wanted to go the other way, we need to add this information back: ne need to provide a failure reason when a value is absent in an Option. Something like this:
import scala.util.{Try, Success, Failure}
def optionToTry[T](opt:Option[T], failure: => Throwable): Try[T] = opt match {
case Some(v) => Success(v)
case None => Failure(failure)
}
With the help of that function, we can rewrite the original expression as:
val res: Try[Int] = for {
strValue <- optionToTry(m.get("a"), new NoSuchElementException("a"))
value <- Try(strValue.trim.toInt)
} yield value
which uses flatMap behind the scenes to compose the two Try instances like this:
val res = optionToTry(m.get("a"), new NoSuchElementException("a"))
.flatMap(strValue => Try(strValue.trim.toInt))
Note that we could save ourselves a bit of coding by using the unsafe map getter like so:
val res: Try[Int] = for {
strValue <- Try(m("a"))
value <- Try(strValue.trim.toInt)
} yield value
but this version would be computationally more expensive given the cost of handling exceptions in the JVM.
I have a sequence of parameters. For each parameter I have to perform DB query, which may or may not return a result. Simply speaking, I need to stop after the first result is non-empty. Of course, I would like to avoid doing unnecessary calls. The caveat is - I need to have this operation(s) contained as a another Future - or any "most reactive" approach.
Speaking of code:
//that what I have
def dbQuery(p:Param): Future[Option[Result]] = {}
//my list of params
val input = Seq(p1,p2,p3)
//that what I need to implements
def getFirstNonEmpty(params:Seq[Param]): Future[Option[Result]]
I know I can possibly just wrap entire function in yet another Future and execute code sequentially (Await? Brrr...), but that not the cleanest solution.
Can I somehow create lazy initialized collection of futures, like
params.map ( p => FutureWhichWontStartUnlessAskedWhichWrapsOtherFuture { dbQuery(p) }).findFirst(!_.isEmpty())
I believe it's possible!
What do you think about something like this?
def getFirstNonEmpty(params: Seq[Param]): Future[Option[Result]] = {
params.foldLeft(Future.successful(Option.empty[Result])) { (accuFtrOpt, param) =>
accuFtrOpt.flatMap {
case None => dbQuery(param)
case result => Future.successful(result)
}
}
}
This might be overkill, but if you are open to using scalaz we can do this using OptionT and foldMap.
With OptionT we sort of combine Future and Option into one structure. We can get the first of two Futures with a non-empty result using OptionT.orElse.
import scalaz._, Scalaz._
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
val someF: Future[Option[Int]] = Future.successful(Some(1))
val noneF: Future[Option[Int]] = Future.successful(None)
val first = OptionT(noneF) orElse OptionT(someF)
first.run // Future[Option[Int]] = Success(Some(1))
We could now get the first non-empty Future from a List with reduce from the standard library (this will however run all the Futures) :
List(noneF, noneF, someF).map(OptionT.apply).reduce(_ orElse _).run
But with a List (or other collection) we can't be sure that there is at least one element, so we need to use fold and pass a start value. Scalaz can do this work for us by using a Monoid. The Monoid[OptionT[Future, Int]] we will use will supply the start value and combine the Futures with the orElse used above.
type Param = Int
type Result = Int
type FutureO[x] = OptionT[Future, x]
def query(p: Param): Future[Option[Result]] =
Future.successful{ println(p); if (p > 2) Some(p) else None }
def getFirstNonEmpty(params: List[Param]): Future[Option[Result]] = {
implicit val monoid = PlusEmpty[FutureO].monoid[Result]
params.foldMap(p => OptionT(query(p))).run
}
val result = getFirstNonEmpty(List(1,2,3,4))
// prints 1, 2, 3
result.foreach(println) // Some(3)
This is an old question, but if someone comes looking for an answer, here is my take. I solved it for a use case that required me to loop through a limited number of futures sequentially and stop when the first of them returned a result.
I did not need a library for my use-case, a light-weight combination of recursion and pattern matching was sufficient. Although the question here does not have the same problem as a sequence of futures, looping through a sequence of parameters would be similar.
Here would be the pseudo-code based on recursion.
I have not compiled this, fix the types being matched/returned.
def getFirstNonEmpty(params: Seq[Param]): Future[Option[Result]] = {
if (params.isEmpty) {
Future.successful(None)
} else {
val head = params.head
dbQuery(head) match {
case Some(v) => Future.successful(Some(v))
case None => getFirstNonEmpty(params.tail)
}
}
}
I've got following problem:
val sth: Future[Seq[T, S]] = for {
x <- whatever: Future[List[T]]
y <- x: List[T]
z <- f(y): Future[Option[S]]
n <- z: Option[S]
} yield y: T -> n: S
I would want to make this code to work(I guess everyone understands the idea as I've added types).
By "to work" I mean, that I would want to stay with the for-comprehension structure and fulfil expected types in the end. I know there are "ugly" ways to do it, but I want to learn how to do it pure :)
As I read the internet I've reached the conclusion that my problem may be solved by the monad transformers & scalaz. Unfortunately, I couldn't find an example to help understand better how should I proceed.
At the moment I've tried scalaz and Eff monad libs, but I guess I still don't understand how it works because I couldn't solve my problem.
I will be grateful for any help.
EDIT: It supposed to be future of sequence, also regarding the "whatever" I get it as a parameter of the function, sorry for misleading you
You could do something like what you need using the scalaz ListT monad transformer
object Test {
import scalaz._
import ListT._
type T = String
type S = Int
val whatever: Future[List[T]] = ??? // you get this somewhere
def f(y: T): Future[Option[S]] = ??? // function that returns future of option
val sth: Future[List[(T, S)]] = (for {
y <- listT(whatever)
// you cannot mix list and option, but you can convert the option to a list of 1 item
n <- listT(f(y).map(_.toList))
} yield y -> n).run
}
N.B.: Since you start with a future, you cannot return a Seq[(T,S)], you can only have a future. You have to call Await.result if you want to block and get the result.
The problem with for comprehension is that it's not some kind of magic monadic "unwrapper", it's just a sequence of map, flatMap and filter.
As you may know map and flatMap operate only on "inner" type, leaving "outer" type of monad unchanged. This means you can't do this:
for {
x <- whatever: Future[List[T]]
y <- x: List[T]
} yield y
inside single for. Instead, you can do something like this:
for (x <- whatever: Future[List[T]])
yield for (y <- x: List[T]) yield y
Which looks kinda ugly.
Back to your case, I's easier to write whole transformation explicitly using map and flatMap, as it gives you greater visibility and control:
whatever.flatMap {
x: List[T] =>
Future.sequence(x.map {
y: T => f(y).map(y -> _)
}).map(_.collect {
case (y, Some(n)) => y -> n
})
}
Also, #trustnoone mentioned, you can't get rid of the Future without explicitly calling Await.
I have two functions which return Futures. I'm trying to feed a modified result from first function into the other using a for-yield comprehension.
This approach works:
val schoolFuture = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- schoolStore.getSchool(sid.get) if sid.isDefined
} yield s
However I'm not happy with having the "if" in there, it seems that I should be able to use a map instead.
But when I try with a map:
val schoolFuture: Future[Option[School]] = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- sid.map(schoolStore.getSchool(_))
} yield s
I get a compile error:
[error] found : Option[scala.concurrent.Future[Option[School]]]
[error] required: scala.concurrent.Future[Option[School]]
[error] s <- sid.map(schoolStore.getSchool(_))
I've played around with a few variations, but haven't found anything attractive that works. Can anyone suggest a nicer comprehension and/or explain what's wrong with my 2nd example?
Here is a minimal but complete runnable example with Scala 2.10:
import concurrent.{Future, Promise}
case class User(userId: Int)
case class UserDetails(userId: Int, schoolId: Option[Int])
case class School(schoolId: Int, name: String)
trait Error
class UserStore {
def getUserDetails(userId: Int): Future[Either[Error, UserDetails]] = Promise.successful(Right(UserDetails(1, Some(1)))).future
}
class SchoolStore {
def getSchool(schoolId: Int): Future[Option[School]] = Promise.successful(Option(School(1, "Big School"))).future
}
object Demo {
import concurrent.ExecutionContext.Implicits.global
val userStore = new UserStore
val schoolStore = new SchoolStore
val user = User(1)
val schoolFuture: Future[Option[School]] = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- sid.map(schoolStore.getSchool(_))
} yield s
}
(Edited to give a correct answer!)
The key here is that Future and Option don't compose inside for because there aren't the correct flatMap signatures. As a reminder, for desugars like so:
for ( x0 <- c0; w1 = d1; x1 <- c1 if p1; ... ; xN <- cN) yield f
c0.flatMap{ x0 =>
val w1 = d1
c1.filter(x1 => p1).flatMap{ x1 =>
... cN.map(xN => f) ...
}
}
(where any if statement throws a filter into the chain--I've given just one example--and the equals statements just set variables before the next part of the chain). Since you can only flatMap other Futures, every statement c0, c1, ... except the last had better produce a Future.
Now, getUserDetails and getSchool both produce Futures, but sid is an Option, so we can't put it on the right-hand side of a <-. Unfortunately, there's no clean out-of-the-box way to do this. If o is an option, we can
o.map(Future.successful).getOrElse(Future.failed(new Exception))
to turn an Option into an already-completed Future. So
for {
ud <- userStore.getUserDetails(user.userId) // RHS is a Future[Either[...]]
sid = ud.right.toOption.flatMap(_.schoolId) // RHS is an Option[Int]
fid <- sid.map(Future.successful).getOrElse(Future.failed(new Exception)) // RHS is Future[Int]
s <- schoolStore.getSchool(fid)
} yield s
will do the trick. Is that better than what you've got? Doubtful. But if you
implicit class OptionIsFuture[A](val option: Option[A]) extends AnyVal {
def future = option.map(Future.successful).getOrElse(Future.failed(new Exception))
}
then suddenly the for-comprehension looks reasonable again:
for {
ud <- userStore.getUserDetails(user.userId)
sid <- ud.right.toOption.flatMap(_.schoolId).future
s <- schoolStore.getSchool(sid)
} yield s
Is this the best way to write this code? Probably not; it relies upon converting a None into an exception simply because you don't know what else to do at that point. This is hard to work around because of the design decisions of Future; I'd suggest that your original code (which invokes a filter) is at least as good of a way to do it.
This answer to a similar question about Promise[Option[A]] might help. Just substitute Future for Promise.
I'm inferring the following types for getUserDetails and getSchool from your question:
getUserDetails: UserID => Future[Either[??, UserDetails]]
getSchool: SchoolID => Future[Option[School]]
Since you ignore the failure value from the Either, transforming it to an Option instead, you effectively have two values of type A => Future[Option[B]].
Once you've got a Monad instance for Future (there may be one in scalaz, or you could write your own as in the answer I linked), applying the OptionT transformer to your problem would look something like this:
for {
ud <- optionT(getUserDetails(user.userID) map (_.right.toOption))
sid <- optionT(Future.successful(ud.schoolID))
s <- optionT(getSchool(sid))
} yield s
Note that, to keep the types compatible, ud.schoolID is wrapped in an (already completed) Future.
The result of this for-comprehension would have type OptionT[Future, SchoolID]. You can extract a value of type Future[Option[SchoolID]] with the transformer's run method.
What behavior would you like to occur in the case that the Option[School] is None? Would you like the Future to fail? With what kind of exception? Would you like it to never complete? (That sounds like a bad idea).
Anyways, the if clause in a for-expression desugars to a call to the filter method. The contract on Future#filteris thus:
If the current future contains a value which satisfies the predicate,
the new future will also hold that value. Otherwise, the resulting
future will fail with a NoSuchElementException.
But wait:
scala> None.get
java.util.NoSuchElementException: None.get
As you can see, None.get returns the exact same thing.
Thus, getting rid of the if sid.isDefined should work, and this should return a reasonable result:
val schoolFuture = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- schoolStore.getSchool(sid.get)
} yield s
Keep in mind that the result of schoolFuture can be in instance of scala.util.Failure[NoSuchElementException]. But you haven't described what other behavior you'd like.
We've made small wrapper on Future[Option[T]] which acts like one monad (nobody even checked none of monad laws, but there is map, flatMap, foreach, filter and so on) - MaybeLater. It behaves much more than an async option.
There are a lot of smelly code there, but maybe it will be usefull at least as an example.
BTW: there are a lot of open questions(here for ex.)
It's easier to use https://github.com/qifun/stateless-future or https://github.com/scala/async to do A-Normal-Form transform.
I have posted quite several questions about failure handling in Scala and I really thank you all for your answers.
I understand my options when dealing with Either and Scalaz or a for comprehension, and I have another (last?) question:
How to do a fail-fast sequence of operations when the operations are dealing with the outside non-functional world, like a DB?
I mean I have a method like that:
def insertItem(item: Item): Either[Error,Item]
Thanks to Either and these answers, I know how to do it with Either: Chaining method calls with Either and
Method parameters validation in Scala, with for comprehension and monads
But my Item case class is immutable and it doesn't really make sense to return it as a Right since the caller already has the value.
Thus how can I do the same kind of thing with:
def insertItem(item: Item): Option[Error]
In my application, when an user is created, we also create some items for him.
If an item fails to create, then the whole process should stop.
When I use directly Option[Error] in a for comprehension, I don't think I'll get the result I expect.
I guess it makes sense to do something like that:
for {
_ <- insertItem(item1).toLeft("???").right
_ <- insertItem(item2).toLeft("???").right
_ <- insertItem(item3).toLeft("???").right
}
But as the values "???" I put in my Right are never useful, I guess I'm missing the elegant solution which do not involve creating Rights that will never be used.
I think I'm looking for something that will continue the for comprehension only when the result is None, which is kind of weird because I just want to continue to the next operation, and not do a real map operation.
By the way, if possible, I would like both non-Scalaz and Scalaz solutions.
I'm not sure Scalaz handle such things, because it seems more focused on real functional programming and perhaps do not provide code for side-effect behaviors like my use case?
I don't see a principled reason not to use Either[Error, Unit] in a case like this (or at least I've done it, and I don't feel guilty about it). Say we have the following:
def launch(thing: String): Either[String, Unit] = Either.cond(
thing.nonEmpty,
println("Launching the " + thing),
"Need something to launch!"
)
We can show that the right projection monad is appropriately lazy:
scala> for {
| _ <- launch("flowers").right
| _ <- launch("").right
| r <- launch("MISSILES").right
| } yield r
Launching the flowers
res1: Either[String,Unit] = Left(Need something to launch!)
No missiles get launched, as desired.
It's worth noting that if you use Option instead of Either, the operation you're describing is just the sum given the "First" monoid instance for Option (where the addition operation is just orElse). For example, we can write the following with Scalaz 7:
import scalaz._, Scalaz._
def fst[A](x: Option[A]): Option[A] ## Tags.First = Tag(x)
def launch(thing: String): Option[String] = if (thing.isEmpty) Some(
"Need something to launch!"
) else {
println("Launching the " + thing)
None
}
And now:
scala> val r: Option[String] = fst(launch("flowers")) |+| fst(
| launch("")) |+| fst(launch("MISSILES"))
Launching the flowers
r: Option[String] = Some(Need something to launch!)
Once again, no missiles.
We do something similar where I work with scalaz to fail fast with message sending, though this is not idiomatic scalaz:
def insertItem(item: Item): Validation[Error, Unit]
val result = for {
_ <- insertItem(item1)
_ <- insertItem(item2)
_ <- insertItem(item3)
} yield Unit
If you want to chain together methods that contain Option[Error] and only execute the next step if the Option is None, you can use orElse for that.
If you are only interested in the first failure, than an Iterator can be your choice:
case class Item(i: Int)
case class Error(i: Item)
val items = Iterator(1,3,7,-5,-2,0) map Item
def insertItem(item: Item): Option[Error] =
if (item.i < 0) Some(Error(item)) else None
scala> val error = (items map insertItem find (_.isDefined)).flatten
error: Option[Error] = Some(Error(Item(-5)))
If insertItem is not your only method it is possible to call them separated with your values:
val items = Iterator(
() => insertItem(i1),
() => deleteItem(i2),
() => insertItem(i3),
() => updateItem(i4))