How would one transform Future[Option[X]] into Option[Future[X]]?
val futOpt:Future[Option[Int]] = future(Some(1))
val optFut:Option[Future[Int]] = ?
Update:
This is a follow up to this question. I suppose I'm trying to get a grasp on elegantly transforming nested futures. I'm trying to achieve with Options what can be done with Sequences, where you turn a Future[Seq[Future[Seq[X]]]] into Future[Future[Seq[Seq[x]]]] and then flatMap the double layers. As Ionut has clarified, I have phrased the question in flipped order, it was supposed to be Option[Future[X]] -> Future[Option[X]].
Unfortunately, this isn't possible without losing the non-blocking properties of the computation. It's pretty simple if you think about it. You don't know whether the result of the computation is None or Some until that Future has completed, so you have to Await on it. At that point, it makes no sense to have a Future anymore. You can simply return the Option[X], as the Future has completed already.
Take a look here. It always returns Future.successful, which does no computation, just wraps o in a Future for no good reason.
def transform[A](f: Future[Option[A]]): Option[Future[A]] =
Await.result(f, 2.seconds).map(o => Future.successful(o))
So, if in your context makes sense to block, you're better off using this:
def transform[A](f: Future[Option[A]]): Option[A] =
Await.result(f, 2.seconds)
Response for comments:
def transform[A](o: Option[Future[A]]): Future[Option[A]] =
o.map(f => f.map(Option(_))).getOrElse(Future.successful(None))
you could technically you could do this - in snippet below I use A context bound to a monoid so my types hold.
def fut2Opt[A: Monoid](futOpt: Future[Option[A]])(implicit ec: ExecutionContext): Option[Future[A]] =
Try(futOpt.flatMap {
case Some(a) => Future.successful(a)
case None => Future.fromTry(Try(Monoid[A].empty))
}).toOption
Related
I have a db query which returns a Future[String], the implementation does not matter, but the signature is something like this:
def getTicketType(id: Long): Future[String] = {...}
And imagine I have a list of ids which i would want to retrieve ticket types from those ids. so something like this:
val listOfIds: List[Long] = ... (from somewhere else of the code)
val ticketTypesFuture: Future[List[String]] = Future.sequence(listOfIds.map(getTicketType))
So far so good, but there is another function, which is called within the main process, that HAS to return a Boolean or an Option[Boolean] value, since it's result is used in the main process which holds a gigantic for comprehension, combined of some Either[Int, Option[JsValue]]'s. The way I'm doing it right now (which I believe is the worst way of implementing such thing :) ), is this:
def thatFunction(): Boolean = {
// ... val listOfIds, ticketTypesFuture defined above
var result = false // here is the nasty code :)
val futureResult: Future[Boolean] = ticketTypesFuture.map { ticketTypes =>
if (!ticketTypes.forall(someCondition)) {
// some code which returns either true or false
} else false
}
futureResult.omComplete {
case Success(value) => result = value
case _ => result = false
}
result
}
But there must be a better approach to do this, so I would appreciate any help!
The sane option is to go the other way and make your "gigantic for comprehension, combined of some Either[Int, Option[JsValue]]'s" work with futures. Wrap the part before and after the query using Future.apply or Future.successful, and you should be fine. Or if it contains other database/API accesses, make them return Future as well.
If you can't, your choice is:
use Await.result as in Tim's answer, which loses any benefit of futures. If you really want that, consider using a library which doesn't return a future in the first place. But this may be a placeholder until you switch.
use Future#value if you want not to wait and just do something else if the result is not ready. For example you might show some old results, or an empty list until you get data.
(After writing this, I saw #jwvh already said basically the same in a comment, hopefully it still helps to have a more expanded version.)
If you must convert Future[Boolean] to Option[Boolean] then you need to wait for the Future using Await.result. This will throw an error if the Future fails, so wrap it in a Try.
val futureResult: Future[Boolean] = ???
Try(Await.result(futureResult, Duration.Inf)).toOption
But the better solution is to convert the calling code to accept a Future and avoid blocking.
As you can see in the answers and comments, multiple approaches have been discussed, which include:
1- waiting for the Future to complete (using mutation, Await, ...) which are the worst of the approaches, so just don't do that :)
2- mapping on the value like this: futureResult.value.map(t => t.isSuccess && t.get)
and some other solutions.
As #jwh mentioned, another solution is to handle it properly, anything that touches futureResult becomes a Future!
But since I couldn't change all the calculations and functions inside that for comprehension, I placed this Future[Boolean] condition outside of the for loop, and everythin is just fine.
Taking in account that any pattern matching on Future result matches Success[A] and Failure[A] (onComplete(), andThen()) (because they expect Try[A] as an argument, directly or through partial function), could there be a case when I would want to say explicitly that a function is of type Future[Try[A]]?
There are various constructs in Scala which carry failure case in them. There are Option, Either, Try and Future. (Futures main point is to abstract asynchronous operations, an error handling is there for convinience). Scalaz have even more: Validation (Disjunction and Maybe are better Either and Option).
They all have a bit different treatment of erroneous values. Yet Try and Future have very similar, both wrap Throwable. So IMHO, Future[Try[A]] doesn't add much information (about the error). Compare to having Future[Future[A]] or Try[Try[A]]. OTOH Future[Option[A]] or Future[Either[MyError, A]] make sense to me.
There might be sitatuon where you have for example potentially failing f: A => B and g: B => C and you'd like to avoid creating too much tasks in the ExecutionContext:
val a: Future[A] = ???
val c: Future[C] = a.map(f).map(g) // two tasks, not good
val c2: Future[Try[C]] = a.map(x => Try { f(x) } map g ) // one task, maybe better
// get will throw, but exception will be catched by Future's map
// Almost no information is lost compared to `c2`
// You cannot anymore distinguish between possible error in `a`
// and exceptions thrown by `f` or `g`.
val c3: Future[C] = a.map(x => Try { f (x) }.map(g).get)
In this case, I'd rather refactor f and g to have better types, at least: f: A => Option[B] and g: B => Option[C] then, ending up with Future[Option[C]].
Try[A] represents a synchronous computation that may fail.
Future[A] represents an asynchronous computation that may fail.
Under this definitions, Future[Try[A]] represents the result of a synchronous computation (that may fail) executed inside of an asynchronous computation (that may fail).
Does it make sense? Not to me, but I'm open to creative interpretations.
My code heavily uses Akka and untyped actors.
An example of typical logic is as follows:
val myFuture: Future[Any] = akka.pattern.AskSupport.ask(myActorRef, myMessage)
val completedLogic: Future[Unit] = myFuture.map(myFunction)
myFunction then contains a strongly typed signature as follows:
def myFunction(): (Option[MyStronglyTypedClass]) => Unit = {
(myOption: Option[MyStronglyTypedClass]) => myOption foreach (myObject => // do some logic
}
I know that myFuture will always contain an instance of MyStronglyTypedClass or null for this particular actor. I will also know this for other actor/future/function combinations.
My problem comes when I look to create an implicit conversion from the Future[Any] to the Option[MyStronglyTypedClass] or Option[MyOtherStronglyTypedClass]
The implicit conversion will just do a null check and one other piece of logic before creating the Option
How do I go about performing this implicit conversion from Any to a subtype, or is it even possible?
You should, instead, convert to Future[Option[MyStronglyTypedClass]]:
def asMyStronglyTypedClass(x: Any): Option[MyStronglyTypedClass] = x match {
case null => None
case ...
}
// this will fail if myFuture fails or asMyStronglyTypedClass throws
val typedFuture = myFuture.map(asMyStronglyTypedClass)
and do what you want with this future. E.g.
typedFuture.onSuccess(myFunction)
EDIT: I missed you already have a map. In this case the issue is that you don't need to convert Future[Any] to Option, but its result (i.e. Any). You can write e.g. myFuture.map(asMyStronglyTypedClass).map(myFunction) or myFuture.map(x => myFunction(asMyStronglyTypedClass(x))). You could also make asMyStronglyTypedClass implicit and write myFuture.map(x => myFunction(x)). I still think it isn't a good idea, as it could get applied somewhere you don't expect.
If you really want to write myFuture.map(myFunction), you'll need a different implicit conversion to make the compiler understand:
implicit def contraMap(f: Option[MyStronglyTypedClass] => Unit): Any => Unit =
x => f(asMyStronglyTypedClass(x))
Of course, these can be made generic over your types, as mentioned in the comments.
Impliciy converting from Any to something else is something that you should never do because it effectively switches off Scala's type system. Converting the type of a future in a safe fashion can be done using
myFuture.mapTo[Option[MyStronglyTypedClass]]
I'm writing a programming language interpreter.
I have need of the right code idiom to both evaluate a sequence of expressions to get a sequence of their values, and propagate state from one evaluator to the next to the next as the evaluations take place. I'd like a functional programming idiom for this.
It's not a fold because the results come out like a map. It's not a map because of the state prop across.
What I have is this code which I'm using to try to figure this out. Bear with a few lines of test rig first:
// test rig
class MonadLearning extends JUnit3Suite {
val d = List("1", "2", "3") // some expressions to evaluate.
type ResType = Int
case class State(i : ResType) // trivial state for experiment purposes
val initialState = State(0)
// my stub/dummy "eval" function...obviously the real one will be...real.
def computeResultAndNewState(s : String, st : State) : (ResType, State) = {
val State(i) = st
val res = s.toInt + i
val newStateInt = i + 1
(res, State(newStateInt))
}
My current solution. Uses a var which is updated as the body of the map is evaluated:
def testTheVarWay() {
var state = initialState
val r = d.map {
s =>
{
val (result, newState) = computeResultAndNewState(s, state)
state = newState
result
}
}
println(r)
println(state)
}
I have what I consider unacceptable solutions using foldLeft which does what I call "bag it as you fold" idiom:
def testTheFoldWay() {
// This startFold thing, requires explicit type. That alone makes it muddy.
val startFold : (List[ResType], State) = (Nil, initialState)
val (r, state) = d.foldLeft(startFold) {
case ((tail, st), s) => {
val (r, ns) = computeResultAndNewState(s, st)
(tail :+ r, ns) // we want a constant-time append here, not O(N). Or could Cons on front and reverse later
}
}
println(r)
println(state)
}
I also have a couple of recursive variations (which are obvious, but also not clear or well motivated), one using streams which is almost tolerable:
def testTheStreamsWay() {
lazy val states = initialState #:: resultStates // there are states
lazy val args = d.toStream // there are arguments
lazy val argPairs = args zip states // put them together
lazy val resPairs : Stream[(ResType, State)] = argPairs.map{ case (d1, s1) => computeResultAndNewState(d1, s1) } // map across them
lazy val (results , resultStates) = myUnzip(resPairs)// Note .unzip causes infinite loop. Had to write my own.
lazy val r = results.toList
lazy val finalState = resultStates.last
println(r)
println(finalState)
}
But, I can't figure out anything as compact or clear as the original 'var' solution above, which I'm willing to live with, but I think somebody who eats/drinks/sleeps monad idioms is going to just say ... use this... (Hopefully!)
With the map-with-accumulator combinator (the easy way)
The higher-order function you want is mapAccumL. It's in Haskell's standard library, but for Scala you'll have to use something like Scalaz.
First the imports (note that I'm using Scalaz 7 here; for previous versions you'd import Scalaz._):
import scalaz._, syntax.std.list._
And then it's a one-liner:
scala> d.mapAccumLeft(initialState, computeResultAndNewState)
res1: (State, List[ResType]) = (State(3),List(1, 3, 5))
Note that I've had to reverse the order of your evaluator's arguments and the return value tuple to match the signatures expected by mapAccumLeft (state first in both cases).
With the state monad (the slightly less easy way)
As Petr Pudlák points out in another answer, you can also use the state monad to solve this problem. Scalaz actually provides a number of facilities that make working with the state monad much easier than the version in his answer suggests, and they won't fit in a comment, so I'm adding them here.
First of all, Scalaz does provide a mapM—it's just called traverse (which is a little more general, as Petr Pudlák notes in his comment). So assuming we've got the following (I'm using Scalaz 7 again here):
import scalaz._, Scalaz._
type ResType = Int
case class Container(i: ResType)
val initial = Container(0)
val d = List("1", "2", "3")
def compute(s: String): State[Container, ResType] = State {
case Container(i) => (Container(i + 1), s.toInt + i)
}
We can write this:
d.traverse[({type L[X] = State[Container, X]})#L, ResType](compute).run(initial)
If you don't like the ugly type lambda, you can get rid of it like this:
type ContainerState[X] = State[Container, X]
d.traverse[ContainerState, ResType](compute).run(initial)
But it gets even better! Scalaz 7 gives you a version of traverse that's specialized for the state monad:
scala> d.traverseS(compute).run(initial)
res2: (Container, List[ResType]) = (Container(3),List(1, 3, 5))
And as if that wasn't enough, there's even a version with the run built in:
scala> d.runTraverseS(initial)(compute)
res3: (Container, List[ResType]) = (Container(3),List(1, 3, 5))
Still not as nice as the mapAccumLeft version, in my opinion, but pretty clean.
What you're describing is a computation within the state monad. I believe that the answer to your question
It's not a fold because the results come out like a map. It's not a map because of the state prop across.
is that it's a monadic map using the state monad.
Values of the state monad are computations that read some internal state, possibly modify it, and return some value. It is often used in Haskell (see here or here).
For Scala, there is a trait in the ScalaZ library called State that models it (see also the source). There are utility methods in States for creating instances of State. Note that from the monadic point of view State is just a monadic value. This may seem confusing at first, because it's described by a function depending on a state. (A monadic function would be something of type A => State[B].)
Next you need is a monadic map function that computes values of your expressions, threading the state through the computations. In Haskell, there is a library method mapM that does just that, when specialized to the state monad.
In Scala, there is no such library function (if it is, please correct me). But it's possible to create one. To give a full example:
import scalaz._;
object StateExample
extends App
with States /* utility methods */
{
// The context that is threaded through the state.
// In our case, it just maps variables to integer values.
class Context(val map: Map[String,Int]);
// An example that returns the requested variable's value
// and increases it's value in the context.
def eval(expression: String): State[Context,Int] =
state((ctx: Context) => {
val v = ctx.map.get(expression).getOrElse(0);
(new Context(ctx.map + ((expression, v + 1)) ), v);
});
// Specialization of Haskell's mapM to our State monad.
def mapState[S,A,B](f: A => State[S,B])(xs: Seq[A]): State[S,Seq[B]] =
state((initState: S) => {
var s = initState;
// process the sequence, threading the state
// through the computation
val ys = for(x <- xs) yield { val r = f(x)(s); s = r._1; r._2 };
// return the final state and the output result
(s, ys);
});
// Example: Try to evaluate some variables, starting from an empty context.
val expressions = Seq("x", "y", "y", "x", "z", "x");
print( mapState(eval)(expressions) ! new Context(Map[String,Int]()) );
}
This way you can create simple functions that take some arguments and return State and then combine them into more complex ones by using State.map or State.flatMap (or perhaps better using for comprehensions), and then you can run the whole computation on a list of expressions by mapM.
See also http://blog.tmorris.net/posts/the-state-monad-for-scala-users/
Edit: See Travis Brown's answer, he described how to use the state monad in Scala much more nicely.
He also asks:
But why, when there's a standard combinator that does exactly what you need in this case?
(I ask this as someone who's been slapped for using the state monad when mapAccumL would do.)
It's because the original question asked:
It's not a fold because the results come out like a map. It's not a map because of the state prop across.
and I believe the proper answer is it is a monadic map using the state monad.
Using mapAccumL is surely faster, both less memory and CPU overhead. But the state monad captures the concept of what is going on, the essence of the problem. I believe in many (if not most) cases this is more important. Once we realize the essence of the problem, we can either use the high-level concepts to nicely describe the solution (perhaps sacrificing speed/memory a little) or optimize it to be fast (or perhaps even manage to do both).
On the other hand, mapAccumL solves this particular problem, but doesn't give us a broader answer. If we need to modify it a little, it might happen it won't work any more. Or, if the library starts to be complex, the code can start to be messy and we won't know how to improve it, how to make the original idea clear again.
For example, in the case of evaluating stateful expressions, the library can become complicated and complex. But if we use the state monad, we can build the library around small functions, each taking some arguments and returning something like State[Context,Result]. These atomic computations can be combined to more complex ones using flatMap method or for comprehensions, and finally we'll construct the desired task. The principle will stay the same across the whole library, and the final task will also be something that returns State[Context,Result].
To conclude: I'm not saying using the state monad is the best solution, and certainly it's not the fastest one. I just believe it is most didactic for a functional programmer - it describes the problem in a clean, abstract way.
You could do this recursively:
def testTheRecWay(xs: Seq[String]) = {
def innerTestTheRecWay(xs: Seq[String], priorState: State = initialState, result: Vector[ResType] = Vector()): Seq[ResType] = {
xs match {
case Nil => result
case x :: tail =>
val (res, newState) = computeResultAndNewState(x, priorState)
innerTestTheRecWay(tail, newState, result :+ res)
}
}
innerTestTheRecWay(xs)
}
Recursion is a common practice in functional programming and is most of the time easier to read, write and understand than loops. In fact scala does not have any loops other than while. fold, map, flatMap, for (which is just sugar for flatMap/map), etc. are all recursive.
This method is tail recursive and will be optimized by the compiler to not build a stack, so it is absolutely safe to use. You can add the #annotation.tailrec annotaion, to force the compiler to apply tail recursion elimination. If your method is not tailrec the compiler will then complain.
edit: renamed inner method to avoid ambiguity
It is quite possible that to know whether a function is defined at some point, a significant part of computing its value has to be done. In a PartialFunction, when implementing isDefined and apply, both methods will have to do that. What to do is this common job is costly?
There is the possibility of caching its result, hoping that apply will be called after isDefined. Definitely ugly.
I often wish that PartialFunction[A,B] would be Function[A, Option[B]], which is clearly isomorphic. Or maybe, there could be another method in PartialFunction, say applyOption(a: A): Option[B]. With some mixins, implementors would have a choice of implementing either isDefined and apply or applyOption. Or all of them to be on the safe side, performance wise. Clients which test isDefined just before calling apply would be encouraged to use applyOption instead.
However, this is not so. Some major methods in the library, among them collect in collections require a PartialFunction. Is there a clean (or not so clean) way to avoid paying for computations repeated between isDefined and apply?
Also, is the applyOption(a: A): Option[B] method reasonable? Does it sound feasible to add it in a future version? Would it be worth it?
Why is caching such a problem? In most cases, you have a local computation, so as long as you write a wrapper for the caching, you needn't worry about it. I have the following code in my utility library:
class DroppedFunction[-A,+B](f: A => Option[B]) extends PartialFunction[A,B] {
private[this] var tested = false
private[this] var arg: A = _
private[this] var ans: Option[B] = None
private[this] def cache(a: A) {
if (!tested || a != arg) {
tested = true
arg = a
ans = f(a)
}
}
def isDefinedAt(a: A) = {
cache(a)
ans.isDefined
}
def apply(a: A) = {
cache(a)
ans.get
}
}
class DroppableFunction[A,B](f: A => Option[B]) {
def drop = new DroppedFunction(f)
}
implicit def function_is_droppable[A,B](f: A => Option[B]) = new DroppableFunction(f)
and then if I have an expensive computation, I write a function method A => Option[B] and do something like (f _).drop to use it in collect or whatnot. (If you wanted to do it inline, you could create a method that takes A=>Option[B] and returns a partial function.)
(The opposite transformation--from PartialFunction to A => Option[B]--is called lifting, hence the "drop"; "unlift" is, I think, a more widely used term for the opposite operation.)
Have a look at this thread, Rethinking PartialFunction. You're not the only one wondering about this.
This is an interesting question, and I'll give my 2 cents.
First of resist the urge for premature optimization. Make sure the partial function is the problem. I was amazed at how fast they are on some cases.
Now assuming there is a problem, where would it come from?
Could be a large number of case clauses
Complex pattern matching
Some complex computation on the if causes
One option I'd try to find ways to fail fast. Break the pattern matching into layer, then chain partial functions. This way you can fail the match early. Also extract repeated sub matching. For example:
Lets assume OddEvenList is an extractor that break a list into a odd list and an even list:
var pf1: PartialFuntion[List[Int],R] = {
case OddEvenList(1::ors, 2::ers) =>
case OddEvenList(3::ors, 4::ors) =>
}
Break to two part, one that matches the split then one that tries to match re result (to avoid repeated computation. However this may require some re-engineering
var pf2: PartialFunction[(List[Int],List[Int],R) = {
case (1 :: ors, 2 :: ers) => R1
case (3 :: ors, 4 :: ors) => R2
}
var pf1: PartialFuntion[List[Int],R] = {
case OddEvenList(ors, ers) if(pf2.isDefinedAt(ors,ers) => pf2(ors,ers)
}
I have used this when progressively reading XML files that hard a rather inconstant format.
Another option is to compose partial functions using andThen. Although a quick test here seamed to indicate that only the first was is actually tests.
There is absolutely nothing wrong with caching mechanism inside the partial function, if:
the function returns always the same input, when passed the same argument
it has no side effects
it is completely hidden from the rest of the world
Such cached function is not distiguishable from a plain old pure partial function...