How to create Async[Future] from Async[IO] - scala

I am trying to implicitly add Async and Sync in my code for doobie repository. The Sync and Async[F] works fine IO. I want to convert them to Future and facing problem
I have tried to create my own Aync from IO
def futureAsync(implicit F: MonadError[Future, Throwable]): Async[Future] = new Async[Future] {
override def async[A](k: (Either[Throwable, A] => Unit) => Unit): Future[A] = IO.async(k).unsafeToFuture()
override def asyncF[A](k: (Either[Throwable, A] => Unit) => Future[Unit]): Future[A] =
throw new Exception("Not implemented Future.asyncF")
override def suspend[A](thunk: => Future[A]): Future[A] = thunk
override def bracketCase[A, B](acquire: Future[A])(use: A => Future[B])(release: (A, ExitCase[Throwable]) => Future[Unit]): Future[B] =
throw new Exception("Not implemented Future.bracketCase")
override def raiseError[A](e: Throwable): Future[A] = F.raiseError(e)
override def handleErrorWith[A](fa: Future[A])(f: Throwable => Future[A]): Future[A] = F.handleErrorWith(fa)(_ => f(new Exception("")))
override def pure[A](x: A): Future[A] = F.pure(x)
override def flatMap[A, B](fa: Future[A])(f: A => Future[B]): Future[B] = F.flatMap(fa)(f)
override def tailRecM[A, B](a: A)(f: A => Future[Either[A, B]]): Future[B] = F.tailRecM(a)(f)
}
I am struck with implementation of two functions in there asyncF and bracketCase
Can some one help?

As Reactormonk says in a comment above, it's not possible to write an instance of Async for Future that has the right semantics, because Async extends Sync, and Sync requires a representation of a computation that can be run repeatedly, while Scala's futures begin running when they're defined and can't be re-run.
An unlawful instance
It's instructive to see this for yourself, though, and I'd encourage you to try to write your own compile-able but (necessarily) unlawful Async[Future] instance without looking at the next block of code. For the sake of the example, though, here's a quick sketch off the top of my head:
import scala.concurrent.{ExecutionContext, Future, Promise}
import scala.util.{Failure, Success}
import cats.effect.{Async, ExitCase, IO}
def futureAsync(implicit c: ExecutionContext): Async[Future] = new Async[Future] {
def async[A](k: (Either[Throwable, A] => Unit) => Unit): Future[A] =
IO.async(k).unsafeToFuture()
def asyncF[A](k: (Either[Throwable, A] => Unit) => Future[Unit]): Future[A] = {
val p = Promise[A]()
val f = k {
case Right(a) => p.success(a)
case Left(e) => p.failure(e)
}
f.flatMap(_ => p.future)
}
def suspend[A](thunk: => Future[A]): Future[A] = Future(thunk).flatten
def bracketCase[A, B](acquire: Future[A])(use: A => Future[B])(
release: (A, ExitCase[Throwable]) => Future[Unit]
): Future[B] = acquire.flatMap { a =>
use(a).transformWith {
case Success(b) => release(a, ExitCase.Completed).map(_ => b)
case Failure(e) => release(a, ExitCase.Error(e)).flatMap(_ => Future.failed(e))
}
}
def raiseError[A](e: Throwable): Future[A] = Future.failed(e)
def handleErrorWith[A](fa: Future[A])(f: Throwable => Future[A]): Future[A] =
fa.recoverWith { case t => f(t) }
def pure[A](x: A): Future[A] = Future.successful(x)
def flatMap[A, B](fa: Future[A])(f: A => Future[B]): Future[B] = fa.flatMap(f)
def tailRecM[A, B](a: A)(f: A => Future[Either[A, B]]): Future[B] = f(a).flatMap {
case Right(b) => Future.successful(b)
case Left(a) => tailRecM(a)(f)
}
}
This will compile just fine, and would probably work for some situations (but please don't actually use it!). We've said it can't have the right semantics, though, and we can show that by using cats-effect's laws module.
Checking the laws
First we need some boilerplate-y stuff you don't really need to worry about:
import cats.kernel.Eq, cats.implicits._
import org.scalacheck.Arbitrary
implicit val throwableEq: Eq[Throwable] = Eq.by[Throwable, String](_.toString)
implicit val nonFatalArbitrary: Arbitrary[Throwable] =
Arbitrary(Arbitrary.arbitrary[Exception].map(identity))
implicit def futureEq[A](implicit A: Eq[A], ec: ExecutionContext): Eq[Future[A]] =
new Eq[Future[A]] {
private def liftToEither(f: Future[A]): Future[Either[Throwable, A]] =
f.map(Right(_)).recover { case e => Left(e) }
def eqv(fx: Future[A], fy: Future[A]): Boolean =
scala.concurrent.Await.result(
liftToEither(fx).zip(liftToEither(fy)).map {
case (rx, ry) => rx === ry
},
scala.concurrent.duration.Duration(1, "second")
)
}
Then we can define a test that checks the Async laws for our instance:
import cats.effect.laws.discipline.{AsyncTests, Parameters}
import org.scalatest.FunSuite
import org.typelevel.discipline.scalatest.Discipline
object FutureAsyncSuite extends FunSuite with Discipline {
implicit val ec: ExecutionContext = ExecutionContext.global
implicit val params: Parameters =
Parameters.default.copy(allowNonTerminationLaws = false)
checkAll(
"Async",
AsyncTests[Future](futureAsync).async[String, String, String]
)
}
And then we can run the law tests:
scala> FutureAsyncSuite.execute()
FutureAsyncSuite:
- Async.async.acquire and release of bracket are uncancelable
- Async.async.ap consistent with product + map
- Async.async.applicative homomorphism
...
You'll see that most of the tests are green; this instance gets a lot of things right.
Where it breaks the law
It does show three failed tests, though, including the following:
- Async.async.repeated sync evaluation not memoized *** FAILED ***
GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation.
(Discipline.scala:14)
Falsified after 1 successful property evaluations.
Location: (Discipline.scala:14)
Occurred when passed generated values (
arg0 = "淳칇멀",
arg1 = org.scalacheck.GenArities$$Lambda$7154/1834868832#1624ea25
)
Label of failing property:
Expected: Future(Success(驅ṇ숆㽝珅뢈矉))
Received: Future(Success(淳칇멀))
If you look at the laws definitions, you'll see that this is a test that defines a Future value with delay and then sequences it multiple times, like this:
val change = F.delay { /* observable side effect here */ }
val read = F.delay(cur)
change *> change *> read
The other two failures are similar "not memoized" violations. These tests should see the side effect happen twice, but in our case it's not possible to write delay or suspend for Future in such a way that that would happen (it's worth trying, though, to convince yourself that this is the case).
What you should do instead
To sum up: you can write an Async[Future] instance that will pass something like 75 of the 78 Async laws tests, but it's not possible to write an instance that will pass all of them, and using an unlawful instance is a really bad idea: both potential users of your code and libraries like Doobie will assume that your instances are lawful, and if you don't live up to this assumption you're opening the door to complex and annoying bugs.
It's worth noting that it's not too hard to write a minimal wrapper for Future that has a lawful Async instance (for example I've got a wrapper for Twitter's future called Rerunnable in my catbird library). You really should just stick with cats.effect.IO, though, and use the provided conversions to convert to and from futures in any parts of your code where you're working with traditional Future-based APIs.

Related

How to convert Free monads to Free applicatives

I am using cats library and want to convert Free Monad to Free Applicative.
We have a lot of code in Free monads.. But now some parts of the application has to run in parallel. There are options to use Tagless or Frees.io instead of Free.. But that will be a huge change...
This is an example DSL:
sealed trait DSLAction[A]
case class GetCustomer(request: Boolean) extends DSLAction[String]
case class GetSize(request: Boolean) extends DSLAction[Int]
val f1: Free[DSLAction, String] = liftF(GetCustomer(true))
val f2: Free[DSLAction, Int] = liftF(GetSize(true))
val f3: Free[DSLAction, Int] = liftF(GetSize(false))
val interpreter: DSLAction ~> Id = {
λ[DSLAction ~> Id] {
case GetCustomer(_: Boolean) => {
"hello"
}
case GetSize(_: Boolean) => {
123
}
}
}
Cats library provides a way to convert FreeApplicative to Free monad using .monad()
However, I want to convert Free to FreeApplicative to use them in for comprehension
I want to define a method toApplicative () that does the job...
type FEF[A] = FreeApplicative[DSLAction, A]
val f1AP: FEF[String] = toApplicative(f1)
val f2AP: FEF[Int] = toApplicative(f2)
val prog = for {
a <- (f1AP, f2AP).mapN { case (l, r) => l + r }.monad
b <- f3
} yield {
(a, b)
}
prog.foldMap(interpreter)
I did try to implement something.. But not sure how to define flatMap and tailRec methods..
Or there may be another way
implicit val myConvertor = new Monad[FEF] {
override def pure[A](x: A): FEF[A] = FreeApplicative.pure[DSLAction,A](x)
override def flatMap[A, B](fa: FEF[A])(f: A => FEF[B]): FEF[B] = ???
override def tailRecM[A, B](a: A)(f: A => FEF[Either[A, B]]): FEF[B] = ???
}
final def toApplicative[F, A](free: Free[DSLAction, A]) =
free.foldMap[FreeApplicative[DSLAction, ?]] {
λ[FunctionK[DSLAction, FEF]](fa => FreeApplicative.lift(fa))
}
Thanks
Try
implicit val myConvertor = new Monad[FEF] {
override def pure[A](x: A): FEF[A] = FreeApplicative.pure[DSLAction,A](x)
override def flatMap[A, B](fa: FEF[A])(f: A => FEF[B]): FEF[B] = toApplicative(Monad[Free[DSLAction, ?]].flatMap(fa.monad)(a => f(a).monad))
override def tailRecM[A, B](a: A)(f: A => FEF[Either[A, B]]): FEF[B] = toApplicative(Monad[Free[DSLAction, ?]].tailRecM(a)(a => f(a).monad))
}
Actually toApplicative doesn't work since it needs Monad[FreeApplicative[DSLAction,?]] and such instance doesn't exist, FreeApplicative[DSLAction,?] is an Applicative, not a Monad (on contrary FreeApplicative#monad works because there is Monad[Free[DSLAction,?]]).
https://typelevel.org/cats/datatypes/freeapplicative.html#differences-from-free
So far everything we’ve been doing has been not much different from
Free - we’ve built an algebra and interpreted it. However, there are
some things FreeApplicative can do that Free cannot.
So I guess there is transform from FreeApplicative[DSLAction,?] to Free[DSLAction,?] but there is no transform from Free[DSLAction,?] to FreeApplicative[DSLAction,?]. Generally you can't get thing that can do more (FreeApplicative can do both parallel and consecutive execution) from thing that can do less (Free can do only consecutive execution).
Free is a Monad, so moreover it's an Applicative but this Applicative instance is different from the Applicative instance that FreeApplicative is.
A quote from #BogdanVakulenko's link at reddit
it's genuinely impossible to turn a monad into an applicative in the
general case because many applicatives (e.g. Const[M: Monoid]) don't
form monads. If your monad preserves the distinction between
applicative operations and monadic operations then it's no longer free

functional parallelism and laziness in Scala

Background
I have been reading the book Functional Programming in Scala, and have some questions regarding the content in Chapter 7: Purely functional parallelism.
Here is the code for the answers in the book: Par.scala, but I am confused about certain part of it.
Here is the first part of the code of Par.scala, which stands for Parallelism:
import java.util.concurrent._
object Par {
type Par[A] = ExecutorService => Future[A]
def unit[A](a: A): Par[A] = (es: ExecutorService) => UnitFuture(a)
private case class UnitFuture[A](get: A) extends Future[A] {
def isDone = true
def get(timeout: Long, units: TimeUnit): A = get
def isCancelled = false
def cancel(evenIfRunning: Boolean): Boolean = false
}
def map2[A, B, C](a: Par[A], b: Par[B])(f: (A, B) => C): Par[C] =
(es: ExecutorService) => {
val af = a(es)
val bf = b(es)
UnitFuture(f(af.get, bf.get))
}
def fork[A](a: => Par[A]): Par[A] =
(es: ExecutorService) => es.submit(new Callable[A] {
def call: A = a(es).get
})
def lazyUnit[A](a: => A): Par[A] =
fork(unit(a))
def run[A](es: ExecutorService)(a: Par[A]): Future[A] = a(es)
def asyncF[A, B](f: A => B): A => Par[B] =
a => lazyUnit(f(a))
def map[A, B](pa: Par[A])(f: A => B): Par[B] =
map2(pa, unit(()))((a, _) => f(a))
}
The simplest possible model for Par[A] might be ExecutorService => Future[A], and run simply returns the Future.
unit promotes a constant value to a parallel computation by returning a UnitFuture, which is a simple implementation of Future that just wraps a constant value.
map2 combines the results of two parallel computations with a binary function.
fork marks a computation for concurrent evaluation. The evaluation won’t actually occur until forced by run. Here is with its simplest and most natural implementation of it. Even though it has its problems, let's first put them aside.
lazyUnit wraps its unevaluated argument in a Par and marks it for concurrent evaluation.
run extracts a value from a Par by actually performing the computation.
asyncF converts any function A => B to one that evaluates its result asynchronously.
Questions
The fork is the function confuses me a lot here, because it takes a lazy argument, which will be evaluated later when it is called. Then my questions are more about when we should use this fork, i.e., when we need lazy-evaluation and when we need to have the value directly.
Here is an exercise from the book:
EXERCISE 7.5
Hard: Write this function, called sequence. No additional primitives are required. Do not call run.
def sequence[A](ps: List[Par[A]]): Par[List[A]]
And here is the answers (offered here).
First
def sequence_simple[A](l: List[Par[A]]): Par[List[A]] =
l.foldRight[Par[List[A]]](unit(List()))((h, t) => map2(h, t)(_ :: _))
What is the different between above code and the following:
def sequence_simple[A](l: List[Par[A]]): Par[List[A]] =
l.foldLeft[Par[List[A]]](unit(List()))((t, h) => map2(h, t)(_ :: _))
Additionally
def sequenceRight[A](as: List[Par[A]]): Par[List[A]] =
as match {
case Nil => unit(Nil)
case h :: t => map2(h, fork(sequenceRight(t)))(_ :: _)
}
def sequenceBalanced[A](as: IndexedSeq[Par[A]]): Par[IndexedSeq[A]] = fork {
if (as.isEmpty) unit(Vector())
else if (as.length == 1) map(as.head)(a => Vector(a))
else {
val (l,r) = as.splitAt(as.length/2)
map2(sequenceBalanced(l), sequenceBalanced(r))(_ ++ _)
}
}
In sequenceRight, fork is used when recursive function is directly called. However, in sequenceBalanced, fork is used outside of the whole function body.
Then, what is the differences or above code and the following (where we switched the places of fork):
def sequenceRight[A](as: List[Par[A]]): Par[List[A]] = fork {
as match {
case Nil => unit(Nil)
case h :: t => map2(h, sequenceRight(t))(_ :: _)
}
}
def sequenceBalanced[A](as: IndexedSeq[Par[A]]): Par[IndexedSeq[A]] =
if (as.isEmpty) unit(Vector())
else if (as.length == 1) map(as.head)(a => Vector(a))
else {
val (l,r) = as.splitAt(as.length/2)
map2(fork(sequenceBalanced(l)), fork(sequenceBalanced(r)))(_ ++ _)
}
Finally, given the sequence defined above, we have the following function:
def parMap[A,B](ps: List[A])(f: A => B): Par[List[B]] = fork {
val fbs: List[Par[B]] = ps.map(asyncF(f))
sequence(fbs)
}
I would like to know, can I also implement the function in the following way, which is by applying the lazyUnit defined in the beginning? Is this implementation lazyUnit(ps.map(f)) lazy?
def parMapByLazyUnit[A, B](ps: List[A])(f: A => B): Par[List[B]] =
lazyUnit(ps.map(f))
I did not completely understand your doubt. But I see a major problem with the following solution,
def parMapByLazyUnit[A, B](ps: List[A])(f: A => B): Par[List[B]] =
lazyUnit(ps.map(f))
To understand the problem lets look at def lazyUnit,
def fork[A](a: => Par[A]): Par[A] =
(es: ExecutorService) => es.submit(new Callable[A] {
def call: A = a(es).get
})
def lazyUnit[A](a: => A): Par[A] =
fork(unit(a))
So... lazyUnit takes an expression of type => A and submits it to ExecutorService to get evaluated. And returns the wrapped result of this parallel computation as Par[A].
In parMap for every element of ps: List[A], we not only have to evaluate the corresponding mapping using the function f: A => B but we have to do these evaluations in parallel.
But our solution lazyUnit(ps.map(f)) will submit the whole { ps.map(f) } evaluation as a single task to our ExecutionService. Which means we are not doing it in parallel.
What we need to do is make sure that for each element a in ps: [A], the function f: A => B is executed as a separate task for our ExecutorService.
Now, as we learned from our implementation is that we can run an expression of type exp: => A by using lazyUnit(exp) to get a result: Par[A].
So, we will do exactly that for every a: A in ps: List[A],
val parMappedTmp = ps.map( a => lazyUnit(f(a) ) )
// or
val parMappedTmp = ps.map( a => asyncF(f)(a) )
// or
val parMappedTmp = ps.map(asyncF(f))
But, Now our parMappedTmp is a List[Par[B]] and whereas we needed a Par[List[B]]
So, you will need a function with the following signature to get what you wanted,
def sequence[A](ps: List[Par[A]]): Par[List[A]]
Once you have it,
val parMapped = sequence(parMappedTmp)

Converting blocking code to using scala futures

My old code looks something like below, where all db calls blocking.
I need help converting this over to using Futures.
def getUserPoints(username: String): Option[Long]
db.getUserPoints(username) match {
case Some(userPoints) => Some(userPoints.total)
case None => {
if (db.getSomething("abc").isEmpty) {
db.somethingElse("asdf") match {
case Some(pointId) => {
db.setPoints(pointId, username)
db.findPointsForUser(username)
}
case _ => None
}
} else {
db.findPointsForUser(username)
}
}
}
}
My new API is below where I am returning Futures.
db.getUserPoints(username: String): Future[Option[UserPoints]]
db.getSomething(s: String): Future[Option[Long]]
db.setPoints(pointId, username): Future[Unit]
db.findPointsForUser(username): Future[Option[Long]]
How can I go about converting the above to use my new API that uses futures.
I tried using a for-compr but started to get wierd errors like Future[Nothing].
var userPointsFut: Future[Long] = for {
userPointsOpt <- db.getUserPoints(username)
userPoints <- userPointsOpt
} yield userPoints.total
But it gets a bit tricky with all the branching and if clauses and trying to convert it over to futures.
I would argue that the first issue with this design is that the port of the blocking call to a Future should not wrap the Option type:
The blocking call:
def giveMeSomethingBlocking(for:Id): Option[T]
Should become:
def giveMeSomethingBlocking(for:Id): Future[T]
And not:
def giveMeSomethingBlocking(for:Id): Future[Option[T]]
The blocking call give either a value Some(value) or None, the non-blocking Future version gives either a Success(value) or Failure(exception) which fully preserves the Option semantics in a non-blocking fashion.
With that in mind, we can model the process in question using combinators on Future. Let's see how:
First, lets refactor the API to something we can work with:
type UserPoints = Long
object db {
def getUserPoints(username: String): Future[UserPoints] = ???
def getSomething(s: String): Future[UserPoints] = ???
def setPoints(pointId:UserPoints, username: String): Future[Unit] = ???
def findPointsForUser(username: String): Future[UserPoints] = ???
}
class PointsNotFound extends Exception("bonk")
class StuffNotFound extends Exception("sthing not found")
Then, the process would look like:
def getUserPoints(username:String): Future[UserPoints] = {
db.getUserPoints(username)
.map(userPoints => userPoints /*.total*/)
.recoverWith{
case ex:PointsNotFound =>
(for {
sthingElse <- db.getSomething("abc")
_ <- db.setPoints(sthingElse, username)
points <- db.findPointsForUser(username)
} yield (points))
.recoverWith{
case ex: StuffNotFound => db.findPointsForUser(username)
}
}
}
Which type-checks correctly.
Edit
Given that the API is set in stone, a way to deal with nested monadic types is to define a MonadTransformer. In simple words, let's make Future[Option[T]] a new monad, let's call it FutureO that can be composed with other of its kind. [1]
case class FutureO[+A](future: Future[Option[A]]) {
def flatMap[B](f: A => FutureO[B])(implicit ec: ExecutionContext): FutureO[B] = {
val newFuture = future.flatMap{
case Some(a) => f(a).future
case None => Future.successful(None)
}
FutureO(newFuture)
}
def map[B](f: A => B)(implicit ec: ExecutionContext): FutureO[B] = {
FutureO(future.map(option => option map f))
}
def recoverWith[U >: A](pf: PartialFunction[Throwable, FutureO[U]])(implicit executor: ExecutionContext): FutureO[U] = {
val futOtoFut: FutureO[U] => Future[Option[U]] = _.future
FutureO(future.recoverWith(pf andThen futOtoFut))
}
def orElse[U >: A](other: => FutureO[U])(implicit executor: ExecutionContext): FutureO[U] = {
FutureO(future.flatMap{
case None => other.future
case _ => this.future
})
}
}
And now we can re-write our process preserving the same structure as the future-based composition.
type UserPoints = Long
object db {
def getUserPoints(username: String): Future[Option[UserPoints]] = ???
def getSomething(s: String): Future[Option[Long]] = ???
def setPoints(pointId: UserPoints, username:String): Future[Unit] = ???
def findPointsForUser(username: String): Future[Option[Long]] = ???
}
class PointsNotFound extends Exception("bonk")
class StuffNotFound extends Exception("sthing not found")
def getUserPoints2(username:String): Future[Option[UserPoints]] = {
val futureOpt = FutureO(db.getUserPoints(username))
.map(userPoints => userPoints /*.total*/)
.orElse{
(for {
sthingElse <- FutureO(db.getSomething("abc"))
_ <- FutureO(db.setPoints(sthingElse, username).map(_ => Some(())))
points <- FutureO(db.findPointsForUser(username))
} yield (points))
.orElse{
FutureO(db.findPointsForUser(username))
}
}
futureOpt.future
}
[1] with acknowledgements to http://loicdescotte.github.io/posts/scala-compose-option-future/

Implicit parameter "chaining" for DSL

I have an idea (vague), to pass (or chain) some implicit value in this manner, not introducing parameters to block f:
def block(x: Int)(f: => Unit)(implicit v: Int) = {
implicit val nv = v + x
f
}
def fun(implicit v: Int) = println(v)
such that if I used something alike:
implicit val ii: Int = 0
block(1) {
block(2) {
fun
}
}
It would print 3.
If I could say def block(x: Int)(f: implicit Int => Unit).
In other words I'm seeking for some design pattern which will allow me to implement this DSL: access some cumulative value inside nested blocks but without explicitly passing it as parameter. Is it possible? (implicits are not necessary, just a hint to emphasize that I don't want to pass that accumulator explicitly). Of course upper code will print 0.
EDIT: One of possible usages: composing http routes, in a following manner
prefix("path") {
prefix("subpath") {
post("action1") { (req, res) => do action }
get("action2") { (req, res) => do action }
}
}
Here post and get will access (how?) accumulated prefix, say List("path", "subpath") or "/path/subpath/".
Consider using DynamicVariable for this. It's really simple to use, and thread-safe:
val acc: DynamicVariable[Int] = new DynamicVariable(0)
def block(x: Int)(f: => Unit) = {
acc.withValue(acc.value + x)(f)
}
def fun = println(acc.value)
Passing state via implicit is dirty and will lead to unexpected and hard to track down bugs. What you're asking to do is build a function that can compose in such a way that nested calls accumulate over some operation and anything else uses that value to execute the function?
case class StateAccum[S](init: S){
val op: S => S
def flatMap[A <: S](f: S => StateAccum[A]) ={
val StateAccum(out) = f(s)
StateAccum(op(init, out))
}
def apply(f: S => A) = f(init)
}
which could allow you do exactly what you're after with a slight change in how you're calling it.
Now, if you really want the nested control structures, your apply would have to use an implicit value to distinguish the types of the return such that it applied the function to one and a flatMap to StateAccum returns. It gets crazy but looks like the following:
def apply[A](f: S => A)(implicit mapper: Mapper[S, A]): mapper.Out = mapper(this, f)
trait Mapper[S, A]{
type Out
def apply(s: StateAccum[S], f: S => A): Out
}
object Mapper extends LowPriorityMapper{
implicit def acuum[S, A <: S] = new Mapper[S, StateAccum[A]]{
type Out = StateAccum[A]
def apply(s: StateAccum[S], f: S => StateAccum[A]) = s.flatMap(f)
}
}
trait LowPriorityMapper{
implicit def acuum[S, A] = new Mapper[S, A]{
type Out = A
def apply(s: StateAccum[S], f: S => A) = f(s.init)
}
}

Combining Futures, Eithers and Options in for comprehensions

I have a collection of methods that return different types:
Either[ErrorResponse, X]
Future[Either[ErrorResponse, X]]
Option[ErrorResponse]
These methods need the result from a previous method to perform their computation. The methods:
type Parameters = Map[String, String]
// allows me to flatmap on an either
implicit def toRightProjection[Failure, Success](e: Either[Failure, Success]) =
e.right
// converts anything to a future
implicit def toFuture[T](t: T) =
Future.successful(t)
// retrieves the request paramters from the given request
def requestParameters(request: RequestHeader): Either[ErrorResponse, Parameters] = ???
// retrieves the response type from the given parameters
def responseType(p: Parameters): Either[ErrorResponse, String] = ???
// retrieves the client id from the given parameters
def clientId(p: Parameters): Either[ErrorResponse, String] = ???
// retrieves the client using the given client id
def client(clientId: String): Future[Either[ErrorResponse, Client]] = ???
// validates the response type of the client
def validateResponseType(client: Client, responseType: String): Option[ErrorResponse] = ???
I can the wire them together with the following for comprehension (note that I wrote down some types to clarify the contents of specific parts of the computation).
val result: Either[ErrorResponse, Future[Either[ErrorResponse, Client]]] =
for {
parameters <- requestParameters(request)
clientId <- clientId(parameters)
responseType <- responseType(parameters)
} yield {
val result: Future[Either[ErrorResponse, Either[ErrorResponse, Client]]] =
for {
errorOrClient <- client(clientId)
client <- errorOrClient
} yield validateResponseType(client, responseType).toLeft(client)
result.map(_.joinRight)
}
val wantedResult: Future[Either[ErrorResponse, Client]] =
result.left.map(Future successful Left(_)).merge
The above code is quite messy and I feel this can be done differently. I read about monads and monad transformers. The concept of those is very new to me and I can not get my head around it.
Most of the examples only deal with two types of results: Either[X, Y] and Future[Either[X, Y]]. I still find it very hard to bend my mind around it.
How can I write a nice and clean for comprehension that replaces the above one?
Something like this would be awesome (I am not sure if that is even possible):
val result: Future[Either[ErrorResponse, Client]] =
for {
parameters <- requestParameters(request)
clientId <- clientId(parameters)
responseType <- responseType(parameters)
client <- client(clientId)
_ <- validateResponseType(client, responseType)
}
OK, here is my attempt at this:
import scalaz._, Scalaz._
implicit val futureMonad = new Monad[Future] {
override def point[A](a: ⇒ A): Future[A] = future(a)
override def bind[A, B](fa: Future[A])(f: A ⇒ Future[B]): Future[B] =
fa.flatMap(f)
}
import EitherT._
val result: EitherT[Future, ErrorResponse, Client] =
for {
parameters <- fromEither(Future(requestParameters(request)))
clientId <- fromEither(Future(clientId(parameters)))
responseType <- fromEither(Future(responseType(parameters)))
client <- fromEither(client(clientId))
response <- fromEither[Future, ErrorResponse, Client](Future(validateResponseType(client, responseType).toLeft(client)))
} yield response
val x: Future[\/[ErrorResponse, Client]] = result.run
scala.util.Either is not a Monad, but the scalaz library has a great implementation.
object Test extends ToIdOps {
import scalaz.{ Monad, Functor, EitherT, \/, -\/, \/- }
import scalaz.syntax.ToIdOps
implicit val FutureFunctor = new Functor[Future] {
def map[A, B](a: Future[A])(f: A => B): Future[B] = a map f
}
implicit val FutureMonad = new Monad[Future] {
def point[A](a: => A): Future[A] = Future(a)
def bind[A, B](fa: Future[A])(f: (A) => Future[B]): Future[B] = fa flatMap f
}
def someMethod: Future[\/[InvalidData, ValidData]] = {
// things went well
ValidData.right // this comes from ToIdOps
// or something went wrong
InvalidData.left
}
def someOtherMethod: Future[\/[InvalidData, ValidData]] // same as above
val seq = for {
d <- EitherT(someMethod)
y <- EitherT(someOtherMethod)
} yield { // whatever}
// you can now Await.result(seq.run, duration)
// you can map or match etc with \/- and -\/
val result = seq.run map {
case -\/(left) => // invalid data
case \/-(right) => // game on
}
}
There is no really clean way to do comprehensions over multiple monad types. In ScalaZ there is OptionT that might help, worth checking out. You could also transform your Eithers to Options or the other way around and be able to have a little bit less of a mess. A third option might be to create your own kind of wrapper that combines Future[Either|Option] into the same monad and then comprehend over that.
For reference I asked aboutish the same question on the play framework mailing list recently and got some good links in the replies: https://groups.google.com/d/topic/play-framework/JmCsXNDvAns/discussion