I have a method
def getInfoFromService(reqParams: Map[String, String]): Future[Either[CustomException, A]]
and I have another function
import cats.implicits._
def processInfoAndModelResponse(input: Future[Either[CustomException, A]]): Option[A] = {
for {
either <- input
resultA <- either
} yield resultA.some
}
So basically, I am trying to convert Future[Either[CustomException, A]]) to Option[A]. The resultA <- either in above code would not work as map is not happy with type.
What you try to do is basically something like
def convert[A](future: Future[Either[CustomException, A]]): Option[A] =
Try(Await.result(future, timeout)).toEither.flatten.toOption
This:
looses information about error, which prevents any debugging (though you could log after .flatten)
blocks async operation which kills any benefit of using Future in the first place
Monads do not work like that. Basically you can handle several monadic calculations with for for the same monad, but Future and Either are different monads.
Try
import scala.concurrent.{Await, Future}
import scala.concurrent.duration._
def processInfoAndModelResponse(input: Future[Either[CustomException, A]]): Option[A] = {
Await.result(input, 1.minute).toOption
}
#MateuszKubuszok's answer is different in that it doesn't throw.
You can't sensibly convert a Future[Either[CustomException, A]] to Option[A], as you don't know what the result will be until the Future has completed.
Future[Either[CustomException, A]] might just bet kept that way, or put in IO as IO[Either[CustomException, A]]. If you don't care about CustomException, you can also capture it Future (or IO) error handling mechanisms, or just discard it entirely.
Some options:
absorb the CustomException in Future's error handling:
val fa: Future[A] = fut.transform(_.map(_.toTry))
ignore the CustomException an make the Future return with None
val fa: Future[Option[A]] = fut.map(_.toOption)
if you really want to block a thread and wait for the result,
Await.result(input, Duration.Inf).toOption
but all the caveats of awaiting apply. See the documentaion, reproduced here:
WARNING: It is strongly discouraged to supply lengthy timeouts since the progress of the calling thread will be suspended—blocked—until either the Awaitable has a result or the timeout expires.
Related
I am trying to write a Cats MTL version of a function that would save an entity to a database. I want this function to read some SaveOperation[F[_]] from environment, execute it and handle possible failure. So far I came up with 2 version of this function: save is the more polymorphic MTL version and save2 uses exact monads in its signature, meaning that I confine myself to use of IO.
type SaveOperation[F[_]] = Employee => F[Int]
def save[F[_] : Monad](employee: Employee)(implicit
A: Ask[F, SaveOperation[F]],
R: Raise[F, AppError]): F[Unit] =
for {
s <- A.ask
rows <- s(employee)
res <- if rows != 1 then R.raise(FailedInsertion)
else ().pure[F]
} yield res
def save2(employee: Employee): Kleisli[IO, SaveOperation[IO], Either[AppError, Unit]] =
Kleisli((saveOperation) => saveOperation(employee)
.handleErrorWith(err => IO.pure(Left(PersistenceError(err))))
.map(rows =>
if rows != 1 then Left(FailedInsertion)
else Right(())
)
)
I can later call those like this:
val repo = new DoobieEmployeeRepository(xa)
val employee = Employee("john", "doe", Set())
type E[A] = Kleisli[IO, SaveOperation[IO], Either[AppError, A]]
println(EmployeeService.save[E](employee).run(repo.save).unsafeRunSync())
println(EmployeeService.save2(employee).run(repo.save).unsafeRunSync())
The problem is that for the call of save I get the following error:
Could not find an instance of Monad for E.
I found:
cats.data.Kleisli.catsDataMonadErrorForKleisli[F, A, E]
But method catsDataMonadErrorForKleisli in class KleisliInstances0_5 does not match type cats.Monad[E].
This error doesn't seem to make sense to me as effectively signatures are exactly the same for both function, so the monad should be there. I suspect the problem is with Ask[F, SaveOperation[F]] parameter as here F is not IO, while SaveOperation needs the IO.
Why can't I use the Kleisli monad for save call?
Update:
If I modify the type to type E[A] = EitherT[[X] =>> Kleisli[IO, SaveOperation[IO], X], AppError, A], I get a new error:
Could not find an implicit instance of Ask[E, SaveOperation[E]]
The right generic type for SaveOperation is supposed to be IO I guess, but I can't figure how to properly provide it through an instance of Ask
I hope you don't mind if I use this opportunity to do a quick tutorial on how to improve your question. It not only increases the chances of someone answering, but also might help you to find the solution yourself.
There are a couple of problems with the code you submitted, and I mean problems in terms of being a question on SO. Perhaps someone might have a ready answer just by looking at it, but let's say they don't, and they want to try it out in a worksheet. Turns out, your code has a lot of unnecessary stuff and doesn't compile.
Here are some steps you could take to make it better:
Strip away the unnecessary custom dependencies like Employee, DoobieEmployeeRepository, error types etc. and replace them with vanilla Scala types like String or Throwable.
Strip away any remaining code as long as you can still reproduce the problem. For example, the implementations of save and save2 are not needed, and neither are Ask and Raise.
Make sure that the code compiles. This includes adding the necessary imports.
By following these guidelines, we arrive at something like this:
import cats._
import cats.data.Kleisli
import cats.effect.IO
type SaveOperation[F[_]] = String => F[Int]
def save[F[_] : Monad](s: String)(): F[Unit] = ???
def save2(s: String): Kleisli[IO, SaveOperation[IO], Either[Throwable, Unit]] = ???
type E[A] = Kleisli[IO, SaveOperation[IO], Either[Throwable, A]]
println(save[E]("Foo")) // problem!
println(save2("Bar"))
That's already much better, because
a) it allows people to quickly try out your code, and
b) less code means less cognitive load and less space for problems.
Now, to check what's happening here, let's go to some docs:
https://typelevel.org/cats/datatypes/kleisli.html#type-class-instances
It has a Monad instance as long as the chosen F[_] does.
That's interesting, so let's try to further reduce our code:
type E[A] = Kleisli[IO, String, Either[Throwable, A]]
implicitly[Monad[E]] // Monad[E] doesn't exist
OK, but what about:
type E[A] = Kleisli[IO, String, A]
implicitly[Monad[E]] // Monad[E] exists!
This is the key finding. And the reason that Monad[E] doesn't exist in the first case is:
Monad[F[_]] expects a type constructor; F[_] is short for A => F[A] (note that this is actually Kleisli[F, A, A] :)). But if we try to "fix" the value type in Kleisli to Either[Throwable, A] or Option[A] or anything like that, then Monad instance doesn't exist any more. The contract was that we would provide the Monad typeclass with some type A => F[A], but now we're actually providing A => F[Either[Throwable, A]]. Monads don't compose so easily, which is why we have monad transformers.
EDIT:
After a bit of clarification, I think I know what you're going after now. Please check this code:
case class Employee(s: String, s2: String)
case class AppError(msg: String)
type SaveOperation[F[_]] = Employee => F[Int]
def save[F[_] : Monad](employee: Employee)(implicit
A: Ask[F, SaveOperation[F]],
R: Raise[F, AppError]): F[Unit] = for {
s <- A.ask
rows <- s(employee)
res <- if (rows != 1) R.raise(AppError("boom"))
else ().pure[F]
} yield res
implicit val askSaveOp = new Ask[IO, SaveOperation[IO]] {
override def applicative: Applicative[IO] =
implicitly[Applicative[IO]]
override def ask[E2 >: SaveOperation[IO]]: IO[E2] = {
val fun = (e: Employee) => IO({println(s"Saved $e!"); 1})
IO(fun)
}
}
implicit val raiseAppErr = new Raise[IO, AppError] {
override def functor: Functor[IO] =
implicitly[Functor[IO]]
override def raise[E2 <: AppError, A](e: E2): IO[A] =
IO.raiseError(new Throwable(e.msg))
}
save[IO](Employee("john", "doe")).unsafeRunSync() // Saved Employee(john,doe)!
I'm not sure why you expected Ask and Raise to already exist, they are referring to custom types Employee and AppError. Perhaps I'm missing something. So what I did here was that I implemented them myself, and I also got rid of your convoluted type E[A] because what you really want as F[_] is simply IO. There is not much point of having a Raise if you also want to have an Either. I think it makes sense to just have the code based around F monad, with Ask and Raise instances that can store the employee and raise error (in my example, error is raised if you return something other than 1 in the implementation of Ask).
Can you check if this is what you're trying to achieve? We're getting close. Perhaps you wanted to have a generic Ask defined for any kind of SaveOperation input, not just Employee? For what it's worth, I've worked with codebases like this and they can quickly blow up into code that's hard to read and maintain. MTL is fine, but I wouldn't want to go more generic than this. I might even prefer to pass the save function as a parameter rather than via Ask instance, but that's a personal preference.
I have a method which returns some ZIO:
def method(...): ZIO[Any with clock, SomeError, Unit]
The method which calls this return Task[Unit]:
def otherMethod(..): Task[Unit] = {
ZIO.effect(method(...))
}
The problem is when I call it with ZIO.effect I don't get result.
How I should convert ZIO to Task to get result?
With ZIO.effect(method(...)) you get back a Task[ZIO[...]] that is rarely what you want (it's conceptually similar to a nested Future[Future[A]]).
In order to turn a ZIO[R, E, A] into a Taks[A], you have to understand that the latter is just a type alias for ZIO[Any, Throwable, A], which suggest that you have to
eliminate the dependency on the environment R (by providing it)
turn the error type E to Throwable if it's not already a sub-type of it (e.g. by .mapError)
This should work:
def otherMethod(..): Task[Unit] =
method(...)
.mapError(someError => new RuntimeException(s"failed with: $someError"))
.provideLayer(Clock.live)
how to implement a tail recursion functions in scala with futures as return values:
Example code
def getInfo(lists: List[Int]): Future[List[Int]] = {
def getStudentIDs(lists: List[Int]): Future[List[Int]] = {
//here a web service call that returns a future WS response
val response=ws.call(someURL)
response.map(r => {
r.status match {
case 200 =>
var newList = lists + (r.json \ ids)
.as[List[Int]] //here add the ws call response json..
getStudentIDs(newList)
case 429 =>Future.sucessful(lists)
case _ => getStudentIDs(lists)
}
})
}
getStudentIDs(List.empty[Int])
}
I think it's an XY-problem. You probably don't want it "tail-recursive" in the sense of "having a #tailrec"-annotation. What you want is stack safety, so that this method does not blow the stack after a few hundred retries to connect to your webservice.
For this, there are libraries, for example Cats.
In Cats, there is a typeclass called Monad, and this typeclass provides a special method for what seems to be exactly what you want:
tailRecM[A, B](a: A)(f: (A) => F[Either[A, B]]): F[B]
Quote from the docs:
Keeps calling f until a scala.util.Right[B] is returned.
Implementations of this method should use constant stack space [...]
There is an implementation of this for Future in FutureInstances available. The implementation seems trivial though, because it mixes in StackSafeMonad.
You could of course look into implementation of StackSafeMonad, and then try to understand why it is sufficient in the case of Future, but you could also just use the library implementation instead and not worry about whether your recursive method can fail with a StackOverflowError.
Here is the problem (code simplified to make it runnable):
import scala.concurrent._
import scala.concurrent.duration._
import annotation.tailrec
import scala.concurrent.ExecutionContext.Implicits.global
def getInfo(lists: List[Int]): Future[List[Int]] = {
#tailrec
def getStudentIDs(lists: List[Int]): Future[List[Int]] = {
Future(List(1, 2, 3)).flatMap(x => getStudentIDs(x ::: lists))
}
getStudentIDs(List.empty[Int])
}
Gives an error:
error: could not optimize #tailrec annotated method getStudentIDs:
it contains a recursive call not in tail position
Future(1).flatMap(x => getStudentIDs(x :: lists))
^
The problem is not only with a Future. The actual problem is that getStudents is not in terminal/tail position - it's called from map. This would be a problem if you use no Futures and use regular map from collections or any other function for that matter. For example:
def getInfo(lists: List[Int]): List[Int] = {
#tailrec
def getStudentIDs(lists: List[Int]): List[Int] = {
List(1).flatMap(x => getStudentIDs(x :: lists))
}
getStudentIDs(List.empty[Int])
}
Gives the same error:
error: could not optimize #tailrec annotated method getStudentIDs:
it contains a recursive call not in tail position
List(1).flatMap(x => getStudentIDs(x :: lists))
^
What makes it difficult here is that you can't just get a result from future directly to use it in getStudents because you don't know if it's completed and it's not a good practice to block on Future and await for result. So you sort of forced to use map. Here is a very bad example of how to make it tail recursive (just for science :)). Don't do it in production code:
def getInfo(lists: List[Int]): Future[List[Int]] = {
#tailrec
def getStudentIDs(lists: List[Int]): Future[List[Int]] = {
val r = Await.ready(Future(List(1, 2, 3)), Duration.Inf).value.get.getOrElse(lists)
getStudentIDs(r ::: lists)
}
getStudentIDs(List.empty[Int])
}
The compiler is happy, but this might lead to many problems - read about Await, blocking and thread pools for more on that.
I think it's probably not a big issue that your function is not tail recursive because you probably don't want to create lots of futures this way anyway. There are other concurrency frameworks you could try if that's really a problem like actors (Akka), etc.
I have a sequence of functions that return a future. I want to execute them sequentially i.e. after the first function future is complete, execute the next function and so on. Is there a way to do it?
ops: Seq[() => Future[Unit]]
You can combine all the futures into a single one with a foldLeft and flatMap:
def executeSequentially(ops: Seq[() => Future[Unit]])(
implicit exec: ExecutionContext
): Future[Unit] =
ops.foldLeft(Future.successful(()))((cur, next) => cur.flatMap(_ => next()))
foldLeft ensures the order from left to right and flatMap gives sequential execution. Functions are executed with the ExecutionContext, so calling executeSequentially is not blocking. And you can add callbacks or await on the resulting Future when/if you need it.
If you are using Twitter Futures, then I guess you won't need to pass ExecutionContext, but the general idea with foldLeft and flatMap should still work.
If given a Seq[Future[T]] you can convert it to a Future[Seq[T]] like so:
Val a: Seq[Future[T]] = ???
val resut: Future[Seq[T]] = Future.sequence(a)
a little less boilerplate than the above :)
I believe this should do it:
import scala.concurrent.{Await, Future}
import scala.concurrent.duration.Duration
def runSequentially(ops: Seq[() => Future[Unit]]): Unit = {
ops.foreach(f => Await.result(f(), Duration.Inf))
}
If you want to wait less then Duration.Inf, or stop at failure - should be easy to do.
Why and how specifically is a Scala Future not a Monad; and would someone please compare it to something that is a Monad, like an Option?
The reason I'm asking is Daniel Westheide's The Neophyte's Guide to Scala Part 8: Welcome to the Future where I asked whether or not a Scala Future was a Monad, and the author responded that it wasn't, which threw off base. I came here to ask for a clarification.
A summary first
Futures can be considered monads if you never construct them with effectful blocks (pure, in-memory computation), or if any effects generated are not considered as part of semantic equivalence (like logging messages). However, this isn't how most people use them in practice. For most people using effectful Futures (which includes most uses of Akka and various web frameworks), they simply aren't monads.
Fortunately, a library called Scalaz provides an abstraction called Task that doesn't have any problems with or without effects.
A monad definition
Let's review briefly what a monad is. A monad must be able to define at least these two functions:
def unit[A](block: => A)
: Future[A]
def bind[A, B](fa: Future[A])(f: A => Future[B])
: Future[B]
And these functions must statisfy three laws:
Left identity: bind(unit(a))(f) ≡ f(a)
Right identity: bind(m) { unit(_) } ≡ m
Associativity: bind(bind(m)(f))(g) ≡ bind(m) { x => bind(f(x))(g) }
These laws must hold for all possible values by definition of a monad. If they don't, then we simply don't have a monad.
There are other ways to define a monad that are more or less the same. This one is popular.
Effects lead to non-values
Almost every usage of Future that I've seen uses it for asychronous effects, input/output with an external system like a web service or a database. When we do this, a Future isn't even a value, and mathematical terms like monads only describe values.
This problem arises because Futures execute immediately upon data contruction. This messes up the ability to substitute expressions with their evaluated values (which some people call "referential transparency"). This is one way to understand why Scala's Futures are inadequate for functional programming with effects.
Here's an illustration of the problem. If we have two effects:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits._
def twoEffects =
( Future { println("hello") },
Future { println("hello") } )
we will have two printings of "hello" upon calling twoEffects:
scala> twoEffects
hello
hello
scala> twoEffects
hello
hello
But if Futures were values, we should be able to factor out the common expression:
lazy val anEffect = Future { println("hello") }
def twoEffects = (anEffect, anEffect)
But this doesn't give us the same effect:
scala> twoEffects
hello
scala> twoEffects
The first call to twoEffects runs the effect and caches the result, so the effect isn't run the second time we call twoEffects.
With Futures, we end up having to think about the evaluation policy of the language. For instance, in the example above, the fact I use a lazy value rather than a strict one makes a difference in the operational semantics. This is exactly the kind of twisted reasoning functional programming is designed to avoid -- and it does it by programming with values.
Without substitution, laws break
In the presense of effects, monad laws break. Superficially, the laws appear to hold for simple cases, but the moment we begin to substitute expressions with their evaluated values, we end up with the same problems we illustrated above. We simply can't talk about mathematical concepts like monads when we don't have values in the first place.
To put it bluntly, if you use effects with your Futures, saying they're monads is not even wrong because they aren't even values.
To see how monad laws break, just factor out your effectful Future:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits._
def unit[A]
(block: => A)
: Future[A] =
Future(block)
def bind[A, B]
(fa: Future[A])
(f: A => Future[B])
: Future[B] =
fa flatMap f
lazy val effect = Future { println("hello") }
Again, it will only run one time, but you need it to run twice -- once for the right-hand side of the law, and another for the left. I'll illustrate the problem for the right identity law:
scala> effect // RHS has effect
hello
scala> bind(effect) { unit(_) } // LHS doesn't
The implicit ExecutionContext
Without putting an ExecutionContext in implicit scope, we can't define either unit or bind in our monad. This is because the Scala API for Futures has these signature:
object Future {
// what we need to define unit
def apply[T]
(body: ⇒ T)
(implicit executor: ExecutionContext)
: Future[T]
}
trait Future {
// what we need to define bind
flatMap[S]
(f: T ⇒ Future[S])
(implicit executor: ExecutionContext)
: Future[S]
}
As a "convenience" to the user, the standard library encourages users to define an execution context in implicit scope, but I think this is a huge hole in the API that just leads to defects. One scope of the computation may have one execution context defined while another scope can have another context defined.
Perhaps you can ignore the problem if you define an instance of unit and bind that pins both operations to a single context and use this instance consistently. But this is not what people do most of the time. Most of the time, people use Futures with for-yield comprehensions that become map and flatMap calls. To make for-yield comprehensions work, an execution context must be defined at some non-global implicit scope (because for-yield doesn't provide a way to specify additional parameters to the map and flatMap calls).
To be clear, Scala lets you use lots of things with for-yield comprehensions that aren't actually monads, so don't believe that you have a monad just because it works with for-yield syntax.
A better way
There's a nice library for Scala called Scalaz that has an abstraction called scalaz.concurrent.Task. This abstraction doesn't run effects upon data construction as the standard library Future does. Furthermore, Task actually is a monad. We compose Task monadically (we can use for-yield comprehensions if we like), and no effects run while we're composing. We have our final program when we have composed a single expression evaluating to Task[Unit]. This ends up being our equivalent of a "main" function, and we can finally run it.
Here's an example illustrating how we can substitute Task expressions with their respective evaluated values:
import scalaz.concurrent.Task
import scalaz.IList
import scalaz.syntax.traverse._
def twoEffects =
IList(
Task delay { println("hello") },
Task delay { println("hello") }).sequence_
We will have two printings of "hello" upon calling twoEffects:
scala> twoEffects.run
hello
hello
And if we factor out the common effect,
lazy val anEffect = Task delay { println("hello") }
def twoEffects =
IList(anEffect, anEffect).sequence_
we get what we'd expect:
scala> twoEffects.run
hello
hello
In fact, it doesn't really matter that whether we use a lazy value or a strict value with Task; we get hello printed out twice either way.
If you want to functionally program, consider using Task everywhere you may use Futures. If an API forces Futures upon you, you can convert the Future to a Task:
import concurrent.
{ ExecutionContext, Future, Promise }
import util.Try
import scalaz.\/
import scalaz.concurrent.Task
def fromScalaDeferred[A]
(future: => Future[A])
(ec: ExecutionContext)
: Task[A] =
Task
.delay { unsafeFromScala(future)(ec) }
.flatMap(identity)
def unsafeToScala[A]
(task: Task[A])
: Future[A] = {
val p = Promise[A]
task.runAsync { res =>
res.fold(p failure _, p success _)
}
p.future
}
private def unsafeFromScala[A]
(future: Future[A])
(ec: ExecutionContext)
: Task[A] =
Task.async(
handlerConversion
.andThen { future.onComplete(_)(ec) })
private def handlerConversion[A]
: ((Throwable \/ A) => Unit)
=> Try[A]
=> Unit =
callback =>
{ t: Try[A] => \/ fromTryCatch t.get }
.andThen(callback)
The "unsafe" functions run the Task, exposing any internal effects as side-effects. So try not to call any of these "unsafe" functions until you've composed one giant Task for your entire program.
I believe a Future is a Monad, with the following definitions:
def unit[A](x: A): Future[A] = Future.successful(x)
def bind[A, B](m: Future[A])(fun: A => Future[B]): Future[B] = fut.flatMap(fun)
Considering the three laws:
Left identity:
Future.successful(a).flatMap(f) is equivalent to f(a). Check.
Right identity:
m.flatMap(Future.successful _) is equivalent to m (minus some possible performance implications). Check.
Associativity
m.flatMap(f).flatMap(g) is equivalent to m.flatMap(x => f(x).flatMap(g)). Check.
Rebuttal to "Without substitution, laws break"
The meaning of equivalent in the monad laws, as I understand it, is you could replace one side of the expression with the other side in your code without changing the behavior of the program. Assuming you always use the same execution context, I think that is the case. In the example #sukant gave, it would have had the same issue if it had used Option instead of Future. I don't think the fact that the futures are evaluated eagerly is relevant.
As the other commenters have suggested, you are mistaken. Scala's Future type has the monadic properties:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits._
def unit[A](block: => A): Future[A] = Future(block)
def bind[A, B](fut: Future[A])(fun: A => Future[B]): Future[B] = fut.flatMap(fun)
This is why you can use for-comprehension syntax with futures in Scala.