Stacking monads Writer and OptionT - scala

I have the following code:
override def getStandsByUser(email: String): Try[Seq[Stand]] =
(for {
user <- OptionT(userService.findOneByEmail(email)): Try[Option[User]]
stands <- OptionT.liftF(standService.list()):[Try[List[Stand]]]
filtered = stands.filter(stand => user.stands.contains(stand.id))
} yield filtered).getOrElse(Seq())
}
I want to add logging on each stage of the processing - so I need to introduce writer monad and stack it with monad transformer OptionT. Could you please suggest how to do that?

The best way to do this is to convert your service calls into using cats-mtl.
For representing Try or Option you can use MonadError and for logging you can use FunctorTell. Now I don't know what exactly you're doing inside your userService or standService, but I wrote some code to demonstrate what the result might look like:
type Log = List[String]
//inside UserService
def findOneByEmail[F[_]](email: String)
(implicit F: MonadError[F, Error], W: FunctorTell[F, Log]): F[User] = ???
//inside StandService
def list[F[_]]()
(implicit F: MonadError[F, Error], W: FunctorTell[F, Log]): F[List[Stand]] = ???
def getStandsByUser[F[_]](email: String)
(implicit F: MonadError[F, Error], W: FunctorTell[F, Log]): F[List[Stand]] =
for {
user <- userService.findOneByEmail(email)
stands <- standService.list()
} yield stands.filter(stand => user.stands.contains(stand.id))
//here we actually run the function
val result =
getStandsByUser[WriterT[OptionT[Try, ?], Log, ?] // yields WriterT[OptionT[Try, ?], Log, List[Stand]]
.run // yields OptionT[Try, (Log, List[Stand])]
.value // yields Try[Option[(Log, List[Stand])]]
This way we can avoid all of the calls to liftF and easily compose our different services even if they will use different monad transformers at runtime.

If you take a look at the definition of cats.data.Writer you will see that it is an alias to cats.data.WriterT with the effect fixed to Id.
What you want to do is use WriterT directly and instead of Id use OptionT[Try, YourType].
Here is a small code example of how that can be achieved:
object Example {
import cats.data._
import cats.implicits._
type MyType[A] = OptionT[Try, A]
def myFunction: MyType[Int] = OptionT(Try(Option(1)))
def main(args: Array[String]): Unit = {
val tmp: WriterT[MyType, List[String], Int] = for {
_ <- WriterT.tell[MyType, List[String]](List("Before first invocation"))
i <- WriterT.liftF[MyType, List[String], Int](myFunction)
_ <- WriterT.tell[MyType, List[String]](List("After second invocation"))
j <- WriterT.liftF[MyType, List[String], Int](myFunction)
_ <- WriterT.tell[MyType, List[String]](List(s"Result is ${i + j}"))
} yield i + j
val result: Try[Option[(List[String], Int)]] = tmp.run.value
println(result)
// Success(Some((List(Before first invocation, After second invocation, Result is 2),2)))
}
}
The type annotations make this a bit ugly, but depending on your use case you might be able to get rid of them. As you can see myFunction returns a result of type OptionT[Try, Int] and WriterT.lift will push that into a writer object that also has a List[String] for your logs.

Related

cats-effect:How to transform Map[x,IO[y]] to IO[Map[x,y]]

I have a map of string to IO like this Map[String, IO[String]], I want to transform it into IO[Map[String, String]]. How to do it?
It would be nice to use unorderedTraverse here, but as codenoodle pointed out, it doesn't work because IO is not a commutative applicative. However there is a type that is, and it's called IO.Par. Like the name suggests, its ap combinator won't execute things sequentially but in parallel, so it's commutative – doing a and then b is not the same as doing b and then a, but doing a and b concurrently is the same as doing b and a concurrently.
So you can use unorderedTraverse using a function that doesn't return IO but IO.Par. However the downside to that is that now you need to convert from IO to IO.Par and then back – hardly an improvement.
To solve this problem, I have added the parUnorderedTraverse method in cats 2.0 that will take care of these conversions for you. And because it all happens in parallel it will also be more efficient! There are also parUnorderedSequence, parUnorderedFlatTraverse and parUnorderedFlatSequence.
I should also point out that this works not only for IO but also for everything else with a Parallel instance, such as Either[A, ?] (where A is a CommutativeSemigroup). It should also be possible for List/ZipList, but nobody appears to have bothered to do it yet.
You'll have to be a little careful with this one. Maps in Scala are unordered, so if you try to use cats's sequence like this…
import cats.instances.map._
import cats.effect.IO
import cats.UnorderedTraverse
object Example1 {
type StringMap[V] = Map[String, V]
val m: StringMap[IO[String]] = Map("1" -> IO{println("1"); "1"})
val n: IO[StringMap[String]] = UnorderedTraverse[StringMap].unorderedSequence[IO, String](m)
}
you'll get the following error:
Error: could not find implicit value for evidence parameter of type cats.CommutativeApplicative[cats.effect.IO]
The issue here is that the IO monad is not actually commutative. Here is the definition of commutativity:
map2(u, v)(f) = map2(v, u)(flip(f)) // Commutativity (Scala)
This definition shows that the result is the same even when the effects happen in a different order.
You can make the above code compile by providing an instance of CommutativeApplicative[IO] but that still doesn't make the IO monad commutative. If you run the following code you can see the side effects are not processed in the same order:
import cats.effect.IO
import cats.CommutativeApplicative
object Example2 {
implicit object FakeEvidence extends CommutativeApplicative[IO] {
override def pure[A](x: A): IO[A] = IO(x)
override def ap[A, B](ff: IO[A => B])(fa: IO[A]): IO[B] =
implicitly[Applicative[IO]].ap(ff)(fa)
}
def main(args: Array[String]): Unit = {
def flip[A, B, C](f: (A, B) => C) = (b: B, a: A) => f(a, b)
val fa = IO{println(1); 1}
val fb = IO{println(true); true}
val f = (a: Int, b: Boolean) => s"$a$b"
println(s"IO is not commutative: ${FakeEvidence.map2(fa, fb)(f).unsafeRunSync()} == ${FakeEvidence.map2(fb, fa)(flip(f)).unsafeRunSync()} (look at the side effects above^^)")
}
}
Which outputs the following:
1
true
true
1
IO is not commutative: 1true == 1true (look at the side effects above^^)
In order to get around this I would suggest making your map something with an order, like a List, where sequence will not require commutativity. The following example is just one way to do this:
import cats.effect.IO
import cats.implicits._
object Example3 {
val m: Map[String, IO[String]] = Map("1" -> IO {println("1"); "1"})
val l: IO[List[(String, String)]] = m.toList.traverse[IO, (String, String)] { case (s, io) => io.map(s2 => (s, s2))}
val n: IO[Map[String, String]] = l.map { _.toMap }
}

Stacking M, Either and Writer

I'm currently stacking Futures and Eithers using EitherT:
type ErrorOr[A] = Either[Error, A]
def getAge: Future[ErrorOr[Int]] = ???
def getDob(age: Int): ErrorOr[LocalDate] = ???
for {
age <- EitherT(getAge)
dob <- EitherT.fromEither[Future](getDob(age))
} yield dob
I would now like to introduce the Writer monad i.e.
type MyWriter[A] = Writer[Vector[String], ErrorOr[A]]
def getAge: Future[MyWriter[Int]] = ???
def getDob(age: Int): MyWriter[LocalDate] = ???
My question is, what is the best way to sequence the getAge and getDob calls? I know monads can be stacked i.e. Future -> Writer -> Either but can I continue to use EitherT in this scenario? if so how?
Yeah, you can continue using both using the WriterT monad transformers like this:
type FutureErrorOr[A] = EitherT[Future, Error, A]
type MyStack[A] = WriterT[FutureErrorOr, Vector[String], A]
If you unpack this type, it's analogous to Future[Either[Error, Writer[Vector[String], A]]
Now the tricky part is lifting your functions into this base monad, so here are some examples:
def getAge: FutureErrorOr[Int] = ???
def getDob(age: Int): ErrorOr[LocalDate] = ???
for {
age <- WriterT.liftF(getAge)
dob <- WriterT.liftF(EitherT.fromEither(getDob(age)))
} yield dob
To make this easier you can have a look at cats-mtl.
This is a slight variation to the approach given by #luka-jacobowitz. With his approach, any logs that occurred right up until a "failure" will be lost. Given the suggested types:
type FutureErrorOr[A] = EitherT[Future, Error, A]
type MyStack[A] = WriterT[FutureErrorOr, Vector[String], A]
We find that if we expand a value of MyStack[A] with the run method of WriterT we get a value of the following type:
FutureErrorOr[(Vector[String], A)]
which is the same thing as:
EitherT[Future, Error, (Vector[String], A)]
which we can then expand further with the value method of EitherT:
Future[Either[Error, (Vector[String], A)]]
Here we can see that the only way to retrieve the tuple containing the logs of the result is if the program was "successful" (ie. right associative). If the program fails, any previous logs that were created while the program was running are inaccessible.
If we take the original example and modify it slightly to log something after each step and we assume that the second step returns a value of type Left[Error]:
val program = for {
age <- WriterT.liftF(getAge)
_ <- WriterT.tell(Vector("Got age!"))
dob <- WriterT.liftF(EitherT.fromEither(getDob(age))) // getDob returns Left[Error]
_ <- WriterT.tell(Vector("Got date of birth!"))
} yield {
dob
}
Then when we evaluate the result, we will only get back the left case containing the error without any logs:
val expanded = program.run.value // Future(Success(Left(Error)))
val result = Await.result(expanded, Duration.apply(2, TimeUnit.SECONDS)) // Left(Error), no logs!!
In order to get the value that results from running our program AND the logs that were generated up until the point where the program failed, we can reorder the suggested monads like this:
type MyWriter[A] = WriterT[Future, Vector[String], A]
type MyStack[A] = EitherT[MyWriter, Error, A]
Now, if we expand MyStack[A] with the value method of EitherT we get a value of the following type:
WriterT[Future, Vector[String], Either[Error, A]]
which we can expand further with the run method of WriterT to give us a tuple containing the logs AND the resulting value:
Future[(Vector[String], Either[Error, A])]
With this approach, we can re-write the program like this:
val program = for {
age <- EitherT(WriterT.liftF(getAge.value))
_ <- EitherT.liftF(WriterT.put(())(Vector("Got age!")))
dob <- EitherT.fromEither(getDob(age))
_ <- EitherT.liftF(WriterT.put(())(Vector("Got date of birth!")))
} yield {
dob
}
And when we run it, we will have access to the resulting logs even if there is a failure during the program's execution:
val expanded = program.value.run // Future(Success((Vector("Got age!), Left(Error))))
val result = Await.result(expanded, Duration.apply(2, TimeUnit.SECONDS)) // (Vector("Got age!), Left(Error))
Admittedly, this solution requires a bit more boilerplate, but we can always define some helpers to aid with this:
implicit class EitherTOps[A](eitherT: FutureErrorOr[A]) {
def lift: EitherT[MyWriter, Error, A] = {
EitherT[MyWriter, Error, A](WriterT.liftF[Future, Vector[String], ErrorOr[A]](eitherT.value))
}
}
implicit class EitherOps[A](either: ErrorOr[A]) {
def lift: EitherT[MyWriter, Error, A] = {
EitherT.fromEither[MyWriter](either)
}
}
def log(msg: String): EitherT[MyWriter, Error, Unit] = {
EitherT.liftF[MyWriter, Error, Unit](WriterT.put[Future, Vector[String], Unit](())(Vector(msg)))
}
val program = for {
age <- getAge.lift
_ <- log("Got age!")
dob <- getDob(age).lift
_ <- log("Got date of birth!")
} yield {
dob
}

Combining Futures, Eithers and Options in for comprehensions

I have a collection of methods that return different types:
Either[ErrorResponse, X]
Future[Either[ErrorResponse, X]]
Option[ErrorResponse]
These methods need the result from a previous method to perform their computation. The methods:
type Parameters = Map[String, String]
// allows me to flatmap on an either
implicit def toRightProjection[Failure, Success](e: Either[Failure, Success]) =
e.right
// converts anything to a future
implicit def toFuture[T](t: T) =
Future.successful(t)
// retrieves the request paramters from the given request
def requestParameters(request: RequestHeader): Either[ErrorResponse, Parameters] = ???
// retrieves the response type from the given parameters
def responseType(p: Parameters): Either[ErrorResponse, String] = ???
// retrieves the client id from the given parameters
def clientId(p: Parameters): Either[ErrorResponse, String] = ???
// retrieves the client using the given client id
def client(clientId: String): Future[Either[ErrorResponse, Client]] = ???
// validates the response type of the client
def validateResponseType(client: Client, responseType: String): Option[ErrorResponse] = ???
I can the wire them together with the following for comprehension (note that I wrote down some types to clarify the contents of specific parts of the computation).
val result: Either[ErrorResponse, Future[Either[ErrorResponse, Client]]] =
for {
parameters <- requestParameters(request)
clientId <- clientId(parameters)
responseType <- responseType(parameters)
} yield {
val result: Future[Either[ErrorResponse, Either[ErrorResponse, Client]]] =
for {
errorOrClient <- client(clientId)
client <- errorOrClient
} yield validateResponseType(client, responseType).toLeft(client)
result.map(_.joinRight)
}
val wantedResult: Future[Either[ErrorResponse, Client]] =
result.left.map(Future successful Left(_)).merge
The above code is quite messy and I feel this can be done differently. I read about monads and monad transformers. The concept of those is very new to me and I can not get my head around it.
Most of the examples only deal with two types of results: Either[X, Y] and Future[Either[X, Y]]. I still find it very hard to bend my mind around it.
How can I write a nice and clean for comprehension that replaces the above one?
Something like this would be awesome (I am not sure if that is even possible):
val result: Future[Either[ErrorResponse, Client]] =
for {
parameters <- requestParameters(request)
clientId <- clientId(parameters)
responseType <- responseType(parameters)
client <- client(clientId)
_ <- validateResponseType(client, responseType)
}
OK, here is my attempt at this:
import scalaz._, Scalaz._
implicit val futureMonad = new Monad[Future] {
override def point[A](a: ⇒ A): Future[A] = future(a)
override def bind[A, B](fa: Future[A])(f: A ⇒ Future[B]): Future[B] =
fa.flatMap(f)
}
import EitherT._
val result: EitherT[Future, ErrorResponse, Client] =
for {
parameters <- fromEither(Future(requestParameters(request)))
clientId <- fromEither(Future(clientId(parameters)))
responseType <- fromEither(Future(responseType(parameters)))
client <- fromEither(client(clientId))
response <- fromEither[Future, ErrorResponse, Client](Future(validateResponseType(client, responseType).toLeft(client)))
} yield response
val x: Future[\/[ErrorResponse, Client]] = result.run
scala.util.Either is not a Monad, but the scalaz library has a great implementation.
object Test extends ToIdOps {
import scalaz.{ Monad, Functor, EitherT, \/, -\/, \/- }
import scalaz.syntax.ToIdOps
implicit val FutureFunctor = new Functor[Future] {
def map[A, B](a: Future[A])(f: A => B): Future[B] = a map f
}
implicit val FutureMonad = new Monad[Future] {
def point[A](a: => A): Future[A] = Future(a)
def bind[A, B](fa: Future[A])(f: (A) => Future[B]): Future[B] = fa flatMap f
}
def someMethod: Future[\/[InvalidData, ValidData]] = {
// things went well
ValidData.right // this comes from ToIdOps
// or something went wrong
InvalidData.left
}
def someOtherMethod: Future[\/[InvalidData, ValidData]] // same as above
val seq = for {
d <- EitherT(someMethod)
y <- EitherT(someOtherMethod)
} yield { // whatever}
// you can now Await.result(seq.run, duration)
// you can map or match etc with \/- and -\/
val result = seq.run map {
case -\/(left) => // invalid data
case \/-(right) => // game on
}
}
There is no really clean way to do comprehensions over multiple monad types. In ScalaZ there is OptionT that might help, worth checking out. You could also transform your Eithers to Options or the other way around and be able to have a little bit less of a mess. A third option might be to create your own kind of wrapper that combines Future[Either|Option] into the same monad and then comprehend over that.
For reference I asked aboutish the same question on the play framework mailing list recently and got some good links in the replies: https://groups.google.com/d/topic/play-framework/JmCsXNDvAns/discussion

How to avoid stair-stepping with Monad Transformers in scala?

I have the following code that uses the Reader monad for configuration and also has to deal with IO[Option[String]] and I've ended up with code that stair-steps in my encode function.
How can I formulate a monad transformer for Reader and OptionT to avoid the ugly nested for comprehensions in my encode function?
def encode(fileName: String): Reader[Config, IO[Unit]] = for {
ffmpegWrapper <- findFfmpegWrapper
ffmpegBin <- findFfmpeg
} yield (for {
w <- ffmpegWrapper
b <- ffmpegBin
stream <- callFfmpeg(getCommand(w, b, fileName)).liftM[OptionT]
} yield stream) map (_ foreach (println)) getOrElse Unit.box {}
def getCommand(ffmpegWrapper: String, ffmpegBin: String,
videoFile: String) = s"$ffmpegWrapper $ffmpegBin $videoFile '-vcodec libx264 -s 1024x576' /tmp/out.mp4"
def callFfmpeg(command: String): IO[Stream[String]] = IO {
Process(command).lines_!
}
def findFile(path:List[String]): OptionT[IO,String] = OptionT[IO,String](IO{path.find(new File(_).exists)})
def findFfmpeg:Reader[Config, OptionT[IO,String]] = Reader {c=>findFile(c.ffmpegLocations)}
def findFfmpegWrapper:Reader[Config, OptionT[IO,String]] = Reader {c=>findFile(c.ffmpegWrapperLocations)}
Thank-you!
If you look at the definition of Reader in the Scalaz source, you'll see this:
type Reader[-E, +A] = ReaderT[Id, E, A]
Which tells us that the Reader monad you are using is just a specialization of a monad transformer where the monad being wrapped is the trivial Id monad. You can use ReaderT directly, but wrapping your OptionT[IO, _] monad instead of just wrapping everything in a Reader. For example, the following should do what you want:
type OptionIO[+A] = OptionT[IO, A]
def findFfmpeg: ReaderT[OptionIO, Config, String] =
Kleisli[OptionIO, Config, String](c => findFile(c.ffmpegLocations))
def findFfmpegWrapper: ReaderT[OptionIO, Config, String] =
Kleisli[OptionIO, Config, String](c => findFile(c.ffmpegWrapperLocations))
def encode(fileName: String): ReaderT[OptionIO, Config, Unit] = (for {
w <- findFfmpegWrapper
b <- findFfmpeg
stream <- Kleisli[OptionIO, Config, Stream[String]](
_ => callFfmpeg(getCommand(w, b, fileName)).liftM[OptionT]
)
} yield stream).map(_ foreach println)
In principle you should be able to replace the part after stream <- with the following:
callFfmpeg(getCommand(w, b, fileName)).liftM[OptionT].liftReaderT[Config]
But for some reason the Unapply machinery that liftReaderT relies on doesn't seem to be working in this case. Writing the Kleisli part out explicitly isn't all that horrible, fortunately.
As a footnote: the nice liftReaderT syntax I mentioned becomes available if you define an UnapplyCo instance like this:
implicit def unapplyMFA1[TC[_[_]], F[+_], M0[F[+_], +_], A0](
implicit TC0: TC[({ type L[x] = M0[F, x] })#L]
): UnapplyCo[TC, M0[F, A0]] {
type M[+X] = M0[F, X]
type A = A0
} = new UnapplyCo[TC, M0[F, A0]] {
type M[+X] = M0[F, X]
type A = A0
def TC = TC0
def leibniz = Leibniz.refl
}
I'm not sure off the top of my head whether there's a reason Scalaz 7 doesn't currently provide this instance, but it's probably worth looking into.

Monad transformer in for comprehensions

Consider:
def xs(c: String): Option[List[Long]] = ...
val ys: Stream[Long] = ...
Now I'd write a method something like:
def method(oc: Option[String]): Option[Long] = for {
c <- oc
list <- xs(c)
} yield{
for {
first <- ys.find(list contains _)
} yield first
}
but of course this doesn't compile, since the inferred type is Option[Option[Long]].
Is there a way in terms of scala syntax and standard library to get an Option[Long]? I know I can pattern match, but the question if it can be done using for comprehensions only just arised.
Thanks to tenshi for the answer, that does the job, however I just encountered another example of my problem:
class T
class U
class A(t: String)(implicit x: T)
def getU(a: A): Option[U] = ...
def getU_2(oc: Option[String]): Option[U] = for{
c <- oc
} yield{
implicit val someImplicit: T = new T
val a = A(c)
getU(a)
}
I can add a in the for as: a <- Some(A(c)) but what about the implicit? Should that imply a design change in my code?
Why are you using 2 nested for comprehensions? Shouldn't one do the job?
def method(oc: Option[String]): Option[Long] =
for {
c <- oc
list <- xs(c)
first <- ys.find(list contains _)
} yield first
Update
About your second example. You can define implicit elsewhere and import it or define it in the beginning of the method, but I guess you want to make it's scope as narrow as possible. In this case you can use block directly in the for comprehension:
def getU_2(oc: Option[String]): Option[U] = for {
c <- oc
a <- {
implicit val someImplicit: T = new T
getU(new A(c))
}
} yield a
or (probably the simplest) provide implicit parameter explicitly:
def getU_2(oc: Option[String]): Option[U] = for {
c <- oc
a <- getU(new A(c)(new T))
} yield a