I have the following code that uses the Reader monad for configuration and also has to deal with IO[Option[String]] and I've ended up with code that stair-steps in my encode function.
How can I formulate a monad transformer for Reader and OptionT to avoid the ugly nested for comprehensions in my encode function?
def encode(fileName: String): Reader[Config, IO[Unit]] = for {
ffmpegWrapper <- findFfmpegWrapper
ffmpegBin <- findFfmpeg
} yield (for {
w <- ffmpegWrapper
b <- ffmpegBin
stream <- callFfmpeg(getCommand(w, b, fileName)).liftM[OptionT]
} yield stream) map (_ foreach (println)) getOrElse Unit.box {}
def getCommand(ffmpegWrapper: String, ffmpegBin: String,
videoFile: String) = s"$ffmpegWrapper $ffmpegBin $videoFile '-vcodec libx264 -s 1024x576' /tmp/out.mp4"
def callFfmpeg(command: String): IO[Stream[String]] = IO {
Process(command).lines_!
}
def findFile(path:List[String]): OptionT[IO,String] = OptionT[IO,String](IO{path.find(new File(_).exists)})
def findFfmpeg:Reader[Config, OptionT[IO,String]] = Reader {c=>findFile(c.ffmpegLocations)}
def findFfmpegWrapper:Reader[Config, OptionT[IO,String]] = Reader {c=>findFile(c.ffmpegWrapperLocations)}
Thank-you!
If you look at the definition of Reader in the Scalaz source, you'll see this:
type Reader[-E, +A] = ReaderT[Id, E, A]
Which tells us that the Reader monad you are using is just a specialization of a monad transformer where the monad being wrapped is the trivial Id monad. You can use ReaderT directly, but wrapping your OptionT[IO, _] monad instead of just wrapping everything in a Reader. For example, the following should do what you want:
type OptionIO[+A] = OptionT[IO, A]
def findFfmpeg: ReaderT[OptionIO, Config, String] =
Kleisli[OptionIO, Config, String](c => findFile(c.ffmpegLocations))
def findFfmpegWrapper: ReaderT[OptionIO, Config, String] =
Kleisli[OptionIO, Config, String](c => findFile(c.ffmpegWrapperLocations))
def encode(fileName: String): ReaderT[OptionIO, Config, Unit] = (for {
w <- findFfmpegWrapper
b <- findFfmpeg
stream <- Kleisli[OptionIO, Config, Stream[String]](
_ => callFfmpeg(getCommand(w, b, fileName)).liftM[OptionT]
)
} yield stream).map(_ foreach println)
In principle you should be able to replace the part after stream <- with the following:
callFfmpeg(getCommand(w, b, fileName)).liftM[OptionT].liftReaderT[Config]
But for some reason the Unapply machinery that liftReaderT relies on doesn't seem to be working in this case. Writing the Kleisli part out explicitly isn't all that horrible, fortunately.
As a footnote: the nice liftReaderT syntax I mentioned becomes available if you define an UnapplyCo instance like this:
implicit def unapplyMFA1[TC[_[_]], F[+_], M0[F[+_], +_], A0](
implicit TC0: TC[({ type L[x] = M0[F, x] })#L]
): UnapplyCo[TC, M0[F, A0]] {
type M[+X] = M0[F, X]
type A = A0
} = new UnapplyCo[TC, M0[F, A0]] {
type M[+X] = M0[F, X]
type A = A0
def TC = TC0
def leibniz = Leibniz.refl
}
I'm not sure off the top of my head whether there's a reason Scalaz 7 doesn't currently provide this instance, but it's probably worth looking into.
Related
I am using cats library and want to convert Free Monad to Free Applicative.
We have a lot of code in Free monads.. But now some parts of the application has to run in parallel. There are options to use Tagless or Frees.io instead of Free.. But that will be a huge change...
This is an example DSL:
sealed trait DSLAction[A]
case class GetCustomer(request: Boolean) extends DSLAction[String]
case class GetSize(request: Boolean) extends DSLAction[Int]
val f1: Free[DSLAction, String] = liftF(GetCustomer(true))
val f2: Free[DSLAction, Int] = liftF(GetSize(true))
val f3: Free[DSLAction, Int] = liftF(GetSize(false))
val interpreter: DSLAction ~> Id = {
λ[DSLAction ~> Id] {
case GetCustomer(_: Boolean) => {
"hello"
}
case GetSize(_: Boolean) => {
123
}
}
}
Cats library provides a way to convert FreeApplicative to Free monad using .monad()
However, I want to convert Free to FreeApplicative to use them in for comprehension
I want to define a method toApplicative () that does the job...
type FEF[A] = FreeApplicative[DSLAction, A]
val f1AP: FEF[String] = toApplicative(f1)
val f2AP: FEF[Int] = toApplicative(f2)
val prog = for {
a <- (f1AP, f2AP).mapN { case (l, r) => l + r }.monad
b <- f3
} yield {
(a, b)
}
prog.foldMap(interpreter)
I did try to implement something.. But not sure how to define flatMap and tailRec methods..
Or there may be another way
implicit val myConvertor = new Monad[FEF] {
override def pure[A](x: A): FEF[A] = FreeApplicative.pure[DSLAction,A](x)
override def flatMap[A, B](fa: FEF[A])(f: A => FEF[B]): FEF[B] = ???
override def tailRecM[A, B](a: A)(f: A => FEF[Either[A, B]]): FEF[B] = ???
}
final def toApplicative[F, A](free: Free[DSLAction, A]) =
free.foldMap[FreeApplicative[DSLAction, ?]] {
λ[FunctionK[DSLAction, FEF]](fa => FreeApplicative.lift(fa))
}
Thanks
Try
implicit val myConvertor = new Monad[FEF] {
override def pure[A](x: A): FEF[A] = FreeApplicative.pure[DSLAction,A](x)
override def flatMap[A, B](fa: FEF[A])(f: A => FEF[B]): FEF[B] = toApplicative(Monad[Free[DSLAction, ?]].flatMap(fa.monad)(a => f(a).monad))
override def tailRecM[A, B](a: A)(f: A => FEF[Either[A, B]]): FEF[B] = toApplicative(Monad[Free[DSLAction, ?]].tailRecM(a)(a => f(a).monad))
}
Actually toApplicative doesn't work since it needs Monad[FreeApplicative[DSLAction,?]] and such instance doesn't exist, FreeApplicative[DSLAction,?] is an Applicative, not a Monad (on contrary FreeApplicative#monad works because there is Monad[Free[DSLAction,?]]).
https://typelevel.org/cats/datatypes/freeapplicative.html#differences-from-free
So far everything we’ve been doing has been not much different from
Free - we’ve built an algebra and interpreted it. However, there are
some things FreeApplicative can do that Free cannot.
So I guess there is transform from FreeApplicative[DSLAction,?] to Free[DSLAction,?] but there is no transform from Free[DSLAction,?] to FreeApplicative[DSLAction,?]. Generally you can't get thing that can do more (FreeApplicative can do both parallel and consecutive execution) from thing that can do less (Free can do only consecutive execution).
Free is a Monad, so moreover it's an Applicative but this Applicative instance is different from the Applicative instance that FreeApplicative is.
A quote from #BogdanVakulenko's link at reddit
it's genuinely impossible to turn a monad into an applicative in the
general case because many applicatives (e.g. Const[M: Monoid]) don't
form monads. If your monad preserves the distinction between
applicative operations and monadic operations then it's no longer free
I have the following code:
override def getStandsByUser(email: String): Try[Seq[Stand]] =
(for {
user <- OptionT(userService.findOneByEmail(email)): Try[Option[User]]
stands <- OptionT.liftF(standService.list()):[Try[List[Stand]]]
filtered = stands.filter(stand => user.stands.contains(stand.id))
} yield filtered).getOrElse(Seq())
}
I want to add logging on each stage of the processing - so I need to introduce writer monad and stack it with monad transformer OptionT. Could you please suggest how to do that?
The best way to do this is to convert your service calls into using cats-mtl.
For representing Try or Option you can use MonadError and for logging you can use FunctorTell. Now I don't know what exactly you're doing inside your userService or standService, but I wrote some code to demonstrate what the result might look like:
type Log = List[String]
//inside UserService
def findOneByEmail[F[_]](email: String)
(implicit F: MonadError[F, Error], W: FunctorTell[F, Log]): F[User] = ???
//inside StandService
def list[F[_]]()
(implicit F: MonadError[F, Error], W: FunctorTell[F, Log]): F[List[Stand]] = ???
def getStandsByUser[F[_]](email: String)
(implicit F: MonadError[F, Error], W: FunctorTell[F, Log]): F[List[Stand]] =
for {
user <- userService.findOneByEmail(email)
stands <- standService.list()
} yield stands.filter(stand => user.stands.contains(stand.id))
//here we actually run the function
val result =
getStandsByUser[WriterT[OptionT[Try, ?], Log, ?] // yields WriterT[OptionT[Try, ?], Log, List[Stand]]
.run // yields OptionT[Try, (Log, List[Stand])]
.value // yields Try[Option[(Log, List[Stand])]]
This way we can avoid all of the calls to liftF and easily compose our different services even if they will use different monad transformers at runtime.
If you take a look at the definition of cats.data.Writer you will see that it is an alias to cats.data.WriterT with the effect fixed to Id.
What you want to do is use WriterT directly and instead of Id use OptionT[Try, YourType].
Here is a small code example of how that can be achieved:
object Example {
import cats.data._
import cats.implicits._
type MyType[A] = OptionT[Try, A]
def myFunction: MyType[Int] = OptionT(Try(Option(1)))
def main(args: Array[String]): Unit = {
val tmp: WriterT[MyType, List[String], Int] = for {
_ <- WriterT.tell[MyType, List[String]](List("Before first invocation"))
i <- WriterT.liftF[MyType, List[String], Int](myFunction)
_ <- WriterT.tell[MyType, List[String]](List("After second invocation"))
j <- WriterT.liftF[MyType, List[String], Int](myFunction)
_ <- WriterT.tell[MyType, List[String]](List(s"Result is ${i + j}"))
} yield i + j
val result: Try[Option[(List[String], Int)]] = tmp.run.value
println(result)
// Success(Some((List(Before first invocation, After second invocation, Result is 2),2)))
}
}
The type annotations make this a bit ugly, but depending on your use case you might be able to get rid of them. As you can see myFunction returns a result of type OptionT[Try, Int] and WriterT.lift will push that into a writer object that also has a List[String] for your logs.
I am currently reading Hutton's and Meijer's paper on parsing combinators in Haskell http://www.cs.nott.ac.uk/~pszgmh/monparsing.pdf. For the sake of it I am trying to implement them in scala. I would like to construct something easy to code, extend and also simple and elegant. I have come up with two solutions for the following haskell code
/* Haskell Code */
type Parser a = String -> [(a,String)]
result :: a -> Parser a
result v = \inp -> [(v,inp)]
zero :: Parser a
zero = \inp -> []
item :: Parser Char
item = \inp -> case inp of
[] -> []
(x:xs) -> [(x,xs)]
/* Scala Code */
object Hutton1 {
type Parser[A] = String => List[(A, String)]
def Result[A](v: A): Parser[A] = str => List((v, str))
def Zero[A]: Parser[A] = str => List()
def Character: Parser[Char] = str => if (str.isEmpty) List() else List((str.head, str.tail))
}
object Hutton2 {
trait Parser[A] extends (String => List[(A, String)])
case class Result[A](v: A) extends Parser[A] {
def apply(str: String) = List((v, str))
}
case object Zero extends Parser[T forSome {type T}] {
def apply(str: String) = List()
}
case object Character extends Parser[Char] {
def apply(str: String) = if (str.isEmpty) List() else List((str.head, str.tail))
}
}
object Hutton extends App {
object T1 {
import Hutton1._
def run = {
val r: List[(Int, String)] = Zero("test") ++ Result(5)("test")
println(r.map(x => x._1 + 1) == List(6))
println(Character("abc") == List(('a', "bc")))
}
}
object T2 {
import Hutton2._
def run = {
val r: List[(Int, String)] = Zero("test") ++ Result(5)("test")
println(r.map(x => x._1 + 1) == List(6))
println(Character("abc") == List(('a', "bc")))
}
}
T1.run
T2.run
}
Question 1
In Haskell, zero is a function value that can be used as it is, expessing all failed parsers whether they are of type Parser[Int] or Parser[String]. In scala we achieve the same by calling the function Zero (1st approach) but in this way I believe that I just generate a different function everytime Zero is called. Is this statement true? Is there a way to mitigate this?
Question 2
In the second approach, the Zero case object is extending Parser with the usage of existential types Parser[T forSome {type T}] . If I replace the type with Parser[_] I get the compile error
Error:(19, 28) class type required but Hutton2.Parser[_] found
case object Zero extends Parser[_] {
^
I thought these two expressions where equivalent. Is this the case?
Question 3
Which approach out of the two do you think that will yield better results in expressing the combinators in terms of elegance and simplicity?
I use scala 2.11.8
Note: I didn't compile it, but I know the problem and can propose two solutions.
The more Haskellish way would be to not use subtyping, but to define zero as a polymorphic value. In that style, I would propose to define parsers not as objects deriving from a function type, but as values of one case class:
final case class Parser[T](run: String => List[(T, String)])
def zero[T]: Parser[T] = Parser(...)
As shown by #Alec, yes, this will produce a new value every time, since a def is compiled to a method.
If you want to use subtyping, you need to make Parser covariant. Then you can give zero a bottom result type:
trait Parser[+A] extends (String => List[(A, String)])
case object Zero extends Parser[Nothing] {...}
These are in some way quite related; in system F_<:, which is the base of what Scala uses, the types _|_ (aka Nothing) and \/T <: Any. T behave the same (this hinted at in Types and Programming Languages, chapter 28). The two possibilities given here are a consequence of this fact.
With existentials I'm not so familiar with, but I think that while unbounded T forSome {type T} will behave like Nothing, Scala does not allow inhertance from an existential type.
Question 1
I think that you are right, and here is why: Zero1 below prints hello every time you use it. The solution, Zero2, involves using a val instead.
def Zero1[A]: Parser[A] = { println("hi"); str => List() }
val Zero2: Parser[Nothing] = str => List()
Question 2
No idea. I'm still just starting out with Scala. Hope someone answers this.
Question 3
The trait one will play better with Scala's for (since you can define custom flatMap and map), which turns out to be (somewhat) like Haskell's do. The following is all you need.
trait Parser[A] extends (String => List[(A, String)]) {
def flatMap[B](f: A => Parser[B]): Parser[B] = {
val p1 = this
new Parser[B] {
def apply(s1: String) = for {
(a,s2) <- p1(s1)
p2 = f(a)
(b,s3) <- p2(s2)
} yield (b,s3)
}
}
def map[B](f: A => B): Parser[B] = {
val p = this
new Parser[B] {
def apply(s1: String) = for ((a,s2) <- p(s1)) yield (f(a),s2)
}
}
}
Of course, to do anything interesting you need more parsers. I'll propose to you one simple parser combinator: Choice(p1: Parser[A], p2: Parser[A]): Parser[A] which tries both parsers. (And rewrite your existing parsers more to my style).
def choice[A](p1: Parser[A], p2: Parser[A]): Parser[A] = new Parser[A] {
def apply(s: String): List[(A,String)] = { p1(s) ++ p2(s) }
}
def unit[A](x: A): Parser[A] = new Parser[A] {
def apply(s: String): List[(A,String)] = List((x,s))
}
val character: Parser[Char] = new Parser[Char] {
def apply(s: String): List[(Char,String)] = List((s.head,s.tail))
}
Then, you can write something like the following:
val parser: Parser[(Char,Char)] = for {
x <- choice(unit('x'),char)
y <- char
} yield (x,y)
And calling parser("xyz") gives you List((('x','x'),"yz"), (('x','y'),"z")).
I have an idea (vague), to pass (or chain) some implicit value in this manner, not introducing parameters to block f:
def block(x: Int)(f: => Unit)(implicit v: Int) = {
implicit val nv = v + x
f
}
def fun(implicit v: Int) = println(v)
such that if I used something alike:
implicit val ii: Int = 0
block(1) {
block(2) {
fun
}
}
It would print 3.
If I could say def block(x: Int)(f: implicit Int => Unit).
In other words I'm seeking for some design pattern which will allow me to implement this DSL: access some cumulative value inside nested blocks but without explicitly passing it as parameter. Is it possible? (implicits are not necessary, just a hint to emphasize that I don't want to pass that accumulator explicitly). Of course upper code will print 0.
EDIT: One of possible usages: composing http routes, in a following manner
prefix("path") {
prefix("subpath") {
post("action1") { (req, res) => do action }
get("action2") { (req, res) => do action }
}
}
Here post and get will access (how?) accumulated prefix, say List("path", "subpath") or "/path/subpath/".
Consider using DynamicVariable for this. It's really simple to use, and thread-safe:
val acc: DynamicVariable[Int] = new DynamicVariable(0)
def block(x: Int)(f: => Unit) = {
acc.withValue(acc.value + x)(f)
}
def fun = println(acc.value)
Passing state via implicit is dirty and will lead to unexpected and hard to track down bugs. What you're asking to do is build a function that can compose in such a way that nested calls accumulate over some operation and anything else uses that value to execute the function?
case class StateAccum[S](init: S){
val op: S => S
def flatMap[A <: S](f: S => StateAccum[A]) ={
val StateAccum(out) = f(s)
StateAccum(op(init, out))
}
def apply(f: S => A) = f(init)
}
which could allow you do exactly what you're after with a slight change in how you're calling it.
Now, if you really want the nested control structures, your apply would have to use an implicit value to distinguish the types of the return such that it applied the function to one and a flatMap to StateAccum returns. It gets crazy but looks like the following:
def apply[A](f: S => A)(implicit mapper: Mapper[S, A]): mapper.Out = mapper(this, f)
trait Mapper[S, A]{
type Out
def apply(s: StateAccum[S], f: S => A): Out
}
object Mapper extends LowPriorityMapper{
implicit def acuum[S, A <: S] = new Mapper[S, StateAccum[A]]{
type Out = StateAccum[A]
def apply(s: StateAccum[S], f: S => StateAccum[A]) = s.flatMap(f)
}
}
trait LowPriorityMapper{
implicit def acuum[S, A] = new Mapper[S, A]{
type Out = A
def apply(s: StateAccum[S], f: S => A) = f(s.init)
}
}
I have a collection of methods that return different types:
Either[ErrorResponse, X]
Future[Either[ErrorResponse, X]]
Option[ErrorResponse]
These methods need the result from a previous method to perform their computation. The methods:
type Parameters = Map[String, String]
// allows me to flatmap on an either
implicit def toRightProjection[Failure, Success](e: Either[Failure, Success]) =
e.right
// converts anything to a future
implicit def toFuture[T](t: T) =
Future.successful(t)
// retrieves the request paramters from the given request
def requestParameters(request: RequestHeader): Either[ErrorResponse, Parameters] = ???
// retrieves the response type from the given parameters
def responseType(p: Parameters): Either[ErrorResponse, String] = ???
// retrieves the client id from the given parameters
def clientId(p: Parameters): Either[ErrorResponse, String] = ???
// retrieves the client using the given client id
def client(clientId: String): Future[Either[ErrorResponse, Client]] = ???
// validates the response type of the client
def validateResponseType(client: Client, responseType: String): Option[ErrorResponse] = ???
I can the wire them together with the following for comprehension (note that I wrote down some types to clarify the contents of specific parts of the computation).
val result: Either[ErrorResponse, Future[Either[ErrorResponse, Client]]] =
for {
parameters <- requestParameters(request)
clientId <- clientId(parameters)
responseType <- responseType(parameters)
} yield {
val result: Future[Either[ErrorResponse, Either[ErrorResponse, Client]]] =
for {
errorOrClient <- client(clientId)
client <- errorOrClient
} yield validateResponseType(client, responseType).toLeft(client)
result.map(_.joinRight)
}
val wantedResult: Future[Either[ErrorResponse, Client]] =
result.left.map(Future successful Left(_)).merge
The above code is quite messy and I feel this can be done differently. I read about monads and monad transformers. The concept of those is very new to me and I can not get my head around it.
Most of the examples only deal with two types of results: Either[X, Y] and Future[Either[X, Y]]. I still find it very hard to bend my mind around it.
How can I write a nice and clean for comprehension that replaces the above one?
Something like this would be awesome (I am not sure if that is even possible):
val result: Future[Either[ErrorResponse, Client]] =
for {
parameters <- requestParameters(request)
clientId <- clientId(parameters)
responseType <- responseType(parameters)
client <- client(clientId)
_ <- validateResponseType(client, responseType)
}
OK, here is my attempt at this:
import scalaz._, Scalaz._
implicit val futureMonad = new Monad[Future] {
override def point[A](a: ⇒ A): Future[A] = future(a)
override def bind[A, B](fa: Future[A])(f: A ⇒ Future[B]): Future[B] =
fa.flatMap(f)
}
import EitherT._
val result: EitherT[Future, ErrorResponse, Client] =
for {
parameters <- fromEither(Future(requestParameters(request)))
clientId <- fromEither(Future(clientId(parameters)))
responseType <- fromEither(Future(responseType(parameters)))
client <- fromEither(client(clientId))
response <- fromEither[Future, ErrorResponse, Client](Future(validateResponseType(client, responseType).toLeft(client)))
} yield response
val x: Future[\/[ErrorResponse, Client]] = result.run
scala.util.Either is not a Monad, but the scalaz library has a great implementation.
object Test extends ToIdOps {
import scalaz.{ Monad, Functor, EitherT, \/, -\/, \/- }
import scalaz.syntax.ToIdOps
implicit val FutureFunctor = new Functor[Future] {
def map[A, B](a: Future[A])(f: A => B): Future[B] = a map f
}
implicit val FutureMonad = new Monad[Future] {
def point[A](a: => A): Future[A] = Future(a)
def bind[A, B](fa: Future[A])(f: (A) => Future[B]): Future[B] = fa flatMap f
}
def someMethod: Future[\/[InvalidData, ValidData]] = {
// things went well
ValidData.right // this comes from ToIdOps
// or something went wrong
InvalidData.left
}
def someOtherMethod: Future[\/[InvalidData, ValidData]] // same as above
val seq = for {
d <- EitherT(someMethod)
y <- EitherT(someOtherMethod)
} yield { // whatever}
// you can now Await.result(seq.run, duration)
// you can map or match etc with \/- and -\/
val result = seq.run map {
case -\/(left) => // invalid data
case \/-(right) => // game on
}
}
There is no really clean way to do comprehensions over multiple monad types. In ScalaZ there is OptionT that might help, worth checking out. You could also transform your Eithers to Options or the other way around and be able to have a little bit less of a mess. A third option might be to create your own kind of wrapper that combines Future[Either|Option] into the same monad and then comprehend over that.
For reference I asked aboutish the same question on the play framework mailing list recently and got some good links in the replies: https://groups.google.com/d/topic/play-framework/JmCsXNDvAns/discussion