How to write for comprehension in cats with IO monad - scala

I have the following code:
import cats.effect.IO
import cats.data.State
import cats.data.StateT
import cats.implicits._
import cats.effect.LiftIO
abstract class Example {
object implicits {
implicit def myEffectLiftIO: LiftIO[IOGameplay] =
new LiftIO[IOGameplay] {
override def liftIO[A](ioa: IO[A]): IOGameplay[A] = {
StateT.liftF(ioa)
}
}
}
type Gameplay[A] = State[GameState, A]
type IOGameplay[A] = StateT[IO, GameState, A]
type EitherDirection[A] = Either[Throwable, A]
type Map = Array[Array[FieldType]]
sealed trait FieldType
case class GameState(map: Map, block: Block)
case class Block(f1: Field, f2: Field)
case class Field()
import implicits._
val L = implicitly[LiftIO[IOGameplay]]
sealed trait GameResult
sealed trait Direction
trait IOMonad {
def println(msg: String): IO[Unit]
def readln(): IO[String]
}
def play(io: IOMonad): StateT[IO, GameState, GameResult] = {
val L = implicitly[LiftIO[IOGameplay]]
for {
// print map to the console
_ <- L.liftIO(io.println("Next move: "))
directionOpt <- L.liftIO(readDirection(io))
direction <- StateT.liftF[IO, GameState, Direction](IO.fromEither(directionOpt))
nextBlock <- IO(nextBlock(direction))
gameResult <- calculate(nextBlock)
} yield {
gameResult
}
}
def readDirection(io: IOMonad): IO[EitherDirection[Direction]]
def nextBlock(direction: Direction): Gameplay[Block]
def calculate(block: Block): Gameplay[GameResult]
}
This is not completely accurate, but I posted the whole block to explain the problem.
Here, I have many transformations on values to produce IO and to transform it to StateT. Is there a more clever way to do this? Maybe I should somehow separate io tasks from the main algorithm, i.e. from this for-comprehension? Or should I do it like this?

One issue is that your Gameplay type is not compatible with IOGameplay, since Gameplay uses the Eval monad. I assume you want this:
type Gameplay[F[_], A] = StateT[F, GameState, A]
type IOGameplay[A] = Gameplay[IO, A]
These methods need to return IOGameplay instances (or you could lift them in your program later):
def nextBlock(direction: Direction): IOGameplay[Block]
def calculate(block: Block): IOGameplay[GameResult]
Then the for-comprehension compiles with minor adjustments:
for {
// print map to the console
_ <- L.liftIO(io.println("Next move: "))
directionOpt <- L.liftIO(readDirection(io))
direction <- StateT.liftF[IO, GameState, Direction](IO.fromEither(directionOpt))
nextBlock <- nextBlock(direction)
gameResult <- calculate(nextBlock)
} yield {
gameResult
}
BTW, what is the intended purpose of the IO effect in this program? User input?

If your goal is to avoid lifting stuff from one monad to the other, then you can make your methods and interfaces polymorphic so that they can work with different monads and not just IO. Here's how to do that for your IOMonad trait:
trait IOMonad[F[_]] {
def println(msg: String): F[Unit]
def readln(): F[String]
}
The idea is to not commit to any specific monad, but to make things work for any monad that provides the features that you need for a specific use case. In the IOMonad example, we need the ability to run synchronous side effects, so we express that by passing a parameter of type Sync[F]:
import cats.effect.Sync
object IOMonad {
def apply[F[_]](implicit F: Sync[F]) = new IOMonad[F] {
def println(msg: String): F[Unit] = F.delay(println(msg))
def readln(): F[String] = F.delay(scala.io.StdIn.readLine())
}
}
The other operations in your program need different capabilities. For instance readDirection needs to do console IO and raise errors of type Throwable. The ability to raise errors is expressed by the MonadError trait, so you get this signature:
def readDirection[F[_]](
io: IOMonad[F])(implicit monErr: MonadError[F, Throwable]
): F[Direction]
It's important to note that we're not passing a Sync[F] here, because we don't need it; the IOMonad[F] object is enough. This is important because it allows you to implement the IOMonad interface in some other way that doesn't necessarily involve side effects, notably for testing.
Another example are nextBlock and calculate. These need manipulate a state of type GameState, and the ability to manipulate state is expressed by the MonadState type:
def nextBlock[F[_]](
direction: Direction)(implicit F: MonadState[F, GameState]
): F[Block]
def calculate[F[_]](
block: Block)(implicit F: MonadState[F, GameState]
): F[GameResult]
MonadState is unfortunately not contained in cats or cats-effect, you need the cats-mtl library for that.
When you put all this together, you end up with a program like this:
import cats.MonadError
import cats.mtl.MonadState
import cats.implicits._
abstract class Example {
type Map = Array[Array[FieldType]]
sealed trait FieldType
case class GameState(map: Map, block: Block)
case class Block(f1: Field, f2: Field)
case class Field()
sealed trait GameResult
sealed trait Direction
trait IOMonad[F[_]] {
def println(msg: String): F[Unit]
def readln(): F[String]
}
def play[F[_]](
io: IOMonad[F])(
implicit merr: MonadError[F, Throwable],
mst: MonadState[F, GameState]
): F[GameResult] = {
for {
// print map to the console
_ <- io.println("Next move: ")
direction <- readDirection(io)
nextBlock <- nextBlock[F](direction)
gameResult <- calculate[F](nextBlock)
} yield gameResult
}
def readDirection[F[_]](
io: IOMonad[F])(
implicit merr: MonadError[F, Throwable]
): F[Direction]
def nextBlock[F[_]](
direction: Direction)(
implicit merr: MonadState[F, GameState]
): F[Block]
def calculate[F[_]](
block: Block)(
implicit mst: MonadState[F, GameState]
): F[GameResult]
}
Note that every single concrete Monad is gone – there is no IO, no State, no Either in the above program, and together with these, any necessity to convert or lift between different monads also went away.
Note however that this style of programming (known as MTL-Style) has its drawbacks.
type inference often doesn't work. In this example you need to pass the F parameter explicitly to nextBlock and calculate, because Scala can't infer it
as mentioned before, cats doesn't include all the necessary type classes like MonadState, so you need additional libraries like cats-mtl
it's somewhat hard to understand for newcomers
This is why parts of the Scala community (notably John de Goes and his ZIO effort) are no longer encouraging MTL-style. Others keep pushing it, because it allows code to be reused with different effect types.

Related

Using typeclasses wtih multiple type parameters

I'm trying to use symulacrum's #typeclass to avoid writing Ops/Syntax boilerplate. I have a trait parameterized with effect and a type:
#typeclass trait Close[F[_], T]{
def close(t: T): F[Unit]
}
With the intention to be used as follows:
trait Stream[F[_], Pipe]{
def open(): F[Pipe]
def drain(): F[Unit]
}
object App{
def runApp[F[_], Pipe: Close[F, ?]](implicit stream: Stream[F, Pipe]) = {
for{
pipe <- stream.open()
_ <- stream.drain(pipe)
_ <- pipe.close()
} yield ()
}
}
The reason I decided to put Close[F[_], T] away was that some Pipes in my application are inherently unclosable so it is kind of wierd to put
close method for all Pipes
This is the error I get:
Error:(32, 4) #typeclass may only be applied to types that take a single type parameter
#typeclass trait Close[F[_], T]
QUESTION: In case of a trait has multiple type parameters (like Close[F[_], T]) do I have to write all the Ops/Syntax boilerplate on my own and symulacrum's #typeclass cannot help here?
On your own.
https://github.com/mpilquist/simulacrum#known-limitations
Known Limitations
Only type classes that abstract over a proper type or a unary type constructor are currently supported.

Scalaz, the purpose of *syntax classes

In scalaz when we define a module, we additionally define implicit, helper functions. Here is an example of definition and how it could be used by a client:
trait Functor[F[_]] {
def map[A,B](fa: F[A])(f: A => B): F[B]
}
object Functor {
def fmap[F[_], A,B](as:F[A])(f:A=>B)
(implicit ff:Functor[F]):F[B] =
ff.map(as)(f)
implicit val listFunctor = new Functor[List] {
def map[A,B](as: List[A])(f: A => B): List[B] = as map f
}
}
...
import com.savdev.NewLibrary._
val r = fmap(List(1,2))(_.toString)
final class FunctorOps[F[_], A](self: F[A])(implicit ff:Functor[F]){
def qmap[B](f:A=>B):F[B] = ff.map(self)(f)
}
trait ToFunctorOps {
implicit def ToFunctorOps[F[_],A](v: F[A])(implicit F0: Functor[F]) =
new FunctorOps[F,A](v)
}
object NewLibrary extends ToFunctorOps
...
import com.savdev.NewLibrary._
val r2 = List(1, 4) qmap (x=>x.toString)
The code is slightly changed. But the idea is that we define:
An abstraction and its API (algebra)
Define helper generic functions that use implicits and implicits themselves
Enrich existing types to be able to use our new abstraction. Implicit convertion is used for that. In scalaz we define a final class for a wrapper and implicit converters in traits
All above, the motivation of it and how it can be used by a client is clear. But in scalaz to each such module definition, there is also a related *Syntax class. I cannot understand the purpose of it. Can you please exlain, why it is needed and HOW it can be used in a client code.
In Scalaz it is defined as:
trait FunctorSyntax[F[_]] {
implicit def ToFunctorOps[A](v: F[A]): FunctorOps[F, A] =
new FunctorOps[F, A](v)(FunctorSyntax.this.F)
def F: Functor[F]
}
UPDATED:
Guys, it seems I am not clear enough, or a topic is more complicated for all of us.
What I need is to understand the difference between two traits:
trait ToFunctorOps {
implicit def ToFunctorOps[F[_],A](v: F[A])(implicit F0: Functor[F]) =
new FunctorOps[F,A](v)
}
vs.
trait FunctorSyntax[F[_]] {
implicit def ToFunctorOps[A](v: F[A]): FunctorOps[F, A] =
new FunctorOps[F, A](v)(FunctorSyntax.this.F)
def F: Functor[F]
}
Both traits define a generic method that creates FunctorOps, both have the same visibility rules.
The first ToFunctorOps trait, it is not generic itself, it defines only generic method with [F[_],A]. As a result, I can compose a lot of such traits into one object and import all of them at once. I gave an example how such traits could be used by a client:
object NewLibrary extends ToFunctorOps
...
import com.savdev.NewLibrary._
val r2 = List(1, 4) qmap (x=>x.toString)
This trait already gives clients a possibility to inject methods implicitly. Why do we need FunctorSyntax? This FunctorSyntax trait is a generic itself on [F[_]]. When I extend it, I must provide a type in the definition. Because F[_] now is used in trait definition, a function has less generic parameters, only [A].
I am asking you guys, if you can help and undestand, give me a code example how this FunctorSyntax trait can be used by a client. Exactly this is not clear.
Right now I see tries to explain the other topics, but not the original:
How to create implicit classes, instead of implicit functions.
Difference between final *Ops class and a trait, including their visibility. Here we compare 2 traits with the same visibility.
Explaining in general method injection, how they help. This functionality is provided already with ToFunctorOps.
Guys, again, please show the community USE CASES via CODE of FunctorSyntax. Code itself is always the best documentation.
Best regards
From what I can see in the scalaz codebase, I think that FunctorSyntax is meant as an alternative way of enabling syntax. They define Functor like this (simplified):
trait Functor {
def map[A, B](fa: F[A])(f: A => B): F[B]
val functorSyntax = new FunctorSyntax[F] { def F = Functor.this }
}
This enables the following way of working:
def foo[F[_]](f: F[String])(implicit F: Functor[F]): F[Int] = {
import F.functorSyntax._
f.map(_.length)
}
Compare to how ToFunctorOps adds syntax:
package scalaz.syntax { // simplified version of the scalaz codebase
object functor extends ToFunctorOps
}
import scalaz.syntax.functor._
def foo[F[_]: Functor](f: F[String]): F[Int] = f.map(_.length)
Here's a use case where you would use functorSyntax:
import org.scalatest.{FreeSpec, Matchers}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{Await, Future}
import scala.concurrent.duration._
import scalaz._
import Scalaz._
import scala.language.postfixOps
class ScalazTest extends FreeSpec with Matchers {
"compose functors" in {
val composedFunctor = Functor[Future] compose Functor[List] compose Functor[Option]
import composedFunctor.functorSyntax._
val actual = Future.successful(List(Some(1), Some(2), None, Some(4))) fmap (x => x * 2)
Await.result(actual, 10 seconds) shouldBe List(Some(2), Some(4), None, Some(8))
}
}
The idea is that you can compose several functor instaces and import the final composed functor's instance in scope and work with it. Notice that fmap is resolved to composedFunctor.functorSyntax in this case and it works on 3 levels of nesting (Future[List[Option[Integer]]]) while still accepting a function that deals with primitive types.

How can I dynamically construct a Scala class without knowing types in advance?

I want to build a simple library in which a developer can define a Scala class that represents command line arguments (to keep it simple, just a single set of required arguments -- no flags or optional arguments). I'd like the library to parse the command line arguments and return an instance of the class. The user of the library would just do something like this:
case class FooArgs(fluxType: String, capacitorCount: Int)
def main(args: Array[String]) {
val argsObject: FooArgs = ArgParser.parse(args).as[FooArgs]
// do real stuff
}
The parser should throw runtime errors if the provided arguments do not match the expected types (e.g. if someone passes the string "bar" at a position where an Int is expected).
How can I dynamically build a FooArgs without knowing its shape in advance? Since FooArgs can have any arity or types, I don't know how to iterate over the command line arguments, cast or convert them to the expected types, and then use the result to construct a FooArgs. Basically, I want to do something along these lines:
// ** notional code - does not compile **
def parse[T](args: Seq[String], klass: Class[T]): T = {
val expectedTypes = klass.getDeclaredFields.map(_.getGenericType)
val typedArgs = args.zip(expectedTypes).map({
case (arg, String) => arg
case (arg, Int) => arg.toInt
case (arg, unknownType) =>
throw new RuntimeException(s"Unsupported type $unknownType")
})
(klass.getConstructor(typedArgs).newInstance _).tupled(typedArgs)
}
Any suggestions on how I can achieve something like this?
When you want to abstract over case class (or Tuple) shape, the standard approach is to get the HList representation of the case class with the help of shapeless library. HList keeps track in its type signature of the types of its elements and their amount. Then you can implement the algorithm you want recursively on the HList. Shapeless also provides a number of helpful transformations of HLists in shapeless.ops.hlist.
For this problem, first we need to define an auxiliary typeclass to parse an argument of some type from String:
trait Read[T] {
def apply(str: String): T
}
object Read {
def make[T](f: String => T): Read[T] = new Read[T] {
def apply(str: String) = f(str)
}
implicit val string: Read[String] = make(identity)
implicit val int: Read[Int] = make(_.toInt)
}
You can define more instances of this typeclass, if you need to support other argument types than String or Int.
Then we can define the actual typeclass that parses a sequence of arguments into some type:
// This is needed, because there seems to be a conflict between
// HList's :: and the standard Scala's ::
import shapeless.{:: => :::, _}
trait ParseArgs[T] {
def apply(args: List[String]): T
}
object ParseArgs {
// Base of the recursion on HList
implicit val hnil: ParseArgs[HNil] = new ParseArgs[HNil] {
def apply(args: List[String]) =
if (args.nonEmpty) sys.error("too many args")
else HNil
}
// A single recursion step on HList
implicit def hlist[T, H <: HList](
implicit read: Read[T], parseRest: ParseArgs[H]
): ParseArgs[T ::: H] = new ParseArgs[T ::: H] {
def apply(args: List[String]) = args match {
case first :: rest => read(first) :: parseRest(rest)
case Nil => sys.error("too few args")
}
}
// The implementation for any case class, based on its HList representation
implicit def caseClass[C, H <: HList](
implicit gen: Generic.Aux[C, H], parse: ParseArgs[H]
): ParseArgs[C] = new ParseArgs[C] {
def apply(args: List[String]) = gen.from(parse(args))
}
}
And lastly we can define some API, that uses this typeclass. For example:
case class ArgParser(args: List[String]) {
def to[C](implicit parseArgs: ParseArgs[C]): C = parseArgs(args)
}
object ArgParser {
def parse(args: Array[String]): ArgParser = ArgParser(args.toList)
}
And a simple test:
scala> ArgParser.parse(Array("flux", "10")).to[FooArgs]
res0: FooArgs = FooArgs(flux,10)
There is a great guide on using shapeless for solving similar problems, which you may find helpful: The Type Astronaut’s Guide to Shapeless

Bounds for type parameter of FunctionK

I'm using cats FreeMonad. Here's a simplified version of the algebra:
sealed trait Op[A]
object Op {
final case class Get[T](name: String) extends Op[T]
type OpF[A] = Free[Op, A]
def get[T](name: String): OpF[T] = liftF[Op, T](Get[T](name))
}
One of the interpreters will be a wrapper around a third-party library, called Client here which its get method's signature is similar to:
class Client {
def get[O <: Resource](name: String)
(implicit f: Format[O], d: Definition[O]): Future[O] = ???
}
My question is how can I encode that requirement in my implementation?
class FutureOp extends (Op ~> Future) {
val client = new Client()
def apply[A](fa: Op[A]): Future[A] =
fa match {
case Get(name: String) =>
client.get[A](name)
}
}
I tried things like introducing bounds to my apply (like apply[A <: Resource : Format : Definition]) which didn't work.
I understand that FunctionK is to transform values of first-order-kinded types, but is there anyway in which I can encode the requirements of the type parameter?
I intend to use it like:
def run[F[_]: Monad, A](intp: Op ~> F, op: OpF[A]): F[A] = op.foldMap(intp)
val p: Op.OpF[Foo] = Op.get[Foo]("foo")
val i = new FutureOp()
run(i, d)
(My original answer contained the same idea, but apparently it did not provide enough implementation details. This time, I wrote a more detailed step-by-step guide with a discussion of each intermediate step. Every section contains a separate compilable code snippet.)
TL;DR
Implicits are required for each type T that occurs in get[T], therefore they must be inserted and stored when the DSL-program is constructed, not when it is executed. This solves the problem with the implicits.
There is a general strategy for gluing a natural transformation ~> from several restricted natural transformations trait RNT[R, F[_ <: R], G[_]]{ def apply[A <: R](x: F[A]): G[A] } using pattern matching. This solves the problem with the A <: Resource type bound. Details below.
In your question, you have two separate problems:
implicit Format and Definition
<: Resource-type bound
I want to treat each of these two problems in isolation, and provide a reusable solution strategy for both. I will then apply both strategies to your problem.
My answer below is structured as follows:
First, I will summarize your question as I understand it.
Then I will explain what to do with the implicits, ignoring the type bound.
Then I will deal with the type bound, this time ignoring the implicits.
Finally, I apply both strategies to your particular problem.
Henceforth, I assume that you have scalaVersion 2.12.4, the dependencies
libraryDependencies += "org.typelevel" %% "cats-core" % "1.0.1"
libraryDependencies += "org.typelevel" %% "cats-free" % "1.0.1"
and that you insert
import scala.language.higherKinds
where appropriate.
Note that the solution strategies are not specific to this particular scala version or the cats library.
The setup
The goal of this section is to make sure that I'm solving the right problem, and also to provide very simple mock-up definitions
of Resource, Format, Client etc., so that this answer is self-contained
and compilable.
I assume that you want to build a little domain specific language using the Free monad.
Ideally, you would like to have a DSL that looks approximately like this (I've used the names DslOp for the operations and Dsl for the generated free monad):
import cats.free.Free
import cats.free.Free.liftF
sealed trait DslOp[A]
case class Get[A](name: String) extends DslOp[A]
type Dsl[A] = Free[DslOp, A]
def get[A](name: String): Dsl[A] = liftF[DslOp, A](Get[A](name))
It defines a single command get that can get objects of type A given a string
name.
Later, you want to interpret this DSL using a get method provided by some Client
that you cannot modify:
import scala.concurrent.Future
trait Resource
trait Format[A <: Resource]
trait Definition[A <: Resource]
object Client {
def get[A <: Resource](name: String)
(implicit f: Format[A], d: Definition[A]): Future[A] = ???
}
Your problem is that the get method of the Client has a type bound, and that
it requires additional implicits.
Dealing with implicits when defining interpreter for the Free monad
Let's first pretend that the get-method in client requires implicits, but
ignore the type bound for now:
import scala.concurrent.Future
trait Format[A]
trait Definition[A]
object Client {
def get[A](name: String)(implicit f: Format[A], d: Definition[A])
: Future[A] = ???
}
Before we write down the solution, let's briefly discuss why you cannot supply all
the necessary implicits when you are calling the apply method in ~>.
When passed to foldMap, the apply of FunctionK is supposed
to be able to cope with arbitrarily long programs of type Dsl[X] to produce Future[X].
Arbitrarily long programs of type Dsl[X] can contain an unlimited number of
get[T1], ..., get[Tn] commands for different types T1, ..., Tn.
For each of those T1, ..., Tn, you have to get a Format[T_i] and Definition[T_i] somewhere.
These implicit arguments must be supplied by the compiler.
When you interpret the entire program of type Dsl[X], only the type X but not the types T1, ..., Tn are available,
so the compiler cannot insert all the necessary Definitions and Formats at the call site.
Therefore, all the Definitions and Formats must be supplied as implicit parameters to get[T_i]
when you are constructing the Dsl-program, not when you are interpreting it.
The solution is to add Format[A] and Definition[A] as members to the Get[A] case class,
and make the definition of get[A] with lift[DslOp, A] accept these two additional implicit
parameters:
import cats.free.Free
import cats.free.Free.liftF
import cats.~>
sealed trait DslOp[A]
case class Get[A](name: String, f: Format[A], d: Definition[A])
extends DslOp[A]
type Dsl[A] = Free[DslOp, A]
def get[A](name: String)(implicit f: Format[A], d: Definition[A])
: Dsl[A] = liftF[DslOp, A](Get[A](name, f, d))
Now, we can define the first approximation of the ~>-interpreter, which at least
can cope with the implicits:
val clientInterpreter_1: (DslOp ~> Future) = new (DslOp ~> Future) {
def apply[A](op: DslOp[A]): Future[A] = op match {
case Get(name, f, d) => Client.get(name)(f, d)
}
}
Type bounds in case classes defining the DSL-operations
Now, let's deal with the type bound in isolation. Suppose that your Client
doesn't need any implicits, but imposes an additional bound on A:
import scala.concurrent.Future
trait Resource
object Client {
def get[A <: Resource](name: String): Future[A] = ???
}
If you tried to write down the clientInterpreter in the same way as in the
previous example, you would notice that the type A is too general, and that
you therefore cannot work with the contents of Get[A] in Client.get.
Instead, you have to find a scope where the additional type information A <: Resource
is not lost. One way to achieve it is to define an accept method on Get itself.
Instead of a completely general natural transformation ~>, this accept method will
be able to work with natural transformations with restricted domain.
Here is a trait to model that:
trait RestrictedNat[R, F[_ <: R], G[_]] {
def apply[A <: R](fa: F[A]): G[A]
}
It looks almost like ~>, but with an additional A <: R restriction. Now we
can define accept in Get:
import cats.free.Free
import cats.free.Free.liftF
import cats.~>
sealed trait DslOp[A]
case class Get[A <: Resource](name: String) extends DslOp[A] {
def accept[G[_]](f: RestrictedNat[Resource, Get, G]): G[A] = f(this)
}
type Dsl[A] = Free[DslOp, A]
def get[A <: Resource](name: String): Dsl[A] =
liftF[DslOp, A](Get[A](name))
and write down the second approximation of our interpreter, without any
nasty type-casts:
val clientInterpreter_2: (DslOp ~> Future) = new (DslOp ~> Future) {
def apply[A](op: DslOp[A]): Future[A] = op match {
case g # Get(name) => {
val f = new RestrictedNat[Resource, Get, Future] {
def apply[X <: Resource](g: Get[X]): Future[X] = Client.get(g.name)
}
g.accept(f)
}
}
}
This idea can be generalized to an arbitrary number of type constructors Get_1, ...,
Get_N, with type restrictions R1, ..., RN. The general idea corresponds to
the construction of a piecewise defined natural transformation from smaller
pieces that work only on certain subtypes.
Applying both solution strategies to your problem
Now we can combine the two general strategies into one solution for
your concrete problem:
import scala.concurrent.Future
import cats.free.Free
import cats.free.Free.liftF
import cats.~>
// Client-definition with both obstacles: implicits + type bound
trait Resource
trait Format[A <: Resource]
trait Definition[A <: Resource]
object Client {
def get[A <: Resource](name: String)
(implicit fmt: Format[A], dfn: Definition[A])
: Future[A] = ???
}
// Solution:
trait RestrictedNat[R, F[_ <: R], G[_]] {
def apply[A <: R](fa: F[A]): G[A]
}
sealed trait DslOp[A]
case class Get[A <: Resource](
name: String,
fmt: Format[A],
dfn: Definition[A]
) extends DslOp[A] {
def accept[G[_]](f: RestrictedNat[Resource, Get, G]): G[A] = f(this)
}
type Dsl[A] = Free[DslOp, A]
def get[A <: Resource]
(name: String)
(implicit fmt: Format[A], dfn: Definition[A])
: Dsl[A] = liftF[DslOp, A](Get[A](name, fmt, dfn))
val clientInterpreter_3: (DslOp ~> Future) = new (DslOp ~> Future) {
def apply[A](op: DslOp[A]): Future[A] = op match {
case g: Get[A] => {
val f = new RestrictedNat[Resource, Get, Future] {
def apply[X <: Resource](g: Get[X]): Future[X] =
Client.get(g.name)(g.fmt, g.dfn)
}
g.accept(f)
}
}
}
Now, the clientInterpreter_3 can cope with both problems: the type-bound-problem is dealt with
by defining a RestrictedNat for each case class that imposes an upper bound on its type arguments,
and the implicits-problem is solved by adding an implicit parameter list to DSL's get-method.
I think I've found a way to solve your problem by combining a ReaderT monad transformer with intersection types:
import scala.concurrent.Future
import cats.~>
import cats.data.ReaderT
import cats.free.Free
object FreeMonads {
sealed trait Op[A]
object Op {
final case class Get[T](name: String) extends Op[T]
type OpF[A] = Free[Op, A]
def get[T](name: String): OpF[T] = Free.liftF[Op, T](Get[T](name))
}
trait Resource
trait Format[A]
trait Definition[A]
trait Client {
def get[O <: Resource](name: String)
(implicit f: Format[O], d: Definition[O]): Future[O]
}
type Result[A] = ReaderT[
Future,
(Format[A with Resource], Definition[A with Resource]),
A,
]
class FutureOp(client: Client) extends (Op ~> Result) {
def apply[A](fa: Op[A]): Result[A] =
fa match {
case Op.Get(name: String) =>
ReaderT {
case (format, definition) =>
// The `Future[A]` type ascription makes Intellij IDEA's type
// checker accept the code.
client.get(name)(format, definition): Future[A]
}
}
}
}
The basic idea behind it is that you produce a Reader from your Op and that Reader receives the values that you can use for the implicit params. This solves the problem of type O having instances for Format and Definition.
The other problem is for O be a subtype of Resource. To solve this, we're just saying that the Format and Definition instances are not just instances of any A, but any A that also happens to be a Resource.
Let me know if you bump into problems when using FutureOp.

What are the limitations on inference of higher-kinded types in Scala?

In the following simplified sample code:
case class One[A](a: A) // An identity functor
case class Twice[F[_], A](a: F[A], b: F[A]) // A functor transformer
type Twice1[F[_]] = ({type L[α] = Twice[F, α]}) // We'll use Twice1[F]#L when we'd like to write Twice[F]
trait Applicative[F[_]] // Members omitted
val applicativeOne: Applicative[One] = null // Implementation omitted
def applicativeTwice[F[_]](implicit inner: Applicative[F]): Applicative[({type L[α] = Twice[F, α]})#L] = null
I can call applicativeTwice on applicativeOne, and type inference works, as soon as I try to call it on applicativeTwice(applicativeOne), inference fails:
val aOK = applicativeTwice(applicativeOne)
val bOK = applicativeTwice[Twice1[One]#L](applicativeTwice(applicativeOne))
val cFAILS = applicativeTwice(applicativeTwice(applicativeOne))
The errors in scala 2.10.0 are
- type mismatch;
found : tools.Two.Applicative[[α]tools.Two.Twice[tools.Two.One,α]]
required: tools.Two.Applicative[F]
- no type parameters for method applicativeTwice:
(implicit inner: tools.Two.Applicative[F])tools.Two.Applicative[[α]tools.Two.Twice[F,α]]
exist so that it can be applied to arguments
(tools.Two.Applicative[[α]tools.Two.Twice[tools.Two.One,α]])
--- because ---
argument expression's type is not compatible with formal parameter type;
found : tools.Two.Applicative[[α]tools.Two.Twice[tools.Two.One,α]]
required: tools.Two.Applicative[?F]
Why wouldn't "?F" match with anything (of the right kind) ?
Ultimately I'd like applicativeTwice to be an implicit function, but I'd have to get the type inference working first.
I have seen similar questions, and the answers pointed to limitations in the type inference algorithms. But this case seems pretty limitative, and must be quite an annoyance in monad transformers, so I suspect I'm missing some trick to work around this.
You've hit a common annoyance: SI-2712. For clarity, I'm going to minimize your code a bit:
import language.higherKinds
object Test {
case class Base[A](a: A)
case class Recursive[F[_], A](fa: F[A])
def main(args: Array[String]): Unit = {
val one = Base(1)
val two = Recursive(one)
val three = Recursive(two) // doesn't compile
println(three)
}
}
This demonstrates the same type error as yours:
argument expression's type is not compatible with formal parameter type;
found : Test.Recursive[Test.Base,Int]
required: ?F
val three = Recursive(two) // doesn't compile
^
First a bit of syntax and terminology you probably already know:
In Scala we say that a plain, unparameterized data type (such as Int) has kind _. It's monomorphic.
Base, on the other hand, is parameterized. we can't use it as the type of a value without providing the type it contains, so we say has kind _[_]. It's rank-1 polymorphic: a type constructor that takes a type.
Recursive goes further still: it has two parameters, F[_] and A. The number of type parameters don't matter here, but their kinds do. F[_] is rank-1 polymorphic, so Recursive is rank-2 polymorphic: it's a type constructor that takes a type constructor.
We call anything rank two or above higher-kinded, and this is where the fun starts.
Scala in general doesn't have trouble with higher-kinded types. This is one of several key features that distinguishes its type system from, say, Java's. But it does have trouble with partial application of type parameters when dealing with higher-kinded types.
Here's the problem: Recursive[F[_], A] has two type parameters. In your sample code, you did the "type lambda" trick to partially apply the first parameter, something like:
val one = Base(1)
val two = Recursive(one)
val three = {
type λ[α] = Recursive[Base, α]
Recursive(two : λ[Int])
}
This convinces the compiler that you're providing something of the correct kind (_[_]) to the Recursive constructor. If Scala had curried type parameter lists, I'd definitely have used that here:
case class Base[A](a: A)
case class Recursive[F[_]][A](fa: F[A]) // curried!
def main(args: Array[String]): Unit = {
val one = Base(1) // Base[Int]
val two = Recursive(one) // Recursive[Base][Int]
val three = Recursive(two) // Recursive[Recursive[Base]][Int]
println(three)
}
Alas, it does not (see SI-4719). So, to the best of my knowledge, the most common way of dealing with this problem is the "unapply trick," due to Miles Sabin. Here is a greatly simplified version of what appears in scalaz:
import language.higherKinds
trait Unapply[FA] {
type F[_]
type A
def apply(fa: FA): F[A]
}
object Unapply {
implicit def unapply[F0[_[_], _], G0[_], A0] = new Unapply[F0[G0, A0]] {
type F[α] = F0[G0, α]
type A = A0
def apply(fa: F0[G0, A0]): F[A] = fa
}
}
In somewhat hand-wavey terms, this Unapply construct is like a "first-class type lambda." We define a trait representing the assertion that some type FA can be decomposed into a type constructor F[_] and a type A. Then in its companion object, we can define implicits to provide specific decompositions for types of various kinds. I've only defined here the specific one that we need to make Recursive fit, but you could write others.
With this extra bit of plumbing, we can now do what we need:
import language.higherKinds
object Test {
case class Base[A](a: A)
case class Recursive[F[_], A](fa: F[A])
object Recursive {
def apply[FA](fa: FA)(implicit u: Unapply[FA]) = new Recursive(u(fa))
}
def main(args: Array[String]): Unit = {
val one = Base(1)
val two = Recursive(one)
val three = Recursive(two)
println(three)
}
}
Ta-da! Now type inference works, and this compiles. As an exercise, I'd suggest you create an additional class:
case class RecursiveFlipped[A, F[_]](fa: F[A])
... which isn't really different from Recursive in any meaningful way, of course, but will again break type inference. Then define the additional plumbing needed to fix it. Good luck!
Edit
You asked for a less simplified version, something aware of type-classes. Some modification is required, but hopefully you can see the similarity. First, here's our upgraded Unapply:
import language.higherKinds
trait Unapply[TC[_[_]], FA] {
type F[_]
type A
def TC: TC[F]
def apply(fa: FA): F[A]
}
object Unapply {
implicit def unapply[TC[_[_]], F0[_[_], _], G0[_], A0](implicit TC0: TC[({ type λ[α] = F0[G0, α] })#λ]) =
new Unapply[TC, F0[G0, A0]] {
type F[α] = F0[G0, α]
type A = A0
def TC = TC0
def apply(fa: F0[G0, A0]): F[A] = fa
}
}
Again, this is completely ripped off from scalaz. Now some sample code using it:
import language.{ implicitConversions, higherKinds }
object Test {
// functor type class
trait Functor[F[_]] {
def map[A, B](fa: F[A])(f: A => B): F[B]
}
// functor extension methods
object Functor {
implicit class FunctorOps[F[_], A](fa: F[A])(implicit F: Functor[F]) {
def map[B](f: A => B) = F.map(fa)(f)
}
implicit def unapply[FA](fa: FA)(implicit u: Unapply[Functor, FA]) =
new FunctorOps(u(fa))(u.TC)
}
// identity functor
case class Id[A](value: A)
object Id {
implicit val idFunctor = new Functor[Id] {
def map[A, B](fa: Id[A])(f: A => B) = Id(f(fa.value))
}
}
// pair functor
case class Pair[F[_], A](lhs: F[A], rhs: F[A])
object Pair {
implicit def pairFunctor[F[_]](implicit F: Functor[F]) = new Functor[({ type λ[α] = Pair[F, α] })#λ] {
def map[A, B](fa: Pair[F, A])(f: A => B) = Pair(F.map(fa.lhs)(f), F.map(fa.rhs)(f))
}
}
def main(args: Array[String]): Unit = {
import Functor._
val one = Id(1)
val two = Pair(one, one) map { _ + 1 }
val three = Pair(two, two) map { _ + 1 }
println(three)
}
}
Note (3 years later, July 2016), scala v2.12.0-M5 is starting to implement SI-2172 (support for higher order unification)
See commit 892a6d6 from Miles Sabin
-Xexperimental mode now only includes -Ypartial-unification
It follows Paul Chiusano's simple algorithm:
// Treat the type constructor as curried and partially applied, we treat a prefix
// as constants and solve for the suffix. For the example in the ticket, unifying
// M[A] with Int => Int this unifies as,
//
// M[t] = [t][Int => t] --> abstract on the right to match the expected arity
// A = Int --> capture the remainder on the left
The test/files/neg/t2712-1.scala includes:
package test
trait Two[A, B]
object Test {
def foo[M[_], A](m: M[A]) = ()
def test(ma: Two[Int, String]) = foo(ma) // should fail with -Ypartial-unification *disabled*
}
And (test/files/neg/t2712-2.scala):
package test
class X1
class X2
class X3
trait One[A]
trait Two[A, B]
class Foo extends Two[X1, X2] with One[X3]
object Test {
def test1[M[_], A](x: M[A]): M[A] = x
val foo = new Foo
test1(foo): One[X3] // fails with -Ypartial-unification enabled
test1(foo): Two[X1, X2] // fails without -Ypartial-unification
}