I'm quite new to functional programming. However, I read about the Free Monad, and I'm trying to use it in a toy project. In this project, I model the stock's portfolio domain. As suggested in many books, I defined an algebra for the PortfolioService and an algebra for the PortfolioRepository.
I want to use the Free monad in the definition of the PortfolioRepository algebra and interpreter. For now, I did not define the PortfolioService algebra in terms of the Free monad.
However, if I do so, in the PortfolioService interpreter, I cannot use the algebra of the PortfolioRepository because of different used monads. For example, I cannot use the monads Either[List[String], Portfolio], and Free[PortfolioRepoF, Portfolio] inside the same for-comprehension :(
I doubt that if I start to use the Free monad to model an algebra, all the other algebra that need to compose with it must be defined in terms of the Free monad.
Is it true?
I am using Scala and Cats 2.2.0.
99% of the time Free monad is interchangeable with Tagless final:
you can pass Free[S, *] as your Monad instance
you can .foldMap Free[S, A] using S ~> F mapping with Monad[F] into F[A]
The only difference is when do you interpret:
tagless interprets immediately, so it require you to pass around type class instances for your F, but since F is a type parameter it gives the impression that it is deferred - because it defers the moment when the type is chosen
free monad lets you create the value immediately with no dependencies on type classes, you can store them as vals in objects, there are no dependencies on type classes. The price you pay is intermediate representation that you ultimately want to discard as soon as you will be able to interpret into useful result. On the other hand it is missing tagless' ability to constrain your operation only to certain algebras (e.g. only Functor, only Applicative, etc to better control effects in dependencies).
Nowadays things moved in favor of tagless final. Free monad is used internally in IO monad implementation (Cats Effect IO, Monix Task, ZIO) and in e.g. Doobie (though from what I heard Doobie's author was thinking about rewriting it into tagless, or at least regretting not using tagless?).
If you want to learn how to use that in modelling there is a book by Gabriel Volpe - Practical FP in Scala that uses tagless final as well as my own small project that uses Cats, FS2, Tapir, tagless etc which can demonstrate some ideas.
If you intend to use Free, then well, there are some challenges:
sealed trait DomainA[A] extends Product with Serializable
object DomainA {
case class Service1(input1: X, input2: Y) extends DomainA[Z]
// ...
def service1(input1: X, input2: Y): Free[DomainA, Z] =
Free.liftF(Service1(input1, input2))
}
val interpreterA: DomainA ~> IO = ...
You use Free[DomainA, *], combine it using .map, .flatMap, etc, interpret it with interpretA.
Then you add another domain, DomainB. And the fun begins:
you cannot just combine Free[DomainA, *] with Free[DomainB, *] because they are different types, you need to align them to make that possible!
so, you have to combine all algebras into one:
type BusinessLogic[A] = EitherK[DomainA, DomainB, A]
implicit val injA: InjectK[DomainA, BusinessLogic] = ...
implicit val injB: InjectK[DomainB, BusinessLogic] = ...
your services cannot hardcode one algebra, you have to inject current algebra into a "bigger" one:
def service1[Total[_]](input1: X, input2: Y)(
implicit inject: InjectK[DomainA, Total]
): Free[Total, Z] =
Free.liftF(inject.inj(Service1(input1, input2)))
your interpreter is also more complex now:
val interpreterTotal: EitherK[DomainA, DomainB, *] ~> IO =
new (EitherK[DomainA, DomainB, *] ~> IO) {
def apply[A](fa: EitherK[DomainA, DomainB, A]) =
fa.run.fold(interpreterA, interpreterB)
}
and it gets more complex with each new added algebra (EitherK[DomainA, EitherK[DomainB, ..., *], *]).
In tagless final there is always a dependency but almost always on one type - F - and empirical evidences of many people shows that is easier to use despite being theoretically equal in power to a free monad. But it is not a scientific argument, so feel free to experiment with free monad on your own. See e.g. this Underscore article about using multiple DSLs at once.
Whether you pick one or the other you are NOT forced to use it everywhere - everything that is Free can be (should be) interpreted into a specific implementation, tagless makes you pass the specific implementation as argument so you can use either for a single component, that is interpreted on its edge.
Related
I am trying wrap my head around Semigroupals in Cats. Following are statements from "Scala with Cats" by Underscore.
cats.Semigroupal is a type class that allows us to combine contexts
trait Semigroupal[F[_]] {
def product[A, B](fa: F[A], fb: F[B]): F[(A, B)]
}
The parameters fa and fb are independent of one another: we can compute them in either order before passing them to product. This is in contrast to flatMap, which imposes a strict order on its parameters.
So basically, we should be able to combine two Either contexts as well but that doesn't seem to work:
import cats.instances.either._
type ErrorOr[A] = Either[Vector[String], A]
Semigroupal[ErrorOr].product(Left(Vector("Error 1")), Left(Vector("Error 2")))
// res3: ErrorOr[Tuple2[Nothing, Nothing]] = Left(Vector("Error 1"))
If the USP of semigroupal is to eagerly execute independent operations, both eithers must be evaluated before being passed to product and yet we can't have a combined result.
We might expect product applied to Either to accumulate errors instead of fail fast. Again, perhaps surprisingly, we find that product implements the same fail‐fast behaviour as flatMap.
Isn't it contrary to the original premise of having an alternative approach to be able to combine any contexts of same type?
To ensure consistent semantics, Cats’ Monad (which extends Semigroupal) provides a standard definition of product in terms of map and flatMap.
Why implement product in terms of map and flatMap? What semantics are being referred to here?
So why bother with Semigroupal at all? The answer is that we can create useful data types that have instances of Semigroupal (and Applicative) but not Monad. This frees us to implement product in different ways.
What does this even mean?
Unfortunately, the book doesn't covers these premises in detail! Neither can I find resources online. Could anyone please explain this? TIA.
So basically, we should be able to combine two Either contexts as well but that doesn't seem to work:
It worked, as you can see the result is a valid result, it type checks.
Semigrupal just implies that given an F[A] and a F[B] it produces an F[(A, B)] it doesn't imply that it would be able to evaluate both independently or not; it may, but it may as well not. Contrary to Monad which does imply that it needs to evaluate F[A] before because to evaluate F[B] it needs the A
Isn't it contrary to the original premise of having an alternative approach to be able to combine any contexts of same type?
Is not really a different approach since Monad[F] <: Semigroupal[F], you can always call product on any Monad. Implementing a function in terms of Semigroupal just means that it is open to more types, but it doesn't change the behavior of each type.
Why implement product in terms of map and flatMap? What semantics are being referred to here?
TL;DR; consistency:
// https://github.com/typelevel/cats/blob/54b3c2a06ff4b31f3c5f84692b1a8a3fbe5ad310/laws/src/main/scala/cats/laws/FlatMapLaws.scala#L18
def flatMapConsistentApply[A, B](fa: F[A], fab: F[A => B]): IsEq[F[B]] =
fab.ap(fa) <-> fab.flatMap(f => fa.map(f))
The above laws implies that for any F[A] and for any F[A => B] as long as there exists a Monad (actually FlatMap) for F then, fab.ap(fa) is the same as fab.flatMap(f => fa.map(f))
Now, why? Multiple reasons:
The most common one is the principle of least surprise, if I have a bunch of eithers and I pass them to a generic function, no matter if it requires Monad or Applicative I expect it to fail fast, since that is the behavior of Either.
Liskov, suppose I have two functions f and g, f expects an Applicative and g a Monad, if g calls f under the hood I would expect calling both to return the same result.
Any Monad must be an Applicative, however an accumulating Applicative version of Either requires a Semigroup for the Left, whereas, the Monad instance doesn't require that.
What does this even mean?
It means that we may define another type (for example, Validated) that would only satisfy the Applicative laws but not the Monad laws, as such it can implement an accumulating version of ap
Bonus, since having this situation of having a type that is a Monad but could implement an Applicative that doesn't require sequencing, is so common. The cats maintainers created Parallel to represent that.
So instead of converting your Eithers into Validateds to combine them using mapN you can just use parMapN directly on the Eithers.
I'm trying to comprehend the idea of free monad, and I'm stumped on the first sentence of cats's docs stating that
A free monad is a construction which allows you to build a monad from
any Functor
My confusion is mainly related to the fact that there is no any functor used\created in the process of "freeing" ADT for KVStoreA. There is no implementation of functor for ADT or something like that. However, documentation for Free[S[_],A] says that S must me a functor:
A free operational monad for some functor S.
Where is that functor that free monad is supposed to be based on?
Thing is, Free monad can be used in a few different ways. In order to use one of them you assume that you have some functor S[_], it will have some Functor[S] defined and you will lift that S[*] to Free[S, *] by allowing some sort of extension to the original S:
Free[S, A] = S[A] or A
This is not valid Scala! They way it is actually implemented is using ADT. This only serves as an explanation of the idea.
This way if you have S[A] and you map it with f: A => B, you would use Functor[S] and receive S[B]. But if you flatMap it with f: A => Free[S, B]... you would still be able to calculate that using Functor[S] and receive S[Free[S, B]]. That's what this code would be be underneath. Remembering that Free[S, A] = S[A] or A you could see that this would turn into S[S[S[...]]] and it could go infinitely nested if not for or A part of Free[S, A] which would let us stop at some point.
Of course we don't calculate it all at once, Free usually uses trampoline to calculate only one thing at a time, so we aren't gonna end with tons of nesting at once.
sealed abstract class Free[S[_], A] extends Product with Serializable {
// ...
}
object Free extends FreeInstances {
/**
* Return from the computation with the given value.
*/
final private[free] case class Pure[S[_], A](a: A) extends Free[S, A]
/** Suspend the computation with the given suspension. */
final private[free] case class Suspend[S[_], A](a: S[A]) extends Free[S, A]
/** Call a subroutine and continue with the given function. */
final private[free] case class FlatMapped[S[_], B, C](c: Free[S, C], f: C => Free[S, B]) extends Free[S, B]
...
}
This trampoline representation implements the same idea as S[A] or A but it allows running computations one step at a time. It also shows that Free[S, *] is S[*] algebra extended by pure and flatMap operations (that assume nothing about S), which justifies calling Free a free algebra.
Actually, until we decide to run the computations we only have lazily evaluated data structure. One way of running it is to call free.run that you have defined for Free in cats - it will use trampoline to evaluate each of this nestings, one at a time, but then it will use Comonad[S] to extract the value - as a result we will get rid of nestings as we go. Comonad[S] is also Functor[S] which is why we don't have to provide it separately to do all these maps.
But that is one way of running things. Another way of running things is just assuming that we are recording the map, flatMap and pure operations using Free. For that we don't have to have Functor[S] at all. We might decide to run things using Comonad[S], but we might also decide something else. What if we translated S[_] to some M[_], but that we knew that this M[_] had a Monad[M]? In this case, we wouldn't have to use Functor[S] at all! We would run translation S ~> M, then, map or flatMap on it, and the S inside of it would be translated using the same S ~> M mechanism. This is what foldMap does - it requires us to provide a natural transformation S ~> M and instance of Monad[M] and Functor[S] is completely unnecessary. While conceptually we can still consider that S as a functor we don't need to rely on any of functors properties, not in the form of Functor[S].
So to finally answer your question - with Free when we are still at the stage of "recording operations" we don't have to assume that S is a functor, so we don't need Functor[S]. When we want to interpret Free[S, A] into something - this is where functor might be actually needed, e.g. if we want to run Free[S, A] into A. But for some interpretations like e.g. foldMap (which is IMHO one of the most popular ones) this is not needed because we treat S as a plain data.
Implicit function types are planned to land in Scala 3.0, and claimed to support "effect capabilities" and composable. It is presented as a better alternative to monad transformers. However, reading further explanation it just seems that implicit function types can only model Reader monad. Am I missing something?
Immediately, it is most trivial to notice that implicit function types allow us to encode the Reader monad primarily allocation free, but losing the convenient for comprehension syntax.
I think what Odersky is referring to when he talks about "a better alternative to monad transformers" is the fact that implicit function types allow you to encode a rather boilerplate free free monad (no pun intended), which is one approach to composing monadic effects.
From the following comment on discourse (emphasis mine):
I guess we both agree that in the future we will have very fine-grained effect systems, so much so that a lot of code with be effectful in some way or another. But then you end up with large parts of your program written in monad structure. The structure itself does not tell you which effects the code has; you need to turn to the types for that.
On the other hand, every time you introduce a new effect category (and it could be as ubiquitous as “not proven to be total”) you need to completely refactor your code into the monadic scheme. And that means you are willing to take a slowdown of (I estimate) 10, risk stackoverflows, be extremely verbose, and have a really hard time composing all your fine grained effects. Or you go free which means better composition but likely even more boilerplate. I can see this work in the sense that you very forcefully tell your users: “don’t use effects, it’s just too painful”. So it could have education value. But if you have to deal with effects, it’s utterly suboptimal in several dimensions.
In his paper, Foundations of Implicit Function Types, Odersky lays out an alternative encoding for free monads using implicit function types which require less boilerplate:
// Free definition
type FreeIFT[A[_], M[_], T] = implicit Natural[A, M] => implicit Monad[M] => M[T]
// GADT defintion
enum KVStoreB[T] {
case Put(key: String, value: Int) extends KVStoreB[Unit]
case Get(key: String) extends KVStoreB[Option[Int]]
}
// Lifted definition
import KVStoreB._
type KVStoreIFT[M[_], T] = FreeIFT[KVStoreB, M, T]
def iftExpr[M[_]]: KVStoreIFT[M, Option[Int]] =
for {
_ <- Put("foo", 2).lift
_ <- Put("bar", 5).lift
n <- Get("foo").lift
} yield n
// Free interpeter
def iftInterpreter = new Natural[KVStoreB, Future] {
def apply[T](fa: KVStoreB[T]): Future[T] = ???
}
// Running the interpreter over the free structure
val iftOutput: Future[Option[Int]] = iftExpr[Future](iftInterpreter)
I know what the monads are and how to use them. What I don't understand is what makes, let's say, Option a monad?
In Haskell a monad Maybe is a monad because it's instantiated from Monad class (which has at least 2 necessary functions return and bind that makes class Monad, indeed, a monad).
But in Scala we've got this:
sealed abstract class Option[+A] extends Product with Serializable { ... }
trait Product extends Any with Equals { ... }
Nothing related to a monad.
If I create my own class in Scala, will it be a monad by default? Why not?
Monad is a concept, an abstract interface if you will, that simply defines a way of composing data.
Option supports composition via flatMap, and that's pretty much everything that is needed to wear the "monad badge".
From a theoretical point of view, it should also:
support a unit operation (return, in Haskell terms) to create a monad out of a bare value, which in case of Option is the Some constructor
respect the monadic laws
but this is not strictly enforced by Scala.
Monads in scala are a much looser concept that in Haskell, and the approach is more practical.
The only thing monads are relevant for, from a language perspective, is the ability of being used in a for-comprehension.
flatMap is a basic requirement, and you can optionally provide map, withFilter and foreach.
However, there's no such thing as strict conformance to a Monad typeclass, like in Haskell.
Here's an example: let's define our own monad.
class MyMonad[A](value: A) {
def map[B](f: A => B) = new MyMonad(f(value))
def flatMap[B](f: A => MyMonad[B]) = f(value)
override def toString = value.toString
}
As you see, we're only implementing map and flatMap (well, and toString as a commodity).
Congratulations, we have a monad! Let's try it out:
scala> for {
a <- new MyMonad(2)
b <- new MyMonad(3)
} yield a + b
// res1: MyMonad[Int] = 5
Nice! We are not doing any filtering, so we don't need to implement withFilter. Also since we're yielding a value, we don't need foreach either. Basically you implement whatever you wish to support, without strict requirements. If you try to filter in a for-comprehension and you haven't implemented withFilter, you'll simply get a compile-time error.
Anything that (partially) implements, through duck-typing, the FilterMonadic trait is considered to be a monad in Scala. This is different than how monads are represented in Haskell, or the Monad typeclass in scalaz. However, in order to benefit of the for comprehension syntactic sugar in Scala, an object has to expose some of the methods defined in the FilterMonadic trait.
Also, in Scala, the equivalent of the Haskell return function is the yield keyword used for producing values out of a for comprehension. The desugaring of yield is a call to the map method of the "monad".
The way I'd put it is that there's an emerging distinction between monads as a design pattern vs. a first-class abstraction. Haskell has the latter, in the form of the Monad type class. But if you have a type that has (or can implement) the monadic operations and obeys the laws, that's a monad as well.
These days you can see monads as a design pattern in Java 8's libraries. The Optional and Stream types in Java 8 come with a static of method that corresponds to Haskell return, and a flatMap method. There is however no Monad type.
Somewhere in between you also have the "duck-typed" approach, as Ionuț G. Stan's answer calls out. C# has this as well—LINQ syntax isn't tied to a specific type, but rather it can be used with any class that implements certain methods.
Scala, per se, does not provide the notion of a monad. You can express a monad as a typeclass but Scala also doesn't provide the notion of a typeclass. But Cats does. So you can create a Monad in Scala with the necessary boiler plate, e.g. traits and implicits cleverly used, or you can use cats which provides a monad trait out of the box. As a comparison, Haskel provides monads as part of the language. Regarding your specific question, an Option can be represented as a monad because it has a flatMap method and a unit method (wrapping a value in a Some or a Future, for example).
I was exchanging emails with an acquaintance that is a big Kotlin, Clojure and Java8 fan and asked him why not Scala. He provided many reasons (Scala is too academic, too many features, not the first time I hear this and I think this is very subjective)
but his biggest pain point was as an example, that he doesn't like a language where he can't understand the implementation of basic data structures, and he gave LinkedList as an example.
I took a look at scala.collection.LinkedList and counted the things I either understand or somewhat understand.
CanBuildFrom - after some effort, I get it, type classes, not the longest suicide note
in history [1]
LinkedListLike - I can't remember where I read it, but I got convinced this is there for a good reason
But then I started to stare at these
GenericTraversableTemplate - now I'm scratching my head as well...
SeqFactory, GenericCompanion - OK, now you lost me, I start to understand his point
Can someone who understand this well please explain GenericTraversableTemplate SeqFactory and GenericCompanion in the context of LinkedList? What they are for, what impact on the end user they have (e.g. I'm sure they are there for a good reason, what is that reason?)
Are they there for a practical reason? or is it a level of abstraction that could have been simplified?
I like Scala collections because I don't have to understand the internals to be able to effectively use them. I don't mind a complex implementation if it helps me to keep my usage simpler. e.g. I don't mind paying the price of a complex library if I get the ability to write cleaner more elegant code using it in return. but it will sure be nice to better understand it.
[1] - Is the Scala 2.8 collections library a case of "the longest suicide note in history"?
I will try to describe the concepts from the point of view of a random pedestrian (I've never contributed a single line to the Scala collection library, so don't hit me too hard if I'm wrong).
Since LinkedList is now deprecated, and because Maps provide a better example, I will use TreeMap as example.
CanBuildFrom
The motivation is this: If we take a TreeMap[Int, Int] and map it with
case (x, y) => (2 * x, y * y * 0.3d)
we get TreeMap[Int, Double]. This type safety alone would already explain the necessity for
simple genericBuilder[X] constructs.
However, if we map it with
case (x, y) => x
we obtain an Iterable[Int] (more precisely: a List[Int]), this is no longer a Map, the type of the container has changed. This is where CBF's come into play:
CanBuildFrom[This, X, That]
can be seen as a kind of "type-level function" that tells us: if we map a collection of type
This with a function that returns values of type X, we can build a That. The most specific
CBF is provided at compile time, in the first case it will be something like
CanBuildFrom[TreeMap[_,_], (X,Y), TreeMap[X,Y]]
in the second case it will be something like
CanBuildFrom[TreeMap[_,_], X, Iterable[X]]
and so we always get the right type of the container. The pattern is pretty general.
Every time you have a generic function
foo[X1, ..., Xn](x1: X1, ..., xn: Xn): Y
where the result type Y depends on X1, ..., Xn, you can introduce an implicit parameter as
follows:
foo[X1, ...., Xn, Y](x1: X1, ..., xn: Xn)(implicit CanFooFrom[X1, ..., Xn, Y]): Y
and then define the type-level function X1, ..., Xn -> Y piecewise by providing multiple
implicit CanFooFrom's.
LinkedListLike
In the class definition, we see something like this:
TreeMap[A, B] extends SortedMap[A, B] with SortedMapLike[A, B, TreeMap[A, B]]
This is Scala's way to express the so-called F-bounded polymorphism.
The motivation is as follows: Suppose we have a dozen (or at least two...) implementations of the trait SortedMap[A, B]. Now we want to implement a method withoutHead, it could look
somewhat like this:
def withoutHead = this.remove(this.head)
If we move the implementation into SortedMap[A, B] itself, the best we can do is this:
def withoutHead: SortedMap[A, B] = this.remove(this.head)
But is this the most specific result type we can get? No, that's too vague.
We would like to return TreeMap[A, B] if the original map is a TreeMap, and
CrazySortedLinkedHashMap (or whatever...) if the original was a CrazySortedLinkedHashMap.
This is why we move the implementation into SortedMapLike, and give the following signature to the withoutHead method:
trait SortedMapLike[A, B, Repr <: SortedMap[A, B]] {
...
def withoutHead: Repr = this.remove(this.head)
}
now because TreeMap[A, B] extends SortedMapLike[A, B, TreeMap[A, B]], the result type of
withoutHead is TreeMap[A,B]. The same holds for CrazySortedLinkedHashMap: we get the exact type back. In Java, you would either have to return SortedMap[A, B] or override the method in each subclass (which turned out to be a maintenance nightmare for the feature-rich traits in Scala)
GenericTraversableTemplate
The type is: GenericTraversableTemplate[+A, +CC[X] <: GenTraversable[X]]
As far as i can tell, this is just a trait that provides implementations of
methods that somehow return regular collections with same container type but
possibly different content type (stuff like flatten, transpose, unzip).
Stuff like foldLeft, reduce, exists are not here because these methods care only about content type, not container type.
Stuff like flatMap is not here, because the container type can change (again, CBF's).
Why is it a separate trait, is there a fundamental reason why it exists?
I don't think so... It probably would be possible to group the godzillion of methods somewhat differently. But this is just what happens naturally: you start to implement a trait, and it turns out that it has very many methods. So instead you group loosely related methods, and put them into 10 different traits with awkward names like "GenTraversableTemplate", and them mix them all into traits/classes where you need them...
GenericCompanion
This is just an abstract class that implements some basic functionality which is common
for companion objects of most collection classes (essentially, it just implements very
simple factory methods apply(varargs) and empty).
For example there is method apply that takes varargs of some type A and returns a collection of type CC[A]:
Array(1, 2, 3, 4) // calls Array.apply[A](elems: A*) on the companion object
List(1, 2, 3, 4) // same for List
The implementation is very simple, it's something like this:
def apply[A](varargs: A*): CC[A] = {
val builder = newBuilder[A]
for (arg <- varargs) builder += arg
builder.result()
}
This is obviously the same for Arrays and Lists and TreeMaps and almost everything else, except 'constrained irregular Collections' like Bitset. So this is just common functionality in a common ancestor class of most companion objects. Nothing special about that.
SeqFactory
Similar to GenericCompanion, but this time more specifically for Sequences.
Adds some common factory methods like fill() and iterate() and tabulate() etc.
Again, nothing particularly rocket-scientific here...
Few general remarks
In general: I don't think that one should attempt to understand every single trait in this library. Rather, one should try to look at the library as a whole. As a whole, it has a very interesting architecture. And in my personal opinion, it's actually a very aesthetic piece of software, but one has to stare at it for quite a while (and try to re-implement the whole architectural pattern several times) to grasp it. On the other hand: for example CBF's are kind of "design pattern" that clearly should be eliminated in successors of this language. The whole story with the scope of implicit CBF's still seems like a total nightmare to me. But many things seemed completely inscrutable at first, and almost always, it ended with an epiphany (which is very specific for Scala: for the majority of other languages, such struggles usually end with the thought "Author of this is a complete idiot").