Related
I am trying wrap my head around Semigroupals in Cats. Following are statements from "Scala with Cats" by Underscore.
cats.Semigroupal is a type class that allows us to combine contexts
trait Semigroupal[F[_]] {
def product[A, B](fa: F[A], fb: F[B]): F[(A, B)]
}
The parameters fa and fb are independent of one another: we can compute them in either order before passing them to product. This is in contrast to flatMap, which imposes a strict order on its parameters.
So basically, we should be able to combine two Either contexts as well but that doesn't seem to work:
import cats.instances.either._
type ErrorOr[A] = Either[Vector[String], A]
Semigroupal[ErrorOr].product(Left(Vector("Error 1")), Left(Vector("Error 2")))
// res3: ErrorOr[Tuple2[Nothing, Nothing]] = Left(Vector("Error 1"))
If the USP of semigroupal is to eagerly execute independent operations, both eithers must be evaluated before being passed to product and yet we can't have a combined result.
We might expect product applied to Either to accumulate errors instead of fail fast. Again, perhaps surprisingly, we find that product implements the same fail‐fast behaviour as flatMap.
Isn't it contrary to the original premise of having an alternative approach to be able to combine any contexts of same type?
To ensure consistent semantics, Cats’ Monad (which extends Semigroupal) provides a standard definition of product in terms of map and flatMap.
Why implement product in terms of map and flatMap? What semantics are being referred to here?
So why bother with Semigroupal at all? The answer is that we can create useful data types that have instances of Semigroupal (and Applicative) but not Monad. This frees us to implement product in different ways.
What does this even mean?
Unfortunately, the book doesn't covers these premises in detail! Neither can I find resources online. Could anyone please explain this? TIA.
So basically, we should be able to combine two Either contexts as well but that doesn't seem to work:
It worked, as you can see the result is a valid result, it type checks.
Semigrupal just implies that given an F[A] and a F[B] it produces an F[(A, B)] it doesn't imply that it would be able to evaluate both independently or not; it may, but it may as well not. Contrary to Monad which does imply that it needs to evaluate F[A] before because to evaluate F[B] it needs the A
Isn't it contrary to the original premise of having an alternative approach to be able to combine any contexts of same type?
Is not really a different approach since Monad[F] <: Semigroupal[F], you can always call product on any Monad. Implementing a function in terms of Semigroupal just means that it is open to more types, but it doesn't change the behavior of each type.
Why implement product in terms of map and flatMap? What semantics are being referred to here?
TL;DR; consistency:
// https://github.com/typelevel/cats/blob/54b3c2a06ff4b31f3c5f84692b1a8a3fbe5ad310/laws/src/main/scala/cats/laws/FlatMapLaws.scala#L18
def flatMapConsistentApply[A, B](fa: F[A], fab: F[A => B]): IsEq[F[B]] =
fab.ap(fa) <-> fab.flatMap(f => fa.map(f))
The above laws implies that for any F[A] and for any F[A => B] as long as there exists a Monad (actually FlatMap) for F then, fab.ap(fa) is the same as fab.flatMap(f => fa.map(f))
Now, why? Multiple reasons:
The most common one is the principle of least surprise, if I have a bunch of eithers and I pass them to a generic function, no matter if it requires Monad or Applicative I expect it to fail fast, since that is the behavior of Either.
Liskov, suppose I have two functions f and g, f expects an Applicative and g a Monad, if g calls f under the hood I would expect calling both to return the same result.
Any Monad must be an Applicative, however an accumulating Applicative version of Either requires a Semigroup for the Left, whereas, the Monad instance doesn't require that.
What does this even mean?
It means that we may define another type (for example, Validated) that would only satisfy the Applicative laws but not the Monad laws, as such it can implement an accumulating version of ap
Bonus, since having this situation of having a type that is a Monad but could implement an Applicative that doesn't require sequencing, is so common. The cats maintainers created Parallel to represent that.
So instead of converting your Eithers into Validateds to combine them using mapN you can just use parMapN directly on the Eithers.
I'm using the Reader monad in Scala as provided by the scalaz library. I'm familiar with this monad as defined in Haskell. The problem is that I cannot find the functions equivalent to return, local, and sequence (among others).
Currently I use constructs that I do not like since I'm repeating myself or making my code a bit obscure.
Regarding return, I'm currently using:
Reader{_ => someValue}
I'd rather just use a construct like unit(someValue), but I could not find anything on the internet. There are tutorials like this one that use the approach above, and which I consider not optimal.
Regarding local I also have to do something similar: instead of typing something like: local f myReader I have to unfold its definition:
Reader{env => myReader.run(f(env))
Finally, sequence is a bit closer to what I would expect (being a Haskell refugee doing Scala):
readers: List[Reader[Env, T]]
readerTs: Reader[Env, List[T]] = readers.sequenceU
My problem with this implementation is that, being relatively new to Scala, the type of sequenceU
final class TraverseOps[F[_],A] private[syntax](val self: F[A])(implicit val F: Traverse[F]) extends Ops[F[A]] {
//...
def sequenceU(implicit G: Unapply[Applicative, A]): G.M[F[G.A]]
appears like rather obscure, and seems like black magic. Ideally I would like to use a sequence operations on Monads.
Is there a better translation of these constructs to Scala available on scalaz or similar library? I'm not married to any Functional library for Scala, so any solution using other libraries will do, although I'd rather have an answer using scalaz, since I already implemented my code using it.
To make the things simpler, I fill in some types. Changing them to defs with generic types should still work.
Also I extracted the ReaderInt type, to avoid confusion with type lambdas.
return / pure / point
Scala does not have automatic typeclass resolution, so you need to provide them implicitly. For Kleisli (being a monad transformer for reader),
Kleisli[Id, ?, ?] is enough
implicit val KA = scalaz.Kleisli.kleisliIdApplicative[Int]
type ReaderInt[A] = Kleisli[Id.Id, Int, A]
val alwaysHello = KA.point("hello")
or with imported syntax:
import scalaz.syntax.applicative._
val alwaysHello = "hello".point[ReaderInt]
So as a general rule, you
1) import the applicative intance, which usually located in scalaz.std.something.somethingInstance
2) import scalaz.syntax.something._
3) then you can write x.point[F], where F is your applicative.
local
Not sure, that it answers your question, but Kleisli has a local method.
val f: String ⇒ Int = _.length
val alwaysEleven = alwaysHello local f
sequencing
The same way, you are free to choose to use syntax for or to specify type classes explicitly.
import scalaz.std.list.listInstance
val initial: List[ReaderInt[String]] = ???
val sequenced: ReaderInt[List[String]] = Traverse[List].sequence[ReaderInt, String](initial)
import scalaz.syntax.traverse._
val z = x.sequence[ReaderInt, String]
I prefer not to use sequenceU, which uses Unapply typelcass to infer the G type, because sometimes scala has troubles of figuring out the right one.
And I personally do not find it messy to put in some types myself.
It may worth to look into cats, though it does not have much yet.
Looking at the <> method in the following scala slick class, from http://slick.typesafe.com/doc/2.1.0/api/index.html#scala.slick.lifted.ToShapedValue, it reminds me of that iconic stackoverflow thread about scala prototypes.
def <>[R, U](f: (U) ⇒ R, g: (R) ⇒ Option[U])
(implicit arg0: ClassTag[R], shape: Shape[_ <: FlatShapeLevel, T, U, _]):
MappedProjection[R, U]
Can someone bold and knowledgeable provide an articulate walkthrough of that long prototype definition, carefully clarifying all of its type covariance/invariance, double parameter lists, and other advanced scala aspects?
This exercise will also greatly help dealing with similarly convoluted prototypes!
Ok, let's take a look:
class ToShapedValue[T](val value: T) extends AnyVal {
...
#inline def <>[R: ClassTag, U](f: (U) ⇒ R, g: (R) ⇒ Option[U])(implicit shape: Shape[_ <: FlatShapeLevel, T, U, _]): MappedProjection[R, U]
}
The class is an AnyVal wrapper; while I can't actually see the implicit conversion from a quick look, it smells like the "pimp my library" pattern. So I'm guessing this is meant to add <> as an "extension method" onto some (or maybe all) types.
#inline is an annotation, a way of putting metadata on, well, anything; this one is a hint to the compiler that this should be inlined. <> is the method name - plenty of things that look like "operators" are simply ordinary methods in scala.
The documentation you link has already expanded the R: ClassTag to ordinary R and an implicit ClassTag[R] - this is a "context bound" and it's simply syntactic sugar. ClassTag is a compiler-generated thing that exists for every (concrete) type and helps with reflection, so this is a hint that the method will probably do some reflection on an R at some point.
Now, the meat: this is a generic method, parameterized by two types: [R, U]. Its arguments are two functions, f: U => R and g: R => Option[U]. This looks a bit like the functional Prism concept - a conversion from U to R that always works, and a conversion from R to U that sometimes doesn't work.
The interesting part of the signature (sort of) is the implicit shape at the end. Shape is described as a "typeclass", so this is probably best thought of as a "constraint": it limits the possible types U and R that we can call this function with, to only those for which an appropriate Shape is available.
Looking at the documentation forShape, we see that the four types are Level, Mixed, Unpacked and Packed. So the constraint is: there must be a Shape, whose "level" is some subtype of FlatShapeLevel, where the Mixed type is T and the Unpacked type is R (the Packed type can be any type).
So, this is a type-level function that expresses that R is "the unpacked version of" T. To use the example from the Shape documentation again, if T is (Column[Int], Column[(Int, String)], (Int, Option[Double])) then R will be (Int, (Int, String), (Int, Option[Double]) (and it only works for FlatShapeLevel, but I'm going to make a judgement call that that's probably not important). U is, interestingly enough, completely unconstrained.
So this lets us create a MappedProjection[unpacked-version-of-T, U] from any T, by providing conversion functions in both directions. So in a simple version, maybe T is a Column[String] - a representation of a String column in a database - and we want to represent it as some application-specific type, e.g. EmailAddress. So R=String, U=EmailAddress, and we provide conversion functions in both directions: f: EmailAddress => String and g: String => Option[EmailAddress]. It makes sense that it's this way around: every EmailAddress can be represented as a String (at least, they'd better be, if we want to be able to store them in the database), but not every String is a valid EmailAddress. If our database somehow had e.g. "http://www.foo.com/" in the email address column, our g would return None, and Slick could handle this gracefully.
MappedProjection itself is, sadly, undocumented. But I'm guessing it's some kind of lazy representation of a thing we can query; where we had a Column[String], now we have a pseudo-column-thing whose (underlying) type is EmailAddress. So this might allow us to write pseudo-queries like 'select from users where emailAddress.domain = "gmail.com"', which would be impossible to do directly in the database (which doesn't know which part of an email address is the domain), but is easy to do with the help of code. At least, that's my best guess at what it might do.
Arguably the function could be made clearer by using a standard Prism type (e.g. the one from Monocle) rather than passing a pair of functions explicitly. Using the implicit to provide a type-level function is awkward but necessary; in a fully dependently typed language (e.g. Idris), we could write our type-level function as a function (something like def unpackedType(t: Type): Type = ...). So conceptually, this function looks something like:
def <>[U](p: Prism[U, unpackedType(T)]): MappedProjection[unpackedType(T), U]
Hopefully this explains some of the thought process of reading a new, unfamiliar function. I don't know Slick at all, so I have no idea how accurate I am as to what this <> is used for - did I get it right?
I was exchanging emails with an acquaintance that is a big Kotlin, Clojure and Java8 fan and asked him why not Scala. He provided many reasons (Scala is too academic, too many features, not the first time I hear this and I think this is very subjective)
but his biggest pain point was as an example, that he doesn't like a language where he can't understand the implementation of basic data structures, and he gave LinkedList as an example.
I took a look at scala.collection.LinkedList and counted the things I either understand or somewhat understand.
CanBuildFrom - after some effort, I get it, type classes, not the longest suicide note
in history [1]
LinkedListLike - I can't remember where I read it, but I got convinced this is there for a good reason
But then I started to stare at these
GenericTraversableTemplate - now I'm scratching my head as well...
SeqFactory, GenericCompanion - OK, now you lost me, I start to understand his point
Can someone who understand this well please explain GenericTraversableTemplate SeqFactory and GenericCompanion in the context of LinkedList? What they are for, what impact on the end user they have (e.g. I'm sure they are there for a good reason, what is that reason?)
Are they there for a practical reason? or is it a level of abstraction that could have been simplified?
I like Scala collections because I don't have to understand the internals to be able to effectively use them. I don't mind a complex implementation if it helps me to keep my usage simpler. e.g. I don't mind paying the price of a complex library if I get the ability to write cleaner more elegant code using it in return. but it will sure be nice to better understand it.
[1] - Is the Scala 2.8 collections library a case of "the longest suicide note in history"?
I will try to describe the concepts from the point of view of a random pedestrian (I've never contributed a single line to the Scala collection library, so don't hit me too hard if I'm wrong).
Since LinkedList is now deprecated, and because Maps provide a better example, I will use TreeMap as example.
CanBuildFrom
The motivation is this: If we take a TreeMap[Int, Int] and map it with
case (x, y) => (2 * x, y * y * 0.3d)
we get TreeMap[Int, Double]. This type safety alone would already explain the necessity for
simple genericBuilder[X] constructs.
However, if we map it with
case (x, y) => x
we obtain an Iterable[Int] (more precisely: a List[Int]), this is no longer a Map, the type of the container has changed. This is where CBF's come into play:
CanBuildFrom[This, X, That]
can be seen as a kind of "type-level function" that tells us: if we map a collection of type
This with a function that returns values of type X, we can build a That. The most specific
CBF is provided at compile time, in the first case it will be something like
CanBuildFrom[TreeMap[_,_], (X,Y), TreeMap[X,Y]]
in the second case it will be something like
CanBuildFrom[TreeMap[_,_], X, Iterable[X]]
and so we always get the right type of the container. The pattern is pretty general.
Every time you have a generic function
foo[X1, ..., Xn](x1: X1, ..., xn: Xn): Y
where the result type Y depends on X1, ..., Xn, you can introduce an implicit parameter as
follows:
foo[X1, ...., Xn, Y](x1: X1, ..., xn: Xn)(implicit CanFooFrom[X1, ..., Xn, Y]): Y
and then define the type-level function X1, ..., Xn -> Y piecewise by providing multiple
implicit CanFooFrom's.
LinkedListLike
In the class definition, we see something like this:
TreeMap[A, B] extends SortedMap[A, B] with SortedMapLike[A, B, TreeMap[A, B]]
This is Scala's way to express the so-called F-bounded polymorphism.
The motivation is as follows: Suppose we have a dozen (or at least two...) implementations of the trait SortedMap[A, B]. Now we want to implement a method withoutHead, it could look
somewhat like this:
def withoutHead = this.remove(this.head)
If we move the implementation into SortedMap[A, B] itself, the best we can do is this:
def withoutHead: SortedMap[A, B] = this.remove(this.head)
But is this the most specific result type we can get? No, that's too vague.
We would like to return TreeMap[A, B] if the original map is a TreeMap, and
CrazySortedLinkedHashMap (or whatever...) if the original was a CrazySortedLinkedHashMap.
This is why we move the implementation into SortedMapLike, and give the following signature to the withoutHead method:
trait SortedMapLike[A, B, Repr <: SortedMap[A, B]] {
...
def withoutHead: Repr = this.remove(this.head)
}
now because TreeMap[A, B] extends SortedMapLike[A, B, TreeMap[A, B]], the result type of
withoutHead is TreeMap[A,B]. The same holds for CrazySortedLinkedHashMap: we get the exact type back. In Java, you would either have to return SortedMap[A, B] or override the method in each subclass (which turned out to be a maintenance nightmare for the feature-rich traits in Scala)
GenericTraversableTemplate
The type is: GenericTraversableTemplate[+A, +CC[X] <: GenTraversable[X]]
As far as i can tell, this is just a trait that provides implementations of
methods that somehow return regular collections with same container type but
possibly different content type (stuff like flatten, transpose, unzip).
Stuff like foldLeft, reduce, exists are not here because these methods care only about content type, not container type.
Stuff like flatMap is not here, because the container type can change (again, CBF's).
Why is it a separate trait, is there a fundamental reason why it exists?
I don't think so... It probably would be possible to group the godzillion of methods somewhat differently. But this is just what happens naturally: you start to implement a trait, and it turns out that it has very many methods. So instead you group loosely related methods, and put them into 10 different traits with awkward names like "GenTraversableTemplate", and them mix them all into traits/classes where you need them...
GenericCompanion
This is just an abstract class that implements some basic functionality which is common
for companion objects of most collection classes (essentially, it just implements very
simple factory methods apply(varargs) and empty).
For example there is method apply that takes varargs of some type A and returns a collection of type CC[A]:
Array(1, 2, 3, 4) // calls Array.apply[A](elems: A*) on the companion object
List(1, 2, 3, 4) // same for List
The implementation is very simple, it's something like this:
def apply[A](varargs: A*): CC[A] = {
val builder = newBuilder[A]
for (arg <- varargs) builder += arg
builder.result()
}
This is obviously the same for Arrays and Lists and TreeMaps and almost everything else, except 'constrained irregular Collections' like Bitset. So this is just common functionality in a common ancestor class of most companion objects. Nothing special about that.
SeqFactory
Similar to GenericCompanion, but this time more specifically for Sequences.
Adds some common factory methods like fill() and iterate() and tabulate() etc.
Again, nothing particularly rocket-scientific here...
Few general remarks
In general: I don't think that one should attempt to understand every single trait in this library. Rather, one should try to look at the library as a whole. As a whole, it has a very interesting architecture. And in my personal opinion, it's actually a very aesthetic piece of software, but one has to stare at it for quite a while (and try to re-implement the whole architectural pattern several times) to grasp it. On the other hand: for example CBF's are kind of "design pattern" that clearly should be eliminated in successors of this language. The whole story with the scope of implicit CBF's still seems like a total nightmare to me. But many things seemed completely inscrutable at first, and almost always, it ended with an epiphany (which is very specific for Scala: for the majority of other languages, such struggles usually end with the thought "Author of this is a complete idiot").
So I'm trying to work through Norvig & Russell's "Artificial Intelligence, A Modern Approach" as a way to learn Scala. I have a pretty good grasp on the language basics at this point, but I still find myself often "fighting" the type system.
Long story short, breadth-first and depth-first search algorithms are the same aside from the mechanics of pushing/popping to their underlying collection. Depth-first would prepend new possibilities and use a Stack, while Breadth-first would append and use a Queue.
To keep my algorithm the same, I created a typeclass called "GiveGrab" (I know, horrible name) with the intention of pimping ... err ... enriching collections with these "default" push (give) and pop-like (grab) operations.For example, grab would result in a call to .dequeue() for queues, and .pop() for stacks.
Here's (a somewhat abbreviated version of) the code:
object Example extends App {
trait GiveGrab[A, M[A]] {
def give(x: A*): M[A]
def grab(): A
}
implicit class GiveGrabQueue[T](q: Queue[T]) extends GiveGrab[T,Queue[T]] {
override def give(x: T*) = q ++= x
override def grab() = q.dequeue()
}
class TestClass[T, X <% GiveGrab[T, Queue[T]]](var storage: X) {}
val test = new TestClass[Int, Queue[Int]](new Queue[Int]())
}
When trying to compile this, I get the following errors:
Error:(18, 39) scala.collection.mutable.Queue[T] takes no type parameters, expected: one
class TestClass[T, X <% GiveGrab[T, Queue[T]]](var storage: X) {}
^
Error:(13, 67) scala.collection.mutable.Queue[T] takes no type parameters, expected: one
implicit class GiveGrabQueue[T](q: Queue[T]) extends GiveGrab[T,Queue[T]] {
^
That said, it took me a lot of trial and error to even get to this point. I'm not sure if my trait is really supposed to be typed
trait GiveGrab[A, M[A]]
or
trait GiveGrab[A, M[_]]
or
trait GiveGrab[A, M]
The error "takes no type parameters, expected: one" doesn't make a whole lot of sense to me at this point, and there's only a handful of other posts about that message (some related to dependent types, and some related to the Play framework).
Somewhat related: is there a good article for understanding Scala type signatures? I read through Programming in Scala 2nd Ed, but it didn't really touch on this sort of type gymnastics (either that, or I just missed it.)
Edit: Typos
What #PatrykĆwiek proposed is not a workaround but actually what you are meant to be doing: M[A] in trait GiveGrab defines a type function. Roughly speaking this means: M is a type where you can apply a single type parameter to yield a concrete type. That the parameter is called A is pure coincidence. The following means the same:
trait GiveGrab[A,M[MyRandomName]] { ... }
In the definition of give, you actually use this type function to create a type, when saying M[A]. Therefore, as #PatrykĆwiek said, you should write Queue instead of Queue[T]. While Queue is precisely one of these type functions, Queue[T] is a concrete type and therefore doesn't apply to the definition of M.
The error message you get says exactly that: In the place of M, you are supposed to put a type that takes a parameter (like Queue), but you have put one which takes none (Queue[T] in your case, another example would be String or Int).