Related
I am trying wrap my head around Semigroupals in Cats. Following are statements from "Scala with Cats" by Underscore.
cats.Semigroupal is a type class that allows us to combine contexts
trait Semigroupal[F[_]] {
def product[A, B](fa: F[A], fb: F[B]): F[(A, B)]
}
The parameters fa and fb are independent of one another: we can compute them in either order before passing them to product. This is in contrast to flatMap, which imposes a strict order on its parameters.
So basically, we should be able to combine two Either contexts as well but that doesn't seem to work:
import cats.instances.either._
type ErrorOr[A] = Either[Vector[String], A]
Semigroupal[ErrorOr].product(Left(Vector("Error 1")), Left(Vector("Error 2")))
// res3: ErrorOr[Tuple2[Nothing, Nothing]] = Left(Vector("Error 1"))
If the USP of semigroupal is to eagerly execute independent operations, both eithers must be evaluated before being passed to product and yet we can't have a combined result.
We might expect product applied to Either to accumulate errors instead of fail fast. Again, perhaps surprisingly, we find that product implements the same fail‐fast behaviour as flatMap.
Isn't it contrary to the original premise of having an alternative approach to be able to combine any contexts of same type?
To ensure consistent semantics, Cats’ Monad (which extends Semigroupal) provides a standard definition of product in terms of map and flatMap.
Why implement product in terms of map and flatMap? What semantics are being referred to here?
So why bother with Semigroupal at all? The answer is that we can create useful data types that have instances of Semigroupal (and Applicative) but not Monad. This frees us to implement product in different ways.
What does this even mean?
Unfortunately, the book doesn't covers these premises in detail! Neither can I find resources online. Could anyone please explain this? TIA.
So basically, we should be able to combine two Either contexts as well but that doesn't seem to work:
It worked, as you can see the result is a valid result, it type checks.
Semigrupal just implies that given an F[A] and a F[B] it produces an F[(A, B)] it doesn't imply that it would be able to evaluate both independently or not; it may, but it may as well not. Contrary to Monad which does imply that it needs to evaluate F[A] before because to evaluate F[B] it needs the A
Isn't it contrary to the original premise of having an alternative approach to be able to combine any contexts of same type?
Is not really a different approach since Monad[F] <: Semigroupal[F], you can always call product on any Monad. Implementing a function in terms of Semigroupal just means that it is open to more types, but it doesn't change the behavior of each type.
Why implement product in terms of map and flatMap? What semantics are being referred to here?
TL;DR; consistency:
// https://github.com/typelevel/cats/blob/54b3c2a06ff4b31f3c5f84692b1a8a3fbe5ad310/laws/src/main/scala/cats/laws/FlatMapLaws.scala#L18
def flatMapConsistentApply[A, B](fa: F[A], fab: F[A => B]): IsEq[F[B]] =
fab.ap(fa) <-> fab.flatMap(f => fa.map(f))
The above laws implies that for any F[A] and for any F[A => B] as long as there exists a Monad (actually FlatMap) for F then, fab.ap(fa) is the same as fab.flatMap(f => fa.map(f))
Now, why? Multiple reasons:
The most common one is the principle of least surprise, if I have a bunch of eithers and I pass them to a generic function, no matter if it requires Monad or Applicative I expect it to fail fast, since that is the behavior of Either.
Liskov, suppose I have two functions f and g, f expects an Applicative and g a Monad, if g calls f under the hood I would expect calling both to return the same result.
Any Monad must be an Applicative, however an accumulating Applicative version of Either requires a Semigroup for the Left, whereas, the Monad instance doesn't require that.
What does this even mean?
It means that we may define another type (for example, Validated) that would only satisfy the Applicative laws but not the Monad laws, as such it can implement an accumulating version of ap
Bonus, since having this situation of having a type that is a Monad but could implement an Applicative that doesn't require sequencing, is so common. The cats maintainers created Parallel to represent that.
So instead of converting your Eithers into Validateds to combine them using mapN you can just use parMapN directly on the Eithers.
I'm new in Scala. I come across with this question:
If side effects (outputs, global variables ...) are not taken into account, how many implementations can the below method have? how can I figure it out in general?
def g[A,B,C](x: A,y: B,f:(A,B) => C): C
You can figure it out in general by going backwards from your return type, so you look at C and try to look for where it might come from, we can see that f itself returns C, but it requires us to give it an A and a B, so now we look for those and we find them only once, which gives us the answer that there's only one way to produce a C and therefore only one way to implement this function. If we had two values of A, we'd get two different implementations.
Now of course this kind of ignores the fact that you also implement it using null or an infinite number of ways using asInstanceOf casts. So if you have to be super exact there are 1* implementations of this function.
This question is ill-posed even without nulls, asInstanceOfs and throwing exceptions. For every natural number n, you can simply append .swap.swap to (x, y) n-times in the following implementation (example for n = 1):
def g[A, B, C](x: A, y: B, f: (A, B) => C): C = {
val (xx, yy) = (x,y).swap.swap
f(xx, yy)
}
You can do the same thing by inserting arbitrarily many identity functions at various places. Or by using _1, _2 projections and (-,-)-constructors (or, in general, repeatedly applying non-productive eta-expansions and beta-reductions of your choice).
I have been looking into FP languages (off and on) for some time and have played with Scala, Haskell, F#, and some others. I like what I see and understand some of the fundamental concepts of FP (with absolutely no background in Category Theory - so don't talk Math, please).
So, given a type M[A] we have map which takes a function A=>B and returns a M[B]. But we also have flatMap which takes a function A=>M[B] and returns a M[B]. We also have flatten which takes a M[M[A]] and returns a M[A].
In addition, many of the sources I have read describe flatMap as map followed by flatten.
So, given that flatMap seems to be equivalent to flatten compose map, what is its purpose? Please don't say it is to support 'for comprehensions' as this question really isn't Scala-specific. And I am less concerned with the syntactic sugar than I am in the concept behind it. The same question arises with Haskell's bind operator (>>=). I believe they both are related to some Category Theory concept but I don't speak that language.
I have watched Brian Beckman's great video Don't Fear the Monad more than once and I think I see that flatMap is the monadic composition operator but I have never really seen it used the way he describes this operator. Does it perform this function? If so, how do I map that concept to flatMap?
BTW, I had a long writeup on this question with lots of listings showing experiments I ran trying to get to the bottom of the meaning of flatMap and then ran into this question which answered some of my questions. Sometimes I hate Scala implicits. They can really muddy the waters. :)
FlatMap, known as "bind" in some other languages, is as you said yourself for function composition.
Imagine for a moment that you have some functions like these:
def foo(x: Int): Option[Int] = Some(x + 2)
def bar(x: Int): Option[Int] = Some(x * 3)
The functions work great, calling foo(3) returns Some(5), and calling bar(3) returns Some(9), and we're all happy.
But now you've run into the situation that requires you to do the operation more than once.
foo(3).map(x => foo(x)) // or just foo(3).map(foo) for short
Job done, right?
Except not really. The output of the expression above is Some(Some(7)), not Some(7), and if you now want to chain another map on the end you can't because foo and bar take an Int, and not an Option[Int].
Enter flatMap
foo(3).flatMap(foo)
Will return Some(7), and
foo(3).flatMap(foo).flatMap(bar)
Returns Some(15).
This is great! Using flatMap lets you chain functions of the shape A => M[B] to oblivion (in the previous example A and B are Int, and M is Option).
More technically speaking; flatMap and bind have the signature M[A] => (A => M[B]) => M[B], meaning they take a "wrapped" value, such as Some(3), Right('foo), or List(1,2,3) and shove it through a function that would normally take an unwrapped value, such as the aforementioned foo and bar. It does this by first "unwrapping" the value, and then passing it through the function.
I've seen the box analogy being used for this, so observe my expertly drawn MSPaint illustration:
This unwrapping and re-wrapping behavior means that if I were to introduce a third function that doesn't return an Option[Int] and tried to flatMap it to the sequence, it wouldn't work because flatMap expects you to return a monad (in this case an Option)
def baz(x: Int): String = x + " is a number"
foo(3).flatMap(foo).flatMap(bar).flatMap(baz) // <<< ERROR
To get around this, if your function doesn't return a monad, you'd just have to use the regular map function
foo(3).flatMap(foo).flatMap(bar).map(baz)
Which would then return Some("15 is a number")
It's the same reason you provide more than one way to do anything: it's a common enough operation that you may want to wrap it.
You could ask the opposite question: why have map and flatten when you already have flatMap and a way to store a single element inside your collection? That is,
x map f
x filter p
can be replaced by
x flatMap ( xi => x.take(0) :+ f(xi) )
x flatMap ( xi => if (p(xi)) x.take(0) :+ xi else x.take(0) )
so why bother with map and filter?
In fact, there are various minimal sets of operations you need to reconstruct many of the others (flatMap is a good choice because of its flexibility).
Pragmatically, it's better to have the tool you need. Same reason why there are non-adjustable wrenches.
The simplest reason is to compose an output set where each entry in the input set may produce more than one (or zero!) outputs.
For example, consider a program which outputs addresses for people to generate mailers. Most people have one address. Some have two or more. Some people, unfortunately, have none. Flatmap is a generalized algorithm to take a list of these people and return all of the addresses, regardless of how many come from each person.
The zero output case is particularly useful for monads, which often (always?) return exactly zero or one results (think Maybe- returns zero results if the computation fails, or one if it succeeds). In that case you want to perform an operation on "all of the results", which it just so happens may be one or many.
The "flatMap", or "bind", method, provides an invaluable way to chain together methods that provide their output wrapped in a Monadic construct (like List, Option, or Future). For example, suppose you have two methods that produce a Future of a result (eg. they make long-running calls to databases or web service calls or the like, and should be used asynchronously):
def fn1(input1: A): Future[B] // (for some types A and B)
def fn2(input2: B): Future[C] // (for some types B and C)
How to combine these? With flatMap, we can do this as simply as:
def fn3(input3: A): Future[C] = fn1(a).flatMap(b => fn2(b))
In this sense, we have "composed" a function fn3 out of fn1 and fn2 using flatMap, which has the same general structure (and so can be composed in turn with further similar functions).
The map method would give us a not-so-convenient - and not readily chainable - Future[Future[C]]. Certainly we can then use flatten to reduce this, but the flatMap method does it in one call, and can be chained as far as we wish.
This is so useful a way of working, in fact, that Scala provides the for-comprehension as essentially a short-cut for this (Haskell, too, provides a short-hand way of writing a chain of bind operations - I'm not a Haskell expert, though, and don't recall the details) - hence the talk you will have come across about for-comprehensions being "de-sugared" into a chain of flatMap calls (along with possible filter calls and a final map call for the yield).
Well, one could argue, you don't need .flatten either. Why not just do something like
#tailrec
def flatten[T](in: Seq[Seq[T], out: Seq[T] = Nil): Seq[T] = in match {
case Nil => out
case head ::tail => flatten(tail, out ++ head)
}
Same can be said about map:
#tailrec
def map[A,B](in: Seq[A], out: Seq[B] = Nil)(f: A => B): Seq[B] = in match {
case Nil => out
case head :: tail => map(tail, out :+ f(head))(f)
}
So, why are .flatten and .map provided by the library? Same reason .flatMap is: convenience.
There is also .collect, which is really just
list.filter(f.isDefinedAt _).map(f)
.reduce is actually nothing more then list.foldLeft(list.head)(f),
.headOption is
list match {
case Nil => None
case head :: _ => Some(head)
}
Etc ...
Why do monads not compose when a Monad is an Applicative and an Applicative is a Functor. You see this inheritance chain in many articles on the web ( Which i have gone through ). But when Functors and Applicatives compose why do Monads break this ?
Can someone provide a simple example in scala which demonstrates this issue ? I know this is asked a lot but kind of hard to understand without a simple example.
First, let's start with a simple problem. Let's say, we need to get a sum of two integers, each wrapped in both Future and Option. Let's take cats library in order to resemble Haskell’s standard library definitions with Scala-syntax.
If we use monad approach (aka flatMap), we need:
both Future and Option should have Monad instances defined over them
we also need monadic transformer OptionT which will work only for Option (precisely F[Option[T]])
So, here is the code (let's forget about for-comprehension and lifting to make it simpler):
val fa = OptionT[Future, Int](Future(Some(1)))
val fb = OptionT[Future, Int](Future(Some(2)))
fa.flatMap(a => fb.map(b => a + b)) //note that a and b are already Int's not Future's
if you look at OptionT.flatMap sources:
def flatMap[B](f: A => OptionT[F, B])(implicit F: Monad[F]): OptionT[F, B] =
flatMapF(a => f(a).value)
def flatMapF[B](f: A => F[Option[B]])(implicit F: Monad[F]): OptionT[F, B] =
OptionT(F.flatMap(value)(_.fold(F.pure[Option[B]](None))(f)))
You'll notice that the code is pretty specific to Option's internal logic and structure (fold, None). Same problem for EitherT, StateT etc.
Important thing here is that there is no FutureT defined in cats, so you can compose Future[Option[T]], but can't do that with Option[Future[T]] (later I'll show that this problem is even more generic).
On the other hand, if you choose composition using Applicative, you'll have to meet only one requirement:
both Future and Option should have Applicative instances defined over them
You don't need any special transformers for Option, basically cats library provides Nested class that works for any Applicative (let's forget about applicative builder's sugar to simplify understanding):
val fa = Nested[Future, Option, Int](Future(Some(1)))
val fb = Nested[Future, Option, Int](Future(Some(1)))
fa.map(x => (y: Int) => y + x).ap(fb)
Let's swap Option and Future:
val fa = Nested[Option, Future, Int](Some(Future(1)))
val fb = Nested[Option, Future, Int](Some(Future(1)))
fa.map(x => (y: Int) => y + x).ap(fb)
Works!
So yes Monad is Applicative, Option[Future[T]] is still a monad (on Future[T] but not on T itself) but it allows you to operate only with Future[T] not T. In order to "merge" Option with Future layers - you have to define monadic transformer FutureT, in order to merge Future with Option - you have to define OptionT. And, OptionT is defined in cats/scalaz, but not FutureT.
In general (from here):
Unfortunately, our real goal, composition of monads, is rather more
difficult. .. In fact, we can actually prove that, in a certain sense,
there is no way to construct a join function with the type above using
only the operations of the two monads (see the appendix for an outline
of the proof). It follows that the only way that we might hope to form
a composition is if there are some additional constructions linking
the two component
And this composition is not even necessary commutative (swappable) as I demonstrated for Option and Future.
As an exercise, you can try to define FutureT's flatMap:
def flatMapF[B](f: A => F[Future[B]])(implicit F: Monad[F]): FutureT[F, B] =
FutureT(F.flatMap(value){ x: Future[A] =>
val r: Future[F[Future[B]] = x.map(f)
//you have to return F[Future[B]] here using only f and F.pure,
//where F can be List, Option whatever
})
basically the problem with such implementation is that you have to "extract" value from r which is impossible here, assuming you can't extract value from Future (there is no comonad defined on it) at least in a "non-blocking" context (like ScalaJs). This basically means that you can't "swap" Future and F, like Future[F[Future[B]] => F[Future[Future[B]. The latter is a natural transformation (morphism between functors), so that explains the first comment on this general answer:
you can compose monads if you can provide a natural transformation swap : N M a -> M N a
Applicatives however don't have such problems - you can easily compose them, but keep in mind that result of composition of two Applicatives may not be a monad (but will always be an applicative). Nested[Future, Option, T] is not a monad on T, regardless that both Option and Future are monads on T. Putting in simple words Nested as a class doesn't have flatMap.
It would be also helpful to read:
http://typelevel.org/cats/tut/applicative.html
http://typelevel.org/cats/tut/apply.html
http://typelevel.org/cats/tut/monad.html
http://typelevel.org/cats/tut/optiont.html
Putting it all together (F and G are monads)
F[G[T]] is a monad on G[T], but not on T
G_TRANSFORMER[F, T] required in order to get a monad on T from F[G[T]].
there is no MEGA_TRANSFORMER[G, F, T] as such transformer can't be build on top of monad - it requires additional operations defined on G (it seems like comonad on G should be enough)
every monad (including G and F) is applicative, but not every applicative is a monad
in theory F[G[T]] is an applicative over both G[T] and T. However scala requires to create NESTED[F, G, T] in order to get composed applicative on T (which is implemented in cats library).
NESTED[F, G, T] is applicative, but not a monad
That means you can compose Future x Option (aka Option[Future[T]]) to one single monad (coz OptionT exists), but you can't compose Option x Future (aka Future[Option[T]]) without knowing that Future is something else besides being a monad (even though they’re inherently applicative functors - applicative is not enough to neither build a monad nor monad transformer on it) . Basically:
OptionT can be seen as non-commutative binary operator defined as OptionT: Monad[Option] x Monad[F] -> OptionT[F, T]; for all Monad[F], T; for some F[T]. Or in general: Merge: Monad[G] x Monad[F] -> Monad[Merge]; for all T, Monad[F]; but only for **some of Monad[G]**, some F[T], G[T];
you can compose any two applicatives into one single applicative Nested: Applicative[F] x Applicative[G] -> Nested[F, G]; for all Applicative[F], Applicative[G], T; for some F[T], G[T],
but you can compose any two monads (inherently functors) only into one applicative (but not into monad).
Tony Morris gave a talk on monad transformers that explains this precise issue very well.
http://tonymorris.github.io/blog/posts/monad-transformers/
He uses haskell, but the examples are easily translatable to scala.
I've been doing dev in F# for a while and I like it. However one buzzword I know doesn't exist in F# is higher-kinded types. I've read material on higher-kinded types, and I think I understand their definition. I'm just not sure why they're useful. Can someone provide some examples of what higher-kinded types make easy in Scala or Haskell, that require workarounds in F#? Also for these examples, what would the workarounds be without higher-kinded types (or vice-versa in F#)? Maybe I'm just so used to working around it that I don't notice the absence of that feature.
(I think) I get that instead of myList |> List.map f or myList |> Seq.map f |> Seq.toList higher kinded types allow you to simply write myList |> map f and it'll return a List. That's great (assuming it's correct), but seems kind of petty? (And couldn't it be done simply by allowing function overloading?) I usually convert to Seq anyway and then I can convert to whatever I want afterwards. Again, maybe I'm just too used to working around it. But is there any example where higher-kinded types really saves you either in keystrokes or in type safety?
So the kind of a type is its simple type. For instance Int has kind * which means it's a base type and can be instantiated by values. By some loose definition of higher-kinded type (and I'm not sure where F# draws the line, so let's just include it) polymorphic containers are a great example of a higher-kinded type.
data List a = Cons a (List a) | Nil
The type constructor List has kind * -> * which means that it must be passed a concrete type in order to result in a concrete type: List Int can have inhabitants like [1,2,3] but List itself cannot.
I'm going to assume that the benefits of polymorphic containers are obvious, but more useful kind * -> * types exist than just the containers. For instance, the relations
data Rel a = Rel (a -> a -> Bool)
or parsers
data Parser a = Parser (String -> [(a, String)])
both also have kind * -> *.
We can take this further in Haskell, however, by having types with even higher-order kinds. For instance we could look for a type with kind (* -> *) -> *. A simple example of this might be Shape which tries to fill a container of kind * -> *.
data Shape f = Shape (f ())
Shape [(), (), ()] :: Shape []
This is useful for characterizing Traversables in Haskell, for instance, as they can always be divided into their shape and contents.
split :: Traversable t => t a -> (Shape t, [a])
As another example, let's consider a tree that's parameterized on the kind of branch it has. For instance, a normal tree might be
data Tree a = Branch (Tree a) a (Tree a) | Leaf
But we can see that the branch type contains a Pair of Tree as and so we can extract that piece out of the type parametrically
data TreeG f a = Branch a (f (TreeG f a)) | Leaf
data Pair a = Pair a a
type Tree a = TreeG Pair a
This TreeG type constructor has kind (* -> *) -> * -> *. We can use it to make interesting other variations like a RoseTree
type RoseTree a = TreeG [] a
rose :: RoseTree Int
rose = Branch 3 [Branch 2 [Leaf, Leaf], Leaf, Branch 4 [Branch 4 []]]
Or pathological ones like a MaybeTree
data Empty a = Empty
type MaybeTree a = TreeG Empty a
nothing :: MaybeTree a
nothing = Leaf
just :: a -> MaybeTree a
just a = Branch a Empty
Or a TreeTree
type TreeTree a = TreeG Tree a
treetree :: TreeTree Int
treetree = Branch 3 (Branch Leaf (Pair Leaf Leaf))
Another place this shows up is in "algebras of functors". If we drop a few layers of abstractness this might be better considered as a fold, such as sum :: [Int] -> Int. Algebras are parameterized over the functor and the carrier. The functor has kind * -> * and the carrier kind * so altogether
data Alg f a = Alg (f a -> a)
has kind (* -> *) -> * -> *. Alg useful because of its relation to datatypes and recursion schemes built atop them.
-- | The "single-layer of an expression" functor has kind `(* -> *)`
data ExpF x = Lit Int
| Add x x
| Sub x x
| Mult x x
-- | The fixed point of a functor has kind `(* -> *) -> *`
data Fix f = Fix (f (Fix f))
type Exp = Fix ExpF
exp :: Exp
exp = Fix (Add (Fix (Lit 3)) (Fix (Lit 4))) -- 3 + 4
fold :: Functor f => Alg f a -> Fix f -> a
fold (Alg phi) (Fix f) = phi (fmap (fold (Alg phi)) f)
Finally, though they're theoretically possible, I've never see an even higher-kinded type constructor. We sometimes see functions of that type such as mask :: ((forall a. IO a -> IO a) -> IO b) -> IO b, but I think you'll have to dig into type prolog or dependently typed literature to see that level of complexity in types.
Consider the Functor type class in Haskell, where f is a higher-kinded type variable:
class Functor f where
fmap :: (a -> b) -> f a -> f b
What this type signature says is that fmap changes the type parameter of an f from a to b, but leaves f as it was. So if you use fmap over a list you get a list, if you use it over a parser you get a parser, and so on. And these are static, compile-time guarantees.
I don't know F#, but let's consider what happens if we try to express the Functor abstraction in a language like Java or C#, with inheritance and generics, but no higher-kinded generics. First try:
interface Functor<A> {
Functor<B> map(Function<A, B> f);
}
The problem with this first try is that an implementation of the interface is allowed to return any class that implements Functor. Somebody could write a FunnyList<A> implements Functor<A> whose map method returns a different kind of collection, or even something else that's not a collection at all but is still a Functor. Also, when you use the map method you can't invoke any subtype-specific methods on the result unless you downcast it to the type that you're actually expecting. So we have two problems:
The type system doesn't allow us to express the invariant that the map method always returns the same Functor subclass as the receiver.
Therefore, there's no statically type-safe manner to invoke a non-Functor method on the result of map.
There are other, more complicated ways you can try, but none of them really works. For example, you could try augment the first try by defining subtypes of Functor that restrict the result type:
interface Collection<A> extends Functor<A> {
Collection<B> map(Function<A, B> f);
}
interface List<A> extends Collection<A> {
List<B> map(Function<A, B> f);
}
interface Set<A> extends Collection<A> {
Set<B> map(Function<A, B> f);
}
interface Parser<A> extends Functor<A> {
Parser<B> map(Function<A, B> f);
}
// …
This helps to forbid implementers of those narrower interfaces from returning the wrong type of Functor from the map method, but since there is no limit to how many Functor implementations you can have, there is no limit to how many narrower interfaces you'll need.
(EDIT: And note that this only works because Functor<B> appears as the result type, and so the child interfaces can narrow it. So AFAIK we can't narrow both uses of Monad<B> in the following interface:
interface Monad<A> {
<B> Monad<B> flatMap(Function<? super A, ? extends Monad<? extends B>> f);
}
In Haskell, with higher-rank type variables, this is (>>=) :: Monad m => m a -> (a -> m b) -> m b.)
Yet another try is to use recursive generics to try and have the interface restrict the result type of the subtype to the subtype itself. Toy example:
/**
* A semigroup is a type with a binary associative operation. Law:
*
* > x.append(y).append(z) = x.append(y.append(z))
*/
interface Semigroup<T extends Semigroup<T>> {
T append(T arg);
}
class Foo implements Semigroup<Foo> {
// Since this implements Semigroup<Foo>, now this method must accept
// a Foo argument and return a Foo result.
Foo append(Foo arg);
}
class Bar implements Semigroup<Bar> {
// Any of these is a compilation error:
Semigroup<Bar> append(Semigroup<Bar> arg);
Semigroup<Foo> append(Bar arg);
Semigroup append(Bar arg);
Foo append(Bar arg);
}
But this sort of technique (which is rather arcane to your run-of-the-mill OOP developer, heck to your run-of-the-mill functional developer as well) still can't express the desired Functor constraint either:
interface Functor<FA extends Functor<FA, A>, A> {
<FB extends Functor<FB, B>, B> FB map(Function<A, B> f);
}
The problem here is this doesn't restrict FB to have the same F as FA—so that when you declare a type List<A> implements Functor<List<A>, A>, the map method can still return a NotAList<B> implements Functor<NotAList<B>, B>.
Final try, in Java, using raw types (unparametrized containers):
interface FunctorStrategy<F> {
F map(Function f, F arg);
}
Here F will get instantiated to unparametrized types like just List or Map. This guarantees that a FunctorStrategy<List> can only return a List—but you've abandoned the use of type variables to track the element types of the lists.
The heart of the problem here is that languages like Java and C# don't allow type parameters to have parameters. In Java, if T is a type variable, you can write T and List<T>, but not T<String>. Higher-kinded types remove this restriction, so that you could have something like this (not fully thought out):
interface Functor<F, A> {
<B> F<B> map(Function<A, B> f);
}
class List<A> implements Functor<List, A> {
// Since F := List, F<B> := List<B>
<B> List<B> map(Function<A, B> f) {
// ...
}
}
And addressing this bit in particular:
(I think) I get that instead of myList |> List.map f or myList |> Seq.map f |> Seq.toList higher kinded types allow you to simply write myList |> map f and it'll return a List. That's great (assuming it's correct), but seems kind of petty? (And couldn't it be done simply by allowing function overloading?) I usually convert to Seq anyway and then I can convert to whatever I want afterwards.
There are many languages that generalize the idea of the map function this way, by modeling it as if, at heart, mapping is about sequences. This remark of yours is in that spirit: if you have a type that supports conversion to and from Seq, you get the map operation "for free" by reusing Seq.map.
In Haskell, however, the Functor class is more general than that; it isn't tied to the notion of sequences. You can implement fmap for types that have no good mapping to sequences, like IO actions, parser combinators, functions, etc.:
instance Functor IO where
fmap f action =
do x <- action
return (f x)
-- This declaration is just to make things easier to read for non-Haskellers
newtype Function a b = Function (a -> b)
instance Functor (Function a) where
fmap f (Function g) = Function (f . g) -- `.` is function composition
The concept of "mapping" really isn't tied to sequences. It's best to understand the functor laws:
(1) fmap id xs == xs
(2) fmap f (fmap g xs) = fmap (f . g) xs
Very informally:
The first law says that mapping with an identity/noop function is the same as doing nothing.
The second law says that any result that you can produce by mapping twice, you can also produce by mapping once.
This is why you want fmap to preserve the type—because as soon as you get map operations that produce a different result type, it becomes much, much harder to make guarantees like this.
I don't want to repeat information in some excellent answers already here, but there's a key point I'd like to add.
You usually don't need higher-kinded types to implement any one particular monad, or functor (or applicative functor, or arrow, or ...). But doing so is mostly missing the point.
In general I've found that when people don't see the usefulness of functors/monads/whatevers, it's often because they're thinking of these things one at a time. Functor/monad/etc operations really add nothing to any one instance (instead of calling bind, fmap, etc I could just call whatever operations I used to implement bind, fmap, etc). What you really want these abstractions for is so you can have code that works generically with any functor/monad/etc.
In a context where such generic code is widely used, this means any time you write a new monad instance your type immediately gains access to a large number of useful operations that have already been written for you. That's the point of seeing monads (and functors, and ...) everywhere; not so that I can use bind rather than concat and map to implement myFunkyListOperation (which gains me nothing in itself), but rather so that when I come to need myFunkyParserOperation and myFunkyIOOperation I can re-use the code I originally saw in terms of lists because it's actually monad-generic.
But to abstract across a parameterised type like a monad with type safety, you need higher-kinded types (as well explained in other answers here).
For a more .NET-specific perspective, I wrote a blog post about this a while back. The crux of it is, with higher-kinded types, you could potentially reuse the same LINQ blocks between IEnumerables and IObservables, but without higher-kinded types this is impossible.
The closest you could get (I figured out after posting the blog) is to make your own IEnumerable<T> and IObservable<T> and extended them both from an IMonad<T>. This would allow you to reuse your LINQ blocks if they're denoted IMonad<T>, but then it's no longer typesafe because it allows you to mix-and-match IObservables and IEnumerables within the same block, which while it may sound intriguing to enable this, you'd basically just get some undefined behavior.
I wrote a later post on how Haskell makes this easy. (A no-op, really--restricting a block to a certain kind of monad requires code; enabling reuse is the default).
The most-used example of higher-kinded type polymorphism in Haskell is the Monad interface. Functor and Applicative are higher-kinded in the same way, so I'll show Functor in order to show something concise.
class Functor f where
fmap :: (a -> b) -> f a -> f b
Now, examine that definition, looking at how the type variable f is used. You'll see that f can't mean a type that has value. You can identify values in that type signature because they're arguments to and results of a functions. So the type variables a and b are types that can have values. So are the type expressions f a and f b. But not f itself. f is an example of a higher-kinded type variable. Given that * is the kind of types that can have values, f must have the kind * -> *. That is, it takes a type that can have values, because we know from previous examination that a and b must have values. And we also know that f a and f b must have values, so it returns a type that must have values.
This makes the f used in the definition of Functor a higher-kinded type variable.
The Applicative and Monad interfaces add more, but they're compatible. This means that they work on type variables with kind * -> * as well.
Working on higher-kinded types introduces an additional level of abstraction - you aren't restricted to just creating abstractions over basic types. You can also create abstractions over types that modify other types.
Why might you care about Applicative? Because of traversals.
class (Functor t, Foldable t) => Traversable t where
traverse :: Applicative f => (a -> f b) -> t a -> f (t b)
type Traversal s t a b = forall f. Applicative f => (a -> f b) -> s -> f t
Once you've written a Traversable instance, or a Traversal for some type, you can use it for an arbitrary Applicative.
Why might you care about Monad? One reason is streaming systems like pipes, conduit, and streaming. These are entirely non-trivial systems for working with effectful streams. With the Monad class, we can reuse all that machinery for whatever we like, rather than having to rewrite it from scratch each time.
Why else might you care about Monad? Monad transformers. We can layer monad transformers however we like to express different ideas. The uniformity of Monad is what makes this all work.
What are some other interesting higher-kinded types? Let's say ... Coyoneda. Want to make repeated mapping fast? Use
data Coyoneda f a = forall x. Coyoneda (x -> a) (f x)
This works or any functor f passed to it. No higher-kinded types? You'll need a custom version of this for each functor. This is a pretty simple example, but there are much trickier ones you might not want to have to rewrite every time.
Recently stated learning a bit about higher kinded types. Although it's an interesting idea, to be able to have a generic that needs another generic but apart from library developers, i do not see any practical use in any real app. I use scala in business app, i have also seen and studied the code of some nicely designed sgstems and libraries like kafka, akka and some financial apps. Nowhere I found any higher kinded type in use.
It seems like they are nice for academia or similar but the market doesn't need it or hasn't reached to a point where HKT has any practical uses or proves to be better than other existing techniques. To me it's something that you can use to impress others or write blog posts but nothing more than that. It's like multiverse or string theory. Looks nice on paper, gives you hours to speak about but nothing real
(sorry if you don't have any interest in theoratical physiss). One prove is that all the answers above, they all brilliantly describe the mechanics fail to cite one true real case where we would need it despite being the fact it's been 6+ years since OP posted it.