I'm reading about scalaz and noticed that we can make a list of Applicatives to be an Applicative of List.
def sequenceA[F[_]: Applicative, A](list: List[F[A]]): F[List[A]] = list match {
case Nil => (Nil: List[A]).point[F]
case x :: xs => (x |#| sequenceA(xs)) {_ :: _}
}
The question is can we do the opposite? Can we transform F[List[A]] to List[F[A]]?
This is possible if the F in question is Traversable. The code you're showing is specialized for List, but in fact it holds for all Traversable functors.
This means, that for every F that's Traversable and any G that's Applicative, we can go from F[G[A]] to G[F[A]] with sequence.
List also has an Applicative, so we can use it as our G, whereas in your example List was used as the F, the Traversable functor.
An example where what you're asking works could be Option. Here's the original direction:
val x: List[Option[Int]] = ...
val y: Option[List[Int]] = x.sequence
and the other direction:
val a: Option[List[Int]] = ...
val b: List[Option[Int]] = a.sequence
We could also write another function specializing to List:
def sequenceT[F[_]: Traverse, A](list: F[List[A]]): List[F[A]] = ...
Related
I've the written the following Haskell function which accepts two monadic values and combine them into a single monadic value (it's just to illustrate the degree of genericity (or generic-ness) that Haskell type-system could support).
combine x y = do
a <- x
b <- y
return (a, b)
and I tested it with three different monads:
main = do
putStrLn $ show $ combine (Just 10) (Just 20) -- Maybe a
putStrLn $ show $ combine [100] [10, 20] -- [] a
a <- combine getLine getLine -- IO a
putStrLn $ show a
And it works great as expected. Now, I want to know if Scala's type-system could allow me to write the above function without compromising the genericity. But I don't know Scala enough (though I wish to explore it). So could anyone help me convert this code into Scala?
I think this is the equivalent:
import cats._
import cats.implicits._
def combine[T, F[_]: Monad](fa: F[T], fb: F[T]) = for {
a <- fa
b <- fb
} yield (a, b)
Where Monad is from a library (cats or scalaz).
combine(Option(10), Option(20)) produces Some((10,20)) and combine(List(100), List(10, 20)) produces List((100,10), (100,20)).
EDIT: The above version is over-constrained, since it requires the two argument types to be the same. def combine[A, B, F[_]: Monad](fa: F[A], fb: F[B]) fixes that.
Your combine function is equivalent to the Scala code
for { a <- x; b <- y } yield (a,b)
So you might try defining a function:
def combine[M[_],A,B](x: M[A], y: M[B]): M[(A,B)] =
for { a <- x; b <- y } yield (a,b)
And the compiler will complain that flatMap is not a member of M[A] and map is not a member of M[B].
The thing with for is that it is a bit of compiler magic that will accept any type that implements functions called map, flatMap, and withFilter. This is in contrast to Haskell in which we can add (or let the compiler infer) a Monad constraint to let do notation work.
To expand on the answer #JoePallas gave, it is possible to make this work. In fact, the following implementation is essentially how GHC implements typeclasses. The cats and scalaz libraries provide all this stuff for you, but this is how the sausage is made:
First define the interface we need:
trait For[M[_]] {
def map[A,B](ma: M[A], f: A => B): M[B]
def flatMap[A,B](ma: M[A],f: A => M[B]): M[B]
def withFilter[A](ma: M[A],q: A => Boolean): M[A]
}
(I'm using the name For and using a slightly different interface than Monad.)
Then we provide an implicit implementation of this trait for every data type we want to support. Here's an example for Option:
implicit val optionFor = new For[Option] {
def map[A,B](ma: Option[A], f: A => B): Option[B] = ma.map(f)
def flatMap[A,B](ma: Option[A],f: A => Option[B]): Option[B] = ma.flatMap(f)
def withFilter[A](ma: Option[A],q: A => Boolean): Option[A] = ma.withFilter(q).map(a => a)
}
Then we provide an implicit conversion to a type that can apply these operations:
implicit class ForOps[M[_], A](val ma: M[A]) extends AnyVal {
def map[B](f: A => B)(implicit m: For[M]): M[B] = m.map(ma,f)
def flatMap[B](f: A => M[B])(implicit m: For[M]): M[B] = m.flatMap(ma, f)
def withFilter(q: A => Boolean)(implicit m: For[M]): M[A] = m.withFilter(ma,q)
}
And finally, we can define combine:
def combine[M[_]: For, A, B](ma: M[A], mb: M[B]): M[(A, B)] =
for { a <- ma; b <- mb } yield (a, b)
The syntax
def f[T: TC] = ???
is sugar for
def f[T](implicit unutterableName: TC[T]) = ???
An implicit argument list, if not given explicitly at the call site, will be automatically filled in by searching for values/functions with the correct types, as long as those are themselves implicit. In this case, we look for a proof that M is a monad. In the body, this value is implicit, and it has no name to access it. Implicit search can still find it. ForOps allows the 3 for operations to automagically appear on the values by using that Monad.
This is really an explicit version of how GHC implements typeclasses. In the simplest case of no optimization:
class Applicative m => Monad m where
return :: a -> m a
(>>=) :: m a -> (a -> m b) -> m b
compiles to
data Monad m = Monad {
monadSubApplicative :: Applicative m
return :: forall a. a -> m a
(>>=) :: forall a. m a -> (a -> m b) -> m b
}
and
instance Monad [] where
return = _
(>>=) = _
becomes
monadList :: Monad []
monadList = Monad {
monadSubApplicative = applicativeList
, return = _
, (>>=) = _
}
You will often hear the word "dictionary" be used to describe the underlying data type and values. And combine is
combine :: Monad m -> m a -> m b -> m (a, b)
combine (Monad _ return (>>=)) ma mb = ma >>= \a -> mb >>= \b -> return (a, b)
However, GHC applies a bunch of restrictions to the system that makes it more predictable and performs more optimization. Scala sacrifices this to allow the programmer to perform more interesting acrobatics.
For good measure, an instance like this:
newtype Compose f g a = Compose { unCompose :: f (g a) }
instance (Functor f, Functor g) => Functor (Compose f g) where
fmap f (Compose fga) = Compose $ fmap (fmap f) fga
would be done like this in Scala (using an implicit def, not a val):
trait Functor[F[_]] { def map[A, B](fa: F[A])(f: A => B): F[B] }
final case class Compose[F[_], G[_], A](val get: F[G[A]]) extends AnyVal
object Compose {
// you usually put these implicits in the associated companions
// because implicit search is picky about where it looks
implicit def functor[F[_], G[_]](implicit
functorF: Functor[F],
functorG: Functor[G]
// type lambda: use a type projection on a refinement type
// to create an anonymous type-level function
// it's universally accepted as a horrendous abuse of syntax
// you can use the kind-projector plugin to avoid writing them (directly)
) : Functor[({type L[X] = Compose[F, G, X]})#L]
= new Functor[({type L[X] = Compose[F, G, X]})#L] {
override def map[A, B](cfga: Compose[F, G, A])(f: A => B): Compose[F, G, B] =
Compose(functorF.map(cfga.get) { ga => functorG.map(ga)(f) })
}
}
Making all this stuff explicit is a bit ugly, but it works quite well.
In functional programming (and programming in general) it is good practice to use the least powerful abstraction you can find. In the example you gave, you actually don't need the power of a monad. The combine function is liftA2 from the applicative type class. Example:
import Data.Maybe
import Control.Applicative
z= Just 1
y= Just 2
liftA2 (,) z y
> Just (1,2)
In Scala you have something similar. An example from the Scalaz library which uses the same abstraction:
import scalaz._, Scalaz._
(Option(1) |#| Option(2))(Tuple2.apply)
> res3: Option[(Int, Int)] = Some((1, 2))
The reason you don't need the monad abstraction is that the values are independent of each other.
For example,
Vector(Some(1), Some(2), Some(3), None).flatMap{
n => n
}
produces a Vector(1, 2, 3) instead of giving an error. As I have seen in other languages, flatMap is used when you have a mapper function that produces nesting so I would expect this to be a valid flatMap:
Vector(1, 2, 3).flatMap{
eachNum => Vector(eachNum)
}
My mapper function produces a Vector which would cause nesting (i.e. Vector(Vector(1), Vector(2), Vector(3), Vector(4))) if I used a map due to the container wrapping. However, flatMap will remove this nesting and flatten it. This makes sense when there is nesting of two identical monads.
However, I do not understand how using a flatMap with a mapper function that returns an Option makes a Vector[Option[Int]] become a Vector[Int]. Is there some sort of transformation going on (I have never seen this before), could someone explain and perhaps point me to some resources?
Thank you very much
we can use reify to see what is going on:
scala> import reflect.runtime.universe._
import reflect.runtime.universe._
scala> val v = Vector(Some(1), Some(2), Some(3), None)
v: scala.collection.immutable.Vector[Option[Int]] = Vector(Some(1), Some(2), Some(3), None)
scala> reify { v.flatMap(x => x) }
res0: reflect.runtime.universe.Expr[scala.collection.immutable.Vector[Int]] =
Expr[scala.collection.immutable.Vector[Int]]($read.v.flatMap(((x) =>
Option.option2Iterable(x)))(Vector.canBuildFrom))
This is showing us that it is using the option2Iterable conversion to convert the Option to Iterable, and Iterable is a subtype of the GenTraversableOnce type that flatMap is expecting.
The function f passed within the flatMap here:
Vector(Some(1), Some(2), Some(3), None).flatMap{
n => n
}
Is a function A => GenTraversableOnce[B] as described in the flatMap implementation:
def flatMap[B, That](f : scala.Function1[A, GenTraversableOnce[B]])
(implicit bf : CanBuildFrom[Repr, B, That])
: That = ???
The function implemented in your example n => n is:
(n: Option[Int]) => n
Where A is Option[Int] and B is Int.
Because CanBuildFrom is defined as trait CanBuildFrom[-From, -Elem, +To]:
From is Repr, and in this case Vector
Elem is B so Int
The result of flatMap is therefore Vector[Int]
Here is a recursive version of append for two lists:
def append[A](x: List[A], y: List[A]): List[A] = x match {
case Nil => y
case h :: t => h :: append(t, y)
}
How to convert it to tail-recursive version?
Try this, though x is prepended in reverse order. is that intended?
import scala.annotation.tailrec
#tailrec
def append[A](x: List[A], y: List[A]): List[A] = x match {
case Nil => y
case h :: t => append(t, h :: y)
}
If you want to prepend x in the order it comes with, you'ld have to do something like this:
import scala.annotation.tailrec
def append[A](x: List[A], y: List[A]): List[A] = {
#tailrec
def innerAppend[A](x: List[A], y: List[A]): List[A] = x match {
case Nil => y
case h :: t => innerAppend(t, h :: y)
}
innerAppend(x.reverse, y)
}
The Scala Standard Library's foldLeft method for sequences is a tail-recursive method that allows for the creation of many core sequence operations which apply a binary operation between their elements.
Using foldLeft, here's how you'd implement your append method:
def append[A](x: List[A], y: List[A]): List[A] =
x.reverse.foldLeft(y)((t, h) => h :: t) //Note the swapped order for h and t
which will return an appended list in the same order as given with the inclusion of reverse:
append(List(1,2,3), List(4,5,6))
res0: List[Int] = List(1, 2, 3, 4, 5, 6)
The lack of #tailrec annotation is not meant to suggest that this is not tail-recursive; rather, #tailrec just won't detect properly when you are implementing a method or function through a tail-recursive method like foldLeft.
Unlike foldLeft, foldRight from the Standard Library will not create a tail-recursive method, though its syntax is easier to understand than having to reverse the input list and swap the parameters. That's why I prefer implementing a foldRightViaFoldLeft method, as it has a more user-friendly syntax, and keeps the tail-recursive properties:
def foldRVL[A,B](l: List[A], z: B)(f: (A,B) => B): B = //"RVL" == "Right Via Left"
l.reverse.foldLeft(z)((b, a) => f(a, b)) //Note again that we are swapping a and b
def append[A](x: List[A], y: List[A]): List[A] =
foldRVL(x, y)(_ :: _) //the swap in foldRVL means that we don't have swap when using it
This is still tail-recursive, but is now easier to read. There are many, many uses for foldRVL, and other fold methods. Use them wherever you can. For more information about folds, and a better explanation about their recursive properties, I recommend reading this post: Scala Code Review: foldLeft and foldRight.
I'd also highly recommend the book, Functional Programming in Scala, from which I learned how to properly fold things.
Update (2018): my prayers were answered in Dotty (Type Lambdas), so the following Q&A is more "Scala 2.x"-related
Just a simple example from Scala:
scala> def f(x: Int) = x
f: (x: Int)Int
scala> (f _)(5)
res0: Int = 5
Let's make it generic:
scala> def f[T](x: T) = x
f: [T](x: T)T
scala> (f _)(5)
<console>:9: error: type mismatch;
found : Int(5)
required: Nothing
(f _)(5)
^
Let's look at eta-expansion of polymorphic method in Scala:
scala> f _
res2: Nothing => Nothing = <function1>
Comparison with Haskell:
Prelude> let f x = x
Prelude> f 5
5
Prelude> f "a"
"a"
Prelude> :t f
f :: t -> t
Haskell did infer correct type [T] => [T] here.
More realistic example?
scala> identity _
res2: Nothing => Nothing = <function1>
Even more realistic:
scala> def f[T](l: List[T]) = l.head
f: [T](l: List[T])T
scala> f _
res3: List[Nothing] => Nothing = <function1>
You can't make alias for identity - have to write your own function. Things like [T,U](t: T, u: U) => t -> u (make tuple) are impossible to use as values. More general - if you want to pass some lambda that rely on generic type (e.g. uses generic function, for example: creates lists, tuples, modify them in some way) - you can't do that.
So, how to solve that problem? Any workaround, solution or reasoning?
P.S. I've used term polymorphic lambda (instead of function) as function is just named lambda
Only methods can be generic on the JVM/Scala, not values. You can make an anonymous instance that implements some interface (and duplicate it for every type-arity you want to work with):
trait ~>[A[_], B[_]] { //exists in scalaz
def apply[T](a: A[T]): B[T]
}
val f = new (List ~> Id) {
def apply[T](a: List[T]) = a.head
}
Or use shapeless' Poly, which supports more complicated type-cases. But yeah, it's a limitation and it requires working around.
P∀scal is a compiler plugin that provides more concise syntax for encoding polymorphic values as objects with a generic method.
The identity function, as a value, has type ∀A. A => A. To translate that into Scala, assume a trait
trait ForAll[F[_]] {
def apply[A]: F[A]
}
Then the identity function has type ForAll[λ[A => A => A]], where I use the kind-projector syntax, or, without kind-projector:
type IdFun[A] = A => A
type PolyId = ForAll[IdFun]
And now comes the P∀scal syntactic sugar:
val id = Λ[Α](a => a) : PolyId
or equivalently
val id = ν[PolyId](a => a)
("ν" is the Greek lowercase letter "Nu", read "new")
These are really just shorthands for
new PolyId {
def apply[A] = a => a
}
Multiple type parameters and parameters of arbitrary kinds are supported by P∀scal, but you need a dedicated variation on the above ForAll trait for each variant.
I really like #Travis Brown 's solution:
import shapeless._
scala> Poly(identity _)
res2: shapeless.PolyDefns.~>[shapeless.Id,shapeless.Id] = fresh$macro$1$2$#797aa352
-
scala> def f[T](x: T) = x
f: [T](x: T)T
scala> Poly(f _)
res3: shapeless.PolyDefns.~>[shapeless.Id,shapeless.Id] = fresh$macro$2$2$#664ea816
-
scala> def f[T](l: List[T]) = l.head
f: [T](l: List[T])T
scala> val ff = Poly(f _)
ff: shapeless.PolyDefns.~>[List,shapeless.Id] = fresh$macro$3$2$#51254c50
scala> ff(List(1,2,3))
res5: shapeless.Id[Int] = 1
scala> ff(List("1","2","3"))
res6: shapeless.Id[String] = 1
Poly constructor (in some cases) will give you eta-expansion into Shapeless2 Poly1 function, which is (more-less) truly generic. However it doesn't work for multi-parameters (even with multi type-parameters), so have to "implement" Poly2 with implicit + at approach (as #som-snytt suggested), something like:
object myF extends Poly2 {
implicit def caseA[T, U] = at[T, U]{ (a, b) => a -> b}
}
scala> myF(1,2)
res15: (Int, Int) = (1,2)
scala> myF("a",2)
res16: (String, Int) = (a,2)
P.S. I would really want to see it as a part of language.
It seems to do this you will need to do a bit type hinting to help the Scala type inference system.
def id[T] : T => T = identity _
So I guess if you try to pass identity as a parameter to a function call and the types of that parameter are generic then there should be no problem.
So a very very common pattern in Scala is a for comprehension as follows:
for {
i <- monadA
j <- monadB
} yield (i, j)
Similarly for 3-tuples, ..., n-tuples. This is becoming so common in my code I'd imagine that scalaz provide some awesome operator to do this for me, e.g. monadA funnyOperator monadB funnyOperator monadC. I've looked around and can't seem to find anything. So I've defined my own implicit class for 2-tuples and 3-tuples but would prefer to use scalaz.
Bonus
In response the currently accepted answer, would love to see someone tell how to make this compile:
import scalaz.Scalaz._
// Like a 1 element list
case class MyMonad[+T](x: T) {
def map[U](f: T => U): MyMonad[U] = MyMonad(f(x))
def flatMap[U](f: T => MyMonad[U]): MyMonad[U] = f(x)
}
val myMonad: MyMonad[(Int, Int)] = (MyMonad(1) |#| MyMonad(2)).tupled
and not give:
error: value |#| is not a member of MyMonad[Int]
Bonus Solution:
You need to "provide an applicative instance" e.g.
implicit def myMonadApplicative: Bind[MyMonad] = new Bind[MyMonad] {
def bind[A, B](fa: MyMonad[A])(f: A => MyMonad[B]): MyMonad[B] = fa.flatMap(f)
def map[A, B](fa: MyMonad[A])(f: A => B): MyMonad[B] = fa.map(f)
}
Given that each Monad is an Applicative you can also use
(monadA |#| monadB).tupled
E.g.
scala> val b: List[(Int, Int)] = (List(1, 2, 3) |#| List(4, 6)).tupled
b: List[(Int, Int)] = List((1,4), (1,6), (2,4), (2,6), (3,4), (3,6))
You can use sequence; I can never remember whether you need shapeless-scalaz for this or not:
(monadA, monadB).sequence
(monadA, monadB, monadC).sequence