This is probably a very naive question about shapeless:
Suppose I have functions A => M[B] and B => M[C]. How can I compose them to get a new function A => M[B::C::HNil] ?
If you want to do this generically you can use Scalaz's Arrow:
import scalaz._, Scalaz._
def andThenButKeep[Arr[_, _]: Arrow, A, B, C](
f: Arr[A, B],
g: Arr[B, C]
): Arr[A, (B, C)] = f >>> (Category[Arr].id &&& g)
Or if you want an HList instead of a tuple:
import scalaz._, Scalaz._
import shapeless._, shapeless.syntax.std.tuple._
def andThenButKeep[Arr[_, _], A, B, C](
f: Arr[A, B],
g: Arr[B, C]
)(implicit Arr: Arrow[Arr]): Arr[A, B :: C :: HNil] =
f >>> (Arr.id &&& g) >>> Arr.arr((_: (B, C)).productElements)
Now you'd wrap your functions in a Kleisli arrow:
type OptionFunc[A, B] = Kleisli[Option, A, B]
val f: OptionFunc[Int, String] = Kleisli(i => Some("a" * i))
val g: OptionFunc[String, Int] = Kleisli(s => Some(s.length))
val c = andThenButKeep(f, g)
And then:
scala> println(c.run(10))
Some(aaaaaaaaaa :: 10 :: HNil)
You could make this a little less fussy about type inference (but also less generic) by restricting the arrow to a Kleisli arrow over your M.
You need to define that M[_] is some form of a Monad so you can flatMap it, I chose to use the scalaz Monad:
import scalaz._, Scalaz._
import shapeless._
def compose[M[_] : Monad, A, B, C](f: A => M[B], g: B => M[C]): A => M[B :: C :: HNil] = {
a => f(a).flatMap(b => g(b).map(c => b :: c :: HNil))
}
This should do the trick
for {
b <- f(a)
c <- g(b)
} yield b :: c :: HNil
which of course expands to
f(a) flatMap { b =>
g(b) map { c => b :: c :: HNil }
}
I have a function that makes use of an implicit view to a Seq[A], you can see it makes use of the head method and preserves types:-
scala> def needSeq[A, C <% Seq[A]](col: C) = { (col.head , col) }
needSeq: [A, C](col: C)(implicit evidence$1: C => Seq[A])(A, C)
scala> needSeq(List(1,2,3))
res0: (Int, List[Int]) = (1,List(1, 2, 3))
scala> needSeq(List("a","b"))
res1: (java.lang.String, List[java.lang.String]) = (a,List(a, b))
scala> needSeq(Array("a","b"))
res2: (java.lang.String, Array[java.lang.String]) = (a,Array(a, b))
I want to write a function that takes functions like needSeq and applies them to arguments
scala> def useFunc[A, C <% Seq[A], R](col: C)(f: C => R) = { f(col) }
useFunc: [A, C, R](col: C)(f: C => R)(implicit evidence$1: C => Seq[A])R
The problem is because only one type (C) is provided in the parameter list there is no implicit view from C => Seq[A] available
scala> useFunc(List(1,2,3))(needSeq)
<console>:10: error: No implicit view available from C => Seq[A].
useFunc(List(1,2,3))(needSeq)
^
How should I write useFunc?
The problem is in definition needSeq..
if you can try to refactor it to..
def needSeq[A](col : Seq[A]) = (col.head , col)
then both of these cases works..
useFunc(List(1,2,3))(needSeq) //> res1: (Int, Seq[Int]) = (1,List(1, 2, 3))
useFunc(List(1,2,3))(x => needSeq(x)) //> res2: (Int, Seq[Int]) = (1,List(1, 2, 3))
I think that solution from #Eastsun
useFunc(List(1,2,3))(x => needSeq(x))
works because the C from
def useFunc[A, C <% Seq[A], R](col: C)(f: C => R)
is now represented by the x and kind of connects the type of the List with the type of parameter that the needSeq takes
or one could say that the two lines above better resemble each other that way :
def useFunc[A, C <% Seq[A], R] (col: C) (f: C => R)
useFunc (List(1,2,3)) (x => needSeq(x))
I'm trying to write a method which accepts any type of collection CC[_] and maps it to a new collection (the same collection type but a different element type) and I am struggling royally. Basically I'm trying to implement map but not on the collection itself.
The Question
I'm trying to implement a method with a signature which looks a bit like:
def map[CC[_], T, U](cct: CC[T], f: T => U): CC[U]
It's usage would be:
map(List(1, 2, 3, 4), (_ : Int).toString) //would return List[String]
I'm interested in an answer which would also work where CC is Array and I'm interested in the reason my attempts (below) have ultimately not worked.
My Attempts
(For the impatient, in what follows, I utterly fail to get this to work. To reiterate, the question is "how can I write such a method?")
I start like this:
scala> def map[T, U, CC[_]](cct: CC[T], f: T => U)(implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] =
| cct map f
^
<console>:9: error: value map is not a member of type parameter CC[T]
cct map f
^
OK, that makes sense - I need to say that CC is traversable!
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)(implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] =
| cct map f
<console>:10: error: type mismatch;
found : Traversable[U]
required: CC[U]
cct map f
^
Err, OK! Maybe if I actually specify that cbf instance. After all, it specifies the return type (To) as CC[U]:
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)(implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] =
| cct.map(t => f(t))(cbf)
<console>:10: error: type mismatch;
found : scala.collection.generic.CanBuildFrom[CC[T],U,CC[U]]
required: scala.collection.generic.CanBuildFrom[Traversable[T],U,CC[U]]
cct.map(t => f(t))(cbf)
^
Err, OK! That's a more specific error. Looks like I can use that!
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)(implicit cbf: CanBuildFrom[Traversable[T], U, CC[U]]): CC[U] =
| cct.map(t => f(t))(cbf)
map: [T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)(implicit cbf: scala.collection.generic.CanBuildFrom[Traversable[T],U,CC[U]])CC[U]
Brilliant. I has me a map! Let's use this thing!
scala> map(List(1, 2, 3, 4), (_ : Int).toString)
<console>:11: error: Cannot construct a collection of type List[java.lang.String] with elements of type java.lang.String based on a collection of type Traversable[Int].
map(List(1, 2, 3, 4), (_ : Int).toString)
^
Say, what?
Observations
I really can't help but think that Tony Morris' observations about this at the time were absolutely spot on. What did he say? He said "Whatever that is, it is not map". Look at how easy this is in scalaz-style:
scala> trait Functor[F[_]] { def fmap[A, B](fa: F[A])(f: A => B): F[B] }
defined trait Functor
scala> def map[F[_]: Functor, A, B](fa: F[A], f: A => B): F[B] = implicitly[Functor[F]].fmap(fa)(f)
map: [F[_], A, B](fa: F[A], f: A => B)(implicit evidence$1: Functor[F])F[B]
Then
scala> map(List(1, 2, 3, 4), (_ : Int).toString)
<console>:12: error: could not find implicit value for evidence parameter of type Functor[List]
map(List(1, 2, 3, 4), (_ : Int).toString)
^
So that
scala> implicit val ListFunctor = new Functor[List] { def fmap[A, B](fa: List[A])(f: A => B) = fa map f }
ListFunctor: java.lang.Object with Functor[List] = $anon$1#4395cbcb
scala> map(List(1, 2, 3, 4), (_ : Int).toString)
res5: List[java.lang.String] = List(1, 2, 3, 4)
Memo to self: listen to Tony!
What you're running into is not necessarily CanBuildFrom itself, or the Array vs. Seq issue. You're running into String which is not higher-kinded, but supports map against its Chars.
SO: First a digression into Scala's collection design.
What you need is a way to infer both the collection type (e.g. String, Array[Int], List[Foo]) and the element type (e.g. Char, Int, Foo corresponding to the above).
Scala 2.10.x has added a few "type classes" to help you. For example, you can do the following:
class FilterMapImpl[A, Repr](val r: GenTraversableLike[A, Repr]) {
final def filterMap[B, That](f: A => Option[B])(implicit cbf: CanBuildFrom[Repr, B, That]): That =
r.flatMap(f(_).toSeq)
}
implicit def filterMap[Repr, A](r: Repr)(implicit fr: IsTraversableOnce[Repr]): FilterMapImpl[fr.A,Repr] =
new FilterMapImpl(fr.conversion(r))
There's two pieces here. FIRST, your class that uses collections needs two type parameters: The specific type of the collection Repr and the type of the elements A.
Next, you define an implicit method which only takes the collection type Repr. You use the IsTraversableOnce (note: there is also an IsTraversableLike) to capture the element type of that collection. You see this used in the type signature FilterMapImpl[Repr, fr.A].
Now, part of this is because Scala does not use the same category for all of its "functor-like" operations. Specifically, map is a useful method for String. I can adjust all characters. However, String can only be a Seq[Char]. If I want to define a Functor, then my category can only contain the type Char and the arrows Char => Char. This logic is captured in CanBuildFrom. However, since a String is a Seq[Char], if you try to use a map in the category supported by Seq's map method, then CanBuildFrom will alter your call to map.
We're essentially defining an "inheritance" relationship for our categories. If you try to use the Functor pattern, we drop the type signature to the most specific category we can retain. Call it what you will; that's a big motivating factor for the current collection design.
End Digression, answer the question
Now, because we're trying to infer a lot of types at the same time, I think this option has the fewest type annotations:
import collection.generic._
def map[Repr](col: Repr)(implicit tr: IsTraversableLike[Repr]) = new {
def apply[U, That](f: tr.A => U)(implicit cbf: CanBuildFrom[Repr, U, That]) =
tr.conversion(col) map f
}
scala> map("HI") apply (_ + 1 toChar )
warning: there were 2 feature warnings; re-run with -feature for details
res5: String = IJ
The important piece to note here is that IsTraversableLike captures a conversion from Repr to TraversableLike that allows you to use the map method.
Option 2
We also split the method call up a bit so that Scala can infer the types Repr and U before we define our anonymous function. To avoid type annotations on anonymous functions, we must have all types known before it shows up. Now, we can still have Scala infer some types, but lose things that are implicitly Traversable if we do this:
import collection.generic._
import collection._
def map[Repr <: TraversableLike[A, Repr], A, U, That](col: Repr with TraversableLike[A,Repr])(f: A => U)(implicit cbf: CanBuildFrom[Repr, U, That]) =
col map f
Notice that we have to use Repr with TraversableLike[A,Repr]. It seems that most F-bounded types require this juggling.
In any case, now let's see what happens on something that extends Traversable:
scala> map(List(40,41))(_ + 1 toChar )
warning: there were 1 feature warnings; re-run with -feature for details
res8: List[Char] = List(), *)
That's great. However, if we want the same usage for Array and String, we have to go to a bit more work:
scala> map(Array('H', 'I'): IndexedSeq[Char])(_ + 1 toChar)(breakOut): Array[Char]
warning: there were 1 feature warnings; re-run with -feature for details
res14: Array[Char] = Array(I, J)
scala> map("HI": Seq[Char])(_ + 1 toChar)(breakOut) : String
warning: there were 1 feature warnings; re-run with -feature for details
res11: String = IJ
There are two pieces to this usage:
We have to use a type annotation for the implicit conversion from String/Array → Seq/IndexedSeq.
We have to use breakOut for our CanBuildFrom and type-annotate the expected return value.
This is solely because the type Repr <: TraversableLike[A,Repr] does not include String or Array, since those use implicit conversions.
Option 3
You can place all the implicits together at the end and require the user to annotate types. Not the most elegant solution, so I think I'll avoid posting it unless you'd really like to see it.
SO, basically if you want to include String and Array[T] as collections, you have to jump through some hoops. This category restriction for map applies to both String and BitSet functors in Scala.
I hope that helps. Ping me if you have any more questions.
There are actually several questions in there...
Let's start with your last attempt:
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)
(implicit cbf: CanBuildFrom[Traversable[T], U, CC[U]]): CC[U] =
cct.map(t => f(t))(cbf)
This one does compiles but does not work because, according to your type signature, it has to look for an implicit CanBuildFrom[Traversable[Int], String, List[String]] in scope, and there just isn't one. If you were to create one by hand, it would work.
Now the previous attempt:
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)
(implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] =
cct.map(t => f(t))(cbf)
<console>:10: error: type mismatch;
found : scala.collection.generic.CanBuildFrom[CC[T],U,CC[U]]
required: scala.collection.generic.CanBuildFrom[Traversable[T],U,CC[U]]
cct.map(t => f(t))(cbf)
^
This one does not compile because the implicit CanBuildFrom in Traversable is hardcoded to accept only a Traversable as From collection. However, as pointed out in the other answer, TraversableLike knows about the actual collection type (it's its second type parameter), so it defines map with the proper CanBuildFrom[CC[T], U, CC[U]] and everybody is happy. Actually, TraversableLike inherits this map method from scala.collection.generic.FilterMonadic, so this is even more generic:
scala> import scala.collection.generic._
import scala.collection.generic._
scala> def map[T, U, CC[T] <: FilterMonadic[T, CC[T]]](cct: CC[T], f: T => U)
| (implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] = cct.map(f)
warning: there were 1 feature warnings; re-run with -feature for details
map: [T, U, CC[T] <: scala.collection.generic.FilterMonadic[T,CC[T]]](cct: CC[T], f: T => U)(implicit cbf: scala.collection.generic.CanBuildFrom[CC[T],U,CC[U]])CC[U]
scala> map(List(1,2,3,4), (_:Int).toString + "k")
res0: List[String] = List(1k, 2k, 3k, 4k)
Finally, the above does not work with arrays because Array is not a FilterMonadic. But there is an implicit conversion from Array to ArrayOps, and the latter implements FilterMonadic. So if you add a view bound in there, you get something that works for arrays as well:
scala> import scala.collection.generic._
import scala.collection.generic._
scala> def map[T, U, CC[T]](cct: CC[T], f: T => U)
| (implicit cbf: CanBuildFrom[CC[T], U, CC[U]],
| ev: CC[T] => FilterMonadic[T,CC[T]]): CC[U] = cct.map(f)
warning: there were 1 feature warnings; re-run with -feature for details
map: [T, U, CC[T]](cct: CC[T], f: T => U)(implicit cbf: scala.collection.generic.CanBuildFrom[CC[T],U,CC[U]], implicit ev: CC[T] => scala.collection.generic.FilterMonadic[T,CC[T]])CC[U]
scala> map(List(1,2,3,4), (_:Int).toString + "k")
res0: List[String] = List(1k, 2k, 3k, 4k)
scala> map(Array(1,2,3,4), (_:Int).toString + "k")
res1: Array[String] = Array(1k, 2k, 3k, 4k)
EDIT:
There is also a way to make it work for String and co: just remove the higher kinds on the input/output collection, using a third one in the middle:
def map[T, U, From, To, Middle](cct: From, f: T => U)
(implicit ev: From => FilterMonadic[T, Middle],
cbf: CanBuildFrom[Middle,U,To]): To = cct.map(f)
This works on String and even on Map[A,B]:
scala> map(Array(42,1,2), (_:Int).toString)
res0: Array[java.lang.String] = Array(42, 1, 2)
scala> map(List(42,1,2), (_:Int).toString)
res1: List[java.lang.String] = List(42, 1, 2)
scala> map("abcdef", (x: Char) => (x + 1).toChar)
res2: String = bcdefg
scala> map(Map(1 -> "a", 2 -> "b", 42 -> "hi!"), (a:(Int, String)) => (a._2, a._1))
res5: scala.collection.immutable.Map[String,Int] = Map(a -> 1, b -> 2, hi! -> 42)
Tested with 2.9.2. But as jsuereth pointed out, there is the wonderful IsTraversableLike in 2.10 that is better fitted for this.
Is this it?
def map[A,B,T[X] <: TraversableLike[X,T[X]]]
(xs: T[A])(f: A => B)(implicit cbf: CanBuildFrom[T[A],B,T[B]]): T[B] = xs.map(f)
map(List(1,2,3))(_.toString)
// List[String] = List(1, 2, 3)
See also this question.
In Haskell, liftM2 can be defined as:
liftM2 :: (Monad m) => (a1 -> a2 -> r) -> m a1 -> m a2 -> m r
liftM2 f m1 m2 = do
x1 <- m1
x2 <- m2
return $ f x1 x2
I'd like to translate this to Scala. My first attempt was the following:
def liftM2[T1, T2, R, M[_]](f: (T1, T2) => R)(ma: M[T1], mb: M[T2]) : M[R] = for {
a <- ma
b <- mb
} yield f(a, b)
This fails in what I guess is the most obvious way possible: "value flatMap is not a member of type parameter M[T1]". Right, I haven't indicated that M[_] is some kind of monad. So the next thing I tried was to define some structural type like:
type Monad[A] = {
def flatMap[B](f: (A) => Monad[B]): Monad[B]
}
... and to have M[A] <: Monad[A]. But that doesn't work, because Scala doesn't have recursive structural types.
So the next few things I tried involved gyrations similar to M[A] <: FilterMonadic[A, _]. Those all failed, probably because I wasn't able to figure out the right implicit-fu for CanBuildFrom.
The most closely-related question I could find here on StackOverflow was this one, touching both on recursive structural types and how to mimic Haskell's typeclasses in Scala. But that approach requires defining an implicit conversion from each type you care about to the trait defining the typeclass, which seems terribly circular in this case...
Is there any good way to do what I'm trying to do?
The usual way to encode type classes in Scala turns out to follow Haskell pretty closely: List doesn't implement a Monad interface (as you might expect in an object-oriented language), but rather we define the type class instance in a separate object.
trait Monad[M[_]] {
def point[A](a: => A): M[A]
def bind[A, B](ma: M[A])(f: A => M[B]): M[B]
def map[A, B](ma: M[A])(f: A => B): M[B] = bind(ma)(a => point(f(a)))
}
implicit object listMonad extends Monad[List] {
def point[A](a: => A) = List(a)
def bind[A, B](ma: List[A])(f: A => List[B]) = ma flatMap f
}
This idea is introduced in Poor Man's Type Classes and explored more deeply in Type Classes as Objects and Implicits. Notice that the point method could not have been defined in an object-oriented interface, as it doesn't have M[A] as one of it's arguments to be converted to the this reference in an OO encoding. (Or put another way: it can't be part of an interface for the same reason a constructor signature can't be represented in an interface.)
You can then write liftM2 as:
def liftM2[M[_], A, B, C](f: (A, B) => C)
(implicit M: Monad[M]): (M[A], M[B]) => M[C] =
(ma, mb) => M.bind(ma)(a => M.map(mb)(b => f(a, b)))
val f = liftM2[List, Int, Int, Int](_ + _)
f(List(1, 2, 3), List(4, 5)) // List(5, 6, 6, 7, 7, 8)
This pattern has been applied extensively in Scalaz. Version 7, currently in development, includes an index of the type classes.
In addition to providing type classes and instances for standard library types, it provides a 'syntactic' layer that allows the more familiar receiver.method(args) style of method invocation. This often affords better type inference (accounting for Scala's left-to-right inference algorithm), and allows use of the for-comprehension syntactic sugar. Below, we use that to rewrite liftM2, based on the map and flatMap methods in MonadV.
// Before Scala 2.10
trait MonadV[M[_], A] {
def self: M[A]
implicit def M: Monad[M]
def flatMap[B](f: A => M[B]): M[B] = M.bind(self)(f)
def map[B](f: A => B): M[B] = M.map(self)(f)
}
implicit def ToMonadV[M[_], A](ma: M[A])
(implicit M0: Monad[M]) =
new MonadV[M, A] {
val M = M0
val self = ma
}
// Or, as of Scala 2.10
implicit class MonadOps[M[_], A](self: M[A])(implicit M: Monad[M]) {
def flatMap[B](f: A => M[B]): M[B] = M.flatMap(self)(f)
def map[B](f: A => B): M[B] = M.map(self)(f)
}
def liftM2[M[_]: Monad, A, B, C](f: (A, B) => C): (M[A], M[B]) => M[C] =
(ma, mb) => for {a <- ma; b <- mb} yield f(a, b)
Update
Yep, its possible to write less generic version of liftM2 for the Scala collections. You just have to feed in all the required CanBuildFrom instances.
scala> def liftM2[CC[X] <: TraversableLike[X, CC[X]], A, B, C]
| (f: (A, B) => C)
| (implicit ba: CanBuildFrom[CC[A], C, CC[C]], bb: CanBuildFrom[CC[B], C, CC[C]])
| : (CC[A], CC[B]) => CC[C] =
| (ca, cb) => ca.flatMap(a => cb.map(b => f(a, b)))
liftM2: [CC[X] <: scala.collection.TraversableLike[X,CC[X]], A, B, C](f: (A, B) => C)(implicit ba: scala.collection.generic.CanBuildFrom[CC[A],C,CC[C]], implicit bb: scala.collection.generic.CanBuildFrom[CC[B],C,CC[C]])(CC[A], CC[B]) => CC[C]
scala> liftM2[List, Int, Int, Int](_ + _)
res0: (List[Int], List[Int]) => List[Int] = <function2>
scala> res0(List(1, 2, 3), List(4, 5))
res1: List[Int] = List(5, 6, 6, 7, 7, 8)