Update (2018): my prayers were answered in Dotty (Type Lambdas), so the following Q&A is more "Scala 2.x"-related
Just a simple example from Scala:
scala> def f(x: Int) = x
f: (x: Int)Int
scala> (f _)(5)
res0: Int = 5
Let's make it generic:
scala> def f[T](x: T) = x
f: [T](x: T)T
scala> (f _)(5)
<console>:9: error: type mismatch;
found : Int(5)
required: Nothing
(f _)(5)
^
Let's look at eta-expansion of polymorphic method in Scala:
scala> f _
res2: Nothing => Nothing = <function1>
Comparison with Haskell:
Prelude> let f x = x
Prelude> f 5
5
Prelude> f "a"
"a"
Prelude> :t f
f :: t -> t
Haskell did infer correct type [T] => [T] here.
More realistic example?
scala> identity _
res2: Nothing => Nothing = <function1>
Even more realistic:
scala> def f[T](l: List[T]) = l.head
f: [T](l: List[T])T
scala> f _
res3: List[Nothing] => Nothing = <function1>
You can't make alias for identity - have to write your own function. Things like [T,U](t: T, u: U) => t -> u (make tuple) are impossible to use as values. More general - if you want to pass some lambda that rely on generic type (e.g. uses generic function, for example: creates lists, tuples, modify them in some way) - you can't do that.
So, how to solve that problem? Any workaround, solution or reasoning?
P.S. I've used term polymorphic lambda (instead of function) as function is just named lambda
Only methods can be generic on the JVM/Scala, not values. You can make an anonymous instance that implements some interface (and duplicate it for every type-arity you want to work with):
trait ~>[A[_], B[_]] { //exists in scalaz
def apply[T](a: A[T]): B[T]
}
val f = new (List ~> Id) {
def apply[T](a: List[T]) = a.head
}
Or use shapeless' Poly, which supports more complicated type-cases. But yeah, it's a limitation and it requires working around.
P∀scal is a compiler plugin that provides more concise syntax for encoding polymorphic values as objects with a generic method.
The identity function, as a value, has type ∀A. A => A. To translate that into Scala, assume a trait
trait ForAll[F[_]] {
def apply[A]: F[A]
}
Then the identity function has type ForAll[λ[A => A => A]], where I use the kind-projector syntax, or, without kind-projector:
type IdFun[A] = A => A
type PolyId = ForAll[IdFun]
And now comes the P∀scal syntactic sugar:
val id = Λ[Α](a => a) : PolyId
or equivalently
val id = ν[PolyId](a => a)
("ν" is the Greek lowercase letter "Nu", read "new")
These are really just shorthands for
new PolyId {
def apply[A] = a => a
}
Multiple type parameters and parameters of arbitrary kinds are supported by P∀scal, but you need a dedicated variation on the above ForAll trait for each variant.
I really like #Travis Brown 's solution:
import shapeless._
scala> Poly(identity _)
res2: shapeless.PolyDefns.~>[shapeless.Id,shapeless.Id] = fresh$macro$1$2$#797aa352
-
scala> def f[T](x: T) = x
f: [T](x: T)T
scala> Poly(f _)
res3: shapeless.PolyDefns.~>[shapeless.Id,shapeless.Id] = fresh$macro$2$2$#664ea816
-
scala> def f[T](l: List[T]) = l.head
f: [T](l: List[T])T
scala> val ff = Poly(f _)
ff: shapeless.PolyDefns.~>[List,shapeless.Id] = fresh$macro$3$2$#51254c50
scala> ff(List(1,2,3))
res5: shapeless.Id[Int] = 1
scala> ff(List("1","2","3"))
res6: shapeless.Id[String] = 1
Poly constructor (in some cases) will give you eta-expansion into Shapeless2 Poly1 function, which is (more-less) truly generic. However it doesn't work for multi-parameters (even with multi type-parameters), so have to "implement" Poly2 with implicit + at approach (as #som-snytt suggested), something like:
object myF extends Poly2 {
implicit def caseA[T, U] = at[T, U]{ (a, b) => a -> b}
}
scala> myF(1,2)
res15: (Int, Int) = (1,2)
scala> myF("a",2)
res16: (String, Int) = (a,2)
P.S. I would really want to see it as a part of language.
It seems to do this you will need to do a bit type hinting to help the Scala type inference system.
def id[T] : T => T = identity _
So I guess if you try to pass identity as a parameter to a function call and the types of that parameter are generic then there should be no problem.
I have a function that makes use of an implicit view to a Seq[A], you can see it makes use of the head method and preserves types:-
scala> def needSeq[A, C <% Seq[A]](col: C) = { (col.head , col) }
needSeq: [A, C](col: C)(implicit evidence$1: C => Seq[A])(A, C)
scala> needSeq(List(1,2,3))
res0: (Int, List[Int]) = (1,List(1, 2, 3))
scala> needSeq(List("a","b"))
res1: (java.lang.String, List[java.lang.String]) = (a,List(a, b))
scala> needSeq(Array("a","b"))
res2: (java.lang.String, Array[java.lang.String]) = (a,Array(a, b))
I want to write a function that takes functions like needSeq and applies them to arguments
scala> def useFunc[A, C <% Seq[A], R](col: C)(f: C => R) = { f(col) }
useFunc: [A, C, R](col: C)(f: C => R)(implicit evidence$1: C => Seq[A])R
The problem is because only one type (C) is provided in the parameter list there is no implicit view from C => Seq[A] available
scala> useFunc(List(1,2,3))(needSeq)
<console>:10: error: No implicit view available from C => Seq[A].
useFunc(List(1,2,3))(needSeq)
^
How should I write useFunc?
The problem is in definition needSeq..
if you can try to refactor it to..
def needSeq[A](col : Seq[A]) = (col.head , col)
then both of these cases works..
useFunc(List(1,2,3))(needSeq) //> res1: (Int, Seq[Int]) = (1,List(1, 2, 3))
useFunc(List(1,2,3))(x => needSeq(x)) //> res2: (Int, Seq[Int]) = (1,List(1, 2, 3))
I think that solution from #Eastsun
useFunc(List(1,2,3))(x => needSeq(x))
works because the C from
def useFunc[A, C <% Seq[A], R](col: C)(f: C => R)
is now represented by the x and kind of connects the type of the List with the type of parameter that the needSeq takes
or one could say that the two lines above better resemble each other that way :
def useFunc[A, C <% Seq[A], R] (col: C) (f: C => R)
useFunc (List(1,2,3)) (x => needSeq(x))
I'm trying to write a method which accepts any type of collection CC[_] and maps it to a new collection (the same collection type but a different element type) and I am struggling royally. Basically I'm trying to implement map but not on the collection itself.
The Question
I'm trying to implement a method with a signature which looks a bit like:
def map[CC[_], T, U](cct: CC[T], f: T => U): CC[U]
It's usage would be:
map(List(1, 2, 3, 4), (_ : Int).toString) //would return List[String]
I'm interested in an answer which would also work where CC is Array and I'm interested in the reason my attempts (below) have ultimately not worked.
My Attempts
(For the impatient, in what follows, I utterly fail to get this to work. To reiterate, the question is "how can I write such a method?")
I start like this:
scala> def map[T, U, CC[_]](cct: CC[T], f: T => U)(implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] =
| cct map f
^
<console>:9: error: value map is not a member of type parameter CC[T]
cct map f
^
OK, that makes sense - I need to say that CC is traversable!
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)(implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] =
| cct map f
<console>:10: error: type mismatch;
found : Traversable[U]
required: CC[U]
cct map f
^
Err, OK! Maybe if I actually specify that cbf instance. After all, it specifies the return type (To) as CC[U]:
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)(implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] =
| cct.map(t => f(t))(cbf)
<console>:10: error: type mismatch;
found : scala.collection.generic.CanBuildFrom[CC[T],U,CC[U]]
required: scala.collection.generic.CanBuildFrom[Traversable[T],U,CC[U]]
cct.map(t => f(t))(cbf)
^
Err, OK! That's a more specific error. Looks like I can use that!
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)(implicit cbf: CanBuildFrom[Traversable[T], U, CC[U]]): CC[U] =
| cct.map(t => f(t))(cbf)
map: [T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)(implicit cbf: scala.collection.generic.CanBuildFrom[Traversable[T],U,CC[U]])CC[U]
Brilliant. I has me a map! Let's use this thing!
scala> map(List(1, 2, 3, 4), (_ : Int).toString)
<console>:11: error: Cannot construct a collection of type List[java.lang.String] with elements of type java.lang.String based on a collection of type Traversable[Int].
map(List(1, 2, 3, 4), (_ : Int).toString)
^
Say, what?
Observations
I really can't help but think that Tony Morris' observations about this at the time were absolutely spot on. What did he say? He said "Whatever that is, it is not map". Look at how easy this is in scalaz-style:
scala> trait Functor[F[_]] { def fmap[A, B](fa: F[A])(f: A => B): F[B] }
defined trait Functor
scala> def map[F[_]: Functor, A, B](fa: F[A], f: A => B): F[B] = implicitly[Functor[F]].fmap(fa)(f)
map: [F[_], A, B](fa: F[A], f: A => B)(implicit evidence$1: Functor[F])F[B]
Then
scala> map(List(1, 2, 3, 4), (_ : Int).toString)
<console>:12: error: could not find implicit value for evidence parameter of type Functor[List]
map(List(1, 2, 3, 4), (_ : Int).toString)
^
So that
scala> implicit val ListFunctor = new Functor[List] { def fmap[A, B](fa: List[A])(f: A => B) = fa map f }
ListFunctor: java.lang.Object with Functor[List] = $anon$1#4395cbcb
scala> map(List(1, 2, 3, 4), (_ : Int).toString)
res5: List[java.lang.String] = List(1, 2, 3, 4)
Memo to self: listen to Tony!
What you're running into is not necessarily CanBuildFrom itself, or the Array vs. Seq issue. You're running into String which is not higher-kinded, but supports map against its Chars.
SO: First a digression into Scala's collection design.
What you need is a way to infer both the collection type (e.g. String, Array[Int], List[Foo]) and the element type (e.g. Char, Int, Foo corresponding to the above).
Scala 2.10.x has added a few "type classes" to help you. For example, you can do the following:
class FilterMapImpl[A, Repr](val r: GenTraversableLike[A, Repr]) {
final def filterMap[B, That](f: A => Option[B])(implicit cbf: CanBuildFrom[Repr, B, That]): That =
r.flatMap(f(_).toSeq)
}
implicit def filterMap[Repr, A](r: Repr)(implicit fr: IsTraversableOnce[Repr]): FilterMapImpl[fr.A,Repr] =
new FilterMapImpl(fr.conversion(r))
There's two pieces here. FIRST, your class that uses collections needs two type parameters: The specific type of the collection Repr and the type of the elements A.
Next, you define an implicit method which only takes the collection type Repr. You use the IsTraversableOnce (note: there is also an IsTraversableLike) to capture the element type of that collection. You see this used in the type signature FilterMapImpl[Repr, fr.A].
Now, part of this is because Scala does not use the same category for all of its "functor-like" operations. Specifically, map is a useful method for String. I can adjust all characters. However, String can only be a Seq[Char]. If I want to define a Functor, then my category can only contain the type Char and the arrows Char => Char. This logic is captured in CanBuildFrom. However, since a String is a Seq[Char], if you try to use a map in the category supported by Seq's map method, then CanBuildFrom will alter your call to map.
We're essentially defining an "inheritance" relationship for our categories. If you try to use the Functor pattern, we drop the type signature to the most specific category we can retain. Call it what you will; that's a big motivating factor for the current collection design.
End Digression, answer the question
Now, because we're trying to infer a lot of types at the same time, I think this option has the fewest type annotations:
import collection.generic._
def map[Repr](col: Repr)(implicit tr: IsTraversableLike[Repr]) = new {
def apply[U, That](f: tr.A => U)(implicit cbf: CanBuildFrom[Repr, U, That]) =
tr.conversion(col) map f
}
scala> map("HI") apply (_ + 1 toChar )
warning: there were 2 feature warnings; re-run with -feature for details
res5: String = IJ
The important piece to note here is that IsTraversableLike captures a conversion from Repr to TraversableLike that allows you to use the map method.
Option 2
We also split the method call up a bit so that Scala can infer the types Repr and U before we define our anonymous function. To avoid type annotations on anonymous functions, we must have all types known before it shows up. Now, we can still have Scala infer some types, but lose things that are implicitly Traversable if we do this:
import collection.generic._
import collection._
def map[Repr <: TraversableLike[A, Repr], A, U, That](col: Repr with TraversableLike[A,Repr])(f: A => U)(implicit cbf: CanBuildFrom[Repr, U, That]) =
col map f
Notice that we have to use Repr with TraversableLike[A,Repr]. It seems that most F-bounded types require this juggling.
In any case, now let's see what happens on something that extends Traversable:
scala> map(List(40,41))(_ + 1 toChar )
warning: there were 1 feature warnings; re-run with -feature for details
res8: List[Char] = List(), *)
That's great. However, if we want the same usage for Array and String, we have to go to a bit more work:
scala> map(Array('H', 'I'): IndexedSeq[Char])(_ + 1 toChar)(breakOut): Array[Char]
warning: there were 1 feature warnings; re-run with -feature for details
res14: Array[Char] = Array(I, J)
scala> map("HI": Seq[Char])(_ + 1 toChar)(breakOut) : String
warning: there were 1 feature warnings; re-run with -feature for details
res11: String = IJ
There are two pieces to this usage:
We have to use a type annotation for the implicit conversion from String/Array → Seq/IndexedSeq.
We have to use breakOut for our CanBuildFrom and type-annotate the expected return value.
This is solely because the type Repr <: TraversableLike[A,Repr] does not include String or Array, since those use implicit conversions.
Option 3
You can place all the implicits together at the end and require the user to annotate types. Not the most elegant solution, so I think I'll avoid posting it unless you'd really like to see it.
SO, basically if you want to include String and Array[T] as collections, you have to jump through some hoops. This category restriction for map applies to both String and BitSet functors in Scala.
I hope that helps. Ping me if you have any more questions.
There are actually several questions in there...
Let's start with your last attempt:
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)
(implicit cbf: CanBuildFrom[Traversable[T], U, CC[U]]): CC[U] =
cct.map(t => f(t))(cbf)
This one does compiles but does not work because, according to your type signature, it has to look for an implicit CanBuildFrom[Traversable[Int], String, List[String]] in scope, and there just isn't one. If you were to create one by hand, it would work.
Now the previous attempt:
scala> def map[T, U, X, CC[X] <: Traversable[X]](cct: CC[T], f: T => U)
(implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] =
cct.map(t => f(t))(cbf)
<console>:10: error: type mismatch;
found : scala.collection.generic.CanBuildFrom[CC[T],U,CC[U]]
required: scala.collection.generic.CanBuildFrom[Traversable[T],U,CC[U]]
cct.map(t => f(t))(cbf)
^
This one does not compile because the implicit CanBuildFrom in Traversable is hardcoded to accept only a Traversable as From collection. However, as pointed out in the other answer, TraversableLike knows about the actual collection type (it's its second type parameter), so it defines map with the proper CanBuildFrom[CC[T], U, CC[U]] and everybody is happy. Actually, TraversableLike inherits this map method from scala.collection.generic.FilterMonadic, so this is even more generic:
scala> import scala.collection.generic._
import scala.collection.generic._
scala> def map[T, U, CC[T] <: FilterMonadic[T, CC[T]]](cct: CC[T], f: T => U)
| (implicit cbf: CanBuildFrom[CC[T], U, CC[U]]): CC[U] = cct.map(f)
warning: there were 1 feature warnings; re-run with -feature for details
map: [T, U, CC[T] <: scala.collection.generic.FilterMonadic[T,CC[T]]](cct: CC[T], f: T => U)(implicit cbf: scala.collection.generic.CanBuildFrom[CC[T],U,CC[U]])CC[U]
scala> map(List(1,2,3,4), (_:Int).toString + "k")
res0: List[String] = List(1k, 2k, 3k, 4k)
Finally, the above does not work with arrays because Array is not a FilterMonadic. But there is an implicit conversion from Array to ArrayOps, and the latter implements FilterMonadic. So if you add a view bound in there, you get something that works for arrays as well:
scala> import scala.collection.generic._
import scala.collection.generic._
scala> def map[T, U, CC[T]](cct: CC[T], f: T => U)
| (implicit cbf: CanBuildFrom[CC[T], U, CC[U]],
| ev: CC[T] => FilterMonadic[T,CC[T]]): CC[U] = cct.map(f)
warning: there were 1 feature warnings; re-run with -feature for details
map: [T, U, CC[T]](cct: CC[T], f: T => U)(implicit cbf: scala.collection.generic.CanBuildFrom[CC[T],U,CC[U]], implicit ev: CC[T] => scala.collection.generic.FilterMonadic[T,CC[T]])CC[U]
scala> map(List(1,2,3,4), (_:Int).toString + "k")
res0: List[String] = List(1k, 2k, 3k, 4k)
scala> map(Array(1,2,3,4), (_:Int).toString + "k")
res1: Array[String] = Array(1k, 2k, 3k, 4k)
EDIT:
There is also a way to make it work for String and co: just remove the higher kinds on the input/output collection, using a third one in the middle:
def map[T, U, From, To, Middle](cct: From, f: T => U)
(implicit ev: From => FilterMonadic[T, Middle],
cbf: CanBuildFrom[Middle,U,To]): To = cct.map(f)
This works on String and even on Map[A,B]:
scala> map(Array(42,1,2), (_:Int).toString)
res0: Array[java.lang.String] = Array(42, 1, 2)
scala> map(List(42,1,2), (_:Int).toString)
res1: List[java.lang.String] = List(42, 1, 2)
scala> map("abcdef", (x: Char) => (x + 1).toChar)
res2: String = bcdefg
scala> map(Map(1 -> "a", 2 -> "b", 42 -> "hi!"), (a:(Int, String)) => (a._2, a._1))
res5: scala.collection.immutable.Map[String,Int] = Map(a -> 1, b -> 2, hi! -> 42)
Tested with 2.9.2. But as jsuereth pointed out, there is the wonderful IsTraversableLike in 2.10 that is better fitted for this.
Is this it?
def map[A,B,T[X] <: TraversableLike[X,T[X]]]
(xs: T[A])(f: A => B)(implicit cbf: CanBuildFrom[T[A],B,T[B]]): T[B] = xs.map(f)
map(List(1,2,3))(_.toString)
// List[String] = List(1, 2, 3)
See also this question.
I am trying to understand mapConserve, which is said to be "Like xs map f, but returns xs unchanged if function f maps all elements to themselves," from List. Yet, it is giving out error.
def map [B] (f: (A) ⇒ B): List[B]
def mapConserve (f: (A) ⇒ A): List[A]
def mapConserve [B >: A <: AnyRef] (f: (A) ⇒ B): List[B]
scala> list map (x=>x)
res105: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
scala> list mapConserve (x=>x)
<console>:12: error: inferred type arguments [Int] do not conform to method mapConserve's type parameter bounds [B >: Int <: AnyRef]
list mapConserve (x=>x)
^
The mapConserve code should satisfy the (A) => A function. If not, it still should satisfy the (A) => B function, since type A can be subtype and supertype of itself. Please enlighten me the purpose of mapConserve and the error.
Actually, mapConserve is defined as
def mapConserve[A <: AnyRef](xs: List[A])(f: A => A): List[A]
def mapConserve[B >: A <: AnyRef](f: A => B): List[B]
so A should be a subtype of AnyRef. Int is a subtype of AnyVal, that brings an error.
scala> val l = List("foo", "bar", "baz")
l: List[java.lang.String] = List(foo, bar, baz)
scala> l.mapConserve(_.toUpperCase)
res4: List[java.lang.String] = List(FOO, BAR, BAZ)
scala> l.mapConserve(identity)
res5: List[java.lang.String] = List(foo, bar, baz)
Update:
The only difference between map and mapConserve, as it is described in the scaladoc:
Builds a new list by applying a function to all elements of this list.
Like xs map f, but returns xs unchanged if function f maps all elements to themselves (as determined by eq).
scala> val xs = List.fill(1000000)("foo")
xs: List[java.lang.String] = List(foo, foo,...)
scala> xs.map(identity) eq xs
res48: Boolean = false
scala> xs.mapConserve(identity) eq xs
res49: Boolean = true
And xs mapConserve identity is about five times faster in my simple benchmark.
I'm trying to enrich Scala collections with my own map method, and I'm close but the implicit conversion doesn't work. Besides that, is there anything else I'm missing here? I'm looking at various other resources on the Web, including SO answers that this question is being marked as duplicating, and many are missing something here and there (e.g. using C[A] <: GenTraversable[A], using b() instead of b(xs), forgetting about Array, forgetting about BitSet, etc.).
implicit def conv[A,C](xs: C)(implicit ev: C <:< GenTraversableLike[A,C]) = new {
def mymap[B,D](f: A => B)(implicit b: CanBuildFrom[C,B,D]): D = b(xs).result // placeholder
}
scala> conv(List(1,2,3))
res39: java.lang.Object{def mymap[B,D](f: Int => B)(implicit b: scala.collection.generic.CanBuildFrom[List[Int],B,D]): D} = $$$$2c9d7a9074166de3bf8b66cf7c45a3ed$$$$anon$1#3ed0eea6
scala> conv(List(1,2,3))mymap(_+1)
res40: List[Int] = List()
scala> conv(BitSet(1,2,3))mymap(_+1)
res41: scala.collection.immutable.BitSet = BitSet()
scala> conv(BitSet(1,2,3))mymap(_.toFloat)
res42: scala.collection.immutable.Set[Float] = Set()
scala> List(1,2,3)mymap(_+1)
<console>:168: error: Cannot prove that List[Int] <:< scala.collection.IterableLike[A,List[Int]].
List(1,2,3)mymap(_+1)
^
scala> implicit def conv[A, C](xs: C)(implicit ev: C => GenTraversable[A]) = new {
| def mymap[B,D](f: A => B)(implicit b: CanBuildFrom[GenTraversable[A],B,D]): D =
| xs map f
| }
conv: [A, C](xs: C)(implicit ev: C => scala.collection.GenTraversable[A])java.lang.Object{def mymap[B,D](f: A => B)(implicit b: scala.collection.generic.CanBuildFrom[scala.collection.GenTraversable[A],B,D]): D}
scala> conv(Array(1)) mymap (_+1)
res6: scala.collection.GenTraversable[Int] = ArrayBuffer(2)
scala> Array(1) mymap (_+1)
<console>:68: error: No implicit view available from Array[Int] => scala.collection.GenTraversable[A].
Array(1) mymap (_+1)
^
I've answered this very question about type inference just last week. Here's the code:
implicit def conv[A,C <: GenTraversable[A]](xs: C with GenTraversableLike[A,C]) = new {
def mymap[B,D](f: A => B)(implicit b: CanBuildFrom[C,B,D]): D = {
val builder = b(xs)
xs foreach { x => builder += f(x) }
builder.result
}
}
I could have used GenTraversable instead of GenTraversableLike in this particular case. I prefer the later because it offers more.
The problem is that declaring [A, C <: GenTraversable[A]] does not instruct Scala to infer the type of A from the type of C. Types are inferred based on how they are used in the parameters, and then checked against the boundaries specified by the type parameters.
So when I write xs: C with GenTraversable[A], I let Scala know it should infer A from xs. And writing GenTraversableLike[A, C] tells Scala it should pick a collection that returns C for methods that return the same collection. This means you can call filter and get C back, instead of getting GenTraversable back.
As for wishing to include views, that I don't know how you could accomplish.
I have answered a similar question here. You can also refer to this thread where Rex Kerr explains how to perform such pimping in general.