I am sorry for such non-descriptive title, but I really don't know how to express this better.
class Foo[T]
Seq(new Foo[String], new Foo[Int]).groupBy(_ => 1).map { case (k, Seq(v)) => k -> v }.toMap
<console>:12: error: Cannot prove that (Int, Foo[_146]) forSome { type _146 >: Int with String } <:< (T, U).
WTF?
If I use .mapValues instead of .map, it works. Also, making Foo covariant fixes it too, but in that case I end up with Map[Int,Foo[Any]]
What's going on here? Any ideas?
Without variance, you create a somewhat "nonsensical" sequence:
class Foo[T]
val in = Seq(new Foo[String], new Foo[Int]) // Seq[_ >: Int with String]]
There is simply no common LUB between Foo[String] and Foo[Int]. You may assign it an existential type:
val in = Seq[Foo[_]](new Foo[String], new Foo[Int])
Then we can try to continue:
val in = Seq[Foo[_]](new Foo[String], new Foo[Int])
val g = in.groupBy(_ => 1) // Map[Int, Seq[Foo[_]]]
// the next line would produce a match error, thus make it a `def`
def m = g.map { case (k, Seq(v)) => k -> v } // Iterable[(Int, Foo[_])]
def p = m.toMap // cannot prove that (Int, Foo[_]) <:< (T, U)
Again the existential type bites you here in disallowing a useful inference for the value type. You can enforce it again:
def p = m.toMap[Int, Foo[_]] // Map[Int,Foo[_]]
AFAIK, Scalac will not infer existential types for you.
If you are thinking you have Foo[Any] here, you need to add a variance annotation:
class Foo[+T]
val in = Seq(new Foo[String], new Foo[Int]) // Seq[Foo[Any]]
def m = in.groupBy(_=>1).map {case (k,Seq(v)) => k->v}.toMap // Map[Int,Foo[Any]]
Related
I've implemented a custom collection class which is basically a Map with implicit integer keys and values that are subclasses of AnyRef. It uses the Int keys as index for underlying array structure. Here is the class declaration signature (class instantiation is done in companion object, hence private constructor):
class ArrayMap[T >: Null <: AnyRef: ClassTag] private (private var data: Array[T]) { self =>
...
}
Now I want to add required methods for for-comprehension. I've defined two different map functions. One that returns a List and the other one returns the same data type (ArrayMap).
def map[X](f: (Int, T) => X): List[X] = { ... }
def map[X >: Null <: AnyRef: ClassTag](f: (Int, T) => X): ArrayMap[X] = { ... }
def foreach(f: (Int, T) => Unit): Unit = { ... }
def flatMap[X >: Null <: AnyRef: ClassTag](f: (Int, T) => Iterable[(Int, X)]): ArrayMap[X] = { ... }
def filter(p: (Int, T) => Boolean): ArrayMap[T] = { ... }
No implicit is defined. Above functions work as expected when used separately. The problem is in for-comprehensions. For loop either picks the first map which returns List or throws a mysterious error. The following example produces error:
val map = ArrayMap.empty[Integer]
map(0) = 0
map(1) = 1
map(5) = 2
map(6) = 3
map(10) = 4
val rs: ArrayMap[String] = for (e <- map) yield e._2.toString
Above code throws:
Error:(293, 41) missing parameter type
val rs: ArrayMap[String] = for (e <- map) yield e._2.toString
What am I missing?
[UPDATE]
The full implementation is available as a gist here.
The problem is related to a type mismatch, you defined the function to pass to map as a function of two arguments (Int & T) to X. while in your for comprehension you treat it as a function of one argument (a tuple (Int, T)) to X.
The simplest solution is to redefine your map function signature. e.g.
import scala.reflect.ClassTag
class ArrayMap[T >: Null <: AnyRef: ClassTag] (val data: Array[T]) {
// Note the double parenthesis (()).
def map[X >: Null <: AnyRef: ClassTag](f: ((Int, T)) => X): ArrayMap[X] = ???
def withFilter(p: ((Int, T)) => Boolean): ArrayMap[T] = ???
}
With that definition you can make something like
val map: ArrayMap[java.lang.Integer] = new ArrayMap(Array(1, 2, 3))
// Note I use lazy val to avoid the NotImplementedException.
lazy val rs1: ArrayMap[String] = map.map(tuple => tuple._2.toString)
lazy val rs2: ArrayMap[String] = map.map { case (_, v) => v.toString }
lazy val rs3: ArrayMap[String] = for {
tuple <- map
} yield tuple._2.toString
lazy val rs4: ArrayMap[String] = for {
(_, v) <- map
} yield v.toString
See the full signature of map in Scala Map as a reference.
def isEven(x: Int) =
if (x % 2 == 0) x.successNel else "not even".failureNel
In the above code, the return type of isEven is inferred as scalaz.Validation[scalaz.NonEmptyList[_ <: String],Int]
Whereas explicitly specifying the return type works,
def isEven(x: Int): ValidationNel[String, Int] =
if (x % 2 == 0) x.successNel else "not even".failureNel
Can someone explain what's going on behind the scenes here? Is this a limitation of Scala's type inference system?
I'm trying to use the above function in the following expression,
(isEven(4) |#| isEven(6) |#| isEven(8) |#| isEven(10)) {_ + _ + _ + _}
and only the second variant (with explicit return type) works.
scalaz.NonEmptyList[String] is a sub type of scalaz.NonEmptyList[_ <: String], meaning scalaz.ValidationNEL[String,Int] is a sub type of scalaz.Validation[scalaz.NonEmptyList[_ <: String],Int]. The compiler just give you a more precise type in this case...
Edit: the raisin bread operator requires an Applicative instance. In this case I suppose that scalaz provide such instance for ValidationNel[String, ?], but not for the more precise type inferred for your isEven.
A simpler manifestation of that can be observed with Option:
val some = Some(1)
val none = None
(some |#| none)(_ + _) // Cannot find implicit Applicative[Some] :(
You can view that as an intersect limitation of local type inference. You could also say that FP doesn't always play well with OO. I would suggest following good practices and annotate all your public method and implicits with explicit types.
I will try to explain more details about :
why scalaz.Validation[scalaz.NonEmptyList[_ <: String],Int] this can not work with |#|?
isEven(4) |#| isEven(6) ...
First of all isEven(4) is trying to implicitly convert it from Validation[+E, +A] type to ApplyOps[F0.M,F0.A]type with |#| operator. as the implicit method in ApplySyntax:
implicit def ToApplyOpsUnapply[FA](v: FA)(implicit F0: Unapply[Apply, FA]) =
new ApplyOps[F0.M,F0.A](F0(v))(F0.TC)
As the above code, for this implicit conversion, we also need an implicitly F0: Unapply[Apply, FA] variable. for UnApply implicits, it's in the UnApply:
implicit def unapplyMFA[TC[_[_]], M0[_[_], _], F0[_], A0](implicit TC0: TC[M0[F0, ?]]): Unapply[TC, M0[F0, A0]] {
type M[X] = M0[F0, X]
type A = A0
} =
new Unapply[TC, M0[F0, A0]] {
type M[X] = M0[F0, X]
type A = A0
def TC = TC0
def leibniz = refl
}
As the Validation type, it's using the unapplyMFA implicits variable. In there we also found it's looking for another implicits TC0: TC[M0[F0, ?]] variable. as the before ToApplyOpsUnapply, In there TC0 will be Apply type, that's also can be Applicative type. so it will try to look for the Validation to Applicative implicits, it's in the Validation.scala
implicit def ValidationApplicative[L: Semigroup]: Applicative[Validation[L, ?]] =
new Applicative[Validation[L, ?]] {
override def map[A, B](fa: Validation[L, A])(f: A => B) =
fa map f
def point[A](a: => A) =
Success(a)
def ap[A, B](fa: => Validation[L, A])(f: => Validation[L, A => B]) =
fa ap f
}
for the Semigroup definition:
trait Semigroup[F] { self => //F is invariant, unlike Option[+A]
So the problem is: F is type invariant, not like the Option[+A], compiler could not find an appropriate Applicative implicits for this type Validaiton.
There is a small demo for how to enable type variant for this:
def isEven(x: Int): Validation[NonEmptyList[_ <: String], Int] =
if (x % 2 == 0) x.successNel else "not even".failureNel
val res: String = foo(isEven(4))
def foo[FA](v: FA)(implicit F0: Unapply[Apply, FA]): String = "foo bar"
implicit def ValidationApplicative[L]: Applicative[Validation[L, ?]] =
new Applicative[Validation[L, ?]] {
override def map[A, B](fa: Validation[L, A])(f: A => B) =
fa map f
def point[A](a: => A) =
Success(a)
def ap[A, B](fa: => Validation[L, A])(f: => Validation[L, A => B]) =
//fa ap f
null
}
In there we just unbind L from Semigroup, so now this is type variant. just for fun;).
I asked this question earlier: Combine a PartialFunction with a regular function
and then realized, that I haven't actually asked it right.
So, here goes another attempt.
If I do this:
val foo = PartialFunction[Int, String] { case 1 => "foo" }
val bar = foo orElse { case x => x.toString }
it does not compile: error: missing parameter type for expanded function
The argument types of an anonymous function must be fully known. (SLS 8.5)
Expected type was: PartialFunction[?,?]
But this works fine:
val x: Seq[String] = List(1,2,3).collect { case x => x.toString }
The question is what is the difference? The type of the argument is the same in both cases: PartialFunction[Int, String]. The value passed in is literally identical. Why one does one case work, but not the other?
You need to specify the type for bar because the compiler is unable to infer it. This compiles:
val foo = PartialFunction[Int, String] { case 1 => "foo" }
val bar : (Int => String) = foo orElse { case x => x.toString }
In the case of List(1,2,3).collect{case x => x.toString} the compiler is able to infer the input type of the partial function based off of how theList was typed.
final override def collect[B, That](pf: PartialFunction[A, B])(implicit bf: CanBuildFrom[List[A], B, That])
Based on the type parameters the compiler can infer that you are passing a correctly typed partial function. That's why List(1,2,3).collect{case x:String => x.toString} does not compile nor does List(1,2,3).collect{case x:Int => x.toString; case x: String => x.toString}.
Since List is covariant the compiler is able to infer that the partial function {case x => x.toString} is a partial function on Int. You'll notice that List(1,2,3).collect{case x => x.length} does not compile because the compiler is inferring that you're operating on either an Int or a subclass of Int.
Also keep in mind that the {case x => x.toString} is just syntactic sugar. If we do something like the below then your example works as expected
val f = new PartialFunction[Int, String](){
override def isDefinedAt(x: Int): Boolean = true
override def apply(v1: Int): String = v1.toString
}
val foo = PartialFunction[Int, String] { case 1 => "foo" }
val bar = foo orElse f //This compiles fine.
List(1,2,3).collect{f} // This works as well.
So the only logical answer from my perspective is that the syntactic sugar that is able to generate a PartialFunction instance for {case x => x.toString} does not have enough information at compile time to be able to adequately type it as a PartialFunction[Int, String] in your orElse case.
You can use the library Extractor.scala.
import com.thoughtworks.Extractor._
// Define a PartialFunction
val pf: PartialFunction[Int, String] = {
case 1 => "matched by PartialFunction"
}
// Define an optional function
val f: Int => Option[String] = { i =>
if (i == 2) {
Some("matched by optional function")
} else {
None
}
}
// Convert an optional function to a PartialFunction
val pf2: PartialFunction[Int, String] = f.unlift
util.Random.nextInt(4) match {
case pf.extract(m) => // Convert a PartialFunction to a pattern
println(m)
case f.extract(m) => // Convert an optional function to a pattern
println(m)
case pf2.extract(m) => // Convert a PartialFunction to a pattern
throw new AssertionError("This case should never occur because it has the same condition as `f.extract`.")
case _ =>
println("Not matched")
}
The following Scala code works:
object ReducerTestMain extends App {
type MapOutput = KeyVal[String, Int]
def mapFun(s:String): MapOutput = KeyVal(s, 1)
val red = new ReducerComponent[String, Int]((a: Int, b: Int) => a + b)
val data = List[String]("a", "b", "c", "b", "c", "b")
data foreach {s => red(mapFun(s))}
println(red.mem)
// OUTPUT: Map(a -> 1, b -> 3, c -> 2)
}
class ReducerComponent[K, V](f: (V, V) => V) {
var mem = Map[K, V]()
def apply(kv: KeyVal[K, V]) = {
val KeyVal(k, v) = kv
mem += (k -> (if (mem contains k) f(mem(k), v) else v))
}
}
case class KeyVal[K, V](key: K, value:V)
My problem is I would like to instantiate ReducerComponent like this:
val red = new ReducerComponent[MapOutput, Int]((a: Int, b: Int) => a + b)
or even better:
val red = new ReducerComponent[MapOutput](_ + _)
That means a lot of things:
I would like to type-check that MapOutput is of the type KeyVal[K, C],
I want to type-check that C is the same type used in f,
I also need to "extract" K in order to instantiate mem, and type-check parameters from apply.
Is it a lot to ask? :) I wanted to write something like
class ReducerComponent[KeyVal[K,V]](f: (V, V) => V) {...}
By the time I will instantiate ReducerComponent all I have is f and MapOutput, so inferring V is OK. But then I only have KeyVal[K,V] as a type parameter from a class, which can be different from KeyVal[_,_].
I know what I'm asking is probably crazy if you understand how type inference works, but I don't! And I don't even know what would be a good way to proceed --- apart from making explicit type declarations all the way up in my high-level code. Should I just change all the architecture?
Just write a simple factory:
case class RC[M <: KeyVal[_, _]](){
def apply[K,V](f: (V,V) => V)(implicit ev: KeyVal[K,V] =:= M) = new ReducerComponent[K,V](f)
}
def plus(x: Double, y: Double) = x + y
scala> RC[KeyVal[Int, Double]].apply(plus)
res12: ReducerComponent[Int,Double] = ReducerComponent#7229d116
scala> RC[KeyVal[Int, Double]]()(plus)
res16: ReducerComponent[Int,Double] = ReducerComponent#389f65fe
As you can see, ReducerComponent has appropriate type. Implicit evidence is used here to catch K and V from your M <: KeyVal[_, _].
P.S. The version above requires to specify parameter types explicitly for your f, like (_: Double) + (_: Double). If you want to avoid this:
case class RC[M <: KeyVal[_, _]](){
def factory[K,V](implicit ev: KeyVal[K,V] =:= M) = new {
def apply(f: (V,V) => V) = new ReducerComponent[K,V](f)
}
}
scala> RC[KeyVal[Int, Double]].factory.apply(_ + _)
res5: ReducerComponent[Int,Double] = ReducerComponent#3dc04400
scala> val f = RC[KeyVal[Int, Double]].factory
f: AnyRef{def apply(f: (Double, Double) => Double): ReducerComponent[Int,Double]} = RC$$anon$1#19388ff6
scala> f(_ + _)
res13: ReducerComponent[Int,Double] = ReducerComponent#24d8ae83
Update. If you want to generelize keyval - use type function:
type KV[K,V] = KeyVal[K,V] //may be anything, may implement `type KV[K,V]` from some supertrait
case class RC[M <: KV[_, _]](){
def factory[K,V](implicit ev: KV[K,V] =:= M) = new {
def apply(f: (V,V) => V) = new ReducerComponent[K,V](f)
}
}
But keep in mind that apply from your question still takes KeyVal[K,V].
You can also pass KV into some class:
class Builder[KV[_,_]] {
case class RC[M <: KV[_, _]](){
def factory[K,V](implicit ev: KV[K,V] =:= M) = new {
def apply(f: (V,V) => V) = new ReducerComponent[K,V](f)
}
}
}
scala> val b = new Builder[KeyVal]
scala> val f = b.RC[KeyVal[Int, Double]].factory
scala> f(_ + _)
res2: ReducerComponent[Int,Double] = ReducerComponent#54d9c993
You will need path-dependent types for this. I recommend the following:
First, write a trait that has your relevant types as members, so you can access them within definitions:
trait KeyValAux {
type K
type V
type KV = KeyVal[K, V]
}
Now you can create a factory for ReducerComponent:
object ReducerComponent {
def apply[T <: KeyValAux](f: (T#V, T#V) => T#V) =
new ReducerComponent[T#K, T#V](f)
}
Note that here, we can simply access the members of the type. We can't do this for type parameters.
Now, define your MapOutput in terms of KeyValAux (maybe another name is more appropriate for your use case):
type MapOutput = KeyValAux { type K = String; type V = Int }
def mapFun(s:String): MapOutput#KV = KeyVal(s, 1)
val red = ReducerComponent[MapOutput](_ + _)
UPDATE
As #dk14 mentions in the comments, if you still want the type-parameter syntax, you could do the following:
trait OutputSpec[KK, VV] extends KeyValAux {
type K = KK
type V = VV
}
You can then write:
type MapOutput = OutputSpec[String, Int]
Alternatively, you can write OutputSpec as a type function:
type OutputSpec[KK, VV] = KeyValAux { type K = KK; type V = VV }
This will not generate an additional unused class.
I would like to map the elements of a Scala tuple (or triple, ...) using a single function returning type R. The result should be a tuple (or triple, ...) with elements of type R.
OK, if the elements of the tuple are from the same type, the mapping is not a problem:
scala> implicit def t2mapper[A](t: (A,A)) = new { def map[R](f: A => R) = (f(t._1),f(t._2)) }
t2mapper: [A](t: (A, A))java.lang.Object{def map[R](f: (A) => R): (R, R)}
scala> (1,2) map (_ + 1)
res0: (Int, Int) = (2,3)
But is it also possible to make this solution generic, i.e. to map tuples that contain elements of different types in the same manner?
Example:
class Super(i: Int)
object Sub1 extends Super(1)
object Sub2 extends Super(2)
(Sub1, Sub2) map (_.i)
should return
(1,2): (Int, Int)
But I could not find a solution so that the mapping function determines the super type of Sub1 and Sub2. I tried to use type boundaries, but my idea failed:
scala> implicit def t2mapper[A,B](t: (A,B)) = new { def map[X >: A, X >: B, R](f: X => R) = (f(t._1),f(t._2)) }
<console>:8: error: X is already defined as type X
implicit def t2mapper[A,B](t: (A,B)) = new { def map[X >: A, X >: B, R](f: X => R) = (f(t._1),f(t._2)) }
^
<console>:8: error: type mismatch;
found : A
required: X
Note: implicit method t2mapper is not applicable here because it comes after the application point and it lacks an explicit result type
implicit def t2mapper[A,B](t: (A,B)) = new { def map[X >: A, X >: B, R](f: X => R) = (f(t._1),f(t._2)) }
Here X >: B seems to override X >: A. Does Scala not support type boundaries regarding multiple types? If yes, why not?
I think this is what you're looking for:
implicit def t2mapper[X, A <: X, B <: X](t: (A,B)) = new {
def map[R](f: X => R) = (f(t._1), f(t._2))
}
scala> (Sub1, Sub2) map (_.i)
res6: (Int, Int) = (1,2)
A more "functional" way to do this would be with 2 separate functions:
implicit def t2mapper[A, B](t: (A, B)) = new {
def map[R](f: A => R, g: B => R) = (f(t._1), g(t._2))
}
scala> (1, "hello") map (_ + 1, _.length)
res1: (Int, Int) = (2,5)
I’m not a scala type genius but maybe this works:
implicit def t2mapper[X, A<:X, B<:X](t: (A,B)) = new { def map[A, B, R](f: X => R) = (f(t._1),f(t._2)) }
This can easily be achieved using shapeless, although you'll have to define the mapping function first before doing the map:
object fun extends Poly1 {
implicit def value[S <: Super] = at[S](_.i)
}
(Sub1, Sub2) map fun // typed as (Int, Int), and indeed equal to (1, 2)
(I had to add a val in front of i in the definition of Super, this way: class Super(val i: Int), so that it can be accessed outside)
The deeper question here is "why are you using a Tuple for this?"
Tuples are hetrogenous by design, and can contain an assortment of very different types. If you want a collection of related things, then you should be using ...drum roll... a collection!
A Set or Sequence will have no impact on performance, and would be a much better fit for this kind of work. After all, that's what they're designed for.
For the case when the two functions to be applied are not the same
scala> Some((1, "hello")).map((((_: Int) + 1 -> (_: String).length)).tupled).get
res112: (Int, Int) = (2,5)
The main reason I have supplied this answer is it works for lists of tuples (just change Some to List and remove the get).