Variance of Scala List map function - scala

I have a question that's been bugging me.
Lists in Scala are covariant (List[+A])
Let's say we have these classes:
class A
class B extends A
The map function of List[B] takes a function f: B => C
But I can also use a f: A => C
which is a subclass of f: B => C
and it totally makes sense.
What I am currently confused by is that
the map function should accept only functions that are superclasses of the original function (since functions are contravariant on their arguments), which does not apply in the example I've given.
I know there's something wrong with my logic and I would like to enlightened.

Your error lies in the assumption that map(f: A => C) should only accept functions that are superclasses of A => C.
While in reality, map will accept any function that is a subclass of A => C.
In Scala, a function parameter can always be a subclass of the required type.
The covariance of A in List[A] only tells you that, wherever a List[A] is required, you can provide a List[B], as long as B <: A.
Or, in simpler words: List[B] can be treated as if it was a subclass of List[A].
I have compiled a small example to explain these two behaviours:
class A
class B extends A
// this means: B <: A
val listA: List[A] = List()
val listB: List[B] = List()
// Example 1: List[B] <: List[A]
// Note: Here the List[+T] is our parameter! (Covariance!)
def printListA(list: List[A]): Unit = println(list)
printListA(listA)
printListA(listB)
// Example 2: Function[A, _] <: Function[B, _]
// Note: Here a Function1[-T, +R] is our parameter (Contravariance!)
class C
def fooA(a: A): C = ???
def fooB(b: B): C = ???
listB.map(fooB)
listB.map(fooA)
Try it out!
I hope this helps.

As you already suspected, you are mixing up things here.
On the one hand, you have a List[+A], which tells us something about the relationships between List[A] and List[B], given a relationship between A and B. The fact that List is covariant in A just means that List[B] <: List[A] when B <: A, as you know already know.
On the other hand, List exposes a method map to change its "contents". This method does not really care about List[A], but only about As, so the variance of List is irrelevant here. The bit that is confusing you here is that there is indeed some sub-typing to take into consideration: map accepts an argument (a A => C in this case, but it's not really relevant) and, as usual with methods and functions, you can always substitute its argument with anything that is a subtype of it. In your specific case, any AcceptedType will be ok, as long as AcceptedType <: Function1[A,C]. The variance that matters here is Function's, not List's.

Related

Scala type inference rule over contravarience?

I'm using Circe and noticed something that i am not so confortable with and would like to understand what is going on under the hood ?
Fundamentally it is not really a circe issue. Also i was just playing with circe around to test few thing. So could have decoded in JsonObject straight but that is beside the point.
val jobjectStr = """{
| "idProperty": 1991264,
| "nIndex": 0,
| "sPropertyValue": "0165-5728"
| }""".stripMargin
val jobject = decode[Json](jobjectStr).flatMap{ json =>
json.as[JsonObject]
}
My issue is with the flapMap signature of Either, contravariance and what is happening here:
We have the following types:
decode[Json](jobjectStr): Either[Error, Json]
json.as[JsonObject]: Decoder.Result[JsonObject]
where circe defines
final type Result[A] = Either[DecodingFailure, A]
and
sealed abstract class DecodingFailure(val message: String) extends Error {
Now the signature of flatMap in either is:
def flatMap[A1 >: A, B1](f: B => Either[A1, B1]): Either[A1, B1]
In other words, talking only about type it is like my code is doing
Either[Error, Json] flatMap Either[DecodingFailure, JsonObject]
Hence my issue is: DecodingFailure >: Error is not true
And Indeed the type of the full expression is:
decode[Json](jobjectStr).flatMap{ json =>
json.as[JsonObject]
}: Either[Error, JsonObject]
Hence i'm confused, because my understanding is that the type of the first Parameter of Either is Contravariant in the flatMap Signature. Here there seems to be some wierd least upper bound inferencing going on ... But i am not sure why or if it is even the case.
Any explanation ?
This really isn't a variance issue. A1 >: A is just telling us that the result type, A1, might have to be a super-type of the received type, A, if the compiler has to go looking for a least upper bound (the LUB). (The use of A1 in the f: B => ... description is, I think, a bit confusing.)
Consider the following:
class Base
class SubA extends Base
class SubB extends Base
Either.cond(true, "a string", new SubA)
.flatMap(Either.cond(true, _, new SubB))
//res0: scala.util.Either[Base,String] = Right(a string)
Notice how the result is Either[Base,String] because Base is the LUB of SubA and SubB.
So first of all, we need to understand that the compiler will always try to infer types that allow compilation. The only real way to avoid something to compile is to use implicits.
(not sure if this is part of the language specification, or a compiler implementation detail, or something common to all compilers, or a bug or a feature).
Now, let's start with a simpler example List and ::.
sealed trait List[+A] {
def ::[B >: A](b: B): List[B] = Cons(b, this)
}
final case class Cons[+A](head: A, tail: List[A]) extends List[A]
final case object Nil extends List[Nothing]
So, assuming the compiler will always allow some code like x :: list will always compile. Then, we have three scenarios:
x is of type A and list is a List[A], so it is obvious that the returned value has to be of type List[A].
x is of some type C and list is a List[A], and C is a subtype of A (C <: A). Then, the compiler simply upcast x to be of type A and the process continues as the previous one.
x is of some type D and list is a List[A], and D is not a subtype of A. Then, the compiler finds a new type B which is the LUB between D and A, the compiler finally upcast both x to be of type B and list to be a List[B] (this is possible due covariance) and proceeds like the first one.
Also, note that due to the existence of types like Any and Nothing there is "always" a LUB between two types.
Now let's see Either and flatMap.
sealed trait Either[+L, +R] {
def flatMap[LL >: L, RR](f: R => Either[LL, RR]): Either[LL, RR]
}
final case class Left[+L](l: L) extends Either[L, Nothing]
final case clas Right[+R](r: R) extends Either[Nothing, R]
Now, assuming my left side is an Error, I feel this behaviour of returning the LUB between the two possible lefts is the best, since at the end I would have the first error, or the second error or the final value, so since I do not know which of the two errors it was then that error must be of some type that encapsulates both possible errors.

Difference between Ad-hoc polymorphism and Parametric polymorphism in Scala

So, I have been searching documentation about the main difference between parametric polymorphism and adhoc-polymorphism, but I still have some doubts.
For instance, methods like head in Collections, are clearly parametric polymorphism, since the code used to obtain the head in a List[Int] is the same as in any other List.
List[T] {
def head: T = this match {
case x :: xs => x
case Nil => throw new RuntimeException("Head of empty List.")
}
}
(Not sure if that's the actual implementation of head, but it doesn't matter)
On the other hand, type classes are considered adhoc-polymorphism. Since we can provide different implementations, bounded to the types.
trait Expression[T] {
def evaluate(expr: T): Int
}
object ExpressionEvaluator {
def evaluate[T: Expression](value: T): Int = implicitly[Expression[T]].evaluate(value)
}
implicit val intExpression: Expression[Int] = new Expression[Int] {
override def evaluate(expr: Int): Int = expr
}
ExpressionEvaluator.evaluate(5)
// 5
In the middle, we have methods like filter, that are parametrized, but we can provide different implementations, by providing different functions.
List(1,2,3).filter(_ % 2 == 0)
// List(2)
Are methods like filter, map, etc, considered ad-hoc polymorphism? Why or why not?
The method filter on Lists is an example of parametric polymorphism. The signature is
def filter(p: (A) ⇒ Boolean): List[A]
It works in exactly the same way for all types A. Since it can be parameterized by any type A, it's ordinary parametric polymorphism.
Methods like map make use of both types of polymorphism simultaneously.
Full signature of map is:
final def map[B, That]
(f: (A) ⇒ B)
(implicit bf: CanBuildFrom[List[A], B, That])
: That
This method relies on the presence of an implicit value (the CBF-gizmo), therefore it's ad-hoc polymorphism. However, some of the implicit methods that provide the CBFs of the right type are actually themselves parametrically polymorphic in types A and B. Therefore, unless the compiler manages to find some very special ad-hoc construct like an CanBuildFrom[List[String], Int, BitSet] in the implicit scope, it will sooner or later fall back to something like
implicit def ahComeOnGiveUpAndJustBuildAList[A, B]
: CanBuildFrom[List[A], B, List[B]]
Therefore, I think one could say that it's a kind-of "hybrid parametric-ad-hoc polymorphism" that first attempts to find the most appropriate ad-hoc type class CanBuildFrom[List[A], B, That] in the implicit scope, but eventually falls back to ordinary parametric polymorphism, and returns a one-fits-all CanBuildFrom[List[A], B, List[B]]-solution that is parametrically polymorphic in both A and B.

Scala Option type upper bound don't understand

I'm reading Functional Programming in Scala, and in chapter 04 the authors implement Option on their own. Now, when defining the function getOrElse they use an upper bound to restrict the type of A to a supertype (if a understood correctly)
So, the definition goes:
sealed trait Option[+A] {
def getOrElse[B >: A](default: => B): B = this match {
case None => default
case Some(a) => a
}
}
So, when we have something like
val a = Some(4)
println(a.getOrElse(None)) => println prints a integer value
val b = None
println(b.getOrElse(Some(3)) => println prints a Option[Integer] value
a has type Option[Int], so A would be type Int. B would be type Nothing. Nothing is a subtype of every other type. That means that Option[Nothing] is a subtype of Option[Int] (because of covariance), right?
But with B >: A we said that B has to be a supertype?! So how can we get an Int back? This is a bit confusing for me...
Anyone care to try and clarify?
That means that Option[Nothing] is a subtype of Option[Int] (because of covariance), right?
Correct. Option[Nothing] is an Option[Int].
But with B >: A we said that B has to be a supertype?! So how can we get an Int back?
It doesn't have to be a super-type. It just requires A as a lower-bound. Which means you can still pass Int to getOrElse if A is Int.
But that doesn't mean you can't pass instances of a sub-class. For instance:
class A
class B extends A
class C extends B
scala> Option(new B)
res196: Option[B] = Some(B#661f82ac)
scala> res196.getOrElse(new C)
res197: B = B#661f82ac
scala> res196.getOrElse(new A)
res198: A = B#661f82ac
scala> res196.getOrElse("...")
res199: Object = B#661f82ac
I can still pass an instance of C, because C can be up-cast to B. I can also pass a type higher up the inheritance tree, and getOrElse will return that type, instead. If I pass a type that has nothing to do with the type contained in the Option, then the type with the least upper-bound will be inferred. In the above case, it's Any.
So why is the lower-bound there at all? Why not have:
def getOrElse[B <: A](default: => B): B
This won't work because getOrElse must either return the A that's contained in the Option, or the default B. But if we return the A, and A is not a B, so the type-bound is invalid. Perhaps if getOrElse returned A:
def getOrElse[B <: A](default: => B): A
This would work (if it were really defined that way), but you would be restricted by the type-bounds. So in my above example, you could only pass B or C to getOrElse on an Option[B]. In any case, this is not how it is in the standard library.
The standard library getOrElse allows you to pass anything to it. Say you have Option[A]. If we pass a sub-type of A, then it is up-cast to A. If we pass A, obviously this is okay. And if we pass some other type, then the compiler infers the least upper-bound between the two. In all cases, the type-bound B >: A is met.
Because getOrElse allows you to pass anything to it, many consider it very tricky. For example you could have:
val number = "blah"
// ... lots of code
val result = Option(1).getOrElse(number)
And this will compile. We'll just have an Option[Any] that will probably cause an error somewhere else down the line.

Folding on Type without Monoid Instance

I'm working on this Functional Programming in Scala exercise:
// But what if our list has an element type that doesn't have a Monoid instance?
// Well, we can always map over the list to turn it into a type that does.
As I understand this exercise, it means that, if we have a Monoid of type B, but our input List is of type A, then we need to convert the List[A] to List[B], and then call foldLeft.
def foldMap[A, B](as: List[A], m: Monoid[B])(f: A => B): B = {
val bs = as.map(f)
bs.foldLeft(m.zero)((s, i) => m.op(s, i))
}
Does this understanding and code look right?
First I'd simplify the syntax of the body a bit:
def foldMap[A, B](as: List[A], m: Monoid[B])(f: A => B): B =
as.map(f).foldLeft(m.zero)(m.ops)
Then I'd move the monoid instance into its own implicit parameter list:
def foldMap[A, B](as: List[A])(f: A => B)(implicit m: Monoid[B]): B =
as.map(f).foldLeft(m.zero)(m.ops)
See the original "Type Classes as Objects and Implicits" paper for more detail about how Scala implements type classes using implicit parameter resolution, or this answer by Rex Kerr that I've also linked above.
Next I'd switch the order of the other two parameter lists:
def foldMap[A, B](f: A => B)(as: List[A])(implicit m: Monoid[B]): B =
as.map(f).foldLeft(m.zero)(m.ops)
In general you want to place the parameter lists containing parameters that change the least often first, in order to make partial application more useful. In this case there may only be one possible useful value of A => B for any A and B, but there are lots of values of List[A].
For example, switching the order allows us to write the following (which assumes a monoid instance for Bar):
val fooSum: List[Foo] => Bar = foldMap(fooToBar)
Finally, as a performance optimization (mentioned by stew above), you could avoid creating an intermediate list by moving the application of f into the fold:
def foldMap[A, B](f: A => B)(as: List[A])(implicit m: Monoid[B]): B =
as.foldLeft(m.zero) {
case (acc, a) => m.op(acc, f(a))
}
This is equivalent and more efficient, but to my eye much less clear, so I'd suggest treating it like any optimization—if you need it, use it, but think twice about whether the gains are really worth the loss of clarity.

Why does the scaladoc say HashMap.toArray returns Array[A] instead of Array[(A,B)]?

I was looking at the definition of toArray for hashmaps :
http://www.scala-lang.org/api/current/index.html#scala.collection.immutable.HashMap
It has
toArray: Array[A]
def toArray[B >: (A, B)](implicit arg0: ClassTag[B]): Array[B]
I don't quite understand this - the first bit says you get an Array[A], but the second part says you get Array[B]? Neither of these are what I expect - Array[(A,B)]
When I check it myself :
scala> val x = scala.collection.mutable.HashMap[String, Int]()
x: scala.collection.mutable.HashMap[String,Int] = Map()
scala> x.put("8", 7)
res0: Option[Int] = None
scala> x foreach println
(8,7)
scala> x.toArray
res2: Array[(String, Int)] = Array((8,7))
why isn't it like toList?
toList: scala.List[(A, B)]
The scaladoc has all kinds of subtle bugs. The problem here is that you are seeing the "simplified" version of the method signature (meant as a way to convey the essential part of the signature and hide things such as CanBuildFrom in map/flatMap methods, which are really an implementation detail).
The simplification got a little awry here, and does not seem to make much sense.
If you click on the "full signature" link, you'll see that the real signature looks like:
def toArray[B >: (A, B)](implicit arg0: ClassTag[B]): Array[B]
In fact this is still wrong, as we certainly cannot have a type B where B >: (A, B). It should be more like :
def toArray[C >: (A, B)](implicit arg0: ClassTag[C]): Array[C]
The problem is that there are actually two Bs: the first one comes from the HashMap class declaration itself (HashMap[A, +B]) while the other one comes from the methods toArray defined in its base class
TraversableOnce (def toArray[B >: A](implicit arg0: ClassTag[B]): Array[B]). It just happens that the scaladoc generator failed to dedup the two instances of B
The API you see in the the Scaladoc of toArray:
def toArray[B >: (A, B)](implicit arg0: ClassTag[B]): Array[B]
Is equivalent to:
def toArray[C >: (A, B)](implicit arg0: ClassTag[C]): Array[C]
The choice of the type variable B is indeed unfortunate (and maybe even a Scaladoc bug, I'm not sure if you are allowed to write that).
It basically means you'll get an array of the most specific supertype of (A,B) for which a ClassTag is available. The ClassTag is required in order to create the Array.
This basically means that if at compile-time, the-run time type of the Map you are converting is fully known, you will get a Array[(A,B)]. However, if you have up-casted your Map somewhere, the run-time type of the resulting Array will depend on the up-casted type, and not on the runtime type. This is different behavior than toList and due to the JVMs restrictions on how native arrays can be created.
The Scaladoc is just wrong because it inherits toArray from TraversableOnce, where the type of the collection is A and the return value is B. The Array[A] thing is left over from TraversableOnce where A is whatever TraversableOnce is traversing (in this case, actually (A,B) for a different definition of A and B); and although it fills the (A,B) in properly in the long form, it still uses B as the new return variable instead of a different letter like C.
Kind of confusing! It actually should read
def toArray[C >: (A,B)](...[C]): Array[C]
and the short form should be
toArray: Array[(A,B)]
just like you expect.