How does the Scala compiler synthesize implicit evidence with `<:<`? - scala

Given this (admittedly contrived) code fragment in Scala:
object Main extends App {
class X { def foo = 1 }
def f[A](value: A)(implicit ev: A <:< X) = { value.foo }
println(f(new X()))
}
What does the Scala compiler do to make this pass? I have looked at some code in Predef but I don't understand the implementation. Please give a detailed step by step explanation.

Callsite
Let's look at what the type inferencer does when you write:
f(new X())
It first has to figure out, what the template parameter A of f is. Type inference in Scala goes left to right in argument lists, so trivially (given new X is of type X), we get
f[X](new X)
Now the compiler needs to find an implicit value of type X <:< X (remember, A got resolved to X).
To find implicit values, the compiler looks in various places, amongst others your current scope (in which Predef._ is imported).
The compiler then finds Predef.$conforms:
implicit def $conforms[A]: A <:< A = // some implementation
So this can be used to produce a X <:< X, by invoking it with X as parameter:
f[X](new X)(Predef.$conforms[X])
The actual implementation of $conforms does not matter as far as the type checker is concerned.
Method Implementation
Now lets look at the implementation:
def f[A](value: A)(implicit ev: A <:< X) = { value.foo }
Value is of type A (so something unknown). You want to call foo on value. Since foo is not defined on A, the compiler is looking for an implicit function (or method) that converts A into something that has a foo.
There is such a thing in scope: ev (A <:< B extends A => B).
Therefore, the compiler inserts an implicit conversion using ev:
ev(value).foo
Small Note About Variance
As you might have noticed, <:< is variant in its parameters: <:<[-From, +To]. This can be used to generate actual subtyping evidences. Consider:
class A
class B extends A
val ev1: A <:< A = conforms
val ev2: B <:< A = ev1 // makes sense, works because of variance
// Also
val ev3: B <:< B = conforms
val ev4: B <:< A = ev3 // makes sense, works because of variance
This is notably the reason, why there is no need for a conforms method with two type parameters. Further, note that this behavior is specifically not wanted for =:= (since this is type equivalence), so it is invariant.

Related

Type parameter under self-type doesn't conform to upper bound despite evidence

I have a trait with a self-type annotation that has a type parameter. This trait is from a library and cannot be modified. I want to pass this trait to a function that will require an upper bound for the type parameter. For example, I have this code snippet:
sealed trait Job[K] { self =>
type T
}
case class Encoder[T <: Product]()
def encoder(job: Job[_])(implicit ev: job.T <:< Product): Encoder[job.T] =
new Encoder[job.T]()
This returns an error that Type argument job.T does not conform to upper bound Product and a warning that ev is never used. How should I design the encoder function?
Why it doesn't work?
Your issue has nothing to do with the generalized type constraint. You can remove it and still get the same error. A generalized type constraint is used to constrain the type of arguments the method can receive.
(implicit ev: job.T <:< Product) provides an evidence in scope that matches only if job.T <: Product, allowing only calls to the method with Job arguments where job.T <: Product. This is its purpose.
Your issue is because the Encoder class has its type parameter T <: Product. The generalized type constraint does not treat the type job.T itself as a subtype of Product, as you expected. The evidence only applies to value arguments, not to the type itself, because this is how implicit conversions work.
For example, assuming a value x of type job.T that can be passed to the method as an argument:
def encoder(job: Job[_])(x: job.T)(implicit ev: job.T <:< Product): Unit = {
val y: Product = x // expands to: ev.apply(x)
val z: Encoder[Product] = new Encoder[job.T] // does not compile
}
The first line compiles because x is expanded to ev.apply(x), but the second one cannot be expanded, regardless if Encoder is covariant or not.
First workaround
One workaround you can do is this:
def encoder[U <: Product](job: Job[_])(implicit ev: job.T <:< Product): Encoder[U] =
new Encoder[U]()
The problem with this is that while both type parameters U and T are subtypes of Product, this definition does not says much about the relation between them, and the compiler (and even Intellij) will not infer the correct resulting type, unless you specify it explicitly. For example:
val myjob = new Job[Int] {
type T = (Int, Int)
}
val myencoder: Encoder[Nothing] = encoder(myjob) // infers type Nothing
val myencoder2: Encoder[(Int, Int)] = encoder[(Int, Int)](myjob) // fix
But why use job.T <:< Product if we already have U <: Product. We can instead use the =:= evidence to make sure their types are equal.
def encoder[U <: Product](job: Job[_])(implicit ev: job.T =:= U): Encoder[U] =
new Encoder[U]()
Now the resulting type will be correctly inferred.
Second workaround
A shorter workaround is using a structural type instead:
def encoder(job: Job[_] { type T <: Product }): Encoder[job.T] =
new Encoder[job.T]()
Which is not only cleaner (doesn't require a generalized type constraint), but also avoids the earlier problem.
Both versions work on Scala 2.13.8.
Expanding on Alin's answer, you may also use a type alias to express the same thing like this:
type JobProduct[K, P <: Product] = Job[K] { type T = P }
// Here I personally prefer to use a type parameter rather than an existential
// since I have had troubles with those, but if you don't find issues you may just use
// JobProdut[_, P] instead and remove the K type parameter.
def encoder[K, P <: Product](job: JobProduct[K, P]): Encoder[P] =
new Encoder[P]()
This approach may be more readable to newcomers and allows reuse; however, is essentially the same as what Alin did.

Scala: Why is an abstract type member equal to a type parameter by =:= not equal in use?

I am starting to embrace abstract type members over type parameters - mainly because they seem to work better with type inference for me. However, I am still struggling to understand how to use them from outside of the types they are defined in. For example, I cannot understand why the following Scala program should not compile:
trait Thing
trait Describer {
type X<:Thing
def describe(x:X) = println(x)
}
object Program extends App {
def print[T <: Thing, D <: Describer]
(describer: D, thing:T)
(implicit ev: D#X =:= T)
= describer.describe(thing)
}
Intuitively, I would expect that the requirement D#X =:= T would guarantee that the two types are indeed equal and hence that instances of two could be used interchangeably, but I get this compilation error:
error: type mismatch;
found : thing.type (with underlying type T)
required: describer.X
= describer.describe(thing)
What have I misunderstood? Is there another way to do what I want to do? Or failing that, is it safe to cast thing to the required type (describer.X)?
The type of the parameter of describer.describe is actually describer.X and not D#X. Another thing you have to know is that A =:= B also functions as a conversion from A to B but (at least currently) not the other way around. So the following should work.
def print[T <: Thing]
(describer: Describer, thing: T)
(implicit ev: T =:= describer.X)
= describer.describe(thing)

How does the <:< operator work in Scala?

In Scala there's a class <:< that witnesses a type constraint. From Predef.scala:
sealed abstract class <:<[-From, +To] extends (From => To) with Serializable
private[this] final val singleton_<:< = new <:<[Any,Any] { def apply(x: Any): Any = x }
implicit def $conforms[A]: A <:< A = singleton_<:<.asInstanceOf[A <:< A]
An example of how it's used is in the toMap method of TraversableOnce:
def toMap[T, U](implicit ev: A <:< (T, U)): immutable.Map[T, U] =
What I don't understand is how this works. I understand that A <:< B is syntactically equivalent to the type <:<[A, B]. But I don't get how the compiler can find an implicit of that type if and only if A <: B. I assume that the asInstanceOf call in the definition of $conforms is making this possible somehow, but how? Also, is it significant that a singleton instance of an abstract class is used, instead of just using an object?
Suppose we've got the following simple type hierarchy:
trait Foo
trait Bar extends Foo
We can ask for proof that Bar extends Foo:
val ev = implicitly[Bar <:< Foo]
If we run this in a console with -Xprint:typer, we'll see the following:
private[this] val ev: <:<[Bar,Foo] =
scala.this.Predef.implicitly[<:<[Bar,Foo]](scala.this.Predef.$conforms[Bar]);
So the compiler has picked $conforms[Bar] as the implicit value we've asked for. Of course this value has type Bar <:< Bar, but because <:< is covariant in its second type parameter, this is a subtype of Bar <:< Foo, so it fits the bill.
(There's some magic involved here in the fact that the Scala compiler knows how to find subtypes of the type it's looking for, but it's a fairly generic mechanism and isn't too surprising in its behavior.)
Now suppose we ask for proof that Bar extends String:
val ev = implicitly[Bar <:< String]
If you turn on -Xlog-implicits, you'll see this:
<console>:9: $conforms is not a valid implicit value for <:<[Bar,String] because:
hasMatchingSymbol reported error: type mismatch;
found : <:<[Bar,Bar]
required: <:<[Bar,String]
val ev = implicitly[Bar <:< String]
^
<console>:9: error: Cannot prove that Bar <:< String.
val ev = implicitly[Bar <:< String]
^
The compiler tries the Bar <:< Bar again, but since Bar isn't a String, this isn't a subtype of Bar <:< String, so it's not what we need. But $conforms is the only place the compiler can get <:< instances (unless we've defined our own, which would be dangerous), so it quite properly refuses to compile this nonsense.
To address your second question: the <:<[-From, +To] class is necessary because we need the type parameters for this type class to be useful. The singleton Any <:< Any value could just as well be defined as an object—the decision to use a val and an anonymous class is arguably a little simpler, but it's an implementation detail that you shouldn't ever need to worry about.

Why do lower type bounds change the variance position?

The Scala Language Specification (Section 4.5 on Variance Annotations, p. 44) says
The variance position of a type parameter is the opposite of the variance position
of the enclosing type parameter clause.
The variance position of the lower bound of a type declaration or type parameter
is the opposite of the variance position of the type declaration or parameter.
Using the first point above, it is easy to see (at least formally) that
trait Covariant[+A] {
def problematic[B <: A](x : B)
}
produces the error message
error: covariant type A occurs in contravariant position in type >: Nothing <: A of type B
def problematic[B <: A](x : B)
and using the first and the second point it is easy to see that
trait Contravariant[-A] {
def problematic[B >: A](x : B)
}
produces the error message
error: contravariant type A occurs in covariant position in type >: A <: Any of type B
def problematic[B >: A](x : B)
As I mention, it's easy to see formally (i.e., following the rules for variance annotations) why these errors occur. However, I can not come up with an example illustrating the need for these restrictions. In contrast, it is very easy to come up with examples that illustrate why method parameters should change variance positions, see e.g. Checking Variance Annotations.
So, my question is the following: Assuming, the two pieces of codes above were allowed, what are the examples of problems that arise? This means, I'm looking for examples similar to this one that illustrate what could go wrong in case the two rules cited above were not used. I'm particularly interested in the example involving lower type bounds.
Note that the answer to Scala type bounds & variance leaves this particular question open, whereas the answer given in The "lower bound" will reverse the variance of a type, but why? seems wrong to me.
Edit: I think the first case can be handled as follows (adapting the example cited above). Assume, the following was allowed
trait Queue[+T] {
def head : T
def tail : Queue[T]
def enqueue[U <: T](x : U) : Queue[T]
}
Then we could implement
class QueueImplementation[+T] extends Queue[T] {
/* ... implement Queue here ... */
}
class StrangeIntQueue extends QueueImplementation[Int] {
override def enqueue[U <: Int](x : U) : Queue[Int] = {
println(math.sqrt(x))
super.enqueue(x)
}
}
and use it as
val x : Queue[Any] = new StrangeIntQueue
x.enqueue("abc")
which is clearly troublesome. However, I can not see how to adapt this in order to show that the combination "contravariant type parameter + lower type bound" is also problematic?
Let's suppose we allow for a class to have a type parameter [-T] and a method on that class to have [U >: T]...
for come class hierarchy
Dog <: Mammal <: Animal
class Contra[-X](x: X){
def problem[Y >: X](y: Y): Y = x // X<:Y so this would be valid
}
val cMammal:Contra[Mammal] = new Contra(new Mammal)
val a:Animal = cMammal problem new Animal // Animal >: Mammal, this is fine
val m:Mammal = cMammal problem new Mammal // Mammal >: Mammal, this is fine
val d:Mammal = cMammal problem new Dog // (Dog upcasts to Mammal) >: Mammal, this is fine
val cDog:Contra[Dog] = cMammal // Valid assignment
val a:Animal = cDog problem new Animal // Animal >: Mammal, this is fine
val m:Mammal = cDog problem new Mammal // Mammal >: Mammal, this is fine
val d:Dog = cDog problem new Dog // AAAHHHHHHH!!!!!!
This last line would type check, cDog problem new Dog would actually return a Mammal. This is clearly not a good thing. Thankfully the type system doesn't actually let us do this.
Q.E.D. contravariant type parameter + lower type bound not a good idea to mix.
I hope this example helps.
Use the ++ method from List to see why the restrictions are needed. Due note, this requires that ++ produce a List[B]:
def ++[B](that: GenTraversableOnce[B]): List[B]
with a full signature of
def ++[B >: A, That](that: GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That
So why is it important that [B >: A]. Well, what if we want to combine something such that
trait Foo
trait Bar extends Foo
and we have a method that has a signature
def op(that: List[Foo], other: Foo): List[Foo] = that ++ List(other)
I can pass it a list of type Bar but in order to be able to return it as a List[Foo] I must make the condition that Foo >: Bar so that I can actually do the following
def see(that: List[Bar]): List[Foo] = op(that, myFoo)
which essentially is doing a List[Bar] ++ List[Foo] to return a type of List[Foo] as expressed though a List[Foo] type. That is why the flip happens.
Now if I tried to enforce that Foo <: Bar I would immediately run into the issue that List[Bar] ++ List[Foo] could not return a list of type Foo (not to mention having it conflict with the definition above.) It would only ever be able to return a List of the least upper bound.

what's different between <:< and <: in scala

I already know that:
<: is the Scala syntax type constraint
while <:< is the type that leverage the Scala implicit to reach the type constrait
for example:
object Test {
// the function foo and bar can have the same effect
def foo[A](i:A)(implicit ev : A <:< java.io.Serializable) = i
foo(1) // compile error
foo("hi")
def bar[A <: java.io.Serializable](i:A) = i
bar(1) // compile error
bar("hi")
}
but I want to know when we need to use <: and <:< ?
and if we already have <:, why we need <:< ?
thanks!
The main difference between the two is, that the <: is a constraint on the type, while the <:< is a type for which the compiler has to find evidence, when used as an implicit parameter. What that means for our program is, that in the <: case, the type inferencer will try to find a type that satisfies this constraint. E.g.
def foo[A, B <: A](a: A, b: B) = (a,b)
scala> foo(1, List(1,2,3))
res1: (Any, List[Int]) = (1,List(1, 2, 3))
Here the inferencer finds that Int and List[Int] have the common super type Any, so it infers that for A to satisfy B <: A.
<:< is more restrictive, because the type inferencer runs before the implicit resolution. So the types are already fixed when the compiler tries to find the evidence. E.g.
def bar[A,B](a: A, b: B)(implicit ev: B <:< A) = (a,b)
scala> bar(1,1)
res2: (Int, Int) = (1,1)
scala> bar(1,List(1,2,3))
<console>:9: error: Cannot prove that List[Int] <:< Int.
bar(1,List(1,2,3))
^
1. def bar[A <: java.io.Serializable](i:A) = i
<: - guarantees that instance of i of type parameter A will be subtype of Serializable
2. def foo[A](i:A)(implicit ev : A <:< java.io.Serializable) = i
<:< - guarantees that execution context will contains implicit value (for ev paramenter) of type A what is subtype of Serializable.
This implicit defined in Predef.scala and for foo method and it is prove if instance of type parameter A is subtype of Serializable:
implicit def conforms[A]: A <:< A = singleton_<:<.asInstanceOf[A <:< A]
fictional case of using <:< operator:
class Boo[A](x: A) {
def get: A = x
def div(implicit ev : A <:< Double) = x / 2
def inc(implicit ev : A <:< Int) = x + 1
}
val a = new Boo("hi")
a.get // - OK
a.div // - compile time error String not subtype of Double
a.inc // - compile tile error String not subtype of Int
val b = new Boo(10.0)
b.get // - OK
b.div // - OK
b.inc // - compile time error Double not subtype of Int
val c = new Boo(10)
c.get // - OK
c.div // - compile time error Int not subtype of Double
c.inc // - OK
if we not call methods what not conform <:< condition than all compile and execute.
There are definitely differences between <: and <:<; here is my attempt at explaining which one you should pick.
Let's take two classes:
trait U
class V extends U
The type constraint <: is always used because it drives type inference. That's the only thing it can do: constrain the type on its left-hand side.
The constrained type has to be referenced somewhere, usually in the parameter list (or return type), as in:
def whatever[A <: U](p: A): List[A] = ???
That way, the compiler will throw an error if the input is not a subclass of U, and at the same time allow you to refer to the input's type by name for later use (for example in the return type). Note that if you don't have that second requirement, all this isn't necessary (with exceptions...), as in:
def whatever(p: U): String = ??? // this will obviously only accept T <: U
The Generalized Type Constraint <:< on the other hand, has two uses:
You can use it as an after-the-fact proof that some type was inferred. As in:
class List[+A] {
def sum(implicit ev: A =:= Int) = ???
}
You can create such a list of any type, but sum can only be called when you have the proof that A is actually Int.
You can use the above 'proof' as a way to infer even more types. This allows you to infer types in two steps instead of one.
For example, in the above List class, you could add a flatten method:
def flatten[B](implicit ev: A <:< List[B]): List[B]
This isn't just a proof, this is a way to grab that inner type B with A now fixed.
This can be used within the same method as well: imagine you want to write a utility sort function, and you want both the element type T and the collection type Coll. You could be tempted to write the following:
def sort[T, Coll <: Seq[T]](l: Coll): Coll
But T isn't constrained to be anything in there: it doesn't appear in the arguments nor output type. So T will end up as Nothing, or Any, or whatever the compiler wants, really (usually Nothing). But with this version:
def sort[T, Coll](l: Coll)(implicit ev: Coll <:< Seq[T]): Coll
Now T appears in the parameter's types. There will be two inference runs (one per parameter list): Coll will be inferred to whatever was given, and then, later on, an implicit will be looked for, and if found, T will be inferred with Coll now fixed. This essentially extracts the type parameter T from the previously-inferred Coll.
So essentially, <:< checks (and potentially infers) types as a side-effect of implicit resolution, so it can be used in different places / at different times than type parameter inference. When they happen to do the same thing, stick to <:.
After some thinking, I think it has some different.
for example:
object TestAgain {
class Test[A](a: A) {
def foo[A <: AnyRef] = a
def bar(implicit ev: A <:< AnyRef) = a
}
val test = new Test(1)
test.foo // return 1
test.bar // error: Cannot prove that Int <:< AnyRef.
}
this menas:
the scope of <: is just in the method param generic tpye scope foo[A <: AnyRef]. In the example, the method foo have it's generic tpye A, but not the A in class Test[A]
the scope of <:< , will first find the method's generic type, but the method bar have no param generic type, so it will find the Test[A]'s generic type.
so, I think it's the main difference.