Manifest and abstract type resolution - scala

I am hitting a compiler problem when the compiler needs to solve a manifest for a class with an abstract type parameter. The following snippet show the issue
trait MyStuff
trait SecurityMutatorFactory[X]{
def apply(x1:X,x2:X)
}
object Example{
trait LEdge[N]
{
type L1
}
type MyEdge[X] = LEdge[X] { type L1 = SecurityMutatorFactory[X]}
val a:Manifest[MyEdge[MyStuff]] = implicitly[Manifest[MyEdge[MyStuff]]]
}
As a result, the compiler throws the following type error:
type mismatch;
found : scala.reflect.Manifest[LEdge[MyStuff]]
required: Manifest[MyEdge[MyStuff]]
Note: LEdge[MyStuff] >: MyEdge[MyStuff], but trait Manifest is invariant in type T.
You may wish to investigate a wildcard type such as `_ >: MyEdge[MyStuff]`. (SLS 3.2.10)
val a:Manifest[MyEdge[MyStuff]] = implicitly[Manifest[MyEdge[MyStuff]]]
What is happening at compiler level? ^

As others have suggested the problem comes from
type MyEdge[X] = LEdge[X] { type L1 = SecurityMutatorFactory[X] }
Declarations of the form type F[X] = ... introduce type synonyms, ie new names for existing types. They do not construct new traits or classes. However, LEdge[X] { type L1 = SecurityMutatorFactory[X] } is constructing a new anonymous class. So your example is approximatelly equivalent to
trait MyEdge[X] extends LEdge[X] { type L1 = SecurityMutatorFactory[X] }
(which is what you most probably want) but the original definition in the example is defining a synonym for an anonymous class instead of defining a new class MyEdge[X]. So in the example the new class is not actually called MyEdge. When constructing the implicit manifest, the compiler replaces the type synonym with the underlying type, but fails to construct a manifest for that because that type is anonymous.
Replacing the MyEdge declaration with either a normal extension definition:
trait MyEdge[X] extends LEdge[X] { type L1 = SecurityMutatorFactory[X] }
or with an ordinary type synonym:
type MyEdge[X] = LEdge[X]
both compile successfully.
EDIT
Here is the specific reason why generating implicit manifests for anonymous classes fails.
In the language specification type expessions of the form BaseType { ... } are called refined types.
According to the language specification, the manifest for a refined type is just the manifest of its base class. This however fails to typecheck, because you asked for a Manifest[LEdge[MyStuff]{ type L1 = SecurityMutatorFactory[X] }], but the algorithm is returning Manifest[LEdge[MyStuff]]. This means that you can only construct implicit manifests for types with refined types only in contravariant positions. For example using:
type MyEdge[X] = LEdge[X] { type L1 = SecurityMutatorFactory[X] } => AnyRef
in your example allows it to compile, though it is clearly not what you are after.
The full algorithm for constructing implicit manifests is given at the end of section 7.5 of the language specification. This question is covered by clause 6:
6) If T is a refined type T'{R}, a manifest is generated for T'. (That is, refinements are never reflected in manifests).

Well, I'm not so familiar with that kind of pattern:
type MyEdge[X] = LEdge[X] { type L1 = SecurityMutatorFactory[X]}
but I tend to consider types defined with the type keyword as aliases (concepts) rather than a guarantee about the implementation (EDIT more precisely, I believe that type provides guarantees in terms of prototyping/specifying but that no AST/code is generated until there's an actual need to replace the alias with the traits/classes it's based upon). So even if the compiler claims, in its error message:
LEdge[MyStuff] >: MyEdge[MyStuff]
I'm not sure that, at the bytecode level, it implements MyEdge accordingly, with interfaces/methods/etc. Thus, it might not recognize the wanted relationship between LEdge and MyEdge, eventually:
found : scala.reflect.Manifest[LEdge[MyStuff]]
required: Manifest[MyEdge[MyStuff]]
(and, is the absence of package scala.reflect. a hint? (1))
About your code, how do you use a? Anyway, if the following is your intent, with:
trait MyEdge[X] extends LEdge[X] {
type L1 = SecurityMutatorFactory[X]
}
instead, it does compile (scala 2.10)... (EDIT I just noticed now that dmitry already told that) ...what that does during runtime, I don't know!
As an item of note, Manifest is deprecated after scala 2.9; so you may prefer to use TypeTag[T] as described in the scaladoc.
EDIT:
(1) I suspect that the following happens:
- at the syntactic analysis phase, the compiler registers literally what you specified, that is, the implicitly method shall return a Manifest[MyEdge[MyStuff]].
- by the code generation phase, the aliases are "reconciled" to their nearest classes or traits; in the case of implicitly the result's type Manifest[MyEdge[MyStuff]] becomes trait scala.reflect.Manifest[LEdge[MyStuff]]]
- due to some limitations of type inference involved in Manifest and type "aliasing" within type parameters, however, somehow the specified requirement Manifest[MyEdge[MyStuff]] remains under its raw shape
- (this is pure conjecture, because I've not read the Scala compiler source code for this answer) the compiler would have the proper AST/code on the one hand, but a method prototype/spec that is still under its raw/literal shape on the other hand; that doesn't fit in so it emits an error.
Hoping that helps...

Related

Pattern matching with type parameter bounded to final class

Here is an example
def maybeeq[A <: String](x: A):A = x match {
case z:A => x
}
It produced the following error message during compilation
Error:(27, 12) scrutinee is incompatible with pattern type;
found : A
required: String
case z:A => x
I can put any final class into A's bound to reproduce the error.
Why this compiles for non-final classes but fails on the final? Why type erasure not just replace A with String?
Edited:
Note: such bound allows me to pass String-typed value to 'x' parameter. So 'x' can be just a String and don't have to be subtype of string, so I'm not asking compiler to compile method with incorrect signature. In the real-world code I would just put String instead on A parameter, but from the experimental perspective I'm interested why such extra restriction on top of existing restriction (based on final class nature) is needed.
TBH this is a question about compiler design which can only be answered by those who implemented such check
There is a test in compiler test suite that requires such error to be shown. It has something to do with type information being discarded to a point where a concrete type cannot be assigned to variable, but the reasons for that cannot be understood from git blame of that test.
I'll point out, however, that there is still a number of ways to satisfy A <: String without A being known at compile time to be a String. For one, Null and Nothing satisfy that, being at the bottom of Scala type hierarchy. Those two are explicitly disallowed at type matching. The other example is a bit more involved:
val UhOh: { type T <: String } = new { type T = String }
implicitly[UhOh.T <:< String] // satisfies type bound
implicitly[UhOh.T =:= String] // won't compile - compiler cannot prove the type equality
This is similar to some newtyping patterns, e.g. shapeless.tag
Out of all these possibilities, only one that can do anything reasonable is when A =:= String because String is the only type that can be actually checked at runtime. Oh, except when you use generic type in match - that does not work at all (not without ClassTag in scope at least) because such types are eliminated by erasure.
Final class cannot be extended.so for def maybeeq[A <: String](x: A) is not a correct syntax, since String is final, there should not have any subtype extend from String. the compiler smartly point out this issue.

Issue with existential in Scala

I have an issue working with existentials in Scala. My problem started when creating a mini workflow engine. I started on the idea that it was a directed graph, implemented the model for the latter first and then modeled the Workflow like this:
case class Workflow private(states: List[StateDef], transitions: List[_, _], override val edges: Map[String, List[StateDef]]) extends Digraph[String, StateDef, Transition[_, _]](states, edges) { ... }
In this case class, the first two fields are a list of states which behave as node, transitions which behave as edges.
The Transition parameter types are for the input and output parameters, as this should behave as an executable piece in the workflow, like a function of some sort:
case class Transition[-P, +R](tailState: StateDef, headState: StateDef, action: Action[P, R], condition: Option[Condition[P]] = None) extends Edge[String, StateDef](tailState, headState) {
def execute(param: P): Try[State[R]] = ...
}
I realized soon enough that dealing with a list of transitions in the Workflow object was giving me troubles with its type parameters. I tried to use parameters with [[Any]] and [[Nothing]], but I couldn't make it work (gist [1]).
If I'd do Java, I'd use a wildcard ? and use its 'less type safe and more dynamic' property and Java would have to believe me. But Scala is stricter and with variance and covariance of the Transition parameter types, it's hard to define wildcards and handle these properly. For example, using forSome notation and having a method in Workflow, I would get this error (gist [2]):
Error:(55, 24) type mismatch;
found : List[A$A27.this.Transition[A$A27.this.CreateImage,A$A27.this.Image]]
required: List[A$A27.this.Transition[P forSome { type P },R forSome { type R }]]
lazy val w = Workflow(transitions)
^
Hence then I created an existential type based on a trait (gist [3]), as explained in this article.
trait Transitions {
type Param
type Return
val transition: Transition[Param, Return]
val evidenceParam: StateValue[Param]
val evidenceReturn: StateValue[Return]
}
So now I could plug this existential in my Workflow class like this:
case class Workflow private(states: List[StateDef], transitions: List[Transitions], override val edges: Map[String, List[StateDef]])
extends Digraph[String, StateDef, Transitions](states, edges)
Working in a small file proved to be working (gist [3]). But when I moved on to the real code, my Digraph parent class does not like this Transitions existential. The former needs an Edge[ID, V] type, which Transition complies with but not the Transitions existential of course.
How in Scala does one resolve this situation? It seems troublesome to work with parameter types to get generics in Scala. Is there an easier solution that I haven't tried? Or a magic trick to specify the correct compatible parameter type between Digraph which need an Edge[ID, V] type and not an existential type that basically erase type traces?
I am sorry as this is convoluted, I will try my best to update the question if necessary.
Here are the Gist references for some of my trials and errors:
https://gist.github.com/jimleroyer/943efd00c764880b8119786d9dd6c3a2
https://gist.github.com/jimleroyer/1ce238b3934882ddc02a09485f52f407
https://gist.github.com/jimleroyer/17227b7e334d020a21deb36086b9b978
EDIT-1
Based on #HTNW answer, I've modified the scope of the existentials using forSome and updated the solution: https://gist.github.com/jimleroyer/2cb4ccbec13620585d21d53b4431ce22
I still have an issue though to properly bind the generics with the matchTransition & getTransition methods and without an explicit cast using asInstanceOf. I'll open another question specific to that one issue.
You scoped your existential quantifiers wrong.
R forSome { type R }
is equal to Any, because every single type is a type, so every single type is a subtype of that existential type, and that is the distinguishing feature of Any. Therefore
Transition[P forSome { type P }, R forSome { type R }]
is really
Transition[Any, Any]
and the Transitions end up needing to take Anys as parameter, and you lose all information about the type of the return. Make it
List[Transition[P, R] forSome { type P; type R }] // if every transition can have different types
List[Transition[P, R]] forSome { type P; type R } // if all the transitions need similar types
// The first can also be sugared to
List[Transition[_, _]]
// _ scopes so the forSome is placed outside the nearest enclosing grouping
Also, I don't get where you got the idea that Java's ? is "less safe". Code using it has a higher chance of being unsafe, sure, because ? is limited, but on its own it is perfectly sound (modulo null).

What does `SomeType[_]` mean in scala?

Going through Play Form source code right now and encountered this
def bindFromRequest()(implicit request: play.api.mvc.Request[_]): Form[T] = {
I am guessing it takes request as an implicit parameter (you don't have to call bindFromRequet(request)) of type play.api.mvc.Request[_] and return a generic T type wrapped in Form class. But what does [_] mean.
The notation Foo[_] is a short hand for an existential type:
Foo[A] forSome {type A}
So it differs from a normal type parameter by being existentially quantified. There has to be some type so your code type checks where as if you would use a type parameter for the method or trait it would have to type check for every type A.
For example this is fine:
val list = List("asd");
def doSomething() {
val l: List[_] = list
}
This would not typecheck:
def doSomething[A]() {
val l: List[A] = list
}
So existential types are useful in situations where you get some parameterized type from somewhere but you do not know and care about the parameter (or only about some bounds of it etc.)
In general, you should avoid existential types though, because they get complicated fast. A lot of instances (especially the uses known from Java (called wildcard types there)) can be avoided be using variance annotations when designing your class hierarchy.
The method doesn't take a play.api.mvc.Request, it takes a play.api.mvc.Request parameterized with another type. You could give the type parameter a name:
def bindFromRequest()(implicit request: play.api.mvc.Request[TypeParameter]): Form[T] = {
But since you're not referring to TypeParameter anywhere else it's conventional to use an underscore instead. I believe the underscore is special cased as a 'black hole' here, rather than being a regular name that you could refer to as _ elsewhere in the type signature.

Scala: Type parameters and inheritance

I'm seeing something I do not understand. I have a hierarchy of (say) Vehicles, a corresponding hierarchy of VehicalReaders, and a VehicleReader object with apply methods:
abstract class VehicleReader[T <: Vehicle] {
...
object VehicleReader {
def apply[T <: Vehicle](vehicleId: Int): VehicleReader[T] = apply(vehicleType(vehicleId))
def apply[T <: Vehicle](vehicleType VehicleType): VehicleReader[T] = vehicleType match {
case VehicleType.Car => new CarReader().asInstanceOf[VehicleReader[T]]
...
Note that when you have more than one apply method, you must specify the return type. I have no issues when there is no need to specify the return type.
The cast (.asInstanceOf[VehicleReader[T]]) is the reason for the question - without it the result is compile errors like:
type mismatch;
found : CarReader
required: VehicleReader[T]
case VehicleType.Car => new CarReader()
^
Related questions:
Why cannot the compiler see a CarReader as a VehicleReader[T]?
What is the proper type parameter and return type to use in this situation?
I suspect the root cause here is that VehicleReader is invariant on its type parameter, but making it covariant does not change the result.
I feel like this should be rather simple (i.e., this is easy to accomplish in Java with wildcards).
The problem has a very simple cause and really doesn't have anything to do with variance. Consider even more simple example:
object Example {
def gimmeAListOf[T]: List[T] = List[Int](10)
}
This snippet captures the main idea of your code. But it is incorrect:
val list = Example.gimmeAListOf[String]
What will be the type of list? We asked gimmeAListOf method specifically for List[String], however, it always returns List[Int](10). Clearly, this is an error.
So, to put it in words, when the method has a signature like method[T]: Example[T] it really declares: "for any type T you give me I will return an instance of Example[T]". Such types are sometimes called 'universally quantified', or simply 'universal'.
However, this is not your case: your function returns specific instances of VehicleReader[T] depending on the value of its parameter, e.g. CarReader (which, I presume, extends VehicleReader[Car]). Suppose I wrote something like:
class House extends Vehicle
val reader = VehicleReader[House](VehicleType.Car)
val house: House = reader.read() // Assuming there is a method VehicleReader[T].read(): T
The compiler will happily compile this, but I will get ClassCastException when this code is executed.
There are two possible fixes for this situation available. First, you can use existential (or existentially quantified) type, which can be though as a more powerful version of Java wildcards:
def apply(vehicleType: VehicleType): VehicleReader[_] = ...
Signature for this function basically reads "you give me a VehicleType and I return to you an instance of VehicleReader for some type". You will have an object of type VehicleReader[_]; you cannot say anything about type of its parameter except that this type exists, that's why such types are called existential.
def apply(vehicleType: VehicleType): VehicleReader[T] forSome {type T} = ...
This is an equivalent definition and it is probably more clear from it why these types have such properties - T type is hidden inside parameter, so you don't know anything about it but that it does exist.
But due to this property of existentials you cannot really obtain any information about real type parameters. You cannot get, say, VehicleReader[Car] out of VehicleReader[_] except via direct cast with asInstanceOf, which is dangerous, unless you store a TypeTag/ClassTag for type parameter in VehicleReader and check it before the cast. This is sometimes (in fact, most of time) unwieldy.
That's where the second option comes to the rescue. There is a clear correspondence between VehicleType and VehicleReader[T] in your code, i.e. when you have specific instance of VehicleType you definitely know concrete T in VehicleReader[T] signature:
VehicleType.Car -> CarReader (<: VehicleReader[Car])
VehicleType.Truck -> TruckReader (<: VehicleReader[Truck])
and so on.
Because of this it makes sense to add type parameter to VehicleType. In this case your method will look like
def apply[T <: Vehicle](vehicleType: VehicleType[T]): VehicleReader[T] = ...
Now input type and output type are directly connected, and the user of this method will be forced to provide a correct instance of VehicleType[T] for that T he wants. This rules out the runtime error I have mentioned earlier.
You will still need asInstanceOf cast though. To avoid casting completely you will have to move VehicleReader instantiation code (e.g. yours new CarReader()) to VehicleType, because the only place where you know real value of VehicleType[T] type parameter is where instances of this type are constructed:
sealed trait VehicleType[T <: Vehicle] {
def newReader: VehicleReader[T]
}
object VehicleType {
case object Car extends VehicleType[Car] {
def newReader = new CarReader
}
// ... and so on
}
Then VehicleReader factory method will then look very clean and be completely typesafe:
object VehicleReader {
def apply[T <: Vehicle](vehicleType: VehicleType[T]) = vehicleType.newReader
}

Scalaz 7 - why using type alias results in ambigous typeclass resolution for Reader

Code to test with:
import scalaz.{Reader, Applicative}
class ReaderInstanceTest {
type IntReader[A] = Reader[Int, A]
val a = Applicative[({type l[A] = Reader[Int, A]})#l] // fine
val b = Applicative[IntReader]
// ^ ambigous implicit values
// both method kleisliMonadReader ..
// and method kleisliIdMonadReader ..
}
Is this related to Scala's higher-order unification for type constructor inference ticket? If so (and even if not), could you describe what happens here in the a and b cases?
Do you have guidelines about when to use type lambda and when to use a type alias so that everything works out on the long run without unexpected errors?
Yes, this is related to SI-2712.
kleisliIdMonadReader exists solely to guide type inference; it just forwards to kleisliMonadReader. By providing the type alias IntReader, scalac doesn't need this assistance, and can infer the type arguments for kleisliMonadReader directly. This leads to the ambiguity.
I've just committed a remedy: we can prioritize these implicits relative to each other, by defining one in a subclass.
https://github.com/scalaz/scalaz/commit/6f9ae5f