scala - Any vs underscore in generics - scala

What is the different between the following Generics definitions in Scala:
class Foo[T <: List[_]]
and
class Bar[T <: List[Any]]
My gut tells me they are about the same but that the latter is more explicit. I am finding cases where the former compiles but the latter doesn't, but can't put my finger on the exact difference.
Thanks!
Edit:
Can I throw another into the mix?
class Baz[T <: List[_ <: Any]]

OK, I figured I should have my take on it, instead of just posting comments. Sorry, this is going to be long, if you want the TL;DR skip to the end.
As Randall Schulz said, here _ is a shorthand for an existential type. Namely,
class Foo[T <: List[_]]
is a shorthand for
class Foo[T <: List[Z] forSome { type Z }]
Note that contrary to what Randall Shulz's answer mentions (full disclosure: I got it wrong too in an earlier version of this post, thanks to Jesper Nordenberg for pointing it out) this not the same as:
class Foo[T <: List[Z]] forSome { type Z }
nor is it the same as:
class Foo[T <: List[Z forSome { type Z }]]
Beware, it is easy to get it wrong (as my earlier goof shows): the author of the article referenced by Randall Shulz's answer got it wrong himself (see comments), and fixed it later. My main problem with this article is that in the example shown, the use of existentials is supposed to save us from a typing problem, but it does not. Go check the code, and try to compile compileAndRun(helloWorldVM("Test")) or compileAndRun(intVM(42)). Yep, does not compile. Simply making compileAndRun generic in A would make the code compile, and it would be much simpler.
In short, that's probably not the best article to learn about existentials and what they are good for (the author himself acknowledge in a comment that the article "needs tidying up").
So I would rather recommend reading this article: http://www.artima.com/scalazine/articles/scalas_type_system.html, in particular the sections named "Existential types" and "Variance in Java and Scala".
The important point that you hould get from this article is that existentials are useful (apart from being able to deal with generic java classes) when dealing with non-covariant types.
Here is an example.
case class Greets[T]( private val name: T ) {
def hello() { println("Hello " + name) }
def getName: T = name
}
This class is generic (note also that is is invariant), but we can see that hello really doesn't make any use of the type parameter (unlike getName), so if I get an instance of Greets I should always be able to call it, whatever T is. If I want to define a method that takes a Greets instance and just calls its hello method, I could try this:
def sayHi1( g: Greets[T] ) { g.hello() } // Does not compile
Sure enough, this does not compile, as T comes out of nowhere here.
OK then, let's make the method generic:
def sayHi2[T]( g: Greets[T] ) { g.hello() }
sayHi2( Greets("John"))
sayHi2( Greets('Jack))
Great, this works. We could also use existentials here:
def sayHi3( g: Greets[_] ) { g.hello() }
sayHi3( Greets("John"))
sayHi3( Greets('Jack))
Works too. So all in all, there is no real benefit here from using an existential (as in sayHi3) over type parameter (as in sayHi2).
However, this changes if Greets appears itself as a type parameter to another generic class. Say by example that we want to store several instances of Greets (with different T) in a list. Let's try it:
val greets1: Greets[String] = Greets("John")
val greets2: Greets[Symbol] = Greets('Jack)
val greetsList1: List[Greets[Any]] = List( greets1, greets2 ) // Does not compile
The last line does not compile because Greets is invariant, so a Greets[String] and Greets[Symbol] cannot be treated as a Greets[Any] even though String and Symbol both extends Any.
OK, let's try with an existential, using the shorthand notation _:
val greetsList2: List[Greets[_]] = List( greets1, greets2 ) // Compiles fine, yeah
This compiles fine, and you can do, as expected:
greetsSet foreach (_.hello)
Now, remember that the reason we had a type checking problem in the first place was because Greets is invariant. If it was turned into a covariant class (class Greets[+T]) then everything would have worked out of the box and we would never have needed existentials.
So to sum up, existentials are useful to deal with generic invariant classes, but if the generic class does not need to appear itself as a type parameter to another generic class, chances are that you don't need existentials and simply adding a type parameter to your method will work
Now come back(at last, I know!) to your specific question, regarding
class Foo[T <: List[_]]
Because List is covariant, this is for all intents and purpose the same as just saying:
class Foo[T <: List[Any]]
So in this case, using either notation is really just a matter of style.
However, if you replace List with Set, things change:
class Foo[T <: Set[_]]
Set is invariant and thus we are in the same situation as with the Greets class from my example. Thus the above really is very different from
class Foo[T <: Set[Any]]

The former is a shorthand for an existential type when the code doesn't need to know what the type is or constrain it:
class Foo[T <: List[Z forSome { type Z }]]
This form says that the element type of List is unknown to class Foo rather than your second form, which says specifically that the List's element type is Any.
Check out this brief explanatory blog article on Existential Types in Scala (EDIT: this link is now dead, a snapshot is available at archive.org)

Related

Contravariance in scala

I am new with scala. I am trying to figure out how the whole contravariance relationship works. I understand the concept of covariance and invariant and I also know how I would implement them in practise.
I also understand the concept of contravariance (the reverse of covariance) and how it is implemented in the Function1 trait in Scala. It gives you an abstraction without redefining Function1 implementations for different classes. But, I still don’t get it completely, strange? Now, I am almost there… how can I solve the following problem with contravariance:
class GarbageCan[-A] {
def doSomething(a: A): Unit ={
// do something with 'a' of subtype that is not possible with the supertype
}
}
def setGarbageCanForPlastic(gc: GarbageCan[PlasticItem]): Unit = {
}
The above example is extracted from http://blog.kamkor.me/Covariance-And-Contravariance-In-Scala/. A really good explanation concerning this subject. The hierarchy is as follows: Item (base class) -> PlasticItem (subclass) -> PlasticBottle (subclass of subclass)
The setGarbageCanForPlastic function accepts a GarbageCan with the type of PlasticItem. Because the Parameterized type is contravariant, the following statement is completely legal:
setGarbageCanForPlastic(new GarbageCan[Item])
Now, the doSomething function accepts a Type parameter which is contravariant. How can I work with this type, if I don’t know if the type is a Base class “Item” or subclass “PlasticItem”? I can do something which is permissible within the subclass and not in the base class. If this was an covariant parameter, this would be no problem, a subclass inherits everything from the base class.
I lost it right?... Hope somebody can help me out.
First of all, the doSomething method actually cannot do anything other than essentially discarding a since it does not know what A is. To make it more useful, you would need a bound like class GarbageCan[-A <: Item].
Now, suppose setGarbageCanForPlastic(gc) calls gc.doSomething(new PlasticItem). Since A in GarbageCan[A] is contravariant, we have GarbageCan[Item] <: GarbageCan[PlasticItem] <: GarbageCan[PlasticBottle]. Indeed, the function call setGarbageCanForPlastic(new GarbageCan[Item])) is safe as GarbageCan[Item]'s doSomething can process any Item including a PlasticItem, while the call setGarbageCanForPlastic(new GarbageCan[PlasticBottle])) is unsafe because GarbageCan[PlasticBottle]'s doSomething may not be able to take a PlasticItem which is not necessarily a PlasticBottle.

Variance (covariance and contravariance) in Scala functions [duplicate]

I'm trying to use a covariant type parameter inside a trait to construct a case-class like so:
trait MyTrait[+T] {
private case class MyClass(c: T)
}
compiler says:
error: covariant type T occurs in contravariant position in type T of value c
I then tried the following but it also didn't work:
trait MyTrait[+T] {
private case class MyClass[U <: T](c: U)
}
the error this time is:
error: covariant type T occurs in contravariant position in type >: Nothing <: T of type U
Could somebody explain why the T is in a covariant position here and suggest a solution for this problem?
Thx!
This is a fundamental feature of object-oriented programming that doesn't get as much attention as it deserves.
Suppose you have a collection C[+T]. What the +T means is that if U <: T, then C[U] <: C[T]. Fair enough. But what does it mean to be a subclass? It means that every method should work that worked on the original class. So, suppose you have a method m(t: T). This says you can take any t and do something with it. But C[U] can only do things with U, which might not be all of T! So you have immediately contradicted your claim that C[U] is a subclass of C[T]. It's not. There are things you can do with a C[T] that you can't do with a C[U].
Now, how do you get around this?
One option is to make the class invariant (drop the +). Another option is that if you take a method parameter, to allow any superclass as well: m[S >: T](s: S). Now if T changes to U, it's no big deal: a superclass of T is also a superclass of U, and the method will work. (However, you then have to change your method to be able to handle such things.)
With a case class, it's even harder to get it right unless you make it invariant. I recommend doing that, and pushing the generics and variance elsewhere. But I'd need to see more details to be sure that this would work for your use case.
Almost there. Here:
scala> trait MyTrait[+T] {
| private case class MyClass[U >: T](c: U)
| }
defined trait MyTrait
Which means MyClass[Any] is valid for all T. That is at the root of why one cannot use T in that position, but demonstrating it requires more code than I'm in the mood for at the moment. :-)

Scala reflection: TypeTag from context bound doesn't match type parameter

Let's start with an example code:
import scala.reflect.runtime.universe._
class A[T] {
}
def foo[T: TypeTag](a: A[T]) {
println(typeTag[T])
}
val a = new A[Int]
val b: A[_] = a
foo(a)
foo(b)
The output is:
TypeTag[Int]
TypeTag[_$1]
Well, I have no clue whatsoever what is TypeTag[_$1], but it sure looks fancy. :) Still, I would think foo's expectations of the world are being violated here, as in if foo gets called with an instance of A[Int] then it is guaranteed to have the typetag for Int, and not some other obscure typetag. Also, shouldn't typetags be "concrete" in contrast to weektypetags?
Now of course I see that the compiler cannot tell in the second foo call the type parameter of A. So my expectation wasn't that I magically get typeTag[Int] as the output of the second call, but rather I was hoping for a compile time error.
Where is my missunderstanding?
Some futile speculation
One could argue that what's going on is a call to foo with argument type A[_], or, to make it more explicit, the existential type A[S] forSome { type S }. An then the typeTag captures the "_" somehow. But this doesn't really make sense, as foo expects A[T] for some concrete T, and the above existential type is not A[T] for any concrete T.
Or maybe a call with an argument A[Any]? But then why not TypeTag[Any], and even more severely, A is not covariant, so this would also be plain wrong.
Btw, I'm using scala 2.10.
Type tags are created at compile time, and values are not available at compile time. Once you cast an A[Int] to the existential type A[_], all information about the type parameter has been lost.
If they were created at run-time, based on values, type erasure would make it impossible to know A's parameter. Even something that is known to be an A[Int] at compile-time is at best an A[Object] at run-time (unless A is Array, but let's not go there).
So your speculation is correct, for foo(b), the type parameter is indeed the _ from the existential A[_], and that anonymous type variable prints as _$1.

Casting "hollow" higher kinded type values to avoid instantiations

I caught myself watching a bit of the Scalawags#2 recording, and then there came this part about type erasure and Dick Wall pointing out that reflection will eventually bite you in the feet.
So I was thinking about something that I'm doing quite frequently (and I saw it the implementation of Scala Collections as well). Let's say I have a system with a serializer taking the system as type parameter:
trait Sys[S <: Sys[S]] { type Tx }
trait FooSys extends Sys[FooSys]
trait Serializer[S <: Sys[S], A] {
def read(implicit tx: S#Tx): A
}
Now there are many types A for which serializers can be constructed without value parameters, so essentially the system type parameter is "hollow". And since serializers are heavily invoked in my example, I'm saving instantiation:
object Test {
def serializer[S <: Sys[S]] : Serializer[S, Test[S]] =
anySer.asInstanceOf[Ser[S]]
private val anySer = new Ser[FooSys]
private final class Ser[S <: Sys[S]] extends Serializer[S, Test[S]] {
def read(implicit tx: S#Tx) = new Test[S] {} // (shortened for the example)
}
}
trait Test[S <: Sys[S]]
I know this is correct, but of course, asInstanceOf has a bad smell. Are there any suggestions to this approach? Let me add two things
moving the type parameter from the constructor of trait Serializer to the read method is not an option (there are specific serializers which require value arguments parametrised in S)
adding variance to Serializer's type constructor parameter is not an option
Introduction:
I am a little confused by your example and I might have misunderstood your question, I have a feeling there is a certain type recursion between S and Tx that I am not getting from your question(because if not, S#Tx could be anything and I don't understand the problem with the anySer)
Tentative Answer:
At compile time, for any instance of Ser[T] there will be a well-defined type parameter T, since you want to save it on instantiation, you will have a single anySer Ser[T] for a given specific type A
What you are saying in some way is that a Ser[A] will work as Ser[S] for any S. This can be explained in two ways, according to the relationship between type A and S.
If this conversion is possible for every A<:<S then your serializer is COVARIANT and you can initialize your anySer as a Ser[Nothing] . Since Nothing is subclass of every class in Scala, your anySer will always work as a Ser[Whatever]
If this conversion is possible for every S<:<A then your serializer is CONTRAVARIANT and you can initialize your anySer as a Ser[Any] . Since Any is subclass of every class in Scala, your anySer will always work as a Ser[Whatever]
If it's neither the one of the previous case, then it means that:
def serializer[S <: Sys[S]] : Serializer[S, Test[S]] =
anySer.asInstanceOf[Ser[S]]
Could produce an horrible failure at runtime, because there will some S for which the Serializer won't work. If there are no such S for which this could happen, then your class falls in either 1 or
Comment post-edit
If your types are really invariant, the conversion through a cast breaks the invariance relation. You are basically forcing the type system to perform an un-natural conversion because you know that nothing wrong will happen, on the basis of your own knowledge of the code you have written. If this is the case then casting is the right way to go: you are forcing a different type from the one the compiler can check formally and you are making this explicit. I would even put a big comment saying why you know that operation is legal and the compiler can't guess and eventually attach a beautiful unit test to verify that the "in-formal" relation always holds.
In general, I believe this practice should be used with extreme care. One of the benefits of strongly typed languages is that the compiler performs formal type checking that helps you catch early errors. If you intentionally break it, you give away this big benefit.

Scala: Using representation types

Because traits with representation types are self-referential, declaring that a variable holds an instance of that trait is a little difficult. In this example I simply declare that a variable holds an instance of the trait, declare that a function takes and returns and instance of that trait, and call that function with the variable:
trait Foo[+A <: Foo[A]]
case class Bar() extends Foo[Bar]
case class Grill() extends Foo[Grill]
// Store a generic instance of Foo
val b: Foo[_] = if(true) {
Bar()
} else {
Grill()
}
// Declare a function that take any Foo and returns a Foo of the same type
// that "in" has in the calling context
def echoFoo[A <: Foo[A]](in: A): A = in
// Call said function
val echo = echoFoo(b)
It fails with the error:
inferred type arguments [this.Foo[_$1]] do not conform to method
echoFoo's type parameter bounds [A <: this.Foo[A]]
val echo = echoFoo(b)
^
Now, this makes sense because [_] is like Any (in ways I don't fully understand). What it probably wants is something like Foo[Foo[_]], so that the type parameter conforms to the bounds of A <: Foo[A]. But now there's an inner Foo that has a non-conforming type parameter, suggesting that the solution is something like Foo[Foo[Foo[Foo[..., which is clearly not correct.
So my question can probably be distilled down to: What is the Scala syntax for "This variable holds any legal Foo"?
Self-referential type parameters like this are a bit problematic, because they're not sound. For example, it's possible to define a type like the following:
case class BeerGarden extends Foo[Grill]
As you can see, the A <: Foo[A] bound isn't sufficiently tight. What I prefer in situations like this is to use the cake pattern, and abstract type members:
trait FooModule {
type Foo <: FooLike
def apply(): Foo
trait FooLike {
def echo: Foo
}
}
Now you can use the Foo type recursively and safely:
object Foos {
def echo(foo: FooModule#Foo) = foo.echo
}
Obviously, this isn't an ideal solution to all the problems you might want to solve with such types, but the important observation is that FooLike is an extensible trait, so you can always continue to refine FooLike to add the members that you need, without violating the bound that the type member is intended to enforce. I've found that in every real-world case where the set of types I want to represent is not closed, this is about the best that one can do. The important thing to see is that FooModule abstracts over both the type and the instance constructor, while enforcing the "self-type." You can't abstract over one without abstracting over the other.
Some additional information on this sort of thing (and a bit of a record of my own early struggles with recursive types) is available here:
https://issues.scala-lang.org/browse/SI-2385
While I agree the problem of propagating generics exists, when you hit this problem you should see a big WARNING on your screen because its typically a sign of a bad design. These are general suggestions on the topic.
If you use generics, the type parameter is there for a reason. It lets you interact with a Foo[A] in a type-safe manner by passing in or receiving parameters of type A and allows to put you constraint on A. If you lose the type information, you lose the type-safety and in that case so there is no point of writing a generic class if you do not need the generic anymore: you can change all your signatures to Any and do pattern matching.
In most of the cases, recursive types can be avoided by implementing something like the CanBuildFrom approach for collections, using a "typeclass"
Finally,type-projection (FooModule#Foo) has little application and you might want to look to path-dependent types. However, these have little application as well.