Let's say I have:
trait X {
val x: String
}
Using mix-in, I can define a trait such as
trait XPrinter {
self: X =>
def printX: String = "X is: " + x
}
such that a value/object implementing XPrinter implements x and give its methods such as printX access to the values specified in X such as x.
So far, so good.
I want to know if there is a way of having a trait in the form of:
trait XDependent[T <: X] {
def printX: String = ???
}
So that XDependent instances have access to the value of T.x, with x assumed to be a "static value" glued with the type definition.
Now I understand why T.x can't be accessed in XDependent since a type subtyping X doesn't even have to implement the value of x and T.x might be abstract.
I understand that while Scala offers path-dependent types so that an abstract type defined in X can be used in XDependent, as shown here:
trait X {
type Y //which can be constrained as desired.
}
trait XDependent[T <: X]{
def foo(v:T#Y)
def bar: T#Y
}
it doesn't offer the same thing with values as there is a clear separation between types and values in Scala.
Now I have come across the ideas of value-dependent types and literal-based types. I want to know if the idea of "static value for types", as illustrated above, has much overlap with the these concepts and what the connections are.
I'd also like to know about the different approaches taken in different languages, to blur the separation between types and values, how compatible they are with Scala's type system, and what the complications are in terms of integrating "static values" with the type-system. ie, (Can they be)/ (what happens if they are) overriden by a subtype, etc.
If you can relax the requirement that XDependent has to be a trait, and make it an abstract class instead, then it seems as if a typeclass which provides a single null-ary method x is exactly what you want:
Here is your base trait X (without X.x or anything, that wouldn't be "static"):
trait X
Now you can define a typeclass HasStaticX[T] that guarantees that for a type T we can give some string x:
trait HasStaticX[T] {
def x: String
}
Then you can use it like this:
abstract class XDependent[T <: X : HasStaticX] {
def printX: String = implicitly[HasStaticX[T]].x
}
What HasStaticX does is essentially building a compile-time partial function that can take type T and produce a string-value x associated with T. So, in a way, it's something like a function that takes types and returns values. If this is what you want, then nothing has to be done to for "integrating static values", it just works in the current non-experimental mainstream versions of Scala.
The "value-dependent types" would be exactly the other way round: those would be essentially "functions" that assign types to values.
Related
In this hypothetical, I have a list of operations to be executed. Some of the operations in that list will be more efficient if they can be batched together (eg, lookup up different rows from the same table in a database).
trait Result
trait BatchableOp[T <: BatchableOp[T]] {
def resolve(batch: Vector[T]): Vector[Result]
}
Here we use F-bounded Polymorphism to allow the implementation of the operation to refer to its own type, which is highly convenient.
However, this poses a problem when it comes time to execute:
def execute(operations: Vector[BatchableOp[_]]): Vector[Result] = {
def helper[T <: BatchableOp[T]](clazz: Class[T], batch: Vector[T]): Vector[Result] =
batch.head.resolve(batch)
operations
.groupBy(_.getClass)
.toVector
.flatMap { case (clazz, batch) => helper(clazz, batch)}
}
This results in a compiler error stating inferred type arguments [BatchableOp[_]] do not conform to method helper's type parameter bounds [T <: BatchableOp[T]].
How can the Scala compiler be convinced that the group is all of the same type (which is a subclass of BatchableOp)?
One workaround is to specify the type explicitly, but in this case the type is unknown.
Another workaround relies on enumerating the child types, but I'd like to not have to update the execute method after implementing a new BatchableOp type.
I would like to approach the question systematically, so that the same solution strategy can be applied in similar cases.
First, an obvious remark: you want to work with a vector. The content of the vector can be of different types. The length of the vector is not limited. The number of types of entries of the vector is not limited. Therefore, the compiler cannot prove everything at compile time: you will have to use something like asInstanceOf at some point.
Now to the solution of the actual question:
This here compiles under 2.12.4:
import scala.language.existentials
trait Result
type BOX = BatchableOp[X] forSome { type X <: BatchableOp[X] }
trait BatchableOp[C <: BatchableOp[C]] {
def resolve(batch: Vector[C]): Vector[Result]
// not abstract, needed only once!
def collectSameClassInstances(batch: Vector[BOX]): Vector[C] = {
for (b <- batch if this.getClass.isAssignableFrom(b.getClass))
yield b.asInstanceOf[C]
}
// not abstract either, no additional hassle for subclasses!
def collectAndResolve(batch: Vector[BOX]): Vector[Result] =
resolve(collectSameClassInstances(batch))
}
def execute(operations: Vector[BOX]): Vector[Result] = {
operations
.groupBy(_.getClass)
.toVector
.flatMap{ case (_, batch) =>
batch.head.collectAndResolve(batch)
}
}
The main problem that I see here is that in Scala (unlike in some experimental dependently typed languages) there is no simple way to write down complex computations "under the assumption of existence of a type".
Therefore, it seems difficult / impossible to transform
Vector[BatchOp[T] forSome T]
into a
Vector[BatchOp[T]] forSome T
Here, the first type says: "it's a vector of batchOps, their types are unknown, and can be all different", whereas the second type says: "it's a vector of batchOps of unknown type T, but at least we know that they are all the same".
What you want is something like the following hypothetical language construct:
val vec1: Vector[BatchOp[T] forSome T] = ???
val vec2: Vector[BatchOp[T]] forSome T =
assumingExistsSomeType[C <: BatchOp[C]] yield {
/* `C` now available inside this scope `S` */
vec1.map(_.asInstanceOf[C])
}
Unfortunately, we don't have anything like it for existential types, we can't introduce a helper type C in some scope S such that when C is eliminated, we are left with an existential (at least I don't see a general way to do it).
Therefore, the only interesting question that is to be answered here is:
Given a Vector[BatchOp[X] forSome X] for which I know that there is one common type C such that they all are actually Vector[C], where is the scope in which this C is present as a usable type variable?
It turns out that BatchableOp[C] itself has a type variable C in scope. Therefore, I can add a method collectSameClassInstances to BachableOp[C], and this method will actually have some type C available that it can use in the return type. Then I can immediately pass the result of collectSameClassInstances to the resolve method, and then I get a completely benign Vector[Result] type as output.
Final remark: If you decide to write any code with F-bounded polymorphisms and existentials, at least make sure that you have documented very clearly what exactly you are doing there, and how you will ensure that this combination does not escape in any other parts of the codebase. It doesn't feel like a good idea to expose such interfaces to the users. Keep it localized, make sure these abstractions do not leak anywhere.
Andrey's answer has a key insight that the only scope with the appropriate type variable is on the BatchableOp itself. Here's a reduced version that doesn't rely on importing existentials:
trait Result
trait BatchableOp[T <: BatchableOp[T]] {
def resolve(batch: Vector[T]): Vector[Result]
def unsafeResolve(batch: Vector[BatchableOp[_]]): Vector[Result] = {
resolve(batch.asInstanceOf[Vector[T]])
}
}
def execute(operations: Vector[BatchableOp[_]]): Vector[Result] = {
operations
.groupBy(_.getClass)
.toVector
.flatMap{ case (_, batch) =>
batch.head.unsafeResolve(batch)
}
}
Let's define a trait with an abstract class
object Outer {
trait X {
type T
val empty: T
}
Now we can make an instance of it:
val x = new X {
type T = Int
val empty = 42
}
Scala now recognizes, that x.empty is an Int:
def xEmptyIsInt = x.empty: Int
Now, let's define an other class
case class Y(x: X) extends X {
type T = x.T
val empty = x.empty
}
and make an instance of it
val y = Y(x)
But now Scala, isn't able to infer that y.empty is of type Int. The following
def yEmptyIsInt = y.empty: Int
now produces an error message:
error: type mismatch;
found : y.x.T
required: Int
y.empty : Int
Why is this the case? Is this a scala bug?
We can mitigate this issue by introducing a parameters to the case class:
case class Z[U](x: X { type T = U }) extends X {
type T = U
val empty = x.empty
}
Then it works again
val z = Z(x)
def zEmptyIsInt: Int = z.empty
But we always have to mention all the types inside X at call-site. This ideally should be an implementation detail which leads to the following approach:
case class A[U <: X](z: U) extends X {
type T = z.T
val empty = z.empty
}
This also mitigates the issue
val a = A(x)
def aEmptyIsInt: Int = a.empty
}
So, to summarize, my questions are the following: Why does the simple case doesn't work? Is this a valid workaround? What other problems might come up when we follow one of the two workaround approaches? Is there a better approach?
You've re-used x for different things, so from here on I'll call the object instantiated by val x "instance x" and the x: X used in class Y "parameter x".
"Instance x" is an anonymous subclass of trait X with concrete members overriding trait X's abstract members. As a subclass of X you can pass it to the constructor for case class Y and it will be accepted happily, since as a subclass it is an X.
It seems to me you expect that case class Y will then check at runtime to see if the instance of X it is passed has overridden X's members, and generate an instance of Y whose members have different types depending on what was passed in.
This is emphatically not how Scala works, and would pretty much defeat the purpose of its (static) type system. For example, you wouldn't be able to do anything useful with Y.empty without runtime reflection since it could have any type at all, and at that point you're better off just using a dynamic type system. If you want the benefits of parametricity, free theorems, not needing reflection, etc. then you have to stick to statically determined types (with a small exception for pattern matching). And that's what Scala does here.
What actually happens is that you've told Y's constructor that parameter x is an X, and so the compiler statically determines that x.empty, has the type of X.empty, which is abstract type T. Scala's inference isn't failing; your types are actually mismatched.
Separately, this doesn't really have anything to do with path-dependent types. Here is a good walkthrough, but in short, path-dependent types are bound to their parent's instance, not determined dynamically at runtime.
I just learned Scala. Now I am confused about Contravariance and Covariance.
From this page, I learned something below:
Covariance
Perhaps the most obvious feature of subtyping is the ability to replace a value of a wider type with a value of a narrower type in an expression. For example, suppose I have some types Real, Integer <: Real, and some unrelated type Boolean. I can define a function is_positive :: Real -> Boolean which operates on Real values, but I can also apply this function to values of type Integer (or any other subtype of Real). This replacement of wider (ancestor) types with narrower (descendant) types is called covariance. The concept of covariance allows us to write generic code and is invaluable when reasoning about inheritance in object-oriented programming languages and polymorphism in functional languages.
However, I also saw something from somewhere else:
scala> class Animal
defined class Animal
scala> class Dog extends Animal
defined class Dog
scala> class Beagle extends Dog
defined class Beagle
scala> def foo(x: List[Dog]) = x
foo: (x: List[Dog])List[Dog] // Given a List[Dog], just returns it
scala> val an: List[Animal] = foo(List(new Beagle))
an: List[Animal] = List(Beagle#284a6c0)
Parameter x of foo is contravariant; it expects an argument of type List[Dog], but we give it a List[Beagle], and that's okay
[What I think is the second example should also prove Covariance. Because from the first example, I learned that "apply this function to values of type Integer (or any other subtype of Real)". So correspondingly, here we apply this function to values of type List[Beagle](or any other subtype of List[Dog]). But to my surprise, the second example proves Cotravariance]
I think two are talking the same thing, but one proves Covariance and the other Contravariance. I also saw this question from SO. However I am still confused. Did I miss something or one of the examples is wrong?
A Good recent article (August 2016) on that topic is "Cheat Codes for Contravariance and Covariance" by Matt Handler.
It starts from the general concept as presented in "Covariance and Contravariance of Hosts and Visitors" and diagram from Andre Tyukin and anoopelias's answer.
And its concludes with:
Here is how to determine if your type ParametricType[T] can/cannot be covariant/contravariant:
A type can be covariant when it does not call methods on the type that it is generic over.
If the type needs to call methods on generic objects that are passed into it, it cannot be covariant.
Archetypal examples:
Seq[+A], Option[+A], Future[+T]
A type can be contravariant when it does call methods on the type that it is generic over.
If the type needs to return values of the type it is generic over, it cannot be contravariant.
Archetypal examples:
`Function1[-T1, +R]`, `CanBuildFrom[-From, -Elem, +To]`, `OutputChannel[-Msg]`
Regarding contravariance,
Functions are the best example of contravariance
(note that they’re only contravariant on their arguments, and they’re actually covariant on their result).
For example:
class Dachshund(
name: String,
likesFrisbees: Boolean,
val weinerness: Double
) extends Dog(name, likesFrisbees)
def soundCuteness(animal: Animal): Double =
-4.0/animal.sound.length
def weinerosity(dachshund: Dachshund): Double =
dachshund.weinerness * 100.0
def isDogCuteEnough(dog: Dog, f: Dog => Double): Boolean =
f(dog) >= 0.5
Should we be able to pass weinerosity as an argument to isDogCuteEnough? The answer is no, because the function isDogCuteEnough only guarantees that it can pass, at most specific, a Dog to the function f.
When the function f expects something more specific than what isDogCuteEnough can provide, it could attempt to call a method that some Dogs don’t have (like .weinerness on a Greyhound, which is insane).
That you can pass a List[Beagle] to a function expecting a List[Dog] is nothing to do with contravariance of functions, it is still because List is covariant and that List[Beagle] is a List[Dog].
Instead lets say you had a function:
def countDogsLegs(dogs: List[Dog], legCountFunction: Dog => Int): Int
This function counts all the legs in a list of dogs. It takes a function that accepts a dog and returns an int representing how many legs this dog has.
Furthermore lets say we have a function:
def countLegsOfAnyAnimal(a: Animal): Int
that can count the legs of any animal. We can pass our countLegsOfAnyAnimal function to our countDogsLegs function as the function argument, this is because if this thing can count the legs of any animal, it can count legs of dogs, because dogs are animals, this is because functions are contravariant.
If you look at the definition of Function1 (functions of one parameter), it is
trait Function1[-A, +B]
That is that they are contravariant on their input and covariant on their output. So Function1[Animal,Int] <: Function1[Dog,Int] since Dog <: Animal
Variance is used to indicate subtyping in terms of Containers(eg: List). In most of the languages, if a function requests object of Class Animal, passing any class that inherits Animal(eg: Dog) would be valid. However, in terms of Containers, these need not be valid.
If your function wants Container[A], what are the possible values that can be passed to it? If B extends A and passing Container[B] is valid, then it is Covariant(eg: List[+T]). If, A extends B(the inverse case) and passing Container[B] for Container[A] is valid, then it is Contravariant. Else, it is invariant(which is the default). You could refer to an article where I have tried explaining variances in Scala
https://blog.lakshmirajagopalan.com/posts/variance-in-scala/
I found myself really can't understand the difference between "Generic type" and "higher-kinded type".
Scala code:
trait Box[T]
I defined a trait whose name is Box, which is a type constructor that accepts a parameter type T. (Is this sentence correct?)
Can I also say:
Box is a generic type
Box is a higher-kinded type
None of above is correct
When I discuss the code with my colleagues, I often struggle between the word "generic" and "higher-kinde" to express it.
It's probably too late to answer now, and you probably know the difference by now, but I'm going to answer just to offer an alternate perspective, since I'm not so sure that what Greg says is right. Generics is more general than higher kinded types. Lots of languages, such as Java and C# have generics, but few have higher-kinded types.
To answer your specific question, yes, Box is a type constructor with a type parameter T. You can also say that it is a generic type, although it is not a higher kinded type. Below is a broader answer.
This is the Wikipedia definition of generic programming:
Generic programming is a style of computer programming in which algorithms are written in terms of types to-be-specified-later that are then instantiated when needed for specific types provided as parameters. This approach, pioneered by ML in 1973,1 permits writing common functions or types that differ only in the set of types on which they operate when used, thus reducing duplication.
Let's say you define Box like this. It holds an element of some type, and has a few special methods. It also defines a map function, something like Iterable and Option, so you can take a box holding an integer and turn it into a box holding a string, without losing all those special methods that Box has.
case class Box(elem: Any) {
..some special methods
def map(f: Any => Any): Box = Box(f(elem))
}
val boxedNum: Box = Box(1)
val extractedNum: Int = boxedString.elem.asInstanceOf[Int]
val boxedString: Box = boxedNum.map(_.toString)
val extractedString: String = boxedString.elem.asInstanceOf[String]
If Box is defined like this, your code would get really ugly because of all the calls to asInstanceOf, but more importantly, it's not typesafe, because everything is an Any.
This is where generics can be useful. Let's say we define Box like this instead:
case class Box[A](elem: A) {
def map[B](f: A => B): Box[B] = Box(f(elem))
}
Then we can use our map function for all kinds of stuff, like changing the object inside the Box while still making sure it's inside a Box. Here, there's no need for asInstanceOf since the compiler knows the type of your Boxes and what they hold (even the type annotations and type arguments are not necessary).
val boxedNum: Box[Int] = Box(1)
val extractedNum: Int = boxedNum.elem
val boxedString: Box[String] = boxedNum.map[String](_.toString)
val extractedString: String = boxedString.elem
Generics basically lets you abstract over different types, letting you use Box[Int] and Box[String] as different types even though you only have to create one Box class.
However, let's say that you don't have control over this Box class, and it's defined merely as
case class Box[A](elem: A) {
//some special methods, but no map function
}
Let's say this API you're using also defines its own Option and List classes (both accepting a single type parameter representing the type of the elements). Now you want to be able to map over all these types, but since you can't modify them yourself, you'll have to define an implicit class to create an extension method for them. Let's add an implicit class Mappable for the extension method and a typeclass Mapper.
trait Mapper[C[_]] {
def map[A, B](context: C[A])(f: A => B): C[B]
}
implicit class Mappable[C[_], A](context: C[A])(implicit mapper: Mapper[C]) {
def map[B](f: A => B): C[B] = mapper.map(context)(f)
}
You could define implicit Mappers like so
implicit object BoxMapper extends Mapper[Box] {
def map[B](box: Box[A])(f: A => B): Box[B] = Box(f(box.elem))
}
implicit object OptionMapper extends Mapper[Option] {
def map[B](opt: Option[A])(f: A => B): Option[B] = ???
}
implicit object ListMapper extends Mapper[List] {
def map[B](list: List[A])(f: A => B): List[B] = ???
}
//and so on
and use it as if Box, Option, List, etc. have always had map methods.
Here, Mappable and Mapper are higher-kinded types, whereas Box, Option, and List are first-order types. All of them are generic types and type constructors. Int and String, however, are proper types. Here are their kinds, (kinds are to types as types are to values).
//To check the kind of a type, you can use :kind in the REPL
Kind of Int and String: *
Kind of Box, Option, and List: * -> *
Kind of Mappable and Mapper: (* -> *) -> *
Type constructors are somewhat analogous to functions (which are sometimes called value constructors). A proper type (kind *) is analogous to a simple value. It's a concrete type that you can use for return types, as the types of your variables, etc. You can just directly say val x: Int without passing Int any type parameters.
A first-order type (kind * -> *) is like a function that looks like Any => Any. Instead of taking a value and giving you a value, it takes a type and gives you another type. You can't use first-order types directly (val x: List won't work) without giving them type parameters (val x: List[Int] works). This is what generics does - it lets you abstract over types and create new types (the JVM just erases that information at runtime, but languages like C++ literally generate new classes and functions). The type parameter C in Mapper is also of this kind. The underscore type parameter (you could also use something else, like x) lets the compiler know that C is of kind * -> *.
A higher-kinded type/higher-order type is like a higher-order function - it takes another type constructor as a parameter. You can't use a Mapper[Int] above, because C is supposed to be of kind * -> * (so that you can do C[A] and C[B]), whereas Int is merely *. It's only in languages like Scala and Haskell with higher-kinded types that you can create types like Mapper above and other things beyond languages with more limited type systems, like Java.
This answer (and others) on a similar question may also help.
Edit: I've stolen this very helpful image from that same answer:
There is no difference between 'Higher-Kinded Types' and 'Generics'.
Box is a 'structure' or 'context' and T can be any type.
So T is generic in the English sense... we don't know what it will be and we don't care because we aren't going to be operating on T directly.
C# also refers to these as Generics. I suspect they chose this language because of its simplicity (to not scare people away).
I defined a trait:
trait A {
def hello(name:Any):Any
}
Then define a class X to implement it:
class X extends A {
def hello(name:Any): Any = {}
}
It compiled. Then I change the return type in the subclass:
class X extends A {
def hello(name:Any): String = "hello"
}
It also compiled. Then change the parameter type:
class X extends A {
def hello(name:String): Any = {}
}
It can't compiled this time, the error is:
error: class X needs to be abstract, since method hello in trait A of type (name: Any)
Any is not defined
(Note that Any does not match String: class String in package lang is a subclass
of class Any in package scala, but method parameter types must match exactly.)
It seems the parameter should match exactly, but the return type can be a subtype in subclass?
Update: #Mik378, thanks for your answer, but why the following example can't work? I think it doesn't break Liskov:
trait A {
def hello(name:String):Any
}
class X extends A {
def hello(name:Any): Any = {}
}
It's exactly like in Java, to keep Liskov Substitution principle, you can't override a method with a more finegrained parameter.
Indeed, what if your code deals with the A type, referencing an X type under the hood.
According to A, you can pass Any type you want, but B would allow only String.
Therefore => BOOM
Logically, with the same reasonning, a more finegrained return type is allowed since it would be cover whatever the case is by any code dealing with the A class.
You may want to check those parts:
http://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science)#Covariant_method_return_type
and
http://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science)#Contravariant_method_argument_type
UPDATE----------------
trait A {
def hello(name:String):Any
}
class X extends A {
def hello(name:Any): Any = {}
}
It would act as a perfect overloading, not an overriding.
In Scala, it's possible to have methods with the same name but different parameters:
class X extends A {
def hello(name:String) = "String"
def hello(name:Any) = "Any"
}
This is called method overloading, and mirrors the semantics of Java (although my example is unusual - normally overloaded methods would do roughly the same thing, but with different combinations of parameters).
Your code doesn't compile, because parameter types need to match exactly for overriding to work. Otherwise, it interprets your method as a new method with different parameter types.
However, there is no facility within Scala or Java to allow overloading of return types - overloading only depends on the name and parameter types. With return type overloading it would be impossible to determine which overloaded variant to use in all but the simplest of cases:
class X extends A {
def hello: Any = "World"
def hello: String = "Hello"
def doSomething = {
println(hello.toString) // which hello do we call???
}
}
This is why your first example compiles with no problem - there is no ambiguity about which method you are implementing.
Note for JVM pedants - technically, the JVM does distinguish between methods with different return types. However, Java and Scala are careful to only use this facility as an optimisation, and it is not reflected in the semantics of Java or Scala.
This is off the top of my head, but basically for X.hello to fit the requirements of A.hello, you need for the input of X.hello to be a superclass of A.hello's input (covariance) and for the output of X.hello to be a subclass of A.hello's output(contravariance).
Think of this is a specific case of the following
class A
class A' extends A
class B
class B' extends B
f :: A' -> B
g :: A -> B'
the question is "Can I replace f with g in an expression y=f(x) and still typecheck in the same situations?
In that expression, y is of type B and x is of type A'
In y=f(x) we know that y is of type B and x is of type A'
g(x) is still fine because x is of type A' (thus of type A)
y=g(x) is still fine because g(x) is of type B' (thus of type B)
Actually the easiest way to see this is that y is a subtype of B (i.e. implements at least B), meaning that you can use y as a value of type B. You have to do some mental gymnastics in the other thing.
(I just remember that it's one direction on the input, another on the output, and try it out... it becomes obvious if you think about it).