I have read this article on covariance/contravariance: http://julien.richard-foy.fr/blog/2013/02/21/be-friend-with-covariance-and-contravariance/
The examples are very clear. However, I am struggling to understand the conclusions drawn at the end:
If you look at the definitions of Run[+A] and Vet[-A] you may notice
that the type Aappears only in the return type of methods of Run[+A]
and only in the parameters of methods of Vet[-A]. More generally a
type that produces values of type A can be made covariant on A (as you
did with Run[+A]), and a type that consumes values of type A can be
made contravariant on A (as you did with Vet[-A]).
From the above paragraph you can deduce that types that only have
getters can be covariant (in other words, immutable data types can be
covariant, and that’s the case for most of the data types of Scala’s
standard library), but mutable data types are necessarily invariant
(they have getters and setters, so they both produce and consume
values).
Producers: If something produces type A, I can imagine some reference variable of type A being set to an object of type A or any subtypes of A, but not supertypes, so it's appropriate that it can be covariant.
Consumers: If something consumes type A, I guess that means type A may be used as parameters in methods. I'm not clear what relationship this has to covariance or contravariance.
It seems from the examples that specifying a type as covariant/contravariant affects how it can be consumed by other functions but not sure how it affects the classes themselves.
It seems from the examples that specifying a type as covariant/contravariant affects how it can be consumed by other functions but not sure how it affects the classes themselves.
It is right that the article focused on the consequences of variance for users of a class, not for implementers of a class.
The article shows that covariant and contravariant types give more freedom to users (because a function that accepts a Run[Mammal] effectively accepts a Run[Giraffe] or a Run[Zebra]). For implementors, the perspective is dual: covariant and contravariant types give them more constraints.
These constraints are that covariant types can not occur in contravariant positions and vice versa.
Consider for instance this Producer type definition:
trait Producer[+A] {
def produce(): A
}
The type parameter A is covariant. Therefore we can only use it in covariant positions (such as a method return type), but we can not use it in contravariant position (such as a method parameter):
trait Producer[+A] {
def produce(): A
def consume(a: A): Unit // (does not compile because A is in contravariant position)
}
Why is it illegal to do so? What could go wrong if this code compiled? Well, consider the following scenario. First, get some Producer[Zebra]:
val zebraProducer: Producer[Zebra] = …
Then upcast it to a Producer[Mammal] (which is legal, because we claimed the type parameter to be covariant):
val mammalProducer: Producer[Mammal] = zebraProducer
Finally, feed it with a Giraffe (which is legal too because the consume method a Producer[Mammal] accepts a Mammal, and a Giraffe is a Mammal):
mammalProducer.consume(new Giraffe)
However, if you remember well, the mammalProducer was actually a zebraProducer, so its consume implementation actually only accepts a Zebra, not a Giraffe! So, in practice, if it was allowed to use covariant types in contravariant positions (like I did with the consume method), the type system would be unsound. We can construct a similar scenario (leading to an absurdity) if we pretend that a class with a contravariant type parameter can also have a method where it is in covariant position (see at the end for the code).
(Note that several programming languages, e.g. Java or TypeScript, have such unsound type systems.)
In practice, in Scala if we want to use a covariant type parameter in contravariant position, we have to use the following trick:
trait Producer[+A] {
def produce(): A
def consume[B >: A](b: B): Unit
}
In that case, a Producer[Zebra] would not expect to get an actual Zebra passed in the consume method (but any value of a type B, lower-bounded by Zebra), so it would be legal to pass a Giraffe, which is a Mammal, which is a super-type of Zebra.
Appendix: similar scenario for contravariance
Consider the following class Consumer[-A], which has a contravariant type parameter A:
trait Consumer[-A] {
def consume(a: A): Unit
}
Suppose that the type system allowed us to define a method where A is in covariant position:
trait Consumer[-A] {
def consume(a: A): Unit
def produce(): A // (does not actually compile because A is in covariant position)
}
Now we can get an instance of Consumer[Mammal], upcast it to Consumer[Zebra] (because of contravariance) and call the produce method to get a Zebra:
val mammalConsumer: Consumer[Mammal] = …
val zebraConsumer: Consumer[Zebra] = mammalConsumer // legal, because we claimed `A` to be contravariant
val zebra: Zebra = zebraConsumer.produce()
However, our zebraConsumer is actually mammalConsumer, whose method produce can return any Mammal, not just Zebras. So, at the end, zebra might be initialized to some Mammal that is not a Zebra! In order to avoid such absurdities, the type system forbids us to define the produce method in the Consumer class.
I'm confused on how Scala's Any relates to java.lang.Object. I know that in scala, AnyRef corresponds to object, but it seems to make a difference whether the method (which takes java.lang.Object) is defined in a java class or a scala class):
the java class:
public class JavaClass {
public static void method(Object input) {
}
}
the scala application:
object ScalaObject extends App{
def method(input:java.lang.Object) = {}
val a:Any = null
method(a) // does not work
JavaClass.method(a) // does work
}
So if the method is in a java-Class, then the compiler allows me to pass a variable of type Any, why is that?
The compiler tries to "make up" for the difference between Scala's and Java's type systems. In Scala, Object =:= AnyRef (they're aliases) and AnyRef <: Any. Therefore, a Scala method that takes Object or AnyRef cannot take an Any or an AnyVal. If you wanted a method that worked on everything, well, then you would have written Any, right?
However, Java methods that take Object are normally meant to work on all values, whether they be actual Objects or primitives (int, long, etc.), and they work due to the boxing conversion of primitives into Objects. Primitives and Object do not have a common supertype like they do in Scala. The Java type system is not expressive enough to differentiate "I only want actual objects," from "I will take anything, be they object or primitive."
Therefore, the Scala compiler patches this up by turning Java methods of Object into methods of Any. This feature is simply to ease interop between the languages. It won't apply this transformation to Scala code though, because if you wanted that behavior then you would have actually written Any instead of Object.
The reason for that is that Any can be either AnyRef or AnyVal, while method can only accept objects which are AnyRef. If you modify the a type to be AnyRef, it is going to work:
def method(input: java.lang.Object) = {}
val a: AnyRef = new Object
method(a)
In case of calling the static Java method, the Scala compiler will turn Any into Object, which also includes boxing of AnyVal values.
I am trying to understand the following piece of code. but I don't know what the R#X mean. could someone help me?
// define the abstract types and bounds
trait Recurse {
type Next <: Recurse
// this is the recursive function definition
type X[R <: Recurse] <: Int
}
// implementation
trait RecurseA extends Recurse {
type Next = RecurseA
// this is the implementation
type X[R <: Recurse] = R#X[R#Next]
}
object Recurse {
// infinite loop
type C = RecurseA#X[RecurseA]
}
You may gain type from existing instance of a class:
class C {
type someType = Int
}
val c = new C
type t = c.someType
Or may address to the type directly without instantiating an object: C#someType This form is very usefull for type expressions where you have no space to create intermediate variables.
Adding some clarifications as it was suggested in comments.
Disclaimer: I have only partial understanding of how Scala's type system works. I'd tried to read documentation several times, but was able to extract only patchy knowledges from it. But I have rich experience in scala and may predict compilers behavior on individual cases well.
# called type projection and type projection compliments normal hierarchical type access via . In every type expression scala implicitly uses both.
scala reference gives examples of such invisible conversions:
t ə.type#t
Int scala.type#Int
scala.Int scala.type#Int
data.maintable.Node data.maintable.type#Node
As use see, every trivial usage of type projection actually works on type (that is return with .type) not on an object. The main practical difference (I'm bad with definitions) is that object type is something ephemeral as object itself is. Its type may be changed in appropriate circumstances such as inheritance of an abstract class type. In contrast type's type (the definition of the type projection) is as stable as sun. Types (don't mix them with classes) in scala are not first-class citizens and can not be overridden further.
There are different places suitable for putting type expression into. There are also some places where only stable types are allowed. So basically type projection is more constant for terms of type.
When I compile:
object Test extends App {
implicit def pimp[V](xs: Seq[V]) = new {
def dummy(x: V) = x
}
}
I get:
$ fsc -d aoeu go.scala
go.scala:3: error: Parameter type in structural refinement may not refer to an abstract type defined outside that refinement
def dummy(x: V) = x
^
one error found
Why?
(Scala: "Parameter type in structural refinement may not refer to an abstract type defined outside that refinement" doesn't really answer this.)
It's disallowed by the spec. See 3.2.7 Compound Types.
Within a method declaration in a structural refinement, the type of any value parameter may only refer to type parameters or abstract types that are contained inside the refinement. That is, it must refer either to a type parameter of the method
itself, or to a type definition within the refinement. This restriction does not apply
to the function’s result type.
Before Bug 1906 was fixed, the compiler would have compiled this and you'd have gotten a method not found at runtime. This was fixed in revision 19442 and this is why you get this wonderful message.
The question is then, why is this not allowed?
Here is very detailed explanation from Gilles Dubochet from the scala mailing list back in 2007. It roughly boils down to the fact that structural types use reflection and the compiler does not know how to look up the method to call if it uses a type defined outside the refinement (the compiler does not know ahead of time how to fill the second parameter of getMethod in p.getClass.getMethod("pimp", Array(?))
But go look at the post, it will answer your question and some more.
Edit:
Hello list.
I try to define structural types with abstract datatype in function
parameter. ... Any reason?
I have heard about two questions concerning the structural typing
extension of Scala 2.6 lately, and I would like to answer them here.
Why did we change Scala's native values (“int”, etc.) boxing scheme
to Java's (“java.lang.Integer”).
Why is the restriction on parameters for structurally defined
methods (“Parameter type in structural refinement may not refer
to abstract type defined outside that same refinement”) required.
Before I can answer these two questions, I need to speak about the
implementation of structural types.
The JVM's type system is very basic (and corresponds to Java 1.4). That
means that many types that can be represented in Scala cannot be
represented in the VM. Path dependant types (“x.y.A”), singleton types
(“a.type”), compound types (“A with B”) or abstract types are all types
that cannot be represented in the JVM's type system.
To be able to compile to JVM bytecode, the Scala compilers changes the
Scala types of the program to their “erasure” (see section 3.6 of the
reference). Erased types can be represented in the VM's type system and
define a type discipline on the program that is equivalent to that of
the program typed with Scala types (saving some casts), although less
precise. As a side note, the fact that types are erased in the VM
explains why operations on the dynamic representation of types (pattern
matching on types) are very restricted with respect to Scala's type
system.
Until now all type constructs in Scala could be erased in some way.
This isn't true for structural types. The simple structural type “{ def
x: Int }” can't be erased to “Object” as the VM would not allow
accessing the “x” field. Using an interface “interface X { int x{}; }”
as the erased type won't work either because any instance bound by a
value of this type would have to implement that interface which cannot
be done in presence of separate compilation. Indeed (bear with me) any
class that contains a member of the same name than a member defined in
a structural type anywhere would have to implement the corresponding
interface. Unfortunately this class may be defined even before the
structural type is known to exist.
Instead, any reference to a structurally defined member is implemented
as a reflective call, completely bypassing the VM's type system. For
example def f(p: { def x(q: Int): Int }) = p.x(4) will be rewritten
to something like:
def f(p: Object) = p.getClass.getMethod("x", Array(Int)).invoke(p, Array(4))
And now the answers.
“invoke” will use boxed (“java.lang.Integer”) values whenever the
invoked method uses native values (“int”). That means that the above
call must really look like “...invoke(p, Array(new
java.lang.Integer(4))).intValue”.
Integer values in a Scala program are already often boxed (to allow the
“Any” type) and it would be wasteful to unbox them from Scala's own
boxing scheme to rebox them immediately as java.lang.Integer.
Worst still, when a reflective call has the “Any” return type,
what should be done when a java.lang.Integer is returned? The called
method may either be returning an “int” (in which case it should be
unboxed and reboxed as a Scala box) or it may be returning a
java.lang.Integer that should be left untouched.
Instead we decided to change Scala's own boxing scheme to Java's. The
two previous problems then simply disappear. Some performance-related
optimisations we had with Scala's boxing scheme (pre-calculate the
boxed form of the most common numbers) were easy to use with Java
boxing too. In the end, using Java boxing was even a bit faster than
our own scheme.
“getMethod”'s second parameter is an array with the types of the
parameters of the (structurally defined) method to lookup — for
selecting which method to get when the name is overloaded. This is the
one place where exact, static types are needed in the process of
translating a structural member call. Usually, exploitable static types
for a method's parameter are provided with the structural type
definition. In the example above, the parameter type of “x” is known to
be “Int”, which allows looking it up.
Parameter types defined as abstract types where the abstract type is
defined inside the scope of the structural refinement are no problem
either:
def f(p: { def x[T](t: T): Int }) = p.xInt
In this example we know that any instance passed to “f” as “p” will
define “x[T](t: T)” which is necessarily erased to “x(t: Object)”. The
lookup is then correctly done on the erased type:
def f(p: Object) = p.getClass.getMethod("x", Array(Object)).invoke(p,
Array(new java.lang.Integer(4)))
But if an abstract type from outside the structural refinement's scope
is used to define a parameter of a structural method, everything breaks:
def f[T](p: { def x(t: T): Int }, t: T) = p.x(t)
When “f” is called, “T” can be instantiated to any type, for example:
f[Int]({ def x(t: Int) = t }, 4)
f[Any]({ def x(t: Any) = 5 }, 4)
The lookup for the first case would have to be “getMethod("x",
Array(int))” and for the second “getMethod("x", Array(Object))”, and
there is no way to know which one to generate in the body of
“f”: “p.x(t)”.
To allow defining a unique “getMethod” call inside “f”'s body for
any instantiation of “T” would require any object passed to “f” as the
“p” parameter to have the type of “t” erased to “Any”. This would be a
transformation where the type of a class' members depend on how
instances of this class are used in the program. And this is something
we definitely don't want to do (and can't be done with separate
compilation).
Alternatively, if Scala supported run-time types one could use them to
solve this problem. Maybe one day ...
But for now, using abstract types for structural method's parameter
types is simply forbidden.
Sincerely,
Gilles.
Discovered the problem shortly after posting this: I have to define a named class instead of using an anonymous class. (Still would love to hear a better explanation of the reasoning though.)
object Test extends App {
case class G[V](xs: Seq[V]) {
def dummy(x: V) = x
}
implicit def pimp[V](xs: Seq[V]) = G(xs)
}
works.
As I understand from this blog post "type classes" in Scala is just a "pattern" implemented with traits and implicit adapters.
As the blog says if I have trait A and an adapter B -> A then I can invoke a function, which requires argument of type A, with an argument of type B without invoking this adapter explicitly.
I found it nice but not particularly useful. Could you give a use case/example, which shows what this feature is useful for ?
One use case, as requested...
Imagine you have a list of things, could be integers, floating point numbers, matrices, strings, waveforms, etc. Given this list, you want to add the contents.
One way to do this would be to have some Addable trait that must be inherited by every single type that can be added together, or an implicit conversion to an Addable if dealing with objects from a third party library that you can't retrofit interfaces to.
This approach becomes quickly overwhelming when you also want to begin adding other such operations that can be done to a list of objects. It also doesn't work well if you need alternatives (for example; does adding two waveforms concatenate them, or overlay them?) The solution is ad-hoc polymorphism, where you can pick and chose behaviour to be retrofitted to existing types.
For the original problem then, you could implement an Addable type class:
trait Addable[T] {
def zero: T
def append(a: T, b: T): T
}
//yup, it's our friend the monoid, with a different name!
You can then create implicit subclassed instances of this, corresponding to each type that you wish to make addable:
implicit object IntIsAddable extends Addable[Int] {
def zero = 0
def append(a: Int, b: Int) = a + b
}
implicit object StringIsAddable extends Addable[String] {
def zero = ""
def append(a: String, b: String) = a + b
}
//etc...
The method to sum a list then becomes trivial to write...
def sum[T](xs: List[T])(implicit addable: Addable[T]) =
xs.FoldLeft(addable.zero)(addable.append)
//or the same thing, using context bounds:
def sum[T : Addable](xs: List[T]) = {
val addable = implicitly[Addable[T]]
xs.FoldLeft(addable.zero)(addable.append)
}
The beauty of this approach is that you can supply an alternative definition of some typeclass, either controlling the implicit you want in scope via imports, or by explicitly providing the otherwise implicit argument. So it becomes possible to provide different ways of adding waveforms, or to specify modulo arithmetic for integer addition. It's also fairly painless to add a type from some 3rd-party library to your typeclass.
Incidentally, this is exactly the approach taken by the 2.8 collections API. Though the sum method is defined on TraversableLike instead of on List, and the type class is Numeric (it also contains a few more operations than just zero and append)
Reread the first comment there:
A crucial distinction between type classes and interfaces is that for class A to be a "member" of an interface it must declare so at the site of its own definition. By contrast, any type can be added to a type class at any time, provided you can provide the required definitions, and so the members of a type class at any given time are dependent on the current scope. Therefore we don't care if the creator of A anticipated the type class we want it to belong to; if not we can simply create our own definition showing that it does indeed belong, and then use it accordingly. So this not only provides a better solution than adapters, in some sense it obviates the whole problem adapters were meant to address.
I think this is the most important advantage of type classes.
Also, they handle properly the cases where the operations don't have the argument of the type we are dispatching on, or have more than one. E.g. consider this type class:
case class Default[T](val default: T)
object Default {
implicit def IntDefault: Default[Int] = Default(0)
implicit def OptionDefault[T]: Default[Option[T]] = Default(None)
...
}
I think of type classes as the ability to add type safe metadata to a class.
So you first define a class to model the problem domain and then think of metadata to add to it. Things like Equals, Hashable, Viewable, etc. This creates a separation of the problem domain and the mechanics to use the class and opens up subclassing because the class is leaner.
Except for that, you can add type classes anywhere in the scope, not just where the class is defined and you can change implementations. For example, if I calculate a hash code for a Point class by using Point#hashCode, then I'm limited to that specific implementation which may not create a good distribution of values for the specific set of Points I have. But if I use Hashable[Point], then I may provide my own implementation.
[Updated with example]
As an example, here's a use case I had last week. In our product there are several cases of Maps containing containers as values. E.g., Map[Int, List[String]] or Map[String, Set[Int]]. Adding to these collections can be verbose:
map += key -> (value :: map.getOrElse(key, List()))
So I wanted to have a function that wraps this so I could write
map +++= key -> value
The main issue is that the collections don't all have the same methods for adding elements. Some have '+' while others ':+'. I also wanted to retain the efficiency of adding elements to a list, so I didn't want to use fold/map which create new collections.
The solution is to use type classes:
trait Addable[C, CC] {
def add(c: C, cc: CC) : CC
def empty: CC
}
object Addable {
implicit def listAddable[A] = new Addable[A, List[A]] {
def empty = Nil
def add(c: A, cc: List[A]) = c :: cc
}
implicit def addableAddable[A, Add](implicit cbf: CanBuildFrom[Add, A, Add]) = new Addable[A, Add] {
def empty = cbf().result
def add(c: A, cc: Add) = (cbf(cc) += c).result
}
}
Here I defined a type class Addable that can add an element C to a collection CC. I have 2 default implementations: For Lists using :: and for other collections, using the builder framework.
Then using this type class is:
class RichCollectionMap[A, C, B[_], M[X, Y] <: collection.Map[X, Y]](map: M[A, B[C]])(implicit adder: Addable[C, B[C]]) {
def updateSeq[That](a: A, c: C)(implicit cbf: CanBuildFrom[M[A, B[C]], (A, B[C]), That]): That = {
val pair = (a -> adder.add(c, map.getOrElse(a, adder.empty) ))
(map + pair).asInstanceOf[That]
}
def +++[That](t: (A, C))(implicit cbf: CanBuildFrom[M[A, B[C]], (A, B[C]), That]): That = updateSeq(t._1, t._2)(cbf)
}
implicit def toRichCollectionMap[A, C, B[_], M[X, Y] <: col
The special bit is using adder.add to add the elements and adder.empty to create new collections for new keys.
To compare, without type classes I would have had 3 options:
1. to write a method per collection type. E.g., addElementToSubList and addElementToSet etc. This creates a lot of boilerplate in the implementation and pollutes the namespace
2. to use reflection to determine if the sub collection is a List / Set. This is tricky as the map is empty to begin with (of course scala helps here also with Manifests)
3. to have poor-man's type class by requiring the user to supply the adder. So something like addToMap(map, key, value, adder), which is plain ugly
Yet another way I find this blog post helpful is where it describes typeclasses: Monads Are Not Metaphors
Search the article for typeclass. It should be the first match. In this article, the author provides an example of a Monad typeclass.
The forum thread "What makes type classes better than traits?" makes some interesting points:
Typeclasses can very easily represent notions that are quite difficult to represent in the presence of subtyping, such as equality and ordering.
Exercise: create a small class/trait hierarchy and try to implement .equals on each class/trait in such a way that the operation over arbitrary instances from the hierarchy is properly reflexive, symmetric, and transitive.
Typeclasses allow you to provide evidence that a type outside of your "control" conforms with some behavior.
Someone else's type can be a member of your typeclass.
You cannot express "this method takes/returns a value of the same type as the method receiver" in terms of subtyping, but this (very useful) constraint is straightforward using typeclasses. This is the f-bounded types problem (where an F-bounded type is parameterized over its own subtypes).
All operations defined on a trait require an instance; there is always a this argument. So you cannot define for example a fromString(s:String): Foo method on trait Foo in such a way that you can call it without an instance of Foo.
In Scala this manifests as people desperately trying to abstract over companion objects.
But it is straightforward with a typeclass, as illustrated by the zero element in this monoid example.
Typeclasses can be defined inductively; for example, if you have a JsonCodec[Woozle] you can get a JsonCodec[List[Woozle]] for free.
The example above illustrates this for "things you can add together".
One way to look at type classes is that they enable retroactive extension or retroactive polymorphism. There are a couple of great posts by Casual Miracles and Daniel Westheide that show examples of using Type Classes in Scala to achieve this.
Here's a post on my blog
that explores various methods in scala of retroactive supertyping, a kind of retroactive extension, including a typeclass example.
I don't know of any other use case than Ad-hoc polymorhism which is explained here the best way possible.
Both implicits and typeclasses are used for Type-conversion. The major use-case for both of them is to provide ad-hoc polymorphism(i.e) on classes that you can't modify but expect inheritance kind of polymorphism. In case of implicits you could use both an implicit def or an implicit class (which is your wrapper class but hidden from the client). Typeclasses are more powerful as they can add functionality to an already existing inheritance chain(eg: Ordering[T] in scala's sort function).
For more detail you can see https://lakshmirajagopalan.github.io/diving-into-scala-typeclasses/
In scala type classes
Enables ad-hoc polymorphism
Statically typed (i.e. type-safe)
Borrowed from Haskell
Solves the expression problem
Behavior can be extended
- at compile-time
- after the fact
- without changing/recompiling existing code
Scala Implicits
The last parameter list of a method can be marked implicit
Implicit parameters are filled in by the compiler
In effect, you require evidence of the compiler
… such as the existence of a type class in scope
You can also specify parameters explicitly, if needed
Below Example extension on String class with type class implementation extends the class with a new methods even though string is final :)
/**
* Created by nihat.hosgur on 2/19/17.
*/
case class PrintTwiceString(val original: String) {
def printTwice = original + original
}
object TypeClassString extends App {
implicit def stringToString(s: String) = PrintTwiceString(s)
val name: String = "Nihat"
name.printTwice
}
This is an important difference (needed for functional programming):
consider inc:Num a=> a -> a:
a received is the same that is returned, this cannot be done with subtyping
I like to use type classes as a lightweight Scala idiomatic form of Dependency Injection that still works with circular dependencies yet doesn't add a lot of code complexity. I recently rewrote a Scala project from using the Cake Pattern to type classes for DI and achieved a 59% reduction in code size.