I have a question on type design. Why does Int not extend the Ordered trait. Isn't Int ordered by nature?
Instead, the scala library provides implicit 'orderer' methods which convert Int to Ordered[Int]. What are the design choices being made here?
Example taken from the book Programming in Scala
def maxListImpParm[T <% Ordered[T]](elements:List[T]):T= ...
maxListImpParm(List(1,5,10,3)) // works because of implicit methods
Because Int (and some other classes inherited from AnyVal) is ephemeral -- at runtime it usually represented by primitive value which has no notion of class (and thus inheritance) at all. Of course, there are exceptions, like Int boxing to full blown reference class instance when you put item in collection, but typeclass provides one universal solution. Moreover, typeclasses are more flexible than inheritance.
Related
While going through Programming in Scala, i came across:
While you can define your own value classes (see Section 11.4), there
are nine value classes built into Scala: Byte, Short, Char, Int, Long,
Float, Double, Boolean, and Unit. The first eight of these correspond
to Java's primitive types, and their values are represented at run
time as Java's primitive values. The instances of these classes are
all written as literals in Scala. For example, 42 is an instance of
Int, 'x' is an instance of Char, and false an instance ofBoolean. You
cannot create instances of these classes using new. This is enforced
by the "trick" that value classes are all defined to be both abstract
and final.
Due to which new Int gives the error class Int is abstract; cannot be instantiated
val a: Int = new Int in Scala. Java allows new Integer(23).
Question: What is the trick the author is taking about. Why Scala defines value classes to be abstract and final.
What is the trick the author is taking about ?
The "trick" is that
when a class is abstract, you cannot make instances of it (cannot call new).
when a class is final, you cannot make subclasses
when a class is abstract and you cannot make subclasses, then you also cannot make a concrete subclass that you could instantiate
So as a result, value classes cannot be instantiated by application code.
Why Scala defines value classes to be abstract and final.
The point of value classes is that they are defined by their (immutable) value/contents. The object identity is not relevant.
The Scala compiler also tries to optimize value classes by not creating any objects at all where possible (just using unboxed primitives directly). That only works if we can be sure that you can just box and unbox at will.
In Java new Integer(1) and another new Integer(1) are two different objects, but that is not useful for a pure value class (if you want to use these different instances as lock monitor objects or something else where you need object identity, you are just confusing yourself and others and should not have used Integer).
Say I have a custom class which extends AnyVal and uses a Long internally:
case class Instruction(underlying: Long) extends AnyVal
When I add Instructions to a collection which is specialized for Long, do the Instructions need boxing?
(Are there Scala collections which are specialized for Long? I need an indexed sequence.)
Yes, it will be boxed. Unfortunately, value classes lose all their benefits when used as type arguments (generics) or put into collections. They are boxed always when they need to be seen as any other type than exactly the type of the value class itself.
The reason for that limitation is that to retain sound semantics of Scala language, code like this must work:
case class ValueClass(raw: Long) extends AnyVal
val someList: List[Any] = List[ValueClass](ValueClass(42L))
someList.head match {
case ValueClass(raw) => // boxing needed for this match to work...
case _ => ...
}
Specialization doesn't change anything here, any collection (specialized or not) could be passed somewhere where it's seen as Coll[Any] or Coll[T] where information about exact element type is lost.
If you want an IndexedSeq[Long] with unboxed storage, I think scala.collection.mutable.WrappedArray.ofLong is the closest thing to that. It also has its corresponding builder, scala.collection.mutable.ArrayBuilder.ofLong.
For now Scala doesn't support #specialized for collection.
Currently there is no support for specialization of collections. It would be nice to allow this in the new design if we can do it without too much of an impact on the majority of non-specialized collections.
https://www.scala-lang.org/blog/2017/02/28/collections-rework.html
PS: I think this is caused by the Java doesn't support primitive collection, Since Java generiction is bound to the Object type, but primitive type doesn't derive from Object.
As "Programming in Scala: A comprehensive step-by-step Guide" states, in Scala there are not basic types values, just objects: Integers are Int instances and doubles are Double instances. I assume that these classes map to Java's Integer, Double ... classes and, therefore, are mapped as Object subclasses.
In the book, the following type hierarchy (classes as types) is presented:
Few pages after this graph is presented, you can read:
What somehow troubles me is: If Scala´s Double maps to Java's Double which is an specification of java.lang.Object and AnyRef is an alias for java.lang.Object too, should't AnyVal be a subclass of AnyRef?
EDIT
Few pages after that I read that primitive types are not mapped to Java's primitive types wrapper classes unless their "boxed" versions are required; but I am still confused since it seems to me that not all Scala's objects are java.lang.Object sublcasses instances. That is: There are classes in Scala which could be not translated in the JVM as Object subclasses.
Java does not only have types that extend java.lang.Object (aka scala.AnyRef), but primitive types, e.g. int, double, boolean, ... In Scala you find them under scala.Any. So a scala.Int corresponds to a Java int. Not java.lang.Integer; not until boxing occurs, a mechanism on the JVM to be able to pass primitives to generic methods. Both Java and Scala do auto-boxing, that is construct a reference around a primitive type when a reference is needed.
The difference in Scala is, it doesn't treat scala.Int any different from say String, it doesn't matter whether the type corresponds to a JVM primitive or not. You can call methods on scala.Int as if it was any regular object. In the byte-code you will still have primitive types.
This is why Scala is sometimes called a true or more pure object-oriented language than Java.
When I compile:
object Test extends App {
implicit def pimp[V](xs: Seq[V]) = new {
def dummy(x: V) = x
}
}
I get:
$ fsc -d aoeu go.scala
go.scala:3: error: Parameter type in structural refinement may not refer to an abstract type defined outside that refinement
def dummy(x: V) = x
^
one error found
Why?
(Scala: "Parameter type in structural refinement may not refer to an abstract type defined outside that refinement" doesn't really answer this.)
It's disallowed by the spec. See 3.2.7 Compound Types.
Within a method declaration in a structural refinement, the type of any value parameter may only refer to type parameters or abstract types that are contained inside the refinement. That is, it must refer either to a type parameter of the method
itself, or to a type definition within the refinement. This restriction does not apply
to the function’s result type.
Before Bug 1906 was fixed, the compiler would have compiled this and you'd have gotten a method not found at runtime. This was fixed in revision 19442 and this is why you get this wonderful message.
The question is then, why is this not allowed?
Here is very detailed explanation from Gilles Dubochet from the scala mailing list back in 2007. It roughly boils down to the fact that structural types use reflection and the compiler does not know how to look up the method to call if it uses a type defined outside the refinement (the compiler does not know ahead of time how to fill the second parameter of getMethod in p.getClass.getMethod("pimp", Array(?))
But go look at the post, it will answer your question and some more.
Edit:
Hello list.
I try to define structural types with abstract datatype in function
parameter. ... Any reason?
I have heard about two questions concerning the structural typing
extension of Scala 2.6 lately, and I would like to answer them here.
Why did we change Scala's native values (“int”, etc.) boxing scheme
to Java's (“java.lang.Integer”).
Why is the restriction on parameters for structurally defined
methods (“Parameter type in structural refinement may not refer
to abstract type defined outside that same refinement”) required.
Before I can answer these two questions, I need to speak about the
implementation of structural types.
The JVM's type system is very basic (and corresponds to Java 1.4). That
means that many types that can be represented in Scala cannot be
represented in the VM. Path dependant types (“x.y.A”), singleton types
(“a.type”), compound types (“A with B”) or abstract types are all types
that cannot be represented in the JVM's type system.
To be able to compile to JVM bytecode, the Scala compilers changes the
Scala types of the program to their “erasure” (see section 3.6 of the
reference). Erased types can be represented in the VM's type system and
define a type discipline on the program that is equivalent to that of
the program typed with Scala types (saving some casts), although less
precise. As a side note, the fact that types are erased in the VM
explains why operations on the dynamic representation of types (pattern
matching on types) are very restricted with respect to Scala's type
system.
Until now all type constructs in Scala could be erased in some way.
This isn't true for structural types. The simple structural type “{ def
x: Int }” can't be erased to “Object” as the VM would not allow
accessing the “x” field. Using an interface “interface X { int x{}; }”
as the erased type won't work either because any instance bound by a
value of this type would have to implement that interface which cannot
be done in presence of separate compilation. Indeed (bear with me) any
class that contains a member of the same name than a member defined in
a structural type anywhere would have to implement the corresponding
interface. Unfortunately this class may be defined even before the
structural type is known to exist.
Instead, any reference to a structurally defined member is implemented
as a reflective call, completely bypassing the VM's type system. For
example def f(p: { def x(q: Int): Int }) = p.x(4) will be rewritten
to something like:
def f(p: Object) = p.getClass.getMethod("x", Array(Int)).invoke(p, Array(4))
And now the answers.
“invoke” will use boxed (“java.lang.Integer”) values whenever the
invoked method uses native values (“int”). That means that the above
call must really look like “...invoke(p, Array(new
java.lang.Integer(4))).intValue”.
Integer values in a Scala program are already often boxed (to allow the
“Any” type) and it would be wasteful to unbox them from Scala's own
boxing scheme to rebox them immediately as java.lang.Integer.
Worst still, when a reflective call has the “Any” return type,
what should be done when a java.lang.Integer is returned? The called
method may either be returning an “int” (in which case it should be
unboxed and reboxed as a Scala box) or it may be returning a
java.lang.Integer that should be left untouched.
Instead we decided to change Scala's own boxing scheme to Java's. The
two previous problems then simply disappear. Some performance-related
optimisations we had with Scala's boxing scheme (pre-calculate the
boxed form of the most common numbers) were easy to use with Java
boxing too. In the end, using Java boxing was even a bit faster than
our own scheme.
“getMethod”'s second parameter is an array with the types of the
parameters of the (structurally defined) method to lookup — for
selecting which method to get when the name is overloaded. This is the
one place where exact, static types are needed in the process of
translating a structural member call. Usually, exploitable static types
for a method's parameter are provided with the structural type
definition. In the example above, the parameter type of “x” is known to
be “Int”, which allows looking it up.
Parameter types defined as abstract types where the abstract type is
defined inside the scope of the structural refinement are no problem
either:
def f(p: { def x[T](t: T): Int }) = p.xInt
In this example we know that any instance passed to “f” as “p” will
define “x[T](t: T)” which is necessarily erased to “x(t: Object)”. The
lookup is then correctly done on the erased type:
def f(p: Object) = p.getClass.getMethod("x", Array(Object)).invoke(p,
Array(new java.lang.Integer(4)))
But if an abstract type from outside the structural refinement's scope
is used to define a parameter of a structural method, everything breaks:
def f[T](p: { def x(t: T): Int }, t: T) = p.x(t)
When “f” is called, “T” can be instantiated to any type, for example:
f[Int]({ def x(t: Int) = t }, 4)
f[Any]({ def x(t: Any) = 5 }, 4)
The lookup for the first case would have to be “getMethod("x",
Array(int))” and for the second “getMethod("x", Array(Object))”, and
there is no way to know which one to generate in the body of
“f”: “p.x(t)”.
To allow defining a unique “getMethod” call inside “f”'s body for
any instantiation of “T” would require any object passed to “f” as the
“p” parameter to have the type of “t” erased to “Any”. This would be a
transformation where the type of a class' members depend on how
instances of this class are used in the program. And this is something
we definitely don't want to do (and can't be done with separate
compilation).
Alternatively, if Scala supported run-time types one could use them to
solve this problem. Maybe one day ...
But for now, using abstract types for structural method's parameter
types is simply forbidden.
Sincerely,
Gilles.
Discovered the problem shortly after posting this: I have to define a named class instead of using an anonymous class. (Still would love to hear a better explanation of the reasoning though.)
object Test extends App {
case class G[V](xs: Seq[V]) {
def dummy(x: V) = x
}
implicit def pimp[V](xs: Seq[V]) = G(xs)
}
works.
Given the following:
class TestClass extends TestTrait {
def doesSomething() = methodValue + intValue
}
trait TestTrait {
val intValue = 4
val unusedValue = 5
def methodValue = "method"
def unusedMethod = "unused method"
}
When the above code runs, will TestClass actually have memory allocated to unusedValue or unusedMethod? I've used javap and I know that there exists an unusedValue and an unusedMethod, but I cannot determine if they are actually populated with any sort of state or memory allocation.
Basically, I'm trying to understand if a class ALWAYS gets all that a trait provides, or if the compiler is smart enough to only provide what the class actually uses from the trait?
If a trait always imposes itself on a class, it seems like it could be inefficient, since I expect many programmers will use traits as mixins and therefore wasting memory everywhere.
Thanks to all who read and help me get to the bottom of this!
Generally speaking, in languages like Scala and Java and C++, each class has a table of pointers to its instance methods. If your question is whether the Scala compiler will allocate slots in the method table for unusedMethod then I would say yes it should.
I think your question is whether the Scala compiler will look at the body of TestClass and say "whoa, I only see uses of methodValue and intValue, so being a good compiler I'm going to refrain from allocating space in TestClass's method table for unusedMethod. But it can't really do this in general. The reason is, TestClass will be compiled into a class file TestClass.class and this class may be used in a library by programmers that you don't even know.
And what will they want to do with your class? This:
var x = new TestClass();
print(x.unusedMethod)
See, the thing is the compiler can't predict who is going to use this class in the future, so it puts all methods into its method table, even the ones not called by other methods in the class. This applies to methods declared in the class or picked up via an implemented trait.
If you expect the compiler to do global system-wide static analysis and optimization over a fixed, closed system then I suppose in theory it could whittle away such things, but I suspect that would be a very expensive optimization and not really worth it. If you need this kind of memory savings you would be better off writing smaller traits on your own. :)
It may be easiest to think about how Scala implements traits at the JVM level:
An interface is generated with the same name as the trait, containing all the trait's method signatures
If the trait contains only abstract methods, then nothing more is needed
If the trait contains any concrete methods, then the definition of these will be copied into any class that mixes in the trait
Any vals/vars will also get copied verbatim
It's also worth noting how a hypothetical var bippy: Int is implemented in equivalent java:
private int bippy; //backing field
public int bippy() { return this.bippy; } //getter
public void bippy_$eq(int x) { this.bippy = x; } //setter
For a val, the backing field is final and no setter is generated
When mixing-in a trait, the compiler doesn't analyse usage. For one thing, this would break the contract made by the interface. It would also take an unacceptably long time to perform such an analysis. This means that you will always inherit the cost of the backing fields from any vals/vars that get mixed in.
As you already hinted, if this is a problem then the solution is just use defs in your traits.
There are several other benefits to such an approach and, thanks to the uniform access principle, you can always override such a method with a val further down in the inheritance hierarchy if you need to.