In Scala Reflection, why do constructor params hide getters? - scala

vars in a scala class automatically get getters & setters you can see through scala reflection via members
import scala.reflect.runtime.{universe => ru}
class A(var x: Int)
scala> ru.typeOf[A].members.filter{_.name.toString.contains("x")}
res22: Iterable[reflect.runtime.universe.Symbol] = SynchronizedOps(variable x, method x_=, method x)
However, if you create a subclass, which re-uses the var name in the constructor, the getter is gone:
class B(x:Int, var y: Int) extends A(x)
scala> ru.typeOf[B].members.filter{_.name.toString.contains("x")}
res23: Iterable[reflect.runtime.universe.Symbol] = SynchronizedOps(value x, method x_=)
scala> res23.head.asTerm.isVal
res25: Boolean = true
This seems a little misleading ... after all, B still does have a getter for x (and its not a val)
scala> val b = new B(5,6)
b: B = B#270288ed
scala> b.x
res26: Int = 5
scala> b.x = 7
b.x: Int = 7
scala> b.x
res27: Int = 7
If I try to pretend that the value x I got from members is a getter, I get an error:
scala> val xGetter = res23.head.asTerm
xGetter: reflect.runtime.universe.TermSymbol = value x
scala> val objMirror = ru.runtimeMirror(getClass.getClassLoader).reflect(b)
objMirror: reflect.runtime.universe.InstanceMirror = instance mirror for B#270288ed
scala> val getterMirror = objMirror.reflectField(xGetter)
scala.ScalaReflectionException: Scala field x isn't represented as a Java field, neither it has a Java accessor method
note that private parameters of class constructors don't get mapped onto fields and/or accessors,
unless they are used outside of their declaring constructors.
What is the right workaround here? Is it completely wrong to have a subclass name its constructor args the same as the names in the parent args? Or instead of calling members, do I need to work my up all super-classes to get all getters & setters?
Note that members gives me the inherited getter as long as the subclass doesn't create a constructor w/ the same name:
class Y(var x: Int)
class Z(q:Int, z: Int) extends Y(q)
scala> ru.typeOf[Z].members.filter{_.name.toString.contains("x")}
res28: Iterable[reflect.runtime.universe.Symbol] = SynchronizedOps(method x_=, method x)
EDIT
In case its unclear, I'm really asking:
1) is this a bug in scala reflection?
2) if not, should I:
(a) never have classes use names of constructor fields be the same as the name of fields in base classes? (if so, I'm probably defining all my classes wrong ...)
or
(b) to get all getters & setters, should I just go through the list of all parent classes and use declarations, rather than relying on members to do the right thing, since it doesn't work in this one case?
EDIT 2
in response to #som-snytt's answer, the visible methods of x are really on x in A, not the param in the constructor to B. Eg.:
class A(var x: Int){def showMeX {println(x)}}
class B(x:Int, var y: Int) extends A(x)
scala> val b = new B(5,10)
scala> b.showMeX
5
scala> b.x = 17
b.x: Int = 17
scala> b.showMeX
17
so I don't really think that the either the getter or setter for x has been shadowed, from the perspective of normal user code. Its only been shadowed for reflection code ... and it doesn't make sense to me that there would be two different versions of shadowing.

2) if not, should I: (a) never have classes use names of constructor fields be the same as the name of fields in base classes?
Since they wouldn't let me fix this, that's exactly what I do. I try to give all constructor parameters new names distinct from all inherited names. Here's a typical example within the compiler.
class PackageClassSymbol protected[Symbols] (owner0: Symbol, pos0: Position, name0: TypeName)
Yes, it's ridiculous.
https://groups.google.com/forum/#!topic/scala-language/9jLsT_RRQR0
https://issues.scala-lang.org/browse/SI-3194
https://issues.scala-lang.org/browse/SI-4762
https://issues.scala-lang.org/browse/SI-6880
Oh boy, don't keep pulling that thread...
https://issues.scala-lang.org/browse/SI-7475
https://issues.scala-lang.org/browse/SI-2568
https://issues.scala-lang.org/browse/SI-6794
It shouldn't be hard to see that the odds of any of it being addressed are nil. It's a perfect example of why I quit.
By the way, if you use -Xlint it warns you about this. That's mentioned in SI-4762.
% cat a.scala
class A(var x: Int)
class B(x:Int, var y: Int) extends A(x) {
def z = x
}
% scalac -Xlint a.scala
a.scala:3: warning: private[this] value x in class B shadows mutable x inherited from class A.
Changes to x will not be visible within class B - you may want to give them distinct names.
def z = x
^
one warning found

Well, in:
class B(x:Int, var y: Int) extends A(x)
The x in B is private (not a case class, no val or var specifier), the x in A is public (you specified var). I'm not too familiar with this reflection API, but does it show private members?
If you instead:
scala> class B(val x:Int, var y: Int) extends A(10)
<console>:9: error: overriding variable x in class A of type Int;
value x needs `override' modifier
class B(val x:Int, var y: Int) extends A(10)
^
The only public x is the one in A, which can be shown here:
scala> class B(x:Int, var y: Int) extends A(10)
defined class B
scala> new B(2,3).x
res4: Int = 10
If you want to override the parent member, use override, and change the parent to something that can be overriden.

I guess as long as the answer to my question is not (2a), then if anybody else runs into this, here is a workaround. Maybe it will be unnecessary in the future depending on the answer to (1).
(Some extra stuff here, but maybe useful also)
import scala.reflect.runtime.{universe => ru}
object ReflectionUtils {
def extractGetterSetterPairs(typ: ru.Type): Seq[GetterSetterPair] = {
typ.baseClasses.foldLeft(Seq[GetterSetterPair]()){case (acc, clsSymb) =>
extractGetterSetterPairs(clsSymb.asClass.toType, acc)
}
}
private def extractGetterSetterPairs(typ: ru.Type, acc: Seq[GetterSetterPair]): Seq[GetterSetterPair] = {
val terms = typ.declarations.collect{case x if x.isTerm => x.asTerm}
acc ++ terms.filter{x => x.isGetter}.map{x => x -> x.setter}.
filter{case(g,s) => s.isTerm}.map{case(g,s) =>
GetterSetterPair(g,s.asTerm)
}
}
def termName(t: ru.TermSymbol): String = {
t.name.toString.trim
}
}
case class GetterSetterPair(getter: ru.TermSymbol, setter: ru.TermSymbol) {
val name = ReflectionUtils.termName(getter)
val fieldType = {
//this is way more complicated than it should be. But
// 1) getters for some reason are not instances of ru.MethodType
// java.lang.ClassCastException: scala.reflect.internal.Types$NullaryMethodType cannot be cast to scala.reflect.api.Types$MethodTypeApi
// 2) its a headache to get the types out of setters
val m = setter.typeSignature.
asInstanceOf[ru.MethodType]
m.params.head.typeSignature
}
}

Related

Case class copy does not maintain state of an inherited trait

Say I define a trait and case class that extends it:
trait A {
var list: List[Int] = List()
def add(i: Int) = list = i :: list
}
case class B(str: String) extends A
If I then create an instance of B, modify the list and make a copy of it, the copy takes the values of the list as defined in trait A:
val b = B("foo")
b.add(3)
println("B: " + b.list)
println("Copied: " + b.copy(str = "bar").list)
Output:
B: List(3)
Copied: List()
I know this is probably not good functional programming practice anyway, but is there a way I can force the copy of B to take the altered version of the list for instance b?
You can accomplish this by declaring the list in the case class, and satisfy your trait's type by declaring a method. So:
trait A {
def list: List[Int]
}
case class B(i: Int, var list: List[Int]) extends A
And then your example:
scala> val b = B(2, List(1))
b: B = B(2,List(1))
scala> b.list = List(3)
b.list: List[Int] = List(3)
scala> println("B: " + b.list)
B: List(3)
scala> println("Copied: " + b.copy(i = 4).list)
Copied: List(3)
I don't know what you're doing that requires a mutable list in your class, but I imagine that even though this satisfies your example it might not satisfy whatever problem you're working on since:
scala> val a: A = b
a: A = B(2,List(3))
scala> a.list
res2: List[Int] = List(3)
scala> a.list = List(4)
<console>:15: error: value list_= is not a member of A
a.list = List(4)
When trying to view an instance of B as an A you won't be able to assign the list type. If you were dealing with an instance of A you'd have to do something like
scala> a match {
| case itsAB: B => itsAB.list = List(4); itsAB
| case other => other
| }
res3: A = B(2,List(4))
But either way, this is how you'd do it. Either that or you can follow phongnt's advice to create a non-case class and define a copy method yourself.
In a comment you said:
I failed to mention that the trait attribute needs to be modifiable with a reference to the trait itself
But this sounds like global mutable state to me. If you really need that, and I encourage you to not to then you can use an object:
object A {
#volatile
var list = List.empty[Int]
}
But I doubt this gets at what you're expressing in your question either since there is not an individual list still per instance here. So... looking at your updated question, I can only guess that something like this is what you're trying to accomplish:
object A {
#volatile
var list = List.empty[Int]
}
trait A {
def list = A.list
def add(i: Int) = A.list = i :: A.list
}
case class B(str: String) extends A
Which lets you:
scala> val b = B("hi")
b: B = B(hi)
scala> b.add(0)
scala> b.add(1)
scala> A.list
res4: List[Int] = List(1, 0)
But again, this is global mutable state, it's not a good thing. It's not threadsafe. So if there's a way for you to figure out your problem without doing this... I'd encourage you to do so.
The copy method is generated for you by the compiler when you declare a case class and it is only parameterised with the variables you define in the case class
Hence, you can definitely have copy method do that for you by declaring the variable list in the case class e.g. case class B(i: Int, list: List[Int]). You will also need override val modifier to satisfy scalac. However, scalac doesn't allow you to override a mutable variable, so you need to change the trait too, for example
trait A {
val list: List[Int] = List(1)
}
case class B(i: Int, override val list: List[Int]) extends A

Type parameters cannot be referred in function body in Scala?

I came from C++ world and new to Scala, and this behavior looks unusual.
class G1[A]( val a : A) {
//val c:A = new A //This gives compile error
def fcn1(b: A): Unit = {
//val aobj = new A // This gives compile error
println(a.getClass.getSimpleName)
println(b.getClass.getSimpleName)
}
}
def fcnWithTP[A](): Unit = {
//val a = new A // This gives compile error
//println(a.getClass.getSimpleName)
}
I am not able to crate a object using the type parameter in a class in a function body or a class body. I am only be able to use it in the function parameter.
What is the reason for this? Is this because of type erasure? At run time, the function does not know what the actual type A is, so it cannot create an object of that type?
What is the general rule for this? Does it that mean the type parameter cannot appear in function body or class definition at all? If they can actually appear, what are the examples?
Yes, you're right that this is because of erasure—you don't know anything about A at runtime that you haven't explicitly asserted about it as a constraint in the method signature.
Type erasure on the JVM is only partial, so you can do some horrible things in Scala like ask for the class of a value:
scala> List(1, 2, 3).getClass
res0: Class[_ <: List[Int]] = class scala.collection.immutable.$colon$colon
Once you get to generics, though, everything is erased, so for example you can't tell the following things apart:
scala> List(1, 2, 3).getClass == List("a", "b", "c").getClass
res1: Boolean = true
(In case it's not clear, I think type erasure is unambiguously a good thing, and that the only problem with type erasure on the JVM is that it's not more complete.)
You can write the following:
import scala.reflect.{ ClassTag, classTag }
class G1[A: ClassTag](val a: A) {
val c: A = classTag[A].runtimeClass.newInstance().asInstanceOf[A]
}
And use it like this:
scala> val stringG1: G1[String] = new G1("foo")
stringG1: G1[String] = G1#33d71170
scala> stringG1.c
res2: String = ""
This is a really bad idea, though, since it will crash at runtime for many, many type parameters:
scala> class Foo(i: Int)
defined class Foo
scala> val fooG1: G1[Foo] = new G1(new Foo(0))
java.lang.InstantiationException: Foo
at java.lang.Class.newInstance(Class.java:427)
... 43 elided
Caused by: java.lang.NoSuchMethodException: Foo.<init>()
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.newInstance(Class.java:412)
... 43 more
A better approach is to pass in the constructor:
class G1[A](val a: A)(empty: () => A) {
val c: A = empty()
}
And a much better approach is to use a type class:
trait Empty[A] {
def default: A
}
object Empty {
def instance[A](a: => A): Empty[A] = new Empty[A] {
def default: A = a
}
implicit val stringEmpty: Empty[String] = instance("")
implicit val fooEmpty: Empty[Foo] = instance(new Foo(0))
}
class G1[A: Empty](val a: A) {
val c: A = implicitly[Empty[A]].default
}
And then:
scala> val fooG1: G1[Foo] = new G1(new Foo(10101))
fooG1: G1[Foo] = G1#5a34b5bc
scala> fooG1.c
res0: Foo = Foo#571ccdd0
Here we're referring to A in the definition of G1, but we're only making reference to properties and operations that we've confirmed hold or are available at compile time.
Generics are not the same thing as templates. In C++ Foo<Bar> and Foo<Bat> are two different classes, generated at compile time.
In scala or java, Foo[T] is a single class that has with a type parameter. Consider this:
class Foo(val bar)
class Bar[T] {
val foo = new T // if this was possible ...
}
new Bar[Foo]
In C++, (an equivalent of) this would fail to compile, because there is no accessible constructor of Foo that takes no arguments. The compiler would know that when it tried to instantiate a template for Bar<Foo> class, and fail.
In scala, there is no separate class for Bar[Foo], so, at compilation time, the compiler doesn't know anything about T, other than that it is some type. It has no way of knowing whether calling a constructor (or any other method for that matter) is possible or sensible (you can't instantiate a trait for example, or an abstract class), so new T in that context has to fail: it simply does not make sense.
Roughly speaking, you can use type parameters in places where any type can be used (do declare a return type for example, or a variable), but when you are trying to do something that only works for some types, and not for others, you have to make your type param more specific. For example, this: def foo[T](t: T) = t.intValue does not work, but this: def foo[T <: Number](t: T) = t.intValue does.
Well the compiler does not know how to create an instance of type A. You need to either provide a factory function that returns instance of A, or use Manifest which creates instance of A from reflection.
With factory function:
class G1[A](val a:A)(f: () => A) {
val c:A = f()
}
With Manifest:
class G1[A](val a: A)(implicit m: scala.reflect.Manifest[A]) {
val c: A = m.erasure.newInstance.asInstanceOf[A]
}
When using type parameter, usually you will specify more details on the type A, unless you're implementing some sort of container for A that does not directly interact with A. If you need to interact with A, you need some specification on it. You can say A must be a subclass of B
class G1[A <: B](val a : A)
Now compiler would know A is a subclass of B so you can call all functions defined in B on a:A.

What does treating a method as a function mean in Scala?

By assigning a variable (or value?) a method name with a space and an underscore, you tell scala to treat the method as a function, which apparently means doing more than simply taking the value generated by a call to the method and assigning to the variable. What else is/can go on through such an assignment?
Since Scala runs on the JVM, it's easier to understand in terms of simple Java-like classes without Scala's syntactic sugar.
Remember that Scala functions are essentially members of a class similar to the following (signature deliberately simplified):
class Function[X, Y] {
def apply(x: X): Y
}
Application of a function f to an argument x is desugared into a method application f.apply(x).
Now suppose that you have another class Foo with method bar:
class Foo {
def bar(x: Int): String
}
If you now have an instance foo of type Foo, then whenever its method bar is transformed into a function by writing:
val f = foo.bar(_)
a new instance of an anonymous subclass of Function is created:
val f = new Function[Int, String] {
def apply(x: Int) = foo.bar(x)
}
If you use this syntax inside a class, this is closed over instead of an instance foo.
This is what all those weirdly named classes Main$$anon$1$$anonfun$1 are: they are the anonymous classes that represent functions. The functions can appear quite implicitly (for example, as blocks passed to the for-loops).
That's all there is to it semantically. The rest is just syntactic sugar.
Here is a complete runnable example that demonstrates the conversion of an instance method into a function:
with sugar, from the outside (a)
with sugar, from the inside (b)
without sugar, from the outside (c)
without sugar, from the inside (d)
You can save it into a file and execute with scala <filename.scala>:
/** A simple greeter that prints 'hello name' multiple times */
case class Hey(name: String) { thisHeyInst =>
def hello(x: Int): String = ("hello " + name + " ") * x
def withSugarFromInside = hello(_)
def noSugarFromInside = new Function[Int, String] {
def apply(y: Int) = thisHeyInst.hello(y)
}
}
val heyAlice = Hey("Alice")
val heyBob = Hey("Bob")
val heyCharlie = Hey("Charlie")
val heyDonald = Hey("Donald")
val a = heyAlice.hello(_)
val b = heyBob.withSugarFromInside
val c = new Function[Int, String] { def apply(y: Int) = heyCharlie.hello(y) }
val d = heyDonald.noSugarFromInside
println(a(3))
println(b(3))
println(c(3))
println(d(3))
In all four cases, a greeting is printed three times.
What _ actually does is an eta-conversion. It takes compile-time construction called method and returns runtime construction called anonymous function, which is actually an instance of scala's Function. Exactly the class depends on arity, so it might be Function1, Function2, Function3 and so on. The point here is to make First-class citizen, which may act like a value.
OOP needs a little more than some new object. Before making the code that creates instance, compiler generates a new class (extending FunctionN) in compile-time, but theoretically it shouldn't be necessary a whole new class. For Java 8 it could be native Java-lambdas.
Btw, you may extend Function1 by yourself and even eta-abstract it again:
scala> object f extends (Int => Int) { def apply(a: Int) = a }
scala> f(1)
res0: Int = 1
scala> f.apply _
res1: Int => Int = <function1>
scala> res1(5)
res2: Int = 5
As a conclusion a little copy-paste from #Daniel C. Sobral's answer:
the former can be easily converted into the latter:
val f = m _
Scala will expand that, assuming m type is (List[Int])AnyRef
into (Scala 2.7):
val f = new AnyRef with Function1[List[Int], AnyRef] {
def apply(x$1: List[Int]) = this.m(x$1)
}
On Scala 2.8, it actually uses an AbstractFunction1 class to reduce
class sizes.
Or simply saying val f = m _ is same as val f = (x: List[Int]) => m(x)
To make this answer more modern and precise let's see what's happening using scalac 2.11.2 and javap:
$ echo "object Z{def f(a: Int) = a}" > Z.scala //no eta-abstraction here
$ scalac Z.scala
$ ls
Z$.class Z.class Z.scala
$ echo "object Z{def f(a: Int) = a; val k = f _}" > Z.scala
$ scalac Z.scala
$ ls
Z$$anonfun$1.class Z.class //new anonfun class added for lambda
Z$.class Z.scala
$ javap -c Z\$\$anonfun\$1.class
Compiled from "Z.scala" // I've simplified output a bit
public final class Z$$anonfun$1 extends scala.runtime.AbstractFunction1$mcII$sp implements scala.Serializable {
public final int apply(int);
Code:
calling apply$mcII$sp(int)
public int apply$mcII$sp(int); //that's it
Code:
0: getstatic #25 // reading Field Z$.MODULE$:LZ$;, which points to `object Z`
3: iload_1
4: invokevirtual #28 // calling Method Z$.f
7: ireturn
public final java.lang.Object apply(java.lang.Object); //just boxed version of `apply`
Code:
unboxToInt
calling apply(int) method
boxToInteger
public Z$$anonfun$1();
Code:
AbstractFunction1$mcII$sp."<init>":()V //initialize
}
So it still extends AbstractFunction1
I'll try to provide some examples how a function or method are assigned to values with underscore.
If it's need to reference a zero-argument function
scala> val uuid = java.util.UUID.randomUUID _
uuid: () => java.util.UUID = <function0>
scala> uuid()
res15: java.util.UUID = 3057ef51-8407-44c8-a09e-e2f4396f566e
scala> uuid()
uuid: java.util.UUID = c1e934e4-e722-4279-8a86-004fed8b9090
Check how it's different when one does
scala> val uuid = java.util.UUID.randomUUID
uuid: java.util.UUID = 292708cb-14dc-4ace-a56b-4ed80d7ccfc7
In first case one assigned a reference to a function. And then calling uuid() generates new UUID every time.
In second, function randomUUID has been called and value assigned to a val uuid.
There are some other cases why _ might be useful.
It's possible to use a function with two arguments and create a function with a single argument out of it.
scala> def multiply(n: Int)(m: Int) = n*m
multiply: (n: Int)(m: Int)Int
scala> val by2 = multiply(2) _
by2: Int => Int = <function1>
scala> by2(3)
res16: Int = 6
To be able to do, it's crucial to define function multiply as curried. It's called function currying.

Is there a way to get the direct parents of a ClassSymbol in macro context?

Im trying to get the direct super classes / traits of a ClassSymbol. The method baseClasses() does not work for me as it also includes the super super.. types.
The java.lang.Class.getSuperclass() and java.lang.Class.getInterfaces() would actually be sufficient for my use case, but I can't find a way to go from ClassSymbol to java.lang.Class in macro context!
If you use macro - you can't obtain runtime-object Class for class which does not exist (loaded) in compile-time (so you can't have an access to compiling classess from there). But, you could write your own function:
def getDirectBase(a: ClassSymbol) = {
val base = a.baseClasses.toSet - a //`baseClasses` contains `a` itself
val basebase = base.flatMap {
case x: ClassSymbol => x.baseClasses.toSet - x
}
base -- basebase
}
Results (examples obtained using run-time reflection, but compile-time should be same here):
scala> class A
scala> class B extends A
scala> class C extends B
scala> typeOf[C].typeSymbol.asInstanceOf[ClassSymbol].baseClasses
res37: List[reflect.runtime.universe.Symbol] = List(class C, class B, class A, class Object, class Any)
scala> getDirectBase(typeOf[C].typeSymbol.asInstanceOf[ClassSymbol])
res38: scala.collection.immutable.Set[reflect.runtime.universe.Symbol] = Set(class B)

Adding a field to Scala case class?

I've seen some blogs on the Pimp my Library pattern, and these seem to work well for adding behavior to classes.
But what if I have a case class and I want to add data members to it? As a case class I can't extend it (inheriting from a case class is deprecated/strongly discouraged). Will any of these pimp patterns allow me to add data to a case class?
No - I don't see how you could make this work because the enriched instance is usually thrown away (note: newly the pimp-my-library pattern is called enrich-my-library). For example:
scala> case class X(i: Int, s: String)
defined class X
scala> implicit class Y(x: X) {
| var f: Float = 0F
| }
defined class Y
scala> X(1, "a")
res17: X = X(1,a)
scala> res17.f = 5F
res17.f: Float = 0.0
scala> res17.f
res18: Float = 0.0
You would have to make sure you kept hold of the wrapped instance:
scala> res17: Y
res19: Y = Y#4c2d27de
scala> res19.f = 4
res19.f: Float = 4.0
scala> res19.f
res20: Float = 4.0
However, I find this not useful in practice. You have a wrapper; you're better off making this explicit
This is not the way to do. Just a proof of possibility. With this way you can get plenty of problems.
scala> :paste
// Entering paste mode (ctrl-D to finish)
case class A(i: Int)
class B(a: A){
var s: String = ""
}
object B{
val as = scala.collection.mutable.WeakHashMap[A, B]()
}
implicit def a2b(a: A) = B.as.getOrElseUpdate(a, new B(a))
// Exiting paste mode, now interpreting.
defined class A
defined class B
defined module B
a2b: (a: A)B
scala> val a = A(1)
a: A = A(1)
scala> a.s = "test"
scala> a.s
res0: String = test
WeakHashMap: A hash map with references to entries which are weakly reachable. Entries are removed from this map when the key is no longer (strongly) referenced. This class wraps java.util.WeakHashMap.
Note that due to case class's overridden equals method you get this funny behavior:
scala> A(2).s = "test2"
scala> A(2).s
res2: String = test2
so you should not use case class or use it with override def equals(that: Any) = this eq that.asInstanceOf[AnyRef].