I'm trying to understand implementation of object-private variables in Scala. Scala compiles this class
class Counter{
private[this] var age = 0
}
into the following java byte code:
public class Counter implements scala.ScalaObject {
private int age;
public Counter();
}
But still, because JVM doesn't support object-private fields, we have good-old private field, which can be accessed from other instances of the class. So for me the difference between previous class and the following in terms of hiding private field is not clear.
class Counter2{
private var age = 0
}
public class Counter2 implements scala.ScalaObject {
private int age;
private int age();
private void age_$eq(int);
public Counter2();
}
The JVM is irrelevant. The semantics of Scala are implemented by the Scala compiler, not the JVM. After all, the JVM isn't even the only platform Scala runs on, there are production-ready implementations of Scala on the CLI, and experimental ones on ECMAScript as well as a native one.
Related
After the parser phase of the Scalac process, the following case class
case class ExampleCaseClass(var s:String, var i:Int) extends ContextuallyMutable
takes the intermediate form:
Clazz(case class ExampleCaseClass extends ContextuallyMutable with scala.Product with scala.Serializable {
<caseaccessor> <paramaccessor> var s: String = _;
<caseaccessor> <paramaccessor> var i: Int = _;
def <init>(s: String, i: Int) = {
super.<init>();
()
}
})
However, a run time reflection call:
ExampleCaseClass("Can a Scala compiler plugin transform the autogenerated accessor methods of scala case classes?", 42).getClass.getMethods.foreach(println(_))
reveals many more public methods:
public boolean ExampleCaseClass.equals(java.lang.Object)
public java.lang.String ExampleCaseClass.toString()
public int ExampleCaseClass.hashCode()
public static ExampleCaseClass ExampleCaseClass.apply(java.lang.String,int)
public int ExampleCaseClass.i()
public java.lang.String ExampleCaseClass.s()
public ExampleCaseClass ExampleCaseClass.copy(java.lang.String,int)
public void ExampleCaseClass.i_$eq(int)
public scala.collection.Iterator ExampleCaseClass.productElementNames()
public java.lang.String ExampleCaseClass.productElementName(int)
public void ExampleCaseClass.s_$eq(java.lang.String)
public int ExampleCaseClass.copy$default$2()
public boolean ExampleCaseClass.canEqual(java.lang.Object)
public java.lang.String ExampleCaseClass.productPrefix()
public int ExampleCaseClass.productArity()
public java.lang.Object ExampleCaseClass.productElement(int)
public scala.collection.Iterator ExampleCaseClass.productIterator()
public java.lang.String ExampleCaseClass.copy$default$1()
public static scala.Function1 ExampleCaseClass.tupled()
public static scala.Option ExampleCaseClass.unapply(ExampleCaseClass)
public static scala.Function1 ExampleCaseClass.curried()
public final void java.lang.Object.wait(long,int) throws java.lang.InterruptedException
public final void java.lang.Object.wait() throws java.lang.InterruptedException
public final native void java.lang.Object.wait(long) throws java.lang.InterruptedException
public final native java.lang.Class java.lang.Object.getClass()
public final native void java.lang.Object.notify()
public final native void java.lang.Object.notifyAll()
Clearly some subsequent compiler phase creates the property accessor methods:
public int ExampleCaseClass.i()
public java.lang.String ExampleCaseClass.s()
public void ExampleCaseClass.i_$eq(int)
public void ExampleCaseClass.s_$eq(java.lang.String)
Which compilation phase generates these accessor methods and what manner of compiler plugin (or other means) might prevent or transform them?
The enquirer has already run numerous experiments removing or reshaping the:
<caseaccessor> <paramaccessor> var s: String = _;
<caseaccessor> <paramaccessor> var i: Int = _;
portions of the case class, and also with injecting the desired accessor methods in advance but no combination has met the desired outcome. They either fail to compile because of naming conflicts that arise in subsequent compilation phases, or they alter the parameter names in constructor, apply, and accessor methods.
Can a scala compiler plugin transform synthetic accessors at all? Does the Java Compiler introduce these methods? If so, should the enquirer look to Javac plugins and what analogues might serve the Scala.js and Scala native compilation targets?
Thank you for any consideration.
case class expansion happens in more than one place, see another question.
Instead of writing a new plugin just to disallow using var it would be much better to add a new rule to Wartremover or ScalaFix. As a matter of the fact, these rules already exist:
disallow var with Wartremover - combine it with fatal warnings to fail compilation on var
disallow var with ScalaFix
If you want to add more elaborate rule... it would still be easier just to write your own Wartremover/ScalaFix rule (the latter might be preferred as it is already supported in Scala 3).
And if you really need a custom compiler plugin to mess with code generated by compiler... take a look at better-toString plugin. It adds its own phase after "parser" phase. But I wouldn't hope for removing the autogenerated implementations. At best you can override them manually where specs allows you to.
The enquirer found a Scala 3 solution with persistence and help from examples:
better-tostring a plugin that demonstrates conditional method insertion by Polyvariant.
Compiler Plugin Development in Scala 3 Example and Tutorial by Scala Center dev: Fengyun Liu. The video provided insights into compiler phases and the example shed light on method body generation syntax. In particular, available documentation doesn't readily clarify how to call println from a method body generated as a Tree by a compiler plugin, but Liu's example plugin demonstrated requireModule and requiredMethod.
Scala 3 Compiler Plugin Documentation offers a very nice template for how to start writing a plugin. The final solution looked very similar.
As of 26 Nov 2021, no solution exists for Scala 2.13, but maybe that just means it is time to upgrade.
final class BlockMutatorPlugin extends StandardPlugin {
override val name: String = "BlockMutatorPlugin"
override val description: String = "Scala Compiler Plugin for blocking ContextuallyMutable setter methods."
override def init(options: List[String]): List[PluginPhase] = List(new BlockContextuallyMutableSetters)
}
class BlockContextuallyMutableSetters extends PluginPhase {
val phaseName = "blockGetter"
/* Running this plugin after phases before ElimErasedValueType
resulted in the replacement of the generated setter methods by the synthetic
default versions. By the time that this ElimErasedValueType phase ends, the
defaults already existed, so this plugin could augment them safely. */
override val runsAfter = Set(ElimErasedValueType.name)
private var printBlocked: Tree = _
override def prepareForTemplate(tree: tpd.Template)(using ctx: Context): Context = {
val cnsl = requiredModule("scala.Predef")
val prntln: PreName = "println".toTermName
val say = cnsl.requiredMethod(prntln, List[Types.Type](ctx.definitions.ObjectType))
printBlocked = ref(say).appliedTo(Literal(Constant("Blocked!")))
ctx
}
override def transformTemplate(tree: Template)(using ctx: Context): Tree = {
if (tree.parents.filter(_.symbol.name.toString.equals("ContextuallyMutable")).nonEmpty) {
cpy.Template(tree)(
body = tree.body.collect {
case dd: DefDef if dd.name.isSetterName => DefDef(
dd.symbol.asInstanceOf[Symbols.TermSymbol],
printBlocked
)
case x => x
}
).asInstanceOf[Tree]
} else tree
}
}
The most important part of this effort involved discovering which compiler phase this plugin should follow. Similar efforts in Scala 2.13.6 have failed so far; the only remaining impediment to the Scala 2 solution sought by this original Stack Overflow question. As such, the enquirer will not mark his own answer as the accepted solution unless future edits avail Scala 2. Until that time, your response may claim that designation.
For any inclined to try compile this example, the code above requires the following import statements:
import dotty.tools.dotc.ast.tpd
import tpd.*
import dotty.tools.dotc.core.*
import Names.PreName
import Symbols.{ClassSymbol, requiredMethod, requiredModule}
import Decorators.*
import NameOps.*
import Contexts.Context
import Constants.Constant
import dotty.tools.dotc.plugins.*
import dotty.tools.dotc.transform.ElimErasedValueType
In a scala class, I have defined a protected field and a protected method:
TestProtected.scala:
class TestProtected {
protected var f = 0
protected def m = 1
}
In my opinion, it will convert the protected f and m to a java protected filed and method. But when i use jd-gui to decompiler the TestProtected.class to java file, the result was beyond my expectation:
#ScalaSignature(bytes="...")
public class TestProtected
{
private int f= 0;
public int f() { return this.f; }
public void f$eq(int x$1) { this.f = x$1; }
public int m() {
return 1;
}
}
The f and m in java file is public?
Why?
Thanks!
Firstly, Scala protected is different from Java protected:
Java's protected is visible to same package and subclass, Scala protect is only visible to subclass, so Scala's protected is more restrictive than Java's.
So this is caused by JVM Level access modifier is not same as Scala access modifier, we know Scala runs on JVM, for running on JVM, it needs to generate the compatible byte code in JVM.
There are some other example like:
private
private[this]
private[package]
There are some reference about Scala Access:
Protected and private in Scala become public in Java
Scala Access
Protected Members
I'm not sure, but this may be because companion object of a class should also be able to access protected members of a class in scala. And because companion object is actually a different class from its companioin class, scala has to compile protected members to public in a bytecode.
Trying to get values of all fields of child from parent class like this:
for (field <- this.getClass.getDeclaredFields) {
Logger.debug(field.getName)
field.get(this)
}
and got error
Exception: Class models.Model$$anonfun$4 can not access a member of
class models.Good with modifiers "private"
at line
field.get(this)
In Good class I don't have private fields:
class Good(id: Option[String]) extends Model[realGood](id){
lazy val title: String = this.load[String](realObject.get.title)
lazy val cost: Double = this.load[Double](realObject.get.cost)
}
What's wrong with this code?
As hinted in the comments, Scala's conversion to java bytecode isn't always straightforward (though it's usually pretty predictable, once you get the hang of it). In particular, public fields in Scala compile to a private field with a public getter in java bytecode:
fukaeri:~ dlwh$ cat zzz.scala
class Good(id: Option[String]) {
lazy val title: String = ???
lazy val cost: Double = ???
}
fukaeri:~ dlwh$ scalac zzz.scala
fukaeri:~ dlwh$ javap -private Good
Compiled from "zzz.scala"
public class Good {
private java.lang.String title;
private double cost;
private volatile byte bitmap$0;
private java.lang.String title$lzycompute();
private double cost$lzycompute();
public java.lang.String title();
public double cost();
public Good(scala.Option<java.lang.String>);
}
You can see that Good has private fields for each of your declared public fields, in addition to public getters. Because the fields are lazy val, they also have computation methods for initialization, and there's a bitmap$0 field to ensure that the lazy vals are initialized only once.
In your loop, you can use field.setAccessible(true) to fix your exception.
What's the purpose of Scala object MODULE$?
The following Scala object:
object TestScalaObject {
val TEST_SYMBOL = "*"
def testMethod(x : String) : String = x
}
compiles into two bytecode files TestScalaObject.class and TestScalaObject$.class which if I disassemble to get the equivalent Java code I get:
TestScalaObject.class:
public final class TestScalaObject extends java.lang.Object{
public static final java.lang.String testMethod(java.lang.String);
public static final java.lang.String TEST_SYMBOL();
}
TestScalaObject$.class:
public final class TestScalaObject$ extends java.lang.Object implements scala.ScalaObject{
public static final TestScalaObject$ MODULE$;
public static {};
public java.lang.String TEST_SYMBOL();
public java.lang.String testMethod(java.lang.String);
}
I can see a public static final TestScalaObject$.MODULE$ but what is it used for if I can access everything I need through TestScalaObject.TEST_SYMBOL and TestScalaObject.testMethod() if I ever wanted to do that from Java
MODULE$ holds an instance of the instantiated class. See the Singleton pattern in Java. I don't know of a good source for it, so here's the Wikipedia entry for Singleton.
If I have a linked node in some collection structure I don't really want its next link to be an AtomicReference (I need atomic CAS update) so I declare it as:
#volatile var next: Node[A] = _n
and then in the companion declare:
val updater = AtomicReferenceFieldUpdater.newUpdater(classOf[Link[_]], classOf[Node[_]], "n")
def cas[A](target: Link[A], old: Node[A], newNode: Node[A]) = updater.compareAndSet(target, old, newNode);
At runtime I get the following error:
java.lang.RuntimeException: java.lang.IllegalAccessException:
Class concurrent.Link$ can not access a member of class concurrent.Link
with modifiers "private volatile"
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl.<init>(AtomicReferenceFieldUpdater.java:189)
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater.newUpdater(AtomicReferenceFieldUpdater.java:65)
at concurrent.Link$.<init>(Link.scala:106)
...
So, at runtime the companion object is concurrent.Link$ not concurrent.Link and a different class cannot access the private member of another.
BUT, if I javap -p concurrent.Link
I get:
Compiled from "Link.scala"
public final class concurrent.Link implements concurrent.Node,scala.ScalaObject,scala.Product,java.io.Serializable{
private final java.lang.Object value;
private volatile com.atlassian.util.scala.concurrent.Node node;
public static final boolean cas(com.atlassian.util.scala.concurrent.Link, com.atlassian.util.scala.concurrent.Node, com.atlassian.util.scala.concurrent.Node);
public static final java.util.concurrent.atomic.AtomicReferenceFieldUpdater updater();
So, I have everything but the static instance of the AtomicReferenceFieldUpdater declared on my Link class.
The question is, how do I get an instance of AtomicReferenceFieldUpdater in Scala that points to a volatile var?
The only way I've found so far is to go back to Java (implement an AbstractLink with the next Node field and a static AtomicReferenceFieldUpdater) and inherit from that, which is ugly.
Hard. Since Scala makes fields private, and only accessor methods available, this might not be possible.
When doing this, I eventually decided to do it by creating Java base class with the volatile field and updating through there.
Java file:
public class Base {
volatile Object field = null;
}
Scala file:
class Cls extends Base {
val updater = AtomicReferenceFieldUpdater.newUpdater(classOf[Base[_]], classOf[Object[_]], "field")
def cas[A](old: Object, n: Object) = updater.compareAndSet(this, old, n);
}
I haven't come up with a different approach.
I've been told there is no way to do this in pure scala, and you need to define a class in Java to do this. Fortunately cross-compilation is pretty simple, but still, annoying!