Scala 2 Append A Method To Class Body (Metaprogramming) - scala

Question

I have been stuck on this issue for a week and don't seem to be getting anywhere. I am trying to copy some methods and fields from one class to another.
I have two phases that are involved in this. The first phase scans the code, finds the method defs that need to copied, and save the corresponding Tree
The second phase inserts this tree where needs to go. In order to simplify this question, let's forget about the copying and say that I am trying to insert a simple method def hello(): String = "hello" to the body of some class
The plugin runs after the typer (because I need the package information), and I am having a problem with injecting the type information properly. This results in an assertion exception in the later type checking stage (Full stacktrace at the bottom)
I asked about this in the metaprogramming discord and was pointed to the following resources.
Scala compiler plugin to rewrite method calls
https://contributors.scala-lang.org/t/scala-compiler-plugin-naming-issues-after-typer/2835
But neither yielded successful results unfortunately. I am assuming I have to take special care because the return type is a primitive (?), as the type gets interfaced through Predef
First Attempt:
Results in the error at the very end
tree match {
case pl # ClassDef(mods, name, tparams, e # Template(parent, self, body)) =>
parent.lift(1) match {
case Some(a # TypeTree()) =>
a.original match {
case AppliedTypeTree(Select(This(TypeName(s)), tpt), args) =>
if (tpt.toString == "Policy") {
val insert = q""" def q(): String = {"hello"}""".asInstanceOf[DefDef]
val DefDef(dmodifiers, dname, dtparams, dvparams, dtpt, drhs) = insert
val source = treeCopy.DefDef(insert, dmodifiers, dname, dtparams, dvparams, dtpt, drhs)
val finalCopy = pl.copy(
mods,
name,
tparams,
Template(
parent,
self,
body.:+(
source
)
)
)
localTyper.typed(finalCopy)
} else {
super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
Instead of building the source, I have also tried manually constructing various things.
DefDef(
Modifiers(),
TermName("q"),
List(),
List(List()),
TypeTree().setOriginal(Select(Select(Ident(scala), scala.Predef), TypeName("String"))), //attempt1
Ident(TypeName("String")), //attemp2
TypeTree().setOriginal(Ident(TypeName("String"))), //attempt3
gen.mkAttributedRef(typeOf[String].typeSymbol), //attempt 4
Literal(Constant("hello")))
All resulting in the same error. Note that in the error, the class being printed have the method inserted but the type checker can not make sense of it for some reason
Following the suggestion on the contributors forum, I tried to set the ownership
val source = ... same as above
pl.symbol.owner.info.decls.unlink(pl.symbol)
localTyper.namer.enterDefDef(source)
source.symbol.owner.info.decls.enter(pl.symbol)
val finalCopy = pl.copy(....) //same as above
localTyper.namer.enterClassDef(finalCopy)
finalCopy.symbol.owner.info.decls.enter(finalCopy.symbol)
localTyper.typed(finalCopy)
But this completely screwed up everything and the compiler messed up the symbols and telling me fully implemented classes didn't implement the abstract members thus needed to be declared abstract
I have been going around in circles on this so if anybody have an idea what the best way to append a method to class body after the typer or have somewhat related examples, I would certainly appreciate it
Exception in thread "main" java.lang.AssertionError: assertion failed:
[error] class UserPolicy extends AnyRef with prv.Main.Policy[prv.Main.User] {
[error] <paramaccessor> private[this] val u: prv.Main.User = _;
[error] def <init>(u: prv.Main.User): prv.Main.UserPolicy = {
[error] UserPolicy.super.<init>();
[error] ()
[error] };
[error] private[this] val data: prv.Main.User = UserPolicy.this.u;
[error] <stable> <accessor> def data: prv.Main.User = UserPolicy.this.data;
[error] protected def checkDeclassify(): prv.Main.User = {
[error] def checkExpanded(): prv.Main.User = UserPolicy.this.data;
[error] checkExpanded()
[error] };
[error] def unsafeUnwrap(reason: String): prv.Main.User = UserPolicy.this.data;
[error] def q2(): String = "hello";
[error] def q(): String = "hello"
[error] }
[error] while compiling: <test>
[error] during phase: method-wiring-phase
[error] library version: version 2.13.1
[error] compiler version: version 2.13.1
[error] reconstructed args: -usejavacp
[error] last tree to typer: type UserPolicy
[error] tree position: <unknown>
[error] symbol: <none>
[error] symbol definition: <none> (a NoSymbol)
[error] symbol package: <none>
[error] symbol owners:
[error] call site: <none> in <none>
[error] == Source file context for tree position ==
[error] at scala.reflect.internal.SymbolTable.throwAssertionError(SymbolTable.scala:170)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typedClassDef(Typers.scala:1876)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5794)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5886)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5948)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:254)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:195)
[error] at scala.reflect.api.Trees$Transformer.$anonfun$transformStats$1(Trees.scala:2614)
[error] at scala.reflect.api.Trees$Transformer.transformStats(Trees.scala:2612)
[error] at scala.reflect.internal.Trees$Template.transform(Trees.scala:517)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.$anonfun$transform$1(TypingTransformers.scala:47)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:32)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:333)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:195)
[error] at scala.reflect.api.Trees$Transformer.transformTemplate(Trees.scala:2587)
[error] at scala.reflect.internal.Trees$ModuleDef.$anonfun$transform$3(Trees.scala:370)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:32)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:24)
[error] at scala.reflect.internal.Trees$ModuleDef.transform(Trees.scala:369)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:51)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:333)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:195)
[error] at scala.reflect.api.Trees$Transformer.$anonfun$transformStats$1(Trees.scala:2614)
[error] at scala.reflect.api.Trees$Transformer.transformStats(Trees.scala:2612)
[error] at scala.reflect.internal.Trees$PackageDef.$anonfun$transform$1(Trees.scala:316)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:32)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:24)
[error] at scala.reflect.internal.Trees$PackageDef.transform(Trees.scala:316)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.$anonfun$transform$2(TypingTransformers.scala:49)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:32)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:333)
[error] at privacy.MethodWiring$$anon$3.apply(MethodWire.scala:192)
[error] at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:452)
[error] at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:397)
[error] at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1506)
[error] at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1490)
[error] at scala.tools.nsc.Global$Run.compileSources(Global.scala:1482)
[error] at privacy.AnnotationFinderTest$.delayedEndpoint$privacy$AnnotationFinderTest$1(Test.scala:114)
[error] at privacy.AnnotationFinderTest$delayedInit$body.apply(Test.scala:13)
[error] at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] at scala.App.$anonfun$main$1(App.scala:73)
[error] at scala.App.$anonfun$main$1$adapted(App.scala:73)
[error] at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
[error] at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
[error] at scala.collection.AbstractIterable.foreach(Iterable.scala:921)
[error] at scala.App.main(App.scala:73)
[error] at scala.App.main$(App.scala:71)
[error] at privacy.AnnotationFinderTest$.main(Test.scala:13)
[error] at privacy.AnnotationFinderTest.main(Test.scala)

Answer 1

Posting an answer so the question can be closed. It took me a while but I think I figured it out.
Thanks to #SethTisue for pointing me to TwoTails. I was able to correctly synthesize a method using the part of the code in that repo. However the bottom line is doing something like this after the typer is not trivially possible. Here is the reason why:
Say you are trying to synthesize and append a method m to a class C after the typer. The problem is if you are synthesizing this method, you will eventually invoke it somewhere new C().m. The membership is resolved during the typer, so the typer will never complete and throw an error method m is not a member of C. So,
If you don't require the package information to achieve this, you should do this after the parser
If you need the package information, this gets very tricky. You need to add a few new phases after the parser. I will omit the code because it is very lengthy but here is the gist of it.
Phase 1: Accumulate the list of Class Names you will be appending to and their existing method, skip if it's a pre-known class
Phase 2: Go through the code and accumulate a list of all the symbols that correspond to an instance of this class. ValDef and any parameters to the DefDef. If you have implemented Hindley Milner you will immediately identify the problem that will a way to distinguish similarly named symbols in different scopes. There is a lot of existing literature on this that you can read, I am skipping the details.
Phase 3: Go through the code and accumulate a list of method names that are invoked on C but doesn't yet exist. You need to memorize the parameters and their types as well. Whether you need the return type or not really depends on what you are doing and/or if you want extra soundness/verification in a later step. You can skip this phase if the method you are appending is static and you already know what members will be missing in advance.
Phase 4: Go through the code one last time and append a null method into C that with the proper name and types. Retuning null isn't the best thing, not sure if there is a better alternative.
Later in the typer replace the appended method body with the proper one (the one you are copying)
The actual synthesis looks like this but as I mentioned above, if you actually want this to work, you will need to figure out all the stuff above.
override def transform(tree: Tree): Tree = {
val classesOfInterest = policyTypes.map(a => s"${a.packageName}.${a.typeName}").toList
tree match {
case pl # ClassDef(mods, name, tparams, e # Template(parent, self, body)) =>
parent.lift(1) match {
case Some(a # TypeTree()) =>
val original = a.original
original match {
case AppliedTypeTree(Select(This(TypeName(s)), tpt), args)=>
if (tpt.toString == "Policy") {
val insert = q"""... method to insert""".asInstanceOf[DefDef]
val DefDef(dmodifiers, dname, dtparams, dvparams, dtpt, drhs) = insert
val source = treeCopy.DefDef(insert, dmodifiers, dname, dtparams, dvparams, dtpt, drhs)
//borrow the symbol of another method from the body. This is guaranteed because members like toString will be generated at this point
val xyz = mkNewMethodSymbol(body(body.length).symbol, TermName("q"))
localTyper.typedPos(tree.pos)(
treeCopy.ClassDef(
tree,
mods,
name,
tparams,
treeCopy.Template(
e,
e.parents,
e.self,
e.body :+ localTyper.typed(DefDef(xyz, mkNewMethodRhs(xyz, insert)))
)
)
)
)
} else {
super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
}
def mkNewMethodSymbol(symbol: Symbol, name: TermName): Symbol = {
val flags = METHOD
val methSym = symbol.cloneSymbol(symbol.owner, flags, name)
val param = methSym.newSyntheticValueParam(definitions.IntTpe, TermName("indx"))
methSym.modifyInfo {
case GenPolyType(tparams, MethodType(params, res)) => GenPolyType(tparams, MethodType(params, res))
}
localTyper.namer.enterInScope(methSym)
}