I have been stuck on this issue for a week and don't seem to be getting anywhere. I am trying to copy some methods and fields from one class to another.
I have two phases that are involved in this. The first phase scans the code, finds the method defs that need to copied, and save the corresponding Tree
The second phase inserts this tree where needs to go. In order to simplify this question, let's forget about the copying and say that I am trying to insert a simple method def hello(): String = "hello" to the body of some class
The plugin runs after the typer (because I need the package information), and I am having a problem with injecting the type information properly. This results in an assertion exception in the later type checking stage (Full stacktrace at the bottom)
I asked about this in the metaprogramming discord and was pointed to the following resources.
Scala compiler plugin to rewrite method calls
https://contributors.scala-lang.org/t/scala-compiler-plugin-naming-issues-after-typer/2835
But neither yielded successful results unfortunately. I am assuming I have to take special care because the return type is a primitive (?), as the type gets interfaced through Predef
First Attempt:
Results in the error at the very end
tree match {
case pl # ClassDef(mods, name, tparams, e # Template(parent, self, body)) =>
parent.lift(1) match {
case Some(a # TypeTree()) =>
a.original match {
case AppliedTypeTree(Select(This(TypeName(s)), tpt), args) =>
if (tpt.toString == "Policy") {
val insert = q""" def q(): String = {"hello"}""".asInstanceOf[DefDef]
val DefDef(dmodifiers, dname, dtparams, dvparams, dtpt, drhs) = insert
val source = treeCopy.DefDef(insert, dmodifiers, dname, dtparams, dvparams, dtpt, drhs)
val finalCopy = pl.copy(
mods,
name,
tparams,
Template(
parent,
self,
body.:+(
source
)
)
)
localTyper.typed(finalCopy)
} else {
super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
Instead of building the source, I have also tried manually constructing various things.
DefDef(
Modifiers(),
TermName("q"),
List(),
List(List()),
TypeTree().setOriginal(Select(Select(Ident(scala), scala.Predef), TypeName("String"))), //attempt1
Ident(TypeName("String")), //attemp2
TypeTree().setOriginal(Ident(TypeName("String"))), //attempt3
gen.mkAttributedRef(typeOf[String].typeSymbol), //attempt 4
Literal(Constant("hello")))
All resulting in the same error. Note that in the error, the class being printed have the method inserted but the type checker can not make sense of it for some reason
Following the suggestion on the contributors forum, I tried to set the ownership
val source = ... same as above
pl.symbol.owner.info.decls.unlink(pl.symbol)
localTyper.namer.enterDefDef(source)
source.symbol.owner.info.decls.enter(pl.symbol)
val finalCopy = pl.copy(....) //same as above
localTyper.namer.enterClassDef(finalCopy)
finalCopy.symbol.owner.info.decls.enter(finalCopy.symbol)
localTyper.typed(finalCopy)
But this completely screwed up everything and the compiler messed up the symbols and telling me fully implemented classes didn't implement the abstract members thus needed to be declared abstract
I have been going around in circles on this so if anybody have an idea what the best way to append a method to class body after the typer or have somewhat related examples, I would certainly appreciate it
Exception in thread "main" java.lang.AssertionError: assertion failed:
[error] class UserPolicy extends AnyRef with prv.Main.Policy[prv.Main.User] {
[error] <paramaccessor> private[this] val u: prv.Main.User = _;
[error] def <init>(u: prv.Main.User): prv.Main.UserPolicy = {
[error] UserPolicy.super.<init>();
[error] ()
[error] };
[error] private[this] val data: prv.Main.User = UserPolicy.this.u;
[error] <stable> <accessor> def data: prv.Main.User = UserPolicy.this.data;
[error] protected def checkDeclassify(): prv.Main.User = {
[error] def checkExpanded(): prv.Main.User = UserPolicy.this.data;
[error] checkExpanded()
[error] };
[error] def unsafeUnwrap(reason: String): prv.Main.User = UserPolicy.this.data;
[error] def q2(): String = "hello";
[error] def q(): String = "hello"
[error] }
[error] while compiling: <test>
[error] during phase: method-wiring-phase
[error] library version: version 2.13.1
[error] compiler version: version 2.13.1
[error] reconstructed args: -usejavacp
[error] last tree to typer: type UserPolicy
[error] tree position: <unknown>
[error] symbol: <none>
[error] symbol definition: <none> (a NoSymbol)
[error] symbol package: <none>
[error] symbol owners:
[error] call site: <none> in <none>
[error] == Source file context for tree position ==
[error] at scala.reflect.internal.SymbolTable.throwAssertionError(SymbolTable.scala:170)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typedClassDef(Typers.scala:1876)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5794)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5886)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5948)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:254)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:195)
[error] at scala.reflect.api.Trees$Transformer.$anonfun$transformStats$1(Trees.scala:2614)
[error] at scala.reflect.api.Trees$Transformer.transformStats(Trees.scala:2612)
[error] at scala.reflect.internal.Trees$Template.transform(Trees.scala:517)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.$anonfun$transform$1(TypingTransformers.scala:47)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:32)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:333)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:195)
[error] at scala.reflect.api.Trees$Transformer.transformTemplate(Trees.scala:2587)
[error] at scala.reflect.internal.Trees$ModuleDef.$anonfun$transform$3(Trees.scala:370)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:32)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:24)
[error] at scala.reflect.internal.Trees$ModuleDef.transform(Trees.scala:369)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:51)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:333)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:195)
[error] at scala.reflect.api.Trees$Transformer.$anonfun$transformStats$1(Trees.scala:2614)
[error] at scala.reflect.api.Trees$Transformer.transformStats(Trees.scala:2612)
[error] at scala.reflect.internal.Trees$PackageDef.$anonfun$transform$1(Trees.scala:316)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:32)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:24)
[error] at scala.reflect.internal.Trees$PackageDef.transform(Trees.scala:316)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.$anonfun$transform$2(TypingTransformers.scala:49)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:32)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:333)
[error] at privacy.MethodWiring$$anon$3.apply(MethodWire.scala:192)
[error] at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:452)
[error] at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:397)
[error] at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1506)
[error] at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1490)
[error] at scala.tools.nsc.Global$Run.compileSources(Global.scala:1482)
[error] at privacy.AnnotationFinderTest$.delayedEndpoint$privacy$AnnotationFinderTest$1(Test.scala:114)
[error] at privacy.AnnotationFinderTest$delayedInit$body.apply(Test.scala:13)
[error] at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] at scala.App.$anonfun$main$1(App.scala:73)
[error] at scala.App.$anonfun$main$1$adapted(App.scala:73)
[error] at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
[error] at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
[error] at scala.collection.AbstractIterable.foreach(Iterable.scala:921)
[error] at scala.App.main(App.scala:73)
[error] at scala.App.main$(App.scala:71)
[error] at privacy.AnnotationFinderTest$.main(Test.scala:13)
[error] at privacy.AnnotationFinderTest.main(Test.scala)
Posting an answer so the question can be closed. It took me a while but I think I figured it out.
Thanks to #SethTisue for pointing me to TwoTails. I was able to correctly synthesize a method using the part of the code in that repo. However the bottom line is doing something like this after the typer is not trivially possible. Here is the reason why:
Say you are trying to synthesize and append a method m to a class C after the typer. The problem is if you are synthesizing this method, you will eventually invoke it somewhere new C().m. The membership is resolved during the typer, so the typer will never complete and throw an error method m is not a member of C. So,
If you don't require the package information to achieve this, you should do this after the parser
If you need the package information, this gets very tricky. You need to add a few new phases after the parser. I will omit the code because it is very lengthy but here is the gist of it.
Phase 1: Accumulate the list of Class Names you will be appending to and their existing method, skip if it's a pre-known class
Phase 2: Go through the code and accumulate a list of all the symbols that correspond to an instance of this class. ValDef and any parameters to the DefDef. If you have implemented Hindley Milner you will immediately identify the problem that will a way to distinguish similarly named symbols in different scopes. There is a lot of existing literature on this that you can read, I am skipping the details.
Phase 3: Go through the code and accumulate a list of method names that are invoked on C but doesn't yet exist. You need to memorize the parameters and their types as well. Whether you need the return type or not really depends on what you are doing and/or if you want extra soundness/verification in a later step. You can skip this phase if the method you are appending is static and you already know what members will be missing in advance.
Phase 4: Go through the code one last time and append a null method into C that with the proper name and types. Retuning null isn't the best thing, not sure if there is a better alternative.
Later in the typer replace the appended method body with the proper one (the one you are copying)
The actual synthesis looks like this but as I mentioned above, if you actually want this to work, you will need to figure out all the stuff above.
override def transform(tree: Tree): Tree = {
val classesOfInterest = policyTypes.map(a => s"${a.packageName}.${a.typeName}").toList
tree match {
case pl # ClassDef(mods, name, tparams, e # Template(parent, self, body)) =>
parent.lift(1) match {
case Some(a # TypeTree()) =>
val original = a.original
original match {
case AppliedTypeTree(Select(This(TypeName(s)), tpt), args)=>
if (tpt.toString == "Policy") {
val insert = q"""... method to insert""".asInstanceOf[DefDef]
val DefDef(dmodifiers, dname, dtparams, dvparams, dtpt, drhs) = insert
val source = treeCopy.DefDef(insert, dmodifiers, dname, dtparams, dvparams, dtpt, drhs)
//borrow the symbol of another method from the body. This is guaranteed because members like toString will be generated at this point
val xyz = mkNewMethodSymbol(body(body.length).symbol, TermName("q"))
localTyper.typedPos(tree.pos)(
treeCopy.ClassDef(
tree,
mods,
name,
tparams,
treeCopy.Template(
e,
e.parents,
e.self,
e.body :+ localTyper.typed(DefDef(xyz, mkNewMethodRhs(xyz, insert)))
)
)
)
)
} else {
super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
}
def mkNewMethodSymbol(symbol: Symbol, name: TermName): Symbol = {
val flags = METHOD
val methSym = symbol.cloneSymbol(symbol.owner, flags, name)
val param = methSym.newSyntheticValueParam(definitions.IntTpe, TermName("indx"))
methSym.modifyInfo {
case GenPolyType(tparams, MethodType(params, res)) => GenPolyType(tparams, MethodType(params, res))
}
localTyper.namer.enterInScope(methSym)
}
I have a case class defined as case class IndexList(value: ListBuffer[String])
I'm unable to convert this to JSON using Play API.
I've tried
object IndexList{
implicit val indexListWrites = Json.writes[IndexList]
}
as part of Macro Inception but it fails giving compilation error:
exception during macro expansion:
[error] java.lang.NoSuchMethodError: scala.collection.immutable.$colon$colon.tl$1()Lscala/collection/immutable/List;
I've also tried
object IndexList{
implicit val indexListWrites: Writes[IndexList] = Writes{
(indexList: IndexList) => JsArray(indexList.value)
}
}
but fails with error:
type mismatch;
[error] found : scala.collection.mutable.ListBuffer[String]
[error] required: Seq[play.api.libs.json.JsValue]
Any help regarding either of the methods would be helpful.
If there are other ways to accomplish this, it's fine as well.
I get a weird compilation error in the small Scala exercise I am working on.
I have this method that is supposed to keep on asking user input until a correct answer is provided. Alas I am stumbled at the first case in my pattern matching:
override def guess(guess: Int):Unit = {
val guessIndex = binary(array, guess)
guessIndex match {
case -1 => {
val nextAttempt = StdIn.readLine(s"Please be attentive $guess is outside the search range"
+" (0 to $upperBound). Try again: \n");
val a = validateType[Int](nextAttempt)
guess(a)
}
}
}
The IDE underlines guess(a) with the error "Int doesn't take parameters". Running sbt compile from the console confirms this error:
> compile
[info] Compiling 2 Scala sources to /home/vgorcinschi/Documents/eclipseProjects/Algorithms/Chapter 1 Fundamentals/algorithms1_4_34/target/scala-2.12/classes...
[error] /home/vgorcinschi/Documents/eclipseProjects/Algorithms/Chapter 1 Fundamentals/algorithms1_4_34/src/main/scala/ca/vgorcinschi/algorithms1_4_34/hotandcold/HotAndColdImpl.scala:23: Int does not take parameters
[error] guess(a)
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 0 s, completed 6-May-2017 6:47:58 PM
There are few different Stackoverflow tickets for the same error message, but they're for different scenarios. In mine here it looks like a method who takes an Int parameter is being rejected. If you could please give me a hint this would help me a lot.
Rename the guess parameter (or the method name, so it's something different) - the parameter is the first guess in scope, so the compiler thinks you're trying to call it as a function.
I tried the scala.js example https://github.com/lihaoyi/hands-on-scala-js and its scala.rx with scalatags part in advanced section.
Example compiles are runs fine but when I try to use the latest scala.rx v 0.3.1 I get following compile errors:
[info] Compiling 19 Scala sources to /home/code/workspace/hands-on-scala-js-master/examples/demos/target/scala-2.11/classes...
[error] /home/code/workspace/hands-on-scala-js-master/examples/demos/src/main/scala/advanced/BasicRx.scala:14:
[error] This Rx might leak! Either explicitly mark it unsafe (Rx.unsafe) or make an implicit RxCtx available
[error] in the enclosing scope, for example, by adding (implicit ctx: Ctx.Owner) to line 12: method main
[error] val numChars = Rx{txt().length}
[error] ^
[error] /home/code/workspace/hands-on-scala-js-master/examples/demos/src/main/scala/advanced/BasicRx.scala:15:
[error] This Rx might leak! Either explicitly mark it unsafe (Rx.unsafe) or make an implicit RxCtx available
[error] in the enclosing scope, for example, by adding (implicit ctx: Ctx.Owner) to line 12: method main
[error] val numWords = Rx{
[error] ^
[error] /home/code/workspace/hands-on-scala-js-master/examples/demos/src/main/scala/advanced/BasicRx.scala:55:
[error] This Rx might leak! Either explicitly mark it unsafe (Rx.unsafe) or make an implicit RxCtx available
[error] in the enclosing scope, for example, by adding (implicit ctx: Ctx.Owner) to line 42: method main2
[error] for(fruit <- fruits) yield Rx {
[error] ^
[error] /home/code/workspace/hands-on-scala-js-master/examples/demos/src/main/scala/advanced/BasicRx.scala:65: diverging implicit expansion for type Nothing => scalatags.JsDom.Frag
[error] starting with method rxFrag in object BasicRx
[error] ul(fragments)
[error] ^
[error] /home/code/workspace/hands-on-scala-js-master/examples/demos/src/main/scala/advanced/BasicRx.scala:72: not found: value Obs
[error] Obs(r, skipInitial = true){
[error] ^
[error] /home/code/workspace/hands-on-scala-js-master/examples/demos/src/main/scala/advanced/BasicRx.scala:72: not found: value skipInitial
[error] Obs(r, skipInitial = true){
[error] ^
[warn] /home/code/workspace/hands-on-scala-js-master/examples/demos/src/main/scala/webpage/WeatherSearch.scala:53: non-variable type argument scala.scalajs.js.Dynamic in type pattern scala.scalajs.js.Array[scala.scalajs.js.Dynamic] is unchecked since it is eliminated by erasure
[warn] case jsonlist: js.Array[js.Dynamic] =>
[warn] ^
[warn] one warning found
[error] 6 errors found
[error] (demos/compile:compile) Compilation failed
It seems that ownership concept has somehow changed and also there is no Obs companion object.
I tried to fix these erros in BasicRx.scala:
Old code:
#JSExport
def main(container: html.Div) = {
New code (Leakage fix):
#JSExport
def main(container: html.Div) = {
implicit val ctx: Ctx.Owner = Ctx.Owner.safe()
Old code:
implicit def rxFrag[T <% Frag](r: Rx[T]): Frag = {
def rSafe: dom.Node = span(r()).render
var last = rSafe
Obs(r, skipInitial = true){
val newLast = rSafe
js.Dynamic.global.last = last
last.parentNode.replaceChild(newLast, last)
last = newLast
}
last
}
New code (conversion fix):
implicit def rxFrag[T <% Frag](r: Rx[T]): Frag = {
def rSafe: dom.Node = span(r()).render
var last = rSafe
val thunk = () => {
val newLast = rSafe
js.Dynamic.global.last = last
last.parentNode.replaceChild(newLast, last)
last = newLast
}
new Obs(thunk, r)
last
}
But still get following error:
[info] Compiling 19 Scala sources to /home/code/workspace/hands-on-scala-js-master/examples/demos/target/scala-2.11/classes...
[warn] /home/code/workspace/hands-on-scala-js-master/examples/demos/src/main/scala/webpage/WeatherSearch.scala:53: non-variable type argument scala.scalajs.js.Dynamic in type pattern scala.scalajs.js.Array[scala.scalajs.js.Dynamic] is unchecked since it is eliminated by erasure
[warn] case jsonlist: js.Array[js.Dynamic] =>
[warn] ^
[error] /home/code/workspace/hands-on-scala-js-master/examples/demos/src/main/scala/advanced/BasicRx.scala:74: No implicit Ctx.Data is available here!
[error] def rSafe: dom.Node = span(r()).render
[error] ^
How to fix this one?
And is the leakage fix correctly done?
Updating scalatags to 0.5.4. did not have any affect.
I helped contribute to the 0.3.x branch of scala.rx - and yep, the api is largely changed from 0.2.x. However, I also maintain a more recent version of the "framework" code you are looking at here:
https://github.com/Voltir/framework.rx/blob/master/src/main/scala/framework/Framework.scala
Also, it turns out that in the case of using js.App and its def main(), the correct way to get a Ctx.Owner is to use import rx.Ctx.Owner.Unsafe._. So long as that main function is evaluated a finite number of times (ie once) per page load, it is safe to allow a "leak". Your usage of Ctx.Owner.safe() wont work because the compiler can't prove that main is only called a finite number of times.
One complete example of all of this can be found in the demo project of this library:
https://github.com/Voltir/route.rx
In order to try and understand Scala's type system I'm attempting
to implment a custom implementation for List.foreach method :
package com
object customForEach extends App {
class customForEach[B, A] extends Iterable[A] with collection.Seq[A] {
def foreach[B](f: A ⇒ B) {
var these = this
while (!these.isEmpty) {
f(these.head)
these = these.tail
}
}
def tail = this match {
case h :: t ⇒ t
}
}
}
When I complile this code I receive errors :
[error] \Desktop\Scala\src\main\scala\typeparam.scala:16: constructor cannot be instantiated to expected type;
[error] found : scala.collection.immutable.::[B(in class ::)]
[error] required: com.customForEach.customForEach[B(in class customForEach),A]
[error] case h :: t ? t
[error] ^
[error] \Desktop\Scala\src\main\scala\typeparam.scala:16: not found: value t
[error] case h :: t ? t
[error] ^
[error] \Desktop\Scala\src\main\scala\typeparam.scala:11: type mismatch;
[error] found : Seq[A]
[error] required: com.customForEach.customForEach[B,A]
[error] these = these.tail
[error] ^
[error] three errors found
[error] (compile:compile) Compilation failed
[error] Total time: 0 s, completed 31-Jan-2015 11:53:40
In particular I find it iteresting how println can be composed with List in this fashion : List(1,2,3).foreach(println)
Do I need to add extend another trait in order to access the .tail function ?
For this error :
not found: value t
[error] case h :: t ? t
Shouldn't t be found since it is created using pattern match operator :: ?
There are many reasons why this code won't work. In order to understand the first compiler error not found: value t, you must look at the error immediately before it. :: exists solely for List, but here you do not have a List, only Iterable with Seq. That pattern match can't work, which causes t to become "not found".
There are even larger problems than that, though. Even if you remove your definition of tail (which is unnecessary), you'll then find that you're missing abstract method definitions for apply, iterator, and length from the Seq trait. I imagine you're doing this because you can't extend List, which is sealed. You can copy the implementations of apply, and length from LinearSeqOptimized, then easily implement an iterator method, but there's still another problem: your class does not have a constructor.
Okay, well we'll look at what List does again. List is abstract and has two sub-types, :: and Nil. Nil is just a case object, and :: has a constructor that accepts the head and tail of the List. This isn't going to help you very much, unless you also want to duplicate the code for :: and Nil as well.
Scala collections are very large complicated beasts, and extending them to override one method is not a simple process.