I have been stuck on this issue for a week and don't seem to be getting anywhere. I am trying to copy some methods and fields from one class to another.
I have two phases that are involved in this. The first phase scans the code, finds the method defs that need to copied, and save the corresponding Tree
The second phase inserts this tree where needs to go. In order to simplify this question, let's forget about the copying and say that I am trying to insert a simple method def hello(): String = "hello" to the body of some class
The plugin runs after the typer (because I need the package information), and I am having a problem with injecting the type information properly. This results in an assertion exception in the later type checking stage (Full stacktrace at the bottom)
I asked about this in the metaprogramming discord and was pointed to the following resources.
Scala compiler plugin to rewrite method calls
https://contributors.scala-lang.org/t/scala-compiler-plugin-naming-issues-after-typer/2835
But neither yielded successful results unfortunately. I am assuming I have to take special care because the return type is a primitive (?), as the type gets interfaced through Predef
First Attempt:
Results in the error at the very end
tree match {
case pl # ClassDef(mods, name, tparams, e # Template(parent, self, body)) =>
parent.lift(1) match {
case Some(a # TypeTree()) =>
a.original match {
case AppliedTypeTree(Select(This(TypeName(s)), tpt), args) =>
if (tpt.toString == "Policy") {
val insert = q""" def q(): String = {"hello"}""".asInstanceOf[DefDef]
val DefDef(dmodifiers, dname, dtparams, dvparams, dtpt, drhs) = insert
val source = treeCopy.DefDef(insert, dmodifiers, dname, dtparams, dvparams, dtpt, drhs)
val finalCopy = pl.copy(
mods,
name,
tparams,
Template(
parent,
self,
body.:+(
source
)
)
)
localTyper.typed(finalCopy)
} else {
super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
Instead of building the source, I have also tried manually constructing various things.
DefDef(
Modifiers(),
TermName("q"),
List(),
List(List()),
TypeTree().setOriginal(Select(Select(Ident(scala), scala.Predef), TypeName("String"))), //attempt1
Ident(TypeName("String")), //attemp2
TypeTree().setOriginal(Ident(TypeName("String"))), //attempt3
gen.mkAttributedRef(typeOf[String].typeSymbol), //attempt 4
Literal(Constant("hello")))
All resulting in the same error. Note that in the error, the class being printed have the method inserted but the type checker can not make sense of it for some reason
Following the suggestion on the contributors forum, I tried to set the ownership
val source = ... same as above
pl.symbol.owner.info.decls.unlink(pl.symbol)
localTyper.namer.enterDefDef(source)
source.symbol.owner.info.decls.enter(pl.symbol)
val finalCopy = pl.copy(....) //same as above
localTyper.namer.enterClassDef(finalCopy)
finalCopy.symbol.owner.info.decls.enter(finalCopy.symbol)
localTyper.typed(finalCopy)
But this completely screwed up everything and the compiler messed up the symbols and telling me fully implemented classes didn't implement the abstract members thus needed to be declared abstract
I have been going around in circles on this so if anybody have an idea what the best way to append a method to class body after the typer or have somewhat related examples, I would certainly appreciate it
Exception in thread "main" java.lang.AssertionError: assertion failed:
[error] class UserPolicy extends AnyRef with prv.Main.Policy[prv.Main.User] {
[error] <paramaccessor> private[this] val u: prv.Main.User = _;
[error] def <init>(u: prv.Main.User): prv.Main.UserPolicy = {
[error] UserPolicy.super.<init>();
[error] ()
[error] };
[error] private[this] val data: prv.Main.User = UserPolicy.this.u;
[error] <stable> <accessor> def data: prv.Main.User = UserPolicy.this.data;
[error] protected def checkDeclassify(): prv.Main.User = {
[error] def checkExpanded(): prv.Main.User = UserPolicy.this.data;
[error] checkExpanded()
[error] };
[error] def unsafeUnwrap(reason: String): prv.Main.User = UserPolicy.this.data;
[error] def q2(): String = "hello";
[error] def q(): String = "hello"
[error] }
[error] while compiling: <test>
[error] during phase: method-wiring-phase
[error] library version: version 2.13.1
[error] compiler version: version 2.13.1
[error] reconstructed args: -usejavacp
[error] last tree to typer: type UserPolicy
[error] tree position: <unknown>
[error] symbol: <none>
[error] symbol definition: <none> (a NoSymbol)
[error] symbol package: <none>
[error] symbol owners:
[error] call site: <none> in <none>
[error] == Source file context for tree position ==
[error] at scala.reflect.internal.SymbolTable.throwAssertionError(SymbolTable.scala:170)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typedClassDef(Typers.scala:1876)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5794)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5886)
[error] at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5948)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:254)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:195)
[error] at scala.reflect.api.Trees$Transformer.$anonfun$transformStats$1(Trees.scala:2614)
[error] at scala.reflect.api.Trees$Transformer.transformStats(Trees.scala:2612)
[error] at scala.reflect.internal.Trees$Template.transform(Trees.scala:517)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.$anonfun$transform$1(TypingTransformers.scala:47)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:32)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:333)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:195)
[error] at scala.reflect.api.Trees$Transformer.transformTemplate(Trees.scala:2587)
[error] at scala.reflect.internal.Trees$ModuleDef.$anonfun$transform$3(Trees.scala:370)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:32)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:24)
[error] at scala.reflect.internal.Trees$ModuleDef.transform(Trees.scala:369)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:51)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:333)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:195)
[error] at scala.reflect.api.Trees$Transformer.$anonfun$transformStats$1(Trees.scala:2614)
[error] at scala.reflect.api.Trees$Transformer.transformStats(Trees.scala:2612)
[error] at scala.reflect.internal.Trees$PackageDef.$anonfun$transform$1(Trees.scala:316)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:32)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:24)
[error] at scala.reflect.internal.Trees$PackageDef.transform(Trees.scala:316)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.$anonfun$transform$2(TypingTransformers.scala:49)
[error] at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2625)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:37)
[error] at scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:32)
[error] at privacy.MethodWiring$MethodWiringPhase.transform(MethodWire.scala:333)
[error] at privacy.MethodWiring$$anon$3.apply(MethodWire.scala:192)
[error] at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:452)
[error] at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:397)
[error] at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1506)
[error] at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1490)
[error] at scala.tools.nsc.Global$Run.compileSources(Global.scala:1482)
[error] at privacy.AnnotationFinderTest$.delayedEndpoint$privacy$AnnotationFinderTest$1(Test.scala:114)
[error] at privacy.AnnotationFinderTest$delayedInit$body.apply(Test.scala:13)
[error] at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] at scala.App.$anonfun$main$1(App.scala:73)
[error] at scala.App.$anonfun$main$1$adapted(App.scala:73)
[error] at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
[error] at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
[error] at scala.collection.AbstractIterable.foreach(Iterable.scala:921)
[error] at scala.App.main(App.scala:73)
[error] at scala.App.main$(App.scala:71)
[error] at privacy.AnnotationFinderTest$.main(Test.scala:13)
[error] at privacy.AnnotationFinderTest.main(Test.scala)
Posting an answer so the question can be closed. It took me a while but I think I figured it out.
Thanks to #SethTisue for pointing me to TwoTails. I was able to correctly synthesize a method using the part of the code in that repo. However the bottom line is doing something like this after the typer is not trivially possible. Here is the reason why:
Say you are trying to synthesize and append a method m to a class C after the typer. The problem is if you are synthesizing this method, you will eventually invoke it somewhere new C().m. The membership is resolved during the typer, so the typer will never complete and throw an error method m is not a member of C. So,
If you don't require the package information to achieve this, you should do this after the parser
If you need the package information, this gets very tricky. You need to add a few new phases after the parser. I will omit the code because it is very lengthy but here is the gist of it.
Phase 1: Accumulate the list of Class Names you will be appending to and their existing method, skip if it's a pre-known class
Phase 2: Go through the code and accumulate a list of all the symbols that correspond to an instance of this class. ValDef and any parameters to the DefDef. If you have implemented Hindley Milner you will immediately identify the problem that will a way to distinguish similarly named symbols in different scopes. There is a lot of existing literature on this that you can read, I am skipping the details.
Phase 3: Go through the code and accumulate a list of method names that are invoked on C but doesn't yet exist. You need to memorize the parameters and their types as well. Whether you need the return type or not really depends on what you are doing and/or if you want extra soundness/verification in a later step. You can skip this phase if the method you are appending is static and you already know what members will be missing in advance.
Phase 4: Go through the code one last time and append a null method into C that with the proper name and types. Retuning null isn't the best thing, not sure if there is a better alternative.
Later in the typer replace the appended method body with the proper one (the one you are copying)
The actual synthesis looks like this but as I mentioned above, if you actually want this to work, you will need to figure out all the stuff above.
override def transform(tree: Tree): Tree = {
val classesOfInterest = policyTypes.map(a => s"${a.packageName}.${a.typeName}").toList
tree match {
case pl # ClassDef(mods, name, tparams, e # Template(parent, self, body)) =>
parent.lift(1) match {
case Some(a # TypeTree()) =>
val original = a.original
original match {
case AppliedTypeTree(Select(This(TypeName(s)), tpt), args)=>
if (tpt.toString == "Policy") {
val insert = q"""... method to insert""".asInstanceOf[DefDef]
val DefDef(dmodifiers, dname, dtparams, dvparams, dtpt, drhs) = insert
val source = treeCopy.DefDef(insert, dmodifiers, dname, dtparams, dvparams, dtpt, drhs)
//borrow the symbol of another method from the body. This is guaranteed because members like toString will be generated at this point
val xyz = mkNewMethodSymbol(body(body.length).symbol, TermName("q"))
localTyper.typedPos(tree.pos)(
treeCopy.ClassDef(
tree,
mods,
name,
tparams,
treeCopy.Template(
e,
e.parents,
e.self,
e.body :+ localTyper.typed(DefDef(xyz, mkNewMethodRhs(xyz, insert)))
)
)
)
)
} else {
super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
case _ => super.transform(tree)
}
}
def mkNewMethodSymbol(symbol: Symbol, name: TermName): Symbol = {
val flags = METHOD
val methSym = symbol.cloneSymbol(symbol.owner, flags, name)
val param = methSym.newSyntheticValueParam(definitions.IntTpe, TermName("indx"))
methSym.modifyInfo {
case GenPolyType(tparams, MethodType(params, res)) => GenPolyType(tparams, MethodType(params, res))
}
localTyper.namer.enterInScope(methSym)
}
So this is a pretty specific problem, but might be related to issues other may have experience, though I could not find a good solution.
in our specific case, we are using Elasticsearch. the problematic piece of code is this:
val hit: org.elasticsearch.search.SearchHit = {...}
val innerSystemField: Option[Long] =
Try(hit.field("system.innerField").getValue) match {
case Success(x) => Some(x.asInstanceOf[Long])
case Failure(e) => {
logger.error("exception during innerField retrieval", e)
None
}
}
so, it doesn't really matters we are using Elasticsearch, what matter, is the library API. so, here's the SearchHit interface's field method which returns an instance of SearchHitField. and here's the SearchHitField interface's getValue method. the method is declared as:
<V> V getValue();
and this is where the problem lies. the mapping which we defined Elasticsearch with, guarantee that the returned value will always be a Long. but every once in a while we get a Failure containing a java.lang.ClassCastException: java.lang.Long cannot be cast to scala.runtime.Nothing$. The thing is, that if I explicitly write the type, it won't compile:
[error] /home/me/projects/my-project/src/main/scala/com/org/project/MyElasticsearchCode.scala:123: polymorphic expression cannot be instantiated to expected type;
[error] found : [V]()V
[error] required: Long
[error] Try[Long](hit.field("system.innerField").getValue) match {
[error] ^
[error] one error found
[error] (project/compile:compileIncremental) Compilation failed
[error] Total time: 4 s, completed Jul 27, 2015 4:16:41 PM
So how do I get around this problem?
I find the message of the ClassCastException a bit confusing. Maybe you could try this:
val innerSystemField: Option[Long] =
Try[Any](hit.field("system.innerField").getValue) match {
case Success(x: java.lang.Long) => Some(x.asInstanceOf[Long])
case Failure(e) => {
logger.error("exception during innerField retrieval", e)
None
}
}
In order to try and understand Scala's type system I'm attempting
to implment a custom implementation for List.foreach method :
package com
object customForEach extends App {
class customForEach[B, A] extends Iterable[A] with collection.Seq[A] {
def foreach[B](f: A ⇒ B) {
var these = this
while (!these.isEmpty) {
f(these.head)
these = these.tail
}
}
def tail = this match {
case h :: t ⇒ t
}
}
}
When I complile this code I receive errors :
[error] \Desktop\Scala\src\main\scala\typeparam.scala:16: constructor cannot be instantiated to expected type;
[error] found : scala.collection.immutable.::[B(in class ::)]
[error] required: com.customForEach.customForEach[B(in class customForEach),A]
[error] case h :: t ? t
[error] ^
[error] \Desktop\Scala\src\main\scala\typeparam.scala:16: not found: value t
[error] case h :: t ? t
[error] ^
[error] \Desktop\Scala\src\main\scala\typeparam.scala:11: type mismatch;
[error] found : Seq[A]
[error] required: com.customForEach.customForEach[B,A]
[error] these = these.tail
[error] ^
[error] three errors found
[error] (compile:compile) Compilation failed
[error] Total time: 0 s, completed 31-Jan-2015 11:53:40
In particular I find it iteresting how println can be composed with List in this fashion : List(1,2,3).foreach(println)
Do I need to add extend another trait in order to access the .tail function ?
For this error :
not found: value t
[error] case h :: t ? t
Shouldn't t be found since it is created using pattern match operator :: ?
There are many reasons why this code won't work. In order to understand the first compiler error not found: value t, you must look at the error immediately before it. :: exists solely for List, but here you do not have a List, only Iterable with Seq. That pattern match can't work, which causes t to become "not found".
There are even larger problems than that, though. Even if you remove your definition of tail (which is unnecessary), you'll then find that you're missing abstract method definitions for apply, iterator, and length from the Seq trait. I imagine you're doing this because you can't extend List, which is sealed. You can copy the implementations of apply, and length from LinearSeqOptimized, then easily implement an iterator method, but there's still another problem: your class does not have a constructor.
Okay, well we'll look at what List does again. List is abstract and has two sub-types, :: and Nil. Nil is just a case object, and :: has a constructor that accepts the head and tail of the List. This isn't going to help you very much, unless you also want to duplicate the code for :: and Nil as well.
Scala collections are very large complicated beasts, and extending them to override one method is not a simple process.
import scala.slick.driver.MySQLDriver.simple._
class RichTable[T](tag: Tag, name: String) extends Table[T](tag, name) {
case class QueryExt[B](q: Query[RichTable.this.type, B]) {
def whereEq[C](col: RichTable.this.type => Column[C], c: C) = {
q.filter { fields =>
col(fields) === c
}
}
}
}
Then it complains
[error] /home/jilen/workspace/play-slick/src/main/scala/play/slick/SlickQueryExtension.scala:10: value === is not a member of slick.driver.MySQLDriver.simple.Column[C]
[error] col(fields) === c
[error] ^
[error] /home/jilen/workspace/play-slick/src/main/scala/play/slick/SlickQueryExtension.scala:9: ambiguous implicit values:
[error] both value BooleanColumnCanBeQueryCondition in object CanBeQueryCondition of type => scala.slick.lifted.CanBeQueryCondition[scala.slick.lifted.Column[Boolean]]
[error] and value BooleanOptionColumnCanBeQueryCondition in object CanBeQueryCondition of type => scala.slick.lifted.CanBeQueryCondition[scala.slick.lifted.Column[Option[Boolean]]]
[error] match expected type scala.slick.lifted.CanBeQueryCondition[Nothing]
[error] q.filter { fields =>
[error] ^
[error] two errors found
[error] (compile:compile) Compilation failed
[error] Total time: 0 s, completed Mar 6, 2014 1:21:48 AM
There have been questions about this, but the answers did not work for 2.0
How to parametrize Scala Slick queries by WHERE clause conditions?
Slick doesn't have any information about C, so it doesn't know if it can and how it should map it to a database value and if it can use === on it. So you get a type error. You will have to use Scala's type system to restrict the type to one for which Slick knows how to map it. You can do this by providing a so-called Context Bound, in this case :BaseColumnType.
def whereEq[C:BaseColumnType](col: RichTable.this.type => Column[C], c: C) = {
q.filter { fields =>
col(fields) === c
}
}
BaseColumnType is provided by Slick and using it in this way basically tells the Scala compiler to look for an implicit value of type BaseColumnType[C] in scope, where you call whereEq. Because then it is usually known what C will actually be. Slick comes with BaseColumnType[Int], BaseColumnType[String], etc. so at the call site, the Scala compiler can find one when your C is really an Int or String in that particular call and this way pass the info further to Slick.
Same for LiuTiger's question. abstract class Crud[..., PK:BaseColumnType] should do the trick, a trait doesn't work with context bounds. When implementing an abstract DAO be prepared to face a lot of challenges and get to the edges of your Scala type system skills and learn quite a bit about type inference order, implicit parameters, etc.
I changed one function from:
def submit = Action { request =>
signupForm.bindFromRequest()(request).fold(
// Form has errors
errors => BadRequest(html.signup.form(errors)),
// We got a valid User value, display the summary
user => {
// intensive computation involving database
Ok("okay")
}
)
}
to
def submit = Action { request =>
val result = Akka.future {
signupForm.bindFromRequest()(request).fold(
// Form has errors
errors => BadRequest(html.signup.form(errors)),
// We got a valid User value, display the summary
user => {
// intensive computation involving database
Ok("okay")
}
)
}
Async {
result
}
}
and I get the compilation error of:
[error] found : play.api.mvc.SimpleResult[_ >: java.lang.String with play.api.templates.Html <: java.io.Serializable]
[error] required: play.api.mvc.SimpleResult[_1(in value result)] where type _1(in value result) >: java.lang.String with play.api.templates.Html <: java.io.Serializable
[error] Note: java.io.Serializable >: _1, but class SimpleResult is invariant in type A.
[error] You may wish to define A as -A instead. (SLS 4.5)
[error] signupForm.bindFromRequest()(request).fold(
[error] ^
[error] one error found
The error message seem like it has something to do with variance. Does anyone understand what's going on?
BadRequest is returning the type SimpleResult[Html]
Ok is returning the type SimpleResult[String]
If you make BadRequest and Ok return the same type then it would work.
Try doing Ok(Html("ok")) - or actually render a page.