How to add a new Class in a Scala Compiler Plugin? - scala

In a Scala Compiler Plugin, I'm trying to create a new class that implement a pre-existing trait. So far my code looks like this:
def trait2Impl(original: ClassDef, newName: String): ClassDef = {
val impl = original.impl
// Seems OK to have same self, but does not make sense to me ...
val self = impl.self
// TODO: implement methods ...
val body = impl.body
// We implement original
val parents = original :: impl.parents
val newImpl = treeCopy.Template(impl, parents, self, body)
val name = newTypeName(newName)
// We are a syntheic class, not a user-defined trait
val mods = (original.mods | SYNTHETIC) &~ TRAIT
val tp = original.tparams
val result = treeCopy.ClassDef(original, mods, name, tp, newImpl)
// Same Package?
val owner = original.symbol.owner
// New symbol. What's a Position good for?
val symbol = new TypeSymbol(owner, NoPosition, name)
result.setSymbol(symbol)
symbol.setFlag(SYNTHETIC)
symbol.setFlag(ABSTRACT)
symbol.resetFlag(INTERFACE)
symbol.resetFlag(TRAIT)
owner.info.decls.enter(symbol)
result
}
But it doesn't seem to get added to the package. I suspect that is because actually the package got "traversed" before the trait that causes the generation, and/or because the "override def transform(tree: Tree): Tree" method of the TypingTransformer can only return one Tree, for every Tree that it receives, so it cannot actually produce a new Tree, but only modify one.
So, how do you add a new Class to an existing package? Maybe it would work if I transformed the package when "transform(Tree)" gets it, but I that point I don't know the content of the package yet, so I cannot generate the new Class this early (or could I?). Or maybe it's related to the "Position" parameter of the Symbol?
So far I found several examples where Trees are modified, but none where a completely new Class is created in a Compiler Plugin.

The full source code is here: https://gist.github.com/1794246
The trick is to store the newly created ClassDefs and use them when creating a new PackageDef. Note that you need to deal with both Symbols and trees: a package symbol is just a handle. In order to generate code, you need to generate an AST (just like for a class, where the symbol holds the class name and type, but the code is in the ClassDef trees).
As you noted, package definitions are higher up the tree than classes, so you'd need to recurse first (assuming you'll generate the new class from an existing class). Then, once the subtrees are traversed, you can prepare a new PackageDef (every compilation unit has a package definition, which by default is the empty package) with the new classes.
In the example, assuming the source code is
class Foo {
def foo {
"spring"
}
}
the compiler wraps it into
package <empty> {
class Foo {
def foo {
"spring"
}
}
}
and the plugin transforms it into
package <empty> {
class Foo {
def foo {
"spring"
}
}
package mypackage {
class MyClass extends AnyRef
}
}

Related

Scala macros: How can I get a list of the objects within a given package that inherit some trait?

I have a package foo.bar in which a trait Parent is defined, and a series of objects Child1, Child2, Child3 are defined. I would like to get a List[Parent] containing all child objects defined in foo.bar. How can I write such a macro?
Right now I have the following:
def myMacro(c: blackbox.Context): c.Expr[Set[RuleGroup]] = {
val parentSymbol = c.mirror.staticClass("foo.bar.Parent")
c.mirror.staticPackage("foo.bar").info.members
// get all objects
.filter { sym =>
// remove $ objects
sym.isModule && sym.asModule.moduleClass.asClass.baseClasses.contains(parentSymbol)
}.map { ??? /* retrieve? */ }
???
}
I think this is what you'd be looking for:
.map(sym => c.mirror.reflectModule(sym.asModule).instance.asInstanceOf[Parent])
Later edit:
I have tried doing this in a trait, so not a macro like above, and when calling it with a different package than the one calling it from, it returned an empty collection of objects. Reading through it might have to do with how classloaders work in Scala as they don't have the knowledge of all the classes being loaded, but i see your macro doesn't use a classloader so maybe it still works in your case.
For me it worked using the Reflections library like this in a trait:
import org.reflections.Reflections
import scala.reflect.runtime.universe
import scala.reflect.{ClassTag, classTag}
import scala.collection.JavaConverters._
trait ChildObjects {
def childObjectsOf[Parent: ClassTag](containingPackageFullName: String): Set[Parent] = {
new Reflections(containingPackageFullName)
.getSubTypesOf(classTag[Parent].runtimeClass)
.asScala
.map(cls => {
val mirror = universe.runtimeMirror(cls.getClassLoader)
val moduleSymbol = mirror.moduleSymbol(cls)
mirror.reflectModule(moduleSymbol).instance.asInstanceOf[Parent]
})
.toSet
}
}
If the trait is not sealed you can't do that. Fundamentally if a trait is not sealed, it means new subclasses can be added later under different compilation unit.
If the trait is sealed, than you can use knownDirectSubclasses of ClassSymbolApi but beware of the possible issues the depend on order such as this and this in circe

Scala compiler plugin to rewrite method calls

I'm trying to write a Scala compiler plugin that rewrites method calls from instantiating Set to instantiations of LinkedHashSet. Unfortunately I can't find any working example that does this already. The following code fails with no-symbol does not have an owner:
object DemoErasureComponent extends PluginComponent with TypingTransformers with Transform {
val global: DemoPlugin.this.global.type = DemoPlugin.this.global
import global._
override val runsAfter = List("erasure")
val phaseName = "rewrite-sets"
def newTransformer(unit: CompilationUnit) = new SetTransformer(unit)
class SetTransformer(unit: CompilationUnit) extends TypingTransformer(unit) {
override def transform(tree: Tree): Tree = tree match {
case a#Apply(r#Select(rcvr#Select(predef, set), name), args) if name.toString == "Set" =>
localTyper.typed(treeCopy.Apply(tree, Ident(newTermName("LinkedHashSet")), args))
case t => super.transform(tree)
}
}
For the record, I've found these resources so far:
Scalaxy compiler plugin: https://github.com/ochafik/Scalaxy
Boxer compiler plugin example: https://github.com/retronym/boxer
outdated compiler plugin inside the Scala compiler: http://lampsvn.epfl.ch/trac/scala/browser/scala/trunk/docs/examples/plugintemplate/src/plugintemplate/TemplateTransformComponent.scala
localTyper.typed(treeCopy.Apply(tree, Ident(newTermName("LinkedHashSet")), args))
Here, you are creating a new Apply node via a tree copy, which will copy the type and symbol from tree.
When you typecheck this node, the typer will not recurse into its children as it is already typed, so the Ident will pass through without a type and symbol, which will likely lead to a crash in the code generation phase.
Instead of creating an unattributed Ident and typechecking it, it is more customary to create a fully attrtibuted reference with of the utility methods in TreeGen.
gen.mkAttributedRef(typeOf[scala.collection.immutable.LinkedHashSet].typeSymbol)
The case is also pretty suspicious. You should never need to compare strings like that. It is always better to compare Symbols.
Furthermore, you have to be aware of the tree shapes at the phase you're inserting your compiler plugin. Notice below how after typer, the tree is expanded to include the call to the apply method, and that after erasure, the variable arguments are wrapped in single argument.
% qscalac -Xprint:parser,typer,erasure sandbox/test.scala
[[syntax trees at end of parser]] // test.scala
package <empty> {
object Test extends scala.AnyRef {
def <init>() = {
super.<init>();
()
};
Set(1, 2, 3)
}
}
[[syntax trees at end of typer]] // test.scala
package <empty> {
object Test extends scala.AnyRef {
def <init>(): Test.type = {
Test.super.<init>();
()
};
scala.this.Predef.Set.apply[Int](1, 2, 3)
}
}
[[syntax trees at end of erasure]] // test.scala
package <empty> {
object Test extends Object {
def <init>(): Test.type = {
Test.super.<init>();
()
};
scala.this.Predef.Set().apply(scala.this.Predef.wrapIntArray(Array[Int]{1, 2, 3}))
}
}
Tracking down non-determinism that is caused by making assumptions about the iteration order of HashMaps can be a bit of a nightmare. But I'd caution against this sort of rewriting. If you want a policy within your system to say that non-determistic Sets and Maps should not be used, removing direct usages of them is insufficient. Every call to .toSet or toMap will end up using them anyway.
You might be better served to instrument the bytecode of the standard library (or use a patched standard library) in a test mode to catch all instantiations of these structures (perhaps logging a stack trace). Or, as a more fine grained alternative, find calls to place like HashTrieSet#foreach (although, you'll need to filter out benign usages like the call to foreach within collection.immutable.HashSet(1, 2, 3).count(_ => true).

Why use scala's cake pattern rather than abstract fields?

I have been reading about doing Dependency Injection in scala via the cake pattern. I think I understand it but I must have missed something because I still can't see the point in it! Why is it preferable to declare dependencies via self types rather than just abstract fields?
Given the example in Programming Scala TwitterClientComponent declares dependencies like this using the cake pattern:
//other trait declarations elided for clarity
...
trait TwitterClientComponent {
self: TwitterClientUIComponent with
TwitterLocalCacheComponent with
TwitterServiceComponent =>
val client: TwitterClient
class TwitterClient(val user: TwitterUserProfile) extends Tweeter {
def tweet(msg: String) = {
val twt = new Tweet(user, msg, new Date)
if (service.sendTweet(twt)) {
localCache.saveTweet(twt)
ui.showTweet(twt)
}
}
}
}
How is this better than declaring dependencies as abstract fields as below?
trait TwitterClient(val user: TwitterUserProfile) extends Tweeter {
//abstract fields instead of cake pattern self types
val service: TwitterService
val localCache: TwitterLocalCache
val ui: TwitterClientUI
def tweet(msg: String) = {
val twt = new Tweet(user, msg, new Date)
if (service.sendTweet(twt)) {
localCache.saveTweet(twt)
ui.showTweet(twt)
}
}
}
Looking at instantiation time, which is when DI actually happens (as I understand it), I am struggling to see the advantages of cake, especially when you consider the extra keyboard typing you need to do for the cake declarations (enclosing trait)
//Please note, I have stripped out some implementation details from the
//referenced example to clarify the injection of implemented dependencies
//Cake dependencies injected:
trait TextClient
extends TwitterClientComponent
with TwitterClientUIComponent
with TwitterLocalCacheComponent
with TwitterServiceComponent {
// Dependency from TwitterClientComponent:
val client = new TwitterClient
// Dependency from TwitterClientUIComponent:
val ui = new TwitterClientUI
// Dependency from TwitterLocalCacheComponent:
val localCache = new TwitterLocalCache
// Dependency from TwitterServiceComponent
val service = new TwitterService
}
Now again with abstract fields, more or less the same!:
trait TextClient {
//first of all no need to mixin the components
// Dependency on TwitterClient:
val client = new TwitterClient
// Dependency on TwitterClientUI:
val ui = new TwitterClientUI
// Dependency on TwitterLocalCache:
val localCache = new TwitterLocalCache
// Dependency on TwitterService
val service = new TwitterService
}
I'm sure I must be missing something about cake's superiority! However, at the moment I can't see what it offers over declaring dependencies in any other way (constructor, abstract fields).
Traits with self-type annotation is far more composable than old-fasioned beans with field injection, which you probably had in mind in your second snippet.
Let's look how you will instansiate this trait:
val productionTwitter = new TwitterClientComponent with TwitterUI with FSTwitterCache with TwitterConnection
If you need to test this trait you probably write:
val testTwitter = new TwitterClientComponent with TwitterUI with FSTwitterCache with MockConnection
Hmm, a little DRY violation. Let's improve.
trait TwitterSetup extends TwitterClientComponent with TwitterUI with FSTwitterCache
val productionTwitter = new TwitterSetup with TwitterConnection
val testTwitter = new TwitterSetup with MockConnection
Furthermore if you have a dependency between services in your component (say UI depends on TwitterService) they will be resolved automatically by the compiler.
Think about what happens if TwitterService uses TwitterLocalCache. It would be a lot easier if TwitterService self-typed to TwitterLocalCache because TwitterService has no access to the val localCache you've declared. The Cake pattern (and self-typing) allows for us to inject in a much more universal and flexible manner (among other things, of course).
I was unsure how the actual wiring would work, so I've adapted the simple example in the blog entry you linked to using abstract properties like you suggested.
// =======================
// service interfaces
trait OnOffDevice {
def on: Unit
def off: Unit
}
trait SensorDevice {
def isCoffeePresent: Boolean
}
// =======================
// service implementations
class Heater extends OnOffDevice {
def on = println("heater.on")
def off = println("heater.off")
}
class PotSensor extends SensorDevice {
def isCoffeePresent = true
}
// =======================
// service declaring two dependencies that it wants injected
// via abstract fields
abstract class Warmer() {
val sensor: SensorDevice
val onOff: OnOffDevice
def trigger = {
if (sensor.isCoffeePresent) onOff.on
else onOff.off
}
}
trait PotSensorMixin {
val sensor = new PotSensor
}
trait HeaterMixin {
val onOff = new Heater
}
val warmer = new Warmer with PotSensorMixin with HeaterMixin
warmer.trigger
in this simple case it does work (so the technique you suggest is indeed usable).
However, the same blog shows at least other three methods to achieve the same result; I think the choice is mostly about readability and personal preference. In the case of the technique you suggest IMHO the Warmer class communicates poorly its intent to have dependencies injected. Also to wire up the dependencies, I had to create two more traits (PotSensorMixin and HeaterMixin), but maybe you had a better way in mind to do it.
In this example I think there is no big difference. Self-types can potentially bring more clarity in cases when a trait declares several abstract values, like
trait ThreadPool {
val minThreads: Int
val maxThreads: Int
}
Then instead of depending on several abstract values you just declare dependency on a ThreadPool.
Self-types (as used in Cake pattern) for me are just a way to declare several abstract members at once, giving those a convenient name.

Scala Compiler TypeRef to ClassDef

I am writing a plugin which watches for an #unmatchable annotation and throws a warning if it is found in pattern matching.
I have been able to find the TypeRef, but I can not convert it into a ClassDef so I can inspect the annoations.
I'm guessing that I need to get the root of the tree and use TreeOpts.find in order to get the actual ClassDef. However, I can not find where the root tree is.
EDIT: I need more than the root Compilation units in case a matchable annoation is included in a library.
Here is what I have so far.
class UnmatchablePlugin(val global: Global) extends Plugin {
val name = "unmatchable-check-gen"
val description = "marks a class unmatchable"
val components = List[PluginComponent](UnmatchableComponent)
private object UnmatchableComponent extends PluginComponent with Transform {
val global: UnmatchablePlugin.this.global.type = UnmatchablePlugin.this.global
val runsAfter = List("parser")
// Using the Scala Compiler 2.8.x the runsAfter should be written as below
// val runsAfter = List[String]("parser");
val phaseName = UnmatchablePlugin.this.name
def newTransformer(unit: global.CompilationUnit) = UnmatchableTransformer
object UnmatchableTransformer extends global.Transformer {
override def transform(tree: global.Tree) = {
import global._
tree match {
case cd # global.CaseDef(global.Bind(_, global.Typed(exp,tpt)) , _, _) => {
//Need to turn tpt.tpe.sym into a ClassDef
println("sym: " + tpt.tpe.sym)
tree
}
case t => super.transform(t)
}
}
}
}
}
In general, you cannot turn types/symbols into trees because there might be a symbol that has no tree corresponding to it. It's a case for example when symbol is corresponding to class defined in binary class file.
However, as far as I understand what you are trying to do you don't need ClassDef. Symbols you obtained already has all information about annotations. Check hasAnnotation and getAnnotation methods defined in Symbols.scala (lines 1115-1118).

Dynamic mixin in Scala - is it possible?

What I'd like to achieve is having a proper implementation for
def dynamix[A, B](a: A): A with B
I may know what B is, but don't know what A is (but if B has a self type then I could add some constraints on A).
The scala compiler is happy with the above signature, but I could not yet figure out how the implementation would look like - if it is possible at all.
Some options that came to my mind:
Using reflection/dynamic proxy.
Simplest case: A is an interface on Java level + I can instantiate B and it has no self type. I guess it would not be too hard (unless I run into some nasty, unexpected problems):
create a new B (b), and also a proxy implementing both A and B and using an invocation handler delegating to either a or b.
If B can not be instantiated I could still create a subclass of it, and do as it was described above. If it also has a self type I would probably need some delegation here and there, but it may still work.
But what if A is a concrete type and I can't find a proper interface for it?
Would I run into more problems (e.g. something related to linearization, or special constructs helping Java interoperability)?
Using a kind of wrapping instead of a mixin and return B[A], a is accessible from b.
Unfortunately in this case the caller would need to know how the nesting is done, which could be quite inconvenient if the mixing in/wrapping is done several times (D[C[B[A]]]) as it would need to find the right level of nesting to access the needed functionality, so I don't consider it a solution.
Implementing a compiler plugin. I have zero experience with it but my gut feeling is that it would not be trivial. I think Kevin Wright's autoproxy plugin has a bit similar goal, but it would not be enough for my problem (yet?).
Do you have any other ideas that might work? Which way would you recommend? What kind of "challenges" to expect?
Or should I forget it, because it is not possible with the current Scala constraints?
Intention behind my problem:
Say I have a business workflow, but it's not too strict. Some steps have fixed order, but others do not, but at the end all of them has to be done (or some of them required for further processing).
A bit more concrete example: I have an A, I can add B and C to it. I don't care which is done first, but at the end I'll need an A with B with C.
Comment: I don't know too much about Groovy but SO popped up this question and I guess it's more or less the same as what I'd like, at least conceptional.
I believe this is impossible to do strictly at runtime, because traits are mixed in at compile-time into new Java classes. If you mix a trait with an existing class anonymously you can see, looking at the classfiles and using javap, that an anonymous, name-mangled class is created by scalac:
class Foo {
def bar = 5
}
trait Spam {
def eggs = 10
}
object Main {
def main(args: Array[String]) = {
println((new Foo with Spam).eggs)
}
}
scalac Mixin.scala; ls *.class returns
Foo.class Main$.class Spam$class.class
Main$$anon$1.class Main.class Spam.class
While javap Main\$\$anon\$1 returns
Compiled from "mixin.scala"
public final class Main$$anon$1 extends Foo implements Spam{
public int eggs();
public Main$$anon$1();
}
As you can see, scalac creates a new anonymous class that is loaded at runtime; presumably the method eggs in this anonymous class creates an instance of Spam$class and calls eggs on it, but I'm not completely sure.
However, we can do a pretty hacky trick here:
import scala.tools.nsc._;
import scala.reflect.Manifest
object DynamicClassLoader {
private var id = 0
def uniqueId = synchronized { id += 1; "Klass" + id.toString }
}
class DynamicClassLoader extends
java.lang.ClassLoader(getClass.getClassLoader) {
def buildClass[T, V](implicit t: Manifest[T], v: Manifest[V]) = {
// Create a unique ID
val id = DynamicClassLoader.uniqueId
// what's the Scala code we need to generate this class?
val classDef = "class %s extends %s with %s".
format(id, t.toString, v.toString)
println(classDef)
// fire up a new Scala interpreter/compiler
val settings = new Settings(null)
val interpreter = new Interpreter(settings)
// define this class
interpreter.compileAndSaveRun("<anon>", classDef)
// get the bytecode for this new class
val bytes = interpreter.classLoader.getBytesForClass(id)
// define the bytecode using this classloader; cast it to what we expect
defineClass(id, bytes, 0, bytes.length).asInstanceOf[Class[T with V]]
}
}
val loader = new DynamicClassLoader
val instance = loader.buildClass[Foo, Spam].newInstance
instance.bar
// Int = 5
instance.eggs
// Int = 10
Since you need to use the Scala compiler, AFAIK, this is probably close to the cleanest solution you could do to get this. It's quite slow, but memoization would probably help greatly.
This approach is pretty ridiculous, hacky, and goes against the grain of the language. I imagine all sorts of weirdo bugs could creep in; people who have used Java longer than me warn of the insanity that comes with messing around with classloaders.
I wanted to be able to construct Scala beans in my Spring application context, but I also wanted to be able to specify the mixins to be included in the constructed bean:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:scala="http://www.springframework.org/schema/scala"
xsi:schemaLocation=...>
<scala:bean class="org.cakesolutions.scala.services.UserService" >
<scala:with trait="org.cakesolutions.scala.services.Mixin1" />
<scala:with trait="org.cakesolutions.scala.services.Mixin2" />
<scala:property name="dependency" value="Injected" />
<scala:bean>
</beans>
The difficulty is that Class.forName function does not allow me to specify the mixins. In the end, I extended the above hacky solution to Scala 2.9.1. So, here it is in its full gory; including bits of Spring.
class ScalaBeanFactory(private val beanType: Class[_ <: AnyRef],
private val mixinTypes: Seq[Class[_ <: AnyRef]]) {
val loader = new DynamicClassLoader
val clazz = loader.buildClass(beanType, mixinTypes)
def getTypedObject[T] = getObject.asInstanceOf[T]
def getObject = {
clazz.newInstance()
}
def getObjectType = null
def isSingleton = true
object DynamicClassLoader {
private var id = 0
def uniqueId = synchronized { id += 1; "Klass" + id.toString }
}
class DynamicClassLoader extends java.lang.ClassLoader(getClass.getClassLoader) {
def buildClass(t: Class[_ <: AnyRef], vs: Seq[Class[_ <: AnyRef]]) = {
val id = DynamicClassLoader.uniqueId
val classDef = new StringBuilder
classDef.append("class ").append(id)
classDef.append(" extends ").append(t.getCanonicalName)
vs.foreach(c => classDef.append(" with %s".format(c.getCanonicalName)))
val settings = new Settings(null)
settings.usejavacp.value = true
val interpreter = new IMain(settings)
interpreter.compileString(classDef.toString())
val r = interpreter.classLoader.getResourceAsStream(id)
val o = new ByteArrayOutputStream
val b = new Array[Byte](16384)
Stream.continually(r.read(b)).takeWhile(_ > 0).foreach(o.write(b, 0, _))
val bytes = o.toByteArray
defineClass(id, bytes, 0, bytes.length)
}
}
The code cannot yet deal with constructors with parameters and does not copy annotations from the parent class’s constructor (should it do that?). However, it gives us a good starting point that is usable in the scala Spring namespace. Of course, don’t just take my word for it, verify it in a Specs2 specification:
class ScalaBeanFactorySpec extends Specification {
"getTypedObject mixes-in the specified traits" in {
val f1 = new ScalaBeanFactory(classOf[Cat],
Seq(classOf[Speaking], classOf[Eating]))
val c1 = f1.getTypedObject[Cat with Eating with Speaking]
c1.isInstanceOf[Cat with Eating with Speaking] must_==(true)
c1.speak // in trait Speaking
c1.eat // in trait Eating
c1.meow // in class Cat
}
}