scala proxy macro, issue converting method args to values - scala

I am trying to write a proxy macro using scala macros. I want to be able to proxy a trait X and return instances of X that invoke a function for all methods of X.
Here is what I did so far. Say we want to proxy the trait TheTrait (which is defined below), we can run ProxyMacro.proxy passing a function that will be called for all invocations of the proxy methods.
trait TheTrait
{
def myMethod(x: String)(y: Int): String
}
val proxy = ProxyMacro.proxy[TheTrait] {
case ("myMethod", args) =>
"ok"
}
println(proxy.myMethod("hello")(5))
The implementation so far is this:
package macrotests
import scala.language.experimental.macros
import scala.reflect.macros.whitebox.Context
object ProxyMacro
{
type Implementor = (String, Any) => Any
def proxy[T](implementor: Implementor): T = macro impl[T]
def impl[T: c.WeakTypeTag](c: Context)(implementor: c.Expr[Implementor]): c.Expr[T] = {
import c.universe._
val tpe = weakTypeOf[T]
val decls = tpe.decls.map { decl =>
val termName = decl.name.toTermName
val method = decl.asMethod
val params = method.paramLists.map(_.map(s => internal.valDef(s)))
val paramVars = method.paramLists.flatMap(_.map { s =>
internal.captureVariable(s)
internal.referenceCapturedVariable(s)
})
q""" def $termName (...$params) = {
$implementor (${termName.toString}, List(..${paramVars}) ).asInstanceOf[${method.returnType}]
}"""
}
c.Expr[T] {
q"""
new $tpe {
..$decls
}
"""
}
}
}
But there is a problem. This doesn't compile due to List(..${paramVars}). This should just create a list with all the values of the method arguments.
But I get a compilation issue (not worth pasting it) on that line.
How can I convert the list of method arguments to their values?

showInfo is useful when you debug macro
def showInfo(s: String) =
c.info(c.enclosingPosition, s.split("\n").mkString("\n |---macro info---\n |", "\n |", ""), true)
change
val paramVars = method.paramLists.flatMap(_.map { s =>
internal.captureVariable(s)
internal.referenceCapturedVariable(s)
})(this result is List(x0$1, x1$1))
to
val paramVars = method.paramLists.flatMap(_.map { s =>
s.name
})(this result is List(x, y))

Related

How to get the runtime value of parameter passed to a Scala macro?

I have an ostensibly simple macro problem that I’ve been banging my head against for a few hours, with no luck. Perhaps someone with more experience can help.
I have the following macro:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object MacroObject {
def run(s: String): Unit =
macro runImpl
def runImpl(c: Context)(s: c.Tree): c.Tree = {
import c.universe._
println(s) // <-- I need the macro to know the value of s at compile time
q"()"
}
}
The problem is this: I’d like the macro to know the value s that is passed to it — not an AST of s, but the value of s itself. Specifically, I’d like it to have this behavior:
def runTheMacro(str: String): Unit = MacroObject.run(str)
final val HardCodedString1 = "Hello, world!"
runTheMacro(HardCodedString1) // the macro should print "Hello, world!"
// to the console during macro expansion
final val HardCodedString2 = "So long!"
runTheMacro(HardCodedString2) // the macro should print "So long!"
// to the console during macro expansion
It is guaranteed that the only strings that will be passed to runTheMacro are hard-coded constant values (i.e., known at compile-time).
Is this possible, and how does one do this?
--
Edit: There are also the following constraints:
It must be a blackbox macro.
The macro signature must use c.Trees, not c.Expr[_]s (legacy code; can’t change that part)
I do have a toolbox within the macro at my disposal if needed:
import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox
private val toolbox = currentMirror.mkToolBox()
/** Evaluate the given code fragment at compile time. */
private def eval[A](code: String): A = {
import scala.reflect.runtime.{universe => u}
val uTree: u.Tree = toolbox.parse(code)
toolbox.eval(uTree).asInstanceOf[A]
}
Your eval is runtime reflection's eval, compile-time macro's eval would be c.eval.
"Hello, world!" in
final val HardCodedString1 = "Hello, world!"
runTheMacro(HardCodedString1)
is a runtime value of HardCodedString1.
You can't have access to runtime value at compile time.
At compile time the tree of string HardCodedString1 just doesn't know anything about right hand side of the val tree.
Scala: what can code in Context.eval reference?
If you really need to use a runtime value inside the tree of your program you have to postpone its compilation till runtime
import scala.reflect.runtime.currentMirror
import scala.reflect.runtime.universe._
import scala.tools.reflect.ToolBox
object MacroObject {
val toolbox = currentMirror.mkToolBox()
def run(s: String): Unit = {
toolbox.eval(q"""
println($s)
()
""")
}
}
runTheMacro(HardCodedString1)//Hello, world!
runTheMacro(HardCodedString2)//So long!
Alternatively at compile time you can somehow find the tree of enclosing class and look inside it for the val tree and take its right hand side
def runImpl(c: blackbox.Context)(s: c.Tree): c.Tree = {
import c.universe._
var rhs: Tree = null
val traverser = new Traverser {
override def traverse(tree: Tree): Unit = {
tree match {
case q"$mods val $tname: $tpt = $expr" if tname == TermName("HardCodedString1") =>
rhs = expr
case _ => ()
}
super.traverse(tree)
}
}
traverser.traverse(c.enclosingClass) // deprecated
val rhsStr =
if (rhs != null) c.eval[String](c.Expr(c.untypecheck(rhs.duplicate)))
else c.abort(c.enclosingPosition, "no val HardCodedString1 defined")
println(rhsStr)
q"()"
}
runTheMacro(HardCodedString1)//Warning:scalac: Hello, world!
Or for all such variables
def runImpl(c: blackbox.Context)(s: c.Tree): c.Tree = {
import c.universe._
val sEvaluated =
try {
c.eval[String](c.Expr(c.untypecheck(s.duplicate)))
} catch {
case e: IllegalArgumentException if e.getMessage.startsWith("Could not find proxy") =>
s match {
case q"$sName" =>
var rhs: Tree = null
val traverser = new Traverser {
override def traverse(tree: Tree): Unit = {
tree match {
case q"$mods val $tname: $tpt = $expr" if tname == sName =>
rhs = expr
case _ => ()
}
super.traverse(tree)
}
}
traverser.traverse(c.enclosingClass)
if (rhs != null) c.eval[String](c.Expr(c.untypecheck(rhs.duplicate)))
else c.abort(c.enclosingPosition, s"no val $sName defined")
case _ => c.abort(c.enclosingPosition, s"unsupported tree $s")
}
}
println(sEvaluated)
q"()"
}
MacroObject.run(HardCodedString1) //Warning:scalac: Hello, world!
MacroObject.run(HardCodedString2) //Warning:scalac: So long!
runTheMacro will not work in this case: Error: no val str defined.
To make it work you can make it a macro too
def runTheMacro(str: String): Unit = macro runTheMacroImpl
def runTheMacroImpl(c: blackbox.Context)(str: c.Tree): c.Tree = {
import c.universe._
q"MacroObject.run($str)"
}
runTheMacro(HardCodedString1) //Warning:scalac: Hello, world!
runTheMacro(HardCodedString2) //Warning:scalac: So long!

Quasiquotes and Generics in Scala

I'm using Play Framework and trying to write an action that can parse protobuf request as following:
def AsyncProtoAction(block: ProtoRequestA => Future[Result]): Action[AnyContent] = {
Action.async { request =>
request.contentType match {
case Some("application/protobuf") => request.body.asRaw match {
case Some(raw) => raw.asBytes(1024 * 1024) match {
case Some(rawBytes) =>
val proto = ProtoRequestA.parseFrom(rawBytes)
block(proto)
case _ => ...
}
}
}
}
}
Here, ProtoRequestA is a generated object using ScalaPB. It works, however, I have a lot of protobuf request objects. Then I try to rewrite it using macro:
object Actions {
def AsyncProtoAction[T](block: T => Future[Result]):
Action[AnyContent] = macro asyncProtoAction_impl[T]
def asyncProtoAction_impl[T: c.WeakTypeTag](c: blackbox.Context)
(block: c.Tree) = { import c.universe._
val protoType = weakTypeOf[T]
q"""import play.api.mvc._
Action.async { request =>
request.contentType match {
case Some("application/protobuf") =>
request.body.asRaw match {
case Some(raw) =>
raw.asBytes(1024 * 1024) match {
case Some(rawBytes) =>
val proto = $protoType.parseFrom(rawBytes)
$block(proto)
case _ => ...
}
}
}
}"""
}
}
This doesn't work. The compilation error is value parseFrom is not a member of packageName.ProtoRequestA.
I tried showRaw(tq"$protoType") in REPL, which output String = TypeTree(). I guess the correct output should be String = Select(Ident(TermName("packageName")), TypeName("ProtoRequestA")). So what should I do?
You've identified the problem, which is that when you interpolate the ProtoRequestA type you get something different from the ProtoRequestA companion object. One easy solution is to interpolate the symbol of the companion object, which you can get from the type's symbol. Here's a simplified example:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
def companionObjectBarImpl[A: c.WeakTypeTag](c: Context): c.Tree = {
import c.universe._
q"${ symbolOf[A].companion }.bar"
}
def companionObjectBar[A]: Int = macro companionObjectBarImpl[A]
This macro will work on any type with a companion object with a bar method that returns an Int. For example:
scala> class Foo; object Foo { def bar = 10 }
defined class Foo
defined object Foo
scala> companionObjectBar[Foo]
res0: Int = 10
You should be able to do something similar.

type parameter mismatch with WeakTypeTag reflection + quasiquoting (I think!)

Inspired by travisbrown, I'm trying to use a macro to create some "smart constructors".
Given
package mypkg
sealed trait Hello[A]
case class Ohayo[A,B](a: (A,B)) extends Hello[A]
and
val smartConstructors = FreeMacros.liftConstructors[Hello]
The macro should find all the subclasses of Hello, look at their constructors, and extract a few elements to populate this tree for the "smart constructor":
q"""
def $methodName[..$typeParams](...$paramLists): $baseType =
$companionSymbol[..$typeArgs](...$argLists)
"""
I hoped to get:
val smartConstructors = new {
def ohayo[A, B](a: (A, B)): Hello[A] = Ohayo[A, B](a)
}
but instead get:
error: type mismatch;
found : (A(in class Ohayo), B(in class Ohayo))
required: ((some other)A(in class Ohayo), (some other)B(in class Ohayo))
val liftedConstructors = FreeMacros.liftConstructors[Hello]
At a glance, the tree looks ok to me:
scala> q" new { ..$wellTyped }"
res1: u.Tree =
{
final class $anon extends scala.AnyRef {
def <init>() = {
super.<init>();
()
};
def ohayo[A, B](a: (A, B)): net.arya.constructors.Hello[A] = Ohayo[A, B](a)
};
new $anon()
}
but I guess it invisibly isn't. If I naively try to freshen up the typeParams with info.typeParams.map(p => TypeName(p.name.toString)), I get "can't splice A as type parameter" when I do the quasiquoting.
Where am I going wrong? Thanks for taking a look.
-Arya
import scala.language.experimental.macros
import scala.reflect.api.Universe
import scala.reflect.macros.whitebox
class FreeMacros(val c: whitebox.Context) {
import c.universe._
import FreeMacros._
def liftedImpl[F[_]](implicit t: c.WeakTypeTag[F[_]]): Tree = {
val atc = t.tpe
val childSymbols: Set[ClassSymbol] = subCaseClassSymbols(c.universe)(atc.typeSymbol.asClass)
val wellTyped = childSymbols.map(ctorsForSymbol(c.universe)(atc)).unzip
q"new { ..${wellTyped} }"
}
}
object FreeMacros {
def liftConstructors[F[_]]: Any = macro FreeMacros.liftedImpl[F]
def smartName(name: String): String = (
name.toList match {
case h :: t => h.toLower :: t
case Nil => Nil
}
).mkString
def subCaseClassSymbols(u: Universe)(root: u.ClassSymbol): Set[u.ClassSymbol] = {
val subclasses = root.knownDirectSubclasses
val cast = subclasses.map(_.asInstanceOf[u.ClassSymbol])
val partitioned = mapped.partition(_.isCaseClass)
partitioned match {
case (caseClasses, regularClasses) => caseClasses ++ regularClasses.flatMap(r => subCaseClassSymbols(u)(r))
}
}
def ctorsForSymbol(u: Universe)(atc: u.Type)(caseClass: u.ClassSymbol): (u.DefDef, u.DefDef) = {
import u._
import internal._
// these didn't help
// def clearTypeSymbol(s: Symbol): TypeSymbol = internal.newTypeSymbol(NoSymbol, s.name.toTypeName, s.pos, if(s.isImplicit)Flag.IMPLICIT else NoFlags)
// def clearTypeSymbol2(s: Symbol): TypeSymbol = internal.newTypeSymbol(NoSymbol, s.name.toTypeName, NoPosition, if(s.isImplicit)Flag.IMPLICIT else NoFlags)
// def clearTypeDef(d: TypeDef): TypeDef = internal.typeDef(clearTypeSymbol(d.symbol))
val companionSymbol: Symbol = caseClass.companion
val info: Type = caseClass.info
val primaryCtor: Symbol = caseClass.primaryConstructor
val method = primaryCtor.asMethod
val typeParams = info.typeParams.map(internal.typeDef(_))
// val typeParams = info.typeParams.map(s => typeDef(newTypeSymbol(NoSymbol, s.name.toTypeName, NoPosition, NoFlags)))
// val typeParams = info.typeParams.map(s => internal.typeDef(clearTypeSymbol2(s)))
val typeArgs = info.typeParams.map(_.name)
val paramLists = method.paramLists.map(_.map(internal.valDef(_)))
val argLists = method.paramLists.map(_.map(_.asTerm.name))
val baseType = info.baseType(atc.typeSymbol)
val List(returnType) = baseType.typeArgs
val methodName = TermName(smartName(caseClass.name.toString))
val wellTyped =
q"""
def $methodName[..$typeParams](...$paramLists): $baseType =
$companionSymbol[..$typeArgs](...$argLists)
"""
wellTyped
}
}
P.S. I have been experimenting with toolbox.untypecheck / typecheck per this article but haven't found a working combination.
you need using
clas.typeArgs.map(_.toString).map(name => {
TypeDef(Modifiers(Flag.PARAM),TypeName(name), List(),TypeBoundsTree(EmptyTree, EmptyTree))
}
replace
info.typeParams.map(p => TypeName(p.name.toString))
it si my code
object GetSealedSubClass {
def ol3[T]: Any = macro GetSealedSubClassImpl.ol3[T]
}
class GetSealedSubClassImpl(val c: Context) {
import c.universe._
def showInfo(s: String) =
c.info(c.enclosingPosition, s.split("\n").mkString("\n |---macro info---\n |", "\n |", ""), true)
def ol3[T: c.WeakTypeTag]: c.universe.Tree = {
//get all sub class
val subClass = c.weakTypeOf[T]
.typeSymbol.asClass.knownDirectSubclasses
.map(e => e.asClass.toType)
//check type params must ia s sealed class
if (subClass.size < 1)
c.abort(c.enclosingPosition, s"${c.weakTypeOf[T]} is not a sealed class")
// get sub class constructor params
val subConstructorParams = subClass.map { e =>
//get constructor
e.members.filter(_.isConstructor)
//if the class has many Constructor then you need filter the main Constructor
.head.map(s => s.asMethod)
//get function param list
}.map(_.asMethod.paramLists.head)
.map(_.map(e => q"""${e.name.toTermName}:${e.info} """))
val outfunc = subClass zip subConstructorParams map {
case (clas, parm) =>
q"def smartConstructors[..${
clas.typeArgs.map(_.toString).map(name => {
TypeDef(Modifiers(Flag.PARAM), TypeName(name), List(), TypeBoundsTree(EmptyTree, EmptyTree))
})
}](..${parm})=${clas.typeSymbol.name.toTermName} (..${parm})"
}
val outClass =
q"""
object Term{
..${outfunc}
}
"""
showInfo(show(outClass))
q"""{
$outClass
Term
}
"""
}
}
using like this
sealed trait Hello[A]
case class Ohayo[A, B](a: (A, B)) extends Hello[A]
object GetSealed extends App {
val a = GetSealedSubClass.ol3[Hello[_]]
val b=a.asInstanceOf[ {def smartConstructors[A, B](a: (A, B)): Ohayo[A, B]}].smartConstructors(1, 2).a
println(b)
}

Instantiate class symbol using macro

I'm using the following Scala macro (heavily inspired by the code from this SO question) to get a list of all objects contained in a given package that inherit a specific trait:
object Macros {
def allObjects[T <: AnyRef](packageName: String): List[Any] = macro allObjectsImpl[T]
def allObjectsImpl[T <: AnyRef: c.WeakTypeTag](c: Context)(packageName: c.Expr[String]) = {
import c.universe._
val baseTraitSymbol = c.weakTypeOf[T].typeSymbol
val pkg = packageName.tree match {
case Literal(Constant(name: String)) => c.mirror.staticPackage(name)
}
val types = pkg.typeSignature.members.collect {
case moduleSymbol: ModuleSymbol if moduleSymbol.moduleClass.asClass.baseClasses contains baseTraitSymbol => Ident(moduleSymbol)
}.toList
val listApply = Select(reify(List).tree, TermName("apply"))
c.Expr[List[T]](Apply(listApply, types))
}
}
Which works fine.
I want to change the macro so that instead of getting all the objects in a package, it gets all the concrete classes, and provides a list containing an instance of each of them.
The AST when creating an instance of an Object looks like this:
scala> import scala.reflect.runtime.{universe => u}
import scala.reflect.runtime.{universe=>u}
scala> u showRaw ( u reify {new Object} )
res42: String = Expr(Apply(Select(New(Ident(java.lang.Object)), termNames.CONSTRUCTOR), List()))
So I thought changing my code to this would work:
object Macros {
def allInstances[T <: AnyRef](packageName: String): List[Any] = macro allInstancesImpl[T]
def allInstancesImpl[T <: AnyRef: c.WeakTypeTag](c: Context)(packageName: c.Expr[String]) = {
import c.universe._
val baseTraitSymbol = c.weakTypeOf[T].typeSymbol
val pkg = packageName.tree match {
case Literal(Constant(name: String)) => c.mirror.staticPackage(name)
}
def isConcreteChildClass(child: ClassSymbol, base: Symbol) = {
!child.isAbstract && (child.baseClasses contains base)
}
val types = pkg.typeSignature.members.collect {
case classSymbol: ClassSymbol if isConcreteChildClass(classSymbol, baseTraitSymbol) => {
Apply(Select(New(Ident(classSymbol.primaryConstructor)), termNames.CONSTRUCTOR), List())
}
}.toList
val listApply = Select(reify(List).tree, TermName("apply"))
c.Expr[List[T]](Apply(listApply, types))
}
}
However, when I try to use the updated macro code on a test package, I get the following error:
scala> Macros.allInstances[AnyRef]("test")
<console>:9: error: class type required but ()test.TestClass found
Macros.allInstances[AnyRef]("test")
From what I'm seeing, it looks like the macro is actually returning the constructor itself instead of returning the instance that should get built by the constructor, but I can't figure out what I'm missing.
The problem is in this line (reformatted for clarity):
Apply(
Select(New(Ident(classSymbol.primaryConstructor)), termNames.CONSTRUCTOR),
List()
)
You're essentially selecting the constructor twice. You could just drop primaryConstructor:
Apply(
Select(New(Ident(classSymbol)), termNames.CONSTRUCTOR),
List()
)
Using ApplyConstructor would also work:
ApplyConstructor(Ident(classSymbol), Nil)
Or you could just go with quasiquotes:
q"new ${Ident(classSymbol)}()"
The quasiquote solution is the most future-proof.

Scala macro and type erasure

I'm having some problems with a macro I've written to help me log metrics represented as case class instances to to InfluxDB. I presume I'm having a type erasure problem and that the tyep parameter T is getting lost, but I'm not entirely sure what's going on. (This is also my first exposure to Scala macros.)
import scala.language.experimental.macros
import play.api.libs.json.{JsNumber, JsString, JsObject, JsArray}
abstract class Metric[T] {
def series: String
def jsFields: JsArray = macro MetricsMacros.jsFields[T]
def jsValues: JsArray = macro MetricsMacros.jsValues[T]
}
object Metrics {
case class LoggedMetric(timestamp: Long, series: String, fields: JsArray, values: JsArray)
case object Kick
def log[T](metric: Metric[T]): Unit = {
println(LoggedMetric(
System.currentTimeMillis,
metric.series,
metric.jsFields,
metric.jsValues
))
}
}
And here's an example metric case class:
case class SessionCountMetric(a: Int, b: String) extends Metric[SessionCountMetric] {
val series = "sessioncount"
}
Here's what happens when I try to log it:
scala> val m = SessionCountMetric(1, "a")
m: com.confabulous.deva.SessionCountMetric = SessionCountMetric(1,a)
scala> Metrics.log(m)
LoggedMetric(1411450638296,sessioncount,[],[])
Even though the macro itself seems to work fine:
scala> m.jsFields
res1: play.api.libs.json.JsArray = ["a","b"]
scala> m.jsValues
res2: play.api.libs.json.JsArray = [1,"a"]
Here's the actual macro itself:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object MetricsMacros {
private def fieldNames[T: c.WeakTypeTag](c: Context)= {
val tpe = c.weakTypeOf[T]
tpe.decls.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asTerm.name
}
}
def jsFields[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val names = fieldNames[T](c)
Apply(
q"play.api.libs.json.Json.arr",
names.map(name => Literal(Constant(name.toString))).toList
)
}
def jsValues[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val names = fieldNames[T](c)
Apply(
q"play.api.libs.json.Json.arr",
names.map(name => q"${c.prefix.tree}.$name").toList
)
}
}
Update
I tried Eugene's second suggestion like this:
abstract class Metric[T] {
def series: String
}
trait MetricSerializer[T] {
def fields: Seq[String]
def values(metric: T): Seq[Any]
}
object MetricSerializer {
implicit def materializeSerializer[T]: MetricSerializer[T] = macro MetricsMacros.materializeSerializer[T]
}
object Metrics {
def log[T: MetricSerializer](metric: T): Unit = {
val serializer = implicitly[MetricSerializer[T]]
println(serializer.fields)
println(serializer.values(metric))
}
}
with the macro now looking like this:
object MetricsMacros {
def materializeSerializer[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val tpe = c.weakTypeOf[T]
val names = tpe.decls.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asTerm.name
}
val fields = Apply(
q"Seq",
names.map(name => Literal(Constant(name.toString))).toList
)
val values = Apply(
q"Seq",
names.map(name => q"metric.$name").toList
)
q"""
new MetricSerializer[$tpe] {
def fields = $fields
def values(metric: Metric[$tpe]) = $values
}
"""
}
}
However, when I call Metrics.log -- specifically when it calls implicitly[MetricSerializer[T]] I get the following error:
error: value a is not a member of com.confabulous.deva.Metric[com.confabulous.deva.SessionCountMetric]
Why is it trying to use Metric[com.confabulous.deva.SessionCountMetric] instead of SessionCountMetric?
Conclusion
Fixed it.
def values(metric: Metric[$tpe]) = $values
should have been
def values(metric: $tpe) = $values
You're in a situation that's very close to one described in a recent question: scala macros: defer type inference.
As things stand right now, you'll have to turn log into a macro. An alternative would also to turn Metric.jsFields and Metric.jsValues into JsFieldable and JsValuable type classes materialized by implicit macros at callsites of log (http://docs.scala-lang.org/overviews/macros/implicits.html).