I'm using Play Framework and trying to write an action that can parse protobuf request as following:
def AsyncProtoAction(block: ProtoRequestA => Future[Result]): Action[AnyContent] = {
Action.async { request =>
request.contentType match {
case Some("application/protobuf") => request.body.asRaw match {
case Some(raw) => raw.asBytes(1024 * 1024) match {
case Some(rawBytes) =>
val proto = ProtoRequestA.parseFrom(rawBytes)
block(proto)
case _ => ...
}
}
}
}
}
Here, ProtoRequestA is a generated object using ScalaPB. It works, however, I have a lot of protobuf request objects. Then I try to rewrite it using macro:
object Actions {
def AsyncProtoAction[T](block: T => Future[Result]):
Action[AnyContent] = macro asyncProtoAction_impl[T]
def asyncProtoAction_impl[T: c.WeakTypeTag](c: blackbox.Context)
(block: c.Tree) = { import c.universe._
val protoType = weakTypeOf[T]
q"""import play.api.mvc._
Action.async { request =>
request.contentType match {
case Some("application/protobuf") =>
request.body.asRaw match {
case Some(raw) =>
raw.asBytes(1024 * 1024) match {
case Some(rawBytes) =>
val proto = $protoType.parseFrom(rawBytes)
$block(proto)
case _ => ...
}
}
}
}"""
}
}
This doesn't work. The compilation error is value parseFrom is not a member of packageName.ProtoRequestA.
I tried showRaw(tq"$protoType") in REPL, which output String = TypeTree(). I guess the correct output should be String = Select(Ident(TermName("packageName")), TypeName("ProtoRequestA")). So what should I do?
You've identified the problem, which is that when you interpolate the ProtoRequestA type you get something different from the ProtoRequestA companion object. One easy solution is to interpolate the symbol of the companion object, which you can get from the type's symbol. Here's a simplified example:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
def companionObjectBarImpl[A: c.WeakTypeTag](c: Context): c.Tree = {
import c.universe._
q"${ symbolOf[A].companion }.bar"
}
def companionObjectBar[A]: Int = macro companionObjectBarImpl[A]
This macro will work on any type with a companion object with a bar method that returns an Int. For example:
scala> class Foo; object Foo { def bar = 10 }
defined class Foo
defined object Foo
scala> companionObjectBar[Foo]
res0: Int = 10
You should be able to do something similar.
Related
My task is to print out type information in Java-like notation (using <, > for type arguments notation). In scala 2 I have this small method using scala.reflect.Manifest as a source for type symbol and it's parameters:
def typeOf[T](implicit manifest: Manifest[T]): String = {
def loop[T0](m: Manifest[T0]): String =
if (m.typeArguments.isEmpty) m.runtimeClass.getSimpleName
else {
val typeArguments = m.typeArguments.map(loop(_)).mkString(",")
raw"""${m.runtimeClass.getSimpleName}<$typeArguments>"""
}
loop(manifest)
}
Unfortunately in Scala 3 Manifests are not available. Is there a Scala 3 native way to rewrite this? I'm open to some inline macro stuff. What I have tried so far is
inline def typeOf[T]: String = ${typeOfImpl}
private def typeOfImpl[T: Type](using Quotes): Expr[String] =
import quotes.reflect.*
val tree = TypeTree.of[T]
tree.show
// ^^ call is parameterized with Printer but AFAIK there's no way
// to provide your own implementation for it. You can to chose
// from predefined ones. So how do I proceed from here?
I know that Scala types can't be all represented as Java types. I aim to cover only simple ones that the original method was able to cover. No wildcards or existentials, only fully resolved types like:
List[String] res: List<String>
List[Option[String]] res: List<Option<String>>
Map[String,Option[Int]] res: Map<String,Option<Int>>
I post this answer even though it's not a definitive solution and there's probably a better way but hopefully it can give you some ideas.
I think a good start is using TypeRepr:
val tpr: TypeRepr = TypeRepr.of[T]
val typeParams: List[TypeRepr] = tpr match {
case a: AppliedType => a.args
case _ => Nil
}
Then with a recursive method you should be able to work something out.
Copied from Inspired from https://github.com/gaeljw/typetrees/blob/main/src/main/scala/io/github/gaeljw/typetrees/TypeTreeTagMacros.scala#L12:
private def getTypeString[T](using Type[T], Quotes): Expr[String] = {
import quotes.reflect._
def getTypeStringRec(tpr: TypeRepr)(using Quotes): Expr[String] = {
tpr.asType match {
case '[t] => getTypeString[t]
}
}
val tpr: TypeRepr = TypeRepr.of[T]
val typeParams: List[TypeRepr] = tpr match {
case a: AppliedType => a.args
case _ => Nil
}
val selfTag: Expr[ClassTag[T]] = getClassTag[T]
val argsStrings: Expr[List[String]] =
Expr.ofList(typeParams.map(getTypeStringRec))
'{ /* Compute something using selfTag and argsStrings */ }
}
private def getClassTag[T](using Type[T], Quotes): Expr[ClassTag[T]] = {
import quotes.reflect._
Expr.summon[ClassTag[T]] match {
case Some(ct) =>
ct
case None =>
report.error(
s"Unable to find a ClassTag for type ${Type.show[T]}",
Position.ofMacroExpansion
)
throw new Exception("Error when applying macro")
}
}
The final working solution I came up with was:
def typeOfImpl[T: Type](using Quotes): Expr[String] = {
import quotes.reflect.*
TypeRepr.of[T] match {
case AppliedType(tpr, args) =>
val typeName = Expr(tpr.show)
val typeArguments = Expr.ofList(args.map {
_.asType match {
case '[t] => typeOfImpl[t]
}
})
'{
val tpeName = ${ typeName }
val typeArgs = ${ typeArguments }
typeArgs.mkString(tpeName + "<", ", ", ">")
}
case tpr: TypeRef => Expr(tpr.show)
case other =>
report.errorAndAbort(s"unsupported type: ${other.show}", Position.ofMacroExpansion)
}
}
I am trying to write a proxy macro using scala macros. I want to be able to proxy a trait X and return instances of X that invoke a function for all methods of X.
Here is what I did so far. Say we want to proxy the trait TheTrait (which is defined below), we can run ProxyMacro.proxy passing a function that will be called for all invocations of the proxy methods.
trait TheTrait
{
def myMethod(x: String)(y: Int): String
}
val proxy = ProxyMacro.proxy[TheTrait] {
case ("myMethod", args) =>
"ok"
}
println(proxy.myMethod("hello")(5))
The implementation so far is this:
package macrotests
import scala.language.experimental.macros
import scala.reflect.macros.whitebox.Context
object ProxyMacro
{
type Implementor = (String, Any) => Any
def proxy[T](implementor: Implementor): T = macro impl[T]
def impl[T: c.WeakTypeTag](c: Context)(implementor: c.Expr[Implementor]): c.Expr[T] = {
import c.universe._
val tpe = weakTypeOf[T]
val decls = tpe.decls.map { decl =>
val termName = decl.name.toTermName
val method = decl.asMethod
val params = method.paramLists.map(_.map(s => internal.valDef(s)))
val paramVars = method.paramLists.flatMap(_.map { s =>
internal.captureVariable(s)
internal.referenceCapturedVariable(s)
})
q""" def $termName (...$params) = {
$implementor (${termName.toString}, List(..${paramVars}) ).asInstanceOf[${method.returnType}]
}"""
}
c.Expr[T] {
q"""
new $tpe {
..$decls
}
"""
}
}
}
But there is a problem. This doesn't compile due to List(..${paramVars}). This should just create a list with all the values of the method arguments.
But I get a compilation issue (not worth pasting it) on that line.
How can I convert the list of method arguments to their values?
showInfo is useful when you debug macro
def showInfo(s: String) =
c.info(c.enclosingPosition, s.split("\n").mkString("\n |---macro info---\n |", "\n |", ""), true)
change
val paramVars = method.paramLists.flatMap(_.map { s =>
internal.captureVariable(s)
internal.referenceCapturedVariable(s)
})(this result is List(x0$1, x1$1))
to
val paramVars = method.paramLists.flatMap(_.map { s =>
s.name
})(this result is List(x, y))
I'm having some problems with a macro I've written to help me log metrics represented as case class instances to to InfluxDB. I presume I'm having a type erasure problem and that the tyep parameter T is getting lost, but I'm not entirely sure what's going on. (This is also my first exposure to Scala macros.)
import scala.language.experimental.macros
import play.api.libs.json.{JsNumber, JsString, JsObject, JsArray}
abstract class Metric[T] {
def series: String
def jsFields: JsArray = macro MetricsMacros.jsFields[T]
def jsValues: JsArray = macro MetricsMacros.jsValues[T]
}
object Metrics {
case class LoggedMetric(timestamp: Long, series: String, fields: JsArray, values: JsArray)
case object Kick
def log[T](metric: Metric[T]): Unit = {
println(LoggedMetric(
System.currentTimeMillis,
metric.series,
metric.jsFields,
metric.jsValues
))
}
}
And here's an example metric case class:
case class SessionCountMetric(a: Int, b: String) extends Metric[SessionCountMetric] {
val series = "sessioncount"
}
Here's what happens when I try to log it:
scala> val m = SessionCountMetric(1, "a")
m: com.confabulous.deva.SessionCountMetric = SessionCountMetric(1,a)
scala> Metrics.log(m)
LoggedMetric(1411450638296,sessioncount,[],[])
Even though the macro itself seems to work fine:
scala> m.jsFields
res1: play.api.libs.json.JsArray = ["a","b"]
scala> m.jsValues
res2: play.api.libs.json.JsArray = [1,"a"]
Here's the actual macro itself:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object MetricsMacros {
private def fieldNames[T: c.WeakTypeTag](c: Context)= {
val tpe = c.weakTypeOf[T]
tpe.decls.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asTerm.name
}
}
def jsFields[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val names = fieldNames[T](c)
Apply(
q"play.api.libs.json.Json.arr",
names.map(name => Literal(Constant(name.toString))).toList
)
}
def jsValues[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val names = fieldNames[T](c)
Apply(
q"play.api.libs.json.Json.arr",
names.map(name => q"${c.prefix.tree}.$name").toList
)
}
}
Update
I tried Eugene's second suggestion like this:
abstract class Metric[T] {
def series: String
}
trait MetricSerializer[T] {
def fields: Seq[String]
def values(metric: T): Seq[Any]
}
object MetricSerializer {
implicit def materializeSerializer[T]: MetricSerializer[T] = macro MetricsMacros.materializeSerializer[T]
}
object Metrics {
def log[T: MetricSerializer](metric: T): Unit = {
val serializer = implicitly[MetricSerializer[T]]
println(serializer.fields)
println(serializer.values(metric))
}
}
with the macro now looking like this:
object MetricsMacros {
def materializeSerializer[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val tpe = c.weakTypeOf[T]
val names = tpe.decls.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asTerm.name
}
val fields = Apply(
q"Seq",
names.map(name => Literal(Constant(name.toString))).toList
)
val values = Apply(
q"Seq",
names.map(name => q"metric.$name").toList
)
q"""
new MetricSerializer[$tpe] {
def fields = $fields
def values(metric: Metric[$tpe]) = $values
}
"""
}
}
However, when I call Metrics.log -- specifically when it calls implicitly[MetricSerializer[T]] I get the following error:
error: value a is not a member of com.confabulous.deva.Metric[com.confabulous.deva.SessionCountMetric]
Why is it trying to use Metric[com.confabulous.deva.SessionCountMetric] instead of SessionCountMetric?
Conclusion
Fixed it.
def values(metric: Metric[$tpe]) = $values
should have been
def values(metric: $tpe) = $values
You're in a situation that's very close to one described in a recent question: scala macros: defer type inference.
As things stand right now, you'll have to turn log into a macro. An alternative would also to turn Metric.jsFields and Metric.jsValues into JsFieldable and JsValuable type classes materialized by implicit macros at callsites of log (http://docs.scala-lang.org/overviews/macros/implicits.html).
I am evaluating the possibility and effort to create a macro annotation to turn this:
#txn class Foo(var bar: Int)
into this:
import concurrent.stm.{Ref, InTxn}
class Foo(bar0: Int) {
private val _bar = Ref(bar0)
def bar(implicit tx: InTxn): Int = _bar()
def bar_=(value: Int)(implicit tx: InTxn): Unit = _bar() = value
}
(or perhaps creating a Foo companion object with apply method, no sure yet)
Now what I'm seeing from the ClassDef body is something like this
List(<paramaccessor> var bar: Int = _,
def <init>(bar: Int) = {
super.<init>();
()
})
So bar appears as param-accessor with no init (_), and also as argument to the <init> method.
How would I go about rewriting that body?
import scala.reflect.macros.Context
import scala.language.experimental.macros
import scala.annotation.StaticAnnotation
class txn extends StaticAnnotation {
def macroTransform(annottees: Any*) = macro txnMacro.impl
}
object txnMacro {
def impl(c: Context)(annottees: c.Expr[Any]*): c.Expr[Any] = {
import c.universe._
val inputs = annottees.map(_.tree).toList
def reportInvalidAnnotationTarget(): Unit =
c.error(c.enclosingPosition,
"This annotation can only be used on a class definition")
val expandees: List[Tree] = inputs match {
case cd # ClassDef(mods, nme, tp, tmp # Template(parents, self, body)) :: Nil =>
println(s"Body: $body")
???
case _ =>
reportInvalidAnnotationTarget()
inputs
}
val outputs = expandees
c.Expr[Any](Block(outputs, Literal(Constant(()))))
}
}
As Eugene suggests, using quasiquotes can make this much easier. I haven't put together a full solution, but I tried some pieces so I think something along these lines will work:
case q"class $name extends $parent with ..$traits { ..$body }"=>
val newBody = body.flatMap {
case q"var $varName = $varBody" =>
List(
//put the three defs you need here. some variant of this ...
q"""private val ${stringToTermName("_" + varName.toString)} = Ref(${stringToTermName(varName.name + "0")})""",
//etc ...
)
case other => other
}
q"class $name extends $parent with ..$ttraits { ..${(newBody).toList} }"
You might find my blog post relevant: http://imranrashid.com/posts/learning-scala-macros/
The most relevant code snippet is here:
https://github.com/squito/learn_macros/blob/master/macros/src/main/scala/com/imranrashid/oleander/macros/FillTraitDefs.scala#L78
You may also find this useful for defining the getters & setters:
https://groups.google.com/forum/#!searchin/scala-user/macro|sort:date/scala-user/Ug75FW73mJI/F1ijU0uxZAUJ
I'm trying to achieve a StringContext extension which will allow me to write this:
val tz = zone"Europe/London" //tz is of type java.util.TimeZone
But with the added caveat that it should fail to compile if the supplied time-zone is invalid (assuming that can be determined at compile-time).
Here's a helper function:
def maybeTZ(s: String): Option[java.util.TimeZone] =
java.util.TimeZone.getAvailableIDs collectFirst { case id if id == s =>
java.util.TimeZone.getTimeZone(id)
}
I can create a non-macro implementation very easily:
scala> implicit class TZContext(val sc: StringContext) extends AnyVal {
| def zone(args: Any *) = {
| val s = sc.raw(args.toSeq : _ *)
| maybeTZ(s) getOrElse sys.error(s"Invalid zone: $s")
| }
| }
Then:
scala> zone"UTC"
res1: java.util.TimeZone = sun.util.calendar.ZoneInfo[id="UTC",offset=0,...
So far, so good. Except that this doesn't fail compilation if the timezone is nonsensical (e.g. zone"foobar"); the code falls over at runtime. I'd like to extend it to a macro but, despite reading the docs, I'm really struggling with the details (All of the details, to be precise.)
Can anyone help to get me started here? The all-singing, all-dancing solution should look to see if the StringContext defines any arguments and (if so), defer calculation until runtime, otherwise attempting to parse the zone at compile-time
What have I tried?
Well, macro defs appear to have to be in statically accessible objects. So:
package object oxbow {
implicit class TZContext(val sc: StringContext) extends AnyVal {
def zone(args: Any *) = macro zoneImpl //zoneImpl cannot be in TZContext
}
def zoneImpl(c: reflect.macros.Context)
(args: c.Expr[Any] *): c.Expr[java.util.TimeZone] = {
import c.universe._
//1. How can I access sc from here?
/// ... if I could, would this be right?
if (args.isEmpty) {
val s = sc.raw()
reify(maybeTZ(s) getOrElse sys.error(s"Not valid $s"))
}
else {
//Ok, now I'm stuck. What goes here?
}
}
}
Based on som-snytt's suggestion below, here's the latest attempt:
def zoneImpl(c: reflect.macros.Context)
(args: c.Expr[Any] *): c.Expr[java.util.TimeZone] = {
import c.universe._
val z =
c.prefix.tree match {
case Apply(_, List(Apply(_, List(Literal(Constant(const: String)))))) => gsa.shared.datetime.XTimeZone.getTimeZone(const)
case x => ??? //not sure what to put here
}
c.Expr[java.util.TimeZone](Literal(Constant(z))) //this compiles but doesn't work at the use-site
^^^^^^^^^^^^^^^^^^^
this is wrong. What should it be?
}
At the use-site, a valid zone"UTC" fails to compile with the error:
java.lang.Error: bad constant value: sun.util.calendar.ZoneInfo[id="UTC",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null] of class class sun.util.calendar.ZoneInfo
Presumably I should not have used a Literal(Constant( .. )) to enclose it. What should I have used?
Last example - based on Travis Brown's answer below
def zoneImpl(c: reflect.macros.Context)
(args: c.Expr[Any] *): c.Expr[java.util.TimeZone] = {
import c.universe._
import java.util.TimeZone
val tzExpr: c.Expr[String] = c.prefix.tree match {
case Apply(_, Apply(_, List(tz # Literal(Constant(s: String)))) :: Nil)
if TimeZone.getAvailableIDs contains s => c.Expr(tz)
case Apply(_, Apply(_, List(tz # Literal(Constant(s: String)))) :: Nil) =>
c.abort(c.enclosingPosition, s"Invalid time zone! $s")
case _ => ???
// ^^^ What do I do here? I do not want to abort, I merely wish to
// "carry on as you were". I've tried ...
// c.prefix.tree.asInstanceOf[c.Expr[String]]
// ...but that does not work
}
c.universe.reify(TimeZone.getTimeZone(tzExpr.splice))
}
This is the "song-and-dance" solution that handles interpolation of the timezone:
package object timezone {
import scala.language.implicitConversions
implicit def zoned(sc: StringContext) = new ZoneContext(sc)
}
package timezone {
import scala.language.experimental.macros
import scala.reflect.macros.Context
import java.util.TimeZone
class ZoneContext(sc: StringContext) {
def tz(args: Any*): TimeZone = macro TimeZoned.tzImpl
// invoked if runtime interpolation is required
def tz0(args: Any*): TimeZone = {
val s = sc.s(args: _*)
val z = TimeZoned maybeTZ s getOrElse (throw new RuntimeException(s"Bad timezone $s"))
TimeZone getTimeZone z
}
}
object TimeZoned {
def maybeTZ(s: String): Option[String] =
if (TimeZone.getAvailableIDs contains s) Some(s) else None
def tzImpl(c: Context)(args: c.Expr[Any]*): c.Expr[TimeZone] = {
import c.universe._
c.prefix.tree match {
case Apply(_, List(Apply(_, List(tz #Literal(Constant(const: String)))))) =>
maybeTZ(const) map (
k => reify(TimeZone getTimeZone c.Expr[String](tz).splice)
) getOrElse c.abort(c.enclosingPosition, s"Bad timezone $const")
case x =>
val rts = x.tpe.declaration(newTermName("tz0"))
val rt = treeBuild.mkAttributedSelect(x, rts)
c.Expr[TimeZone](Apply(rt, args.map(_.tree).toList))
}
}
}
}
Usage:
package tztest
import timezone._
object Test extends App {
val delta = 8
//Console println tz"etc/GMT+$delta" //java.lang.RuntimeException: Bad timezone etc/GMT+8
Console println tz"Etc/GMT+$delta"
Console println tz"US/Hawaii"
//Console println tz"US/Nowayi" //error: Bad timezone US/Nowayi
}
The problem is that you can't smuggle a compile-time instance of TimeZone into the code generated by your macro. You can, however, slip a string literal through, so you can generate code that will construct the TimeZone you want at run time, while still checking at compile time to make sure the identifier is available.
The following is a complete working example:
object TimeZoneLiterals {
import java.util.TimeZone
import scala.language.experimental.macros
import scala.reflect.macros.Context
implicit class TZContext(val sc: StringContext) extends AnyVal {
def zone() = macro zoneImpl
}
def zoneImpl(c: reflect.macros.Context)() = {
import c.universe._
val tzExpr = c.prefix.tree match {
case Apply(_, Apply(_, List(tz # Literal(Constant(s: String)))) :: Nil)
if TimeZone.getAvailableIDs contains s => c.Expr(tz)
case _ => c.abort(c.enclosingPosition, "Invalid time zone!")
}
reify(TimeZone.getTimeZone(tzExpr.splice))
}
}
The argument to reify will be the body of the generated method—literally, not after any kind of evaluation, except that the tzExpr.slice bit will be replaced by the compile-time string literal (if, of course, you've found it in the list of available identifiers—otherwise you get a compile-time error).