StringContext and macros: a simple example - scala

I'm trying to achieve a StringContext extension which will allow me to write this:
val tz = zone"Europe/London" //tz is of type java.util.TimeZone
But with the added caveat that it should fail to compile if the supplied time-zone is invalid (assuming that can be determined at compile-time).
Here's a helper function:
def maybeTZ(s: String): Option[java.util.TimeZone] =
java.util.TimeZone.getAvailableIDs collectFirst { case id if id == s =>
java.util.TimeZone.getTimeZone(id)
}
I can create a non-macro implementation very easily:
scala> implicit class TZContext(val sc: StringContext) extends AnyVal {
| def zone(args: Any *) = {
| val s = sc.raw(args.toSeq : _ *)
| maybeTZ(s) getOrElse sys.error(s"Invalid zone: $s")
| }
| }
Then:
scala> zone"UTC"
res1: java.util.TimeZone = sun.util.calendar.ZoneInfo[id="UTC",offset=0,...
So far, so good. Except that this doesn't fail compilation if the timezone is nonsensical (e.g. zone"foobar"); the code falls over at runtime. I'd like to extend it to a macro but, despite reading the docs, I'm really struggling with the details (All of the details, to be precise.)
Can anyone help to get me started here? The all-singing, all-dancing solution should look to see if the StringContext defines any arguments and (if so), defer calculation until runtime, otherwise attempting to parse the zone at compile-time
What have I tried?
Well, macro defs appear to have to be in statically accessible objects. So:
package object oxbow {
implicit class TZContext(val sc: StringContext) extends AnyVal {
def zone(args: Any *) = macro zoneImpl //zoneImpl cannot be in TZContext
}
def zoneImpl(c: reflect.macros.Context)
(args: c.Expr[Any] *): c.Expr[java.util.TimeZone] = {
import c.universe._
//1. How can I access sc from here?
/// ... if I could, would this be right?
if (args.isEmpty) {
val s = sc.raw()
reify(maybeTZ(s) getOrElse sys.error(s"Not valid $s"))
}
else {
//Ok, now I'm stuck. What goes here?
}
}
}
Based on som-snytt's suggestion below, here's the latest attempt:
def zoneImpl(c: reflect.macros.Context)
(args: c.Expr[Any] *): c.Expr[java.util.TimeZone] = {
import c.universe._
val z =
c.prefix.tree match {
case Apply(_, List(Apply(_, List(Literal(Constant(const: String)))))) => gsa.shared.datetime.XTimeZone.getTimeZone(const)
case x => ??? //not sure what to put here
}
c.Expr[java.util.TimeZone](Literal(Constant(z))) //this compiles but doesn't work at the use-site
^^^^^^^^^^^^^^^^^^^
this is wrong. What should it be?
}
At the use-site, a valid zone"UTC" fails to compile with the error:
java.lang.Error: bad constant value: sun.util.calendar.ZoneInfo[id="UTC",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null] of class class sun.util.calendar.ZoneInfo
Presumably I should not have used a Literal(Constant( .. )) to enclose it. What should I have used?
Last example - based on Travis Brown's answer below
def zoneImpl(c: reflect.macros.Context)
(args: c.Expr[Any] *): c.Expr[java.util.TimeZone] = {
import c.universe._
import java.util.TimeZone
val tzExpr: c.Expr[String] = c.prefix.tree match {
case Apply(_, Apply(_, List(tz # Literal(Constant(s: String)))) :: Nil)
if TimeZone.getAvailableIDs contains s => c.Expr(tz)
case Apply(_, Apply(_, List(tz # Literal(Constant(s: String)))) :: Nil) =>
c.abort(c.enclosingPosition, s"Invalid time zone! $s")
case _ => ???
// ^^^ What do I do here? I do not want to abort, I merely wish to
// "carry on as you were". I've tried ...
// c.prefix.tree.asInstanceOf[c.Expr[String]]
// ...but that does not work
}
c.universe.reify(TimeZone.getTimeZone(tzExpr.splice))
}

This is the "song-and-dance" solution that handles interpolation of the timezone:
package object timezone {
import scala.language.implicitConversions
implicit def zoned(sc: StringContext) = new ZoneContext(sc)
}
package timezone {
import scala.language.experimental.macros
import scala.reflect.macros.Context
import java.util.TimeZone
class ZoneContext(sc: StringContext) {
def tz(args: Any*): TimeZone = macro TimeZoned.tzImpl
// invoked if runtime interpolation is required
def tz0(args: Any*): TimeZone = {
val s = sc.s(args: _*)
val z = TimeZoned maybeTZ s getOrElse (throw new RuntimeException(s"Bad timezone $s"))
TimeZone getTimeZone z
}
}
object TimeZoned {
def maybeTZ(s: String): Option[String] =
if (TimeZone.getAvailableIDs contains s) Some(s) else None
def tzImpl(c: Context)(args: c.Expr[Any]*): c.Expr[TimeZone] = {
import c.universe._
c.prefix.tree match {
case Apply(_, List(Apply(_, List(tz #Literal(Constant(const: String)))))) =>
maybeTZ(const) map (
k => reify(TimeZone getTimeZone c.Expr[String](tz).splice)
) getOrElse c.abort(c.enclosingPosition, s"Bad timezone $const")
case x =>
val rts = x.tpe.declaration(newTermName("tz0"))
val rt = treeBuild.mkAttributedSelect(x, rts)
c.Expr[TimeZone](Apply(rt, args.map(_.tree).toList))
}
}
}
}
Usage:
package tztest
import timezone._
object Test extends App {
val delta = 8
//Console println tz"etc/GMT+$delta" //java.lang.RuntimeException: Bad timezone etc/GMT+8
Console println tz"Etc/GMT+$delta"
Console println tz"US/Hawaii"
//Console println tz"US/Nowayi" //error: Bad timezone US/Nowayi
}

The problem is that you can't smuggle a compile-time instance of TimeZone into the code generated by your macro. You can, however, slip a string literal through, so you can generate code that will construct the TimeZone you want at run time, while still checking at compile time to make sure the identifier is available.
The following is a complete working example:
object TimeZoneLiterals {
import java.util.TimeZone
import scala.language.experimental.macros
import scala.reflect.macros.Context
implicit class TZContext(val sc: StringContext) extends AnyVal {
def zone() = macro zoneImpl
}
def zoneImpl(c: reflect.macros.Context)() = {
import c.universe._
val tzExpr = c.prefix.tree match {
case Apply(_, Apply(_, List(tz # Literal(Constant(s: String)))) :: Nil)
if TimeZone.getAvailableIDs contains s => c.Expr(tz)
case _ => c.abort(c.enclosingPosition, "Invalid time zone!")
}
reify(TimeZone.getTimeZone(tzExpr.splice))
}
}
The argument to reify will be the body of the generated method—literally, not after any kind of evaluation, except that the tzExpr.slice bit will be replaced by the compile-time string literal (if, of course, you've found it in the list of available identifiers—otherwise you get a compile-time error).

Related

How to get the runtime value of parameter passed to a Scala macro?

I have an ostensibly simple macro problem that I’ve been banging my head against for a few hours, with no luck. Perhaps someone with more experience can help.
I have the following macro:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object MacroObject {
def run(s: String): Unit =
macro runImpl
def runImpl(c: Context)(s: c.Tree): c.Tree = {
import c.universe._
println(s) // <-- I need the macro to know the value of s at compile time
q"()"
}
}
The problem is this: I’d like the macro to know the value s that is passed to it — not an AST of s, but the value of s itself. Specifically, I’d like it to have this behavior:
def runTheMacro(str: String): Unit = MacroObject.run(str)
final val HardCodedString1 = "Hello, world!"
runTheMacro(HardCodedString1) // the macro should print "Hello, world!"
// to the console during macro expansion
final val HardCodedString2 = "So long!"
runTheMacro(HardCodedString2) // the macro should print "So long!"
// to the console during macro expansion
It is guaranteed that the only strings that will be passed to runTheMacro are hard-coded constant values (i.e., known at compile-time).
Is this possible, and how does one do this?
--
Edit: There are also the following constraints:
It must be a blackbox macro.
The macro signature must use c.Trees, not c.Expr[_]s (legacy code; can’t change that part)
I do have a toolbox within the macro at my disposal if needed:
import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox
private val toolbox = currentMirror.mkToolBox()
/** Evaluate the given code fragment at compile time. */
private def eval[A](code: String): A = {
import scala.reflect.runtime.{universe => u}
val uTree: u.Tree = toolbox.parse(code)
toolbox.eval(uTree).asInstanceOf[A]
}
Your eval is runtime reflection's eval, compile-time macro's eval would be c.eval.
"Hello, world!" in
final val HardCodedString1 = "Hello, world!"
runTheMacro(HardCodedString1)
is a runtime value of HardCodedString1.
You can't have access to runtime value at compile time.
At compile time the tree of string HardCodedString1 just doesn't know anything about right hand side of the val tree.
Scala: what can code in Context.eval reference?
If you really need to use a runtime value inside the tree of your program you have to postpone its compilation till runtime
import scala.reflect.runtime.currentMirror
import scala.reflect.runtime.universe._
import scala.tools.reflect.ToolBox
object MacroObject {
val toolbox = currentMirror.mkToolBox()
def run(s: String): Unit = {
toolbox.eval(q"""
println($s)
()
""")
}
}
runTheMacro(HardCodedString1)//Hello, world!
runTheMacro(HardCodedString2)//So long!
Alternatively at compile time you can somehow find the tree of enclosing class and look inside it for the val tree and take its right hand side
def runImpl(c: blackbox.Context)(s: c.Tree): c.Tree = {
import c.universe._
var rhs: Tree = null
val traverser = new Traverser {
override def traverse(tree: Tree): Unit = {
tree match {
case q"$mods val $tname: $tpt = $expr" if tname == TermName("HardCodedString1") =>
rhs = expr
case _ => ()
}
super.traverse(tree)
}
}
traverser.traverse(c.enclosingClass) // deprecated
val rhsStr =
if (rhs != null) c.eval[String](c.Expr(c.untypecheck(rhs.duplicate)))
else c.abort(c.enclosingPosition, "no val HardCodedString1 defined")
println(rhsStr)
q"()"
}
runTheMacro(HardCodedString1)//Warning:scalac: Hello, world!
Or for all such variables
def runImpl(c: blackbox.Context)(s: c.Tree): c.Tree = {
import c.universe._
val sEvaluated =
try {
c.eval[String](c.Expr(c.untypecheck(s.duplicate)))
} catch {
case e: IllegalArgumentException if e.getMessage.startsWith("Could not find proxy") =>
s match {
case q"$sName" =>
var rhs: Tree = null
val traverser = new Traverser {
override def traverse(tree: Tree): Unit = {
tree match {
case q"$mods val $tname: $tpt = $expr" if tname == sName =>
rhs = expr
case _ => ()
}
super.traverse(tree)
}
}
traverser.traverse(c.enclosingClass)
if (rhs != null) c.eval[String](c.Expr(c.untypecheck(rhs.duplicate)))
else c.abort(c.enclosingPosition, s"no val $sName defined")
case _ => c.abort(c.enclosingPosition, s"unsupported tree $s")
}
}
println(sEvaluated)
q"()"
}
MacroObject.run(HardCodedString1) //Warning:scalac: Hello, world!
MacroObject.run(HardCodedString2) //Warning:scalac: So long!
runTheMacro will not work in this case: Error: no val str defined.
To make it work you can make it a macro too
def runTheMacro(str: String): Unit = macro runTheMacroImpl
def runTheMacroImpl(c: blackbox.Context)(str: c.Tree): c.Tree = {
import c.universe._
q"MacroObject.run($str)"
}
runTheMacro(HardCodedString1) //Warning:scalac: Hello, world!
runTheMacro(HardCodedString2) //Warning:scalac: So long!

Generic String interpolator using StringContext

I'm trying to create some simple custom String interpolator, and I'm successful as long as I don't try to use a type parameter.
import scala.concurrent.Future
object StringImplicits {
implicit class FailureStringContext (val sc : StringContext) extends AnyVal {
// This WORKS, but it's specific to Future :(
def fail[T](args : Any*): Future[T] = {
val orig = sc.s (args : _*)
Future.exception[T](new Exception(orig))
}
// I want this to work for Option,Try,Future!!
def fail[M,T](args:Any*): M[T] = {
val orig = sc.s (args : _*)
// Obviously does not work..
M match {
case Future => Future.exception(new Exception(orig))
case Option => None
case Try => Failure(new Exception(orig))
case _ => ???
}
}
}
}
Can I get this to work? I can't use parametric polymorphism because I'm not the one defining those three types.
What's the equivalent in the type level for that pseudo-code pattern match?
LATEST ATTEMPT
My latest attempt was to use implicitly, but I don't have such implicit! I'd be actually interested to grab a hold of the type that the compiler wants me to return according to type inference.
def fail[T, M[T]](args:Any*): M[T] = {
val orig = sc.s(args: _*)
implicitly[M[T]] match {
case _:Future[T] => Future.exception(new Exception(orig))
case _ => ???
}
}
<console>:18: error: could not find implicit value for parameter e: M[T]
implicitly[M[T]] match {
^
<console>:19: error: value exception is not a member of object scala.concurrent.Future
case _: Future[T] => Future.exception(new Exception(orig))
^
In my opinion the simplest is to rely on good old overloading: just define a different overload for each type that you want to handle.
Now of course, there is the problem of having different overloads with the same signature, and as usual in scala, you can use tricks to work around them. Here we'll add dummy implicit parameters to force each overload to have a distinct signature. Not pretty but it works and will suffice in this case.
import scala.concurrent.Future
import scala.util.{Try, Failure}
implicit class FailureStringContext (val sc : StringContext) extends AnyVal {
def fail[T](args : Any*): Future[T] = {
Future.failed[T](new Exception(sc.s (args : _*)))
}
def fail[T](args : Any*)(implicit dummy: DummyImplicit): Option[T] = {
Option.empty[T]
}
def fail[T](args : Any*)(implicit dummy: DummyImplicit, dummy2: DummyImplicit): Try[T] = {
Failure[T](new Exception(sc.s (args : _*)))
}
}
And tada:
scala> fail"oops": Option[String]
res6: Option[String] = None
scala> fail"oops": Future[String]
res7: scala.concurrent.Future[String] = scala.concurrent.impl.Promise$KeptPromise#6fc1a8f6
scala> fail"oops": Try[String]
res8: scala.util.Try[String] = Failure(java.lang.Exception: oops)

Quasiquotes and Generics in Scala

I'm using Play Framework and trying to write an action that can parse protobuf request as following:
def AsyncProtoAction(block: ProtoRequestA => Future[Result]): Action[AnyContent] = {
Action.async { request =>
request.contentType match {
case Some("application/protobuf") => request.body.asRaw match {
case Some(raw) => raw.asBytes(1024 * 1024) match {
case Some(rawBytes) =>
val proto = ProtoRequestA.parseFrom(rawBytes)
block(proto)
case _ => ...
}
}
}
}
}
Here, ProtoRequestA is a generated object using ScalaPB. It works, however, I have a lot of protobuf request objects. Then I try to rewrite it using macro:
object Actions {
def AsyncProtoAction[T](block: T => Future[Result]):
Action[AnyContent] = macro asyncProtoAction_impl[T]
def asyncProtoAction_impl[T: c.WeakTypeTag](c: blackbox.Context)
(block: c.Tree) = { import c.universe._
val protoType = weakTypeOf[T]
q"""import play.api.mvc._
Action.async { request =>
request.contentType match {
case Some("application/protobuf") =>
request.body.asRaw match {
case Some(raw) =>
raw.asBytes(1024 * 1024) match {
case Some(rawBytes) =>
val proto = $protoType.parseFrom(rawBytes)
$block(proto)
case _ => ...
}
}
}
}"""
}
}
This doesn't work. The compilation error is value parseFrom is not a member of packageName.ProtoRequestA.
I tried showRaw(tq"$protoType") in REPL, which output String = TypeTree(). I guess the correct output should be String = Select(Ident(TermName("packageName")), TypeName("ProtoRequestA")). So what should I do?
You've identified the problem, which is that when you interpolate the ProtoRequestA type you get something different from the ProtoRequestA companion object. One easy solution is to interpolate the symbol of the companion object, which you can get from the type's symbol. Here's a simplified example:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
def companionObjectBarImpl[A: c.WeakTypeTag](c: Context): c.Tree = {
import c.universe._
q"${ symbolOf[A].companion }.bar"
}
def companionObjectBar[A]: Int = macro companionObjectBarImpl[A]
This macro will work on any type with a companion object with a bar method that returns an Int. For example:
scala> class Foo; object Foo { def bar = 10 }
defined class Foo
defined object Foo
scala> companionObjectBar[Foo]
res0: Int = 10
You should be able to do something similar.

Instantiate class symbol using macro

I'm using the following Scala macro (heavily inspired by the code from this SO question) to get a list of all objects contained in a given package that inherit a specific trait:
object Macros {
def allObjects[T <: AnyRef](packageName: String): List[Any] = macro allObjectsImpl[T]
def allObjectsImpl[T <: AnyRef: c.WeakTypeTag](c: Context)(packageName: c.Expr[String]) = {
import c.universe._
val baseTraitSymbol = c.weakTypeOf[T].typeSymbol
val pkg = packageName.tree match {
case Literal(Constant(name: String)) => c.mirror.staticPackage(name)
}
val types = pkg.typeSignature.members.collect {
case moduleSymbol: ModuleSymbol if moduleSymbol.moduleClass.asClass.baseClasses contains baseTraitSymbol => Ident(moduleSymbol)
}.toList
val listApply = Select(reify(List).tree, TermName("apply"))
c.Expr[List[T]](Apply(listApply, types))
}
}
Which works fine.
I want to change the macro so that instead of getting all the objects in a package, it gets all the concrete classes, and provides a list containing an instance of each of them.
The AST when creating an instance of an Object looks like this:
scala> import scala.reflect.runtime.{universe => u}
import scala.reflect.runtime.{universe=>u}
scala> u showRaw ( u reify {new Object} )
res42: String = Expr(Apply(Select(New(Ident(java.lang.Object)), termNames.CONSTRUCTOR), List()))
So I thought changing my code to this would work:
object Macros {
def allInstances[T <: AnyRef](packageName: String): List[Any] = macro allInstancesImpl[T]
def allInstancesImpl[T <: AnyRef: c.WeakTypeTag](c: Context)(packageName: c.Expr[String]) = {
import c.universe._
val baseTraitSymbol = c.weakTypeOf[T].typeSymbol
val pkg = packageName.tree match {
case Literal(Constant(name: String)) => c.mirror.staticPackage(name)
}
def isConcreteChildClass(child: ClassSymbol, base: Symbol) = {
!child.isAbstract && (child.baseClasses contains base)
}
val types = pkg.typeSignature.members.collect {
case classSymbol: ClassSymbol if isConcreteChildClass(classSymbol, baseTraitSymbol) => {
Apply(Select(New(Ident(classSymbol.primaryConstructor)), termNames.CONSTRUCTOR), List())
}
}.toList
val listApply = Select(reify(List).tree, TermName("apply"))
c.Expr[List[T]](Apply(listApply, types))
}
}
However, when I try to use the updated macro code on a test package, I get the following error:
scala> Macros.allInstances[AnyRef]("test")
<console>:9: error: class type required but ()test.TestClass found
Macros.allInstances[AnyRef]("test")
From what I'm seeing, it looks like the macro is actually returning the constructor itself instead of returning the instance that should get built by the constructor, but I can't figure out what I'm missing.
The problem is in this line (reformatted for clarity):
Apply(
Select(New(Ident(classSymbol.primaryConstructor)), termNames.CONSTRUCTOR),
List()
)
You're essentially selecting the constructor twice. You could just drop primaryConstructor:
Apply(
Select(New(Ident(classSymbol)), termNames.CONSTRUCTOR),
List()
)
Using ApplyConstructor would also work:
ApplyConstructor(Ident(classSymbol), Nil)
Or you could just go with quasiquotes:
q"new ${Ident(classSymbol)}()"
The quasiquote solution is the most future-proof.

Scala macro and type erasure

I'm having some problems with a macro I've written to help me log metrics represented as case class instances to to InfluxDB. I presume I'm having a type erasure problem and that the tyep parameter T is getting lost, but I'm not entirely sure what's going on. (This is also my first exposure to Scala macros.)
import scala.language.experimental.macros
import play.api.libs.json.{JsNumber, JsString, JsObject, JsArray}
abstract class Metric[T] {
def series: String
def jsFields: JsArray = macro MetricsMacros.jsFields[T]
def jsValues: JsArray = macro MetricsMacros.jsValues[T]
}
object Metrics {
case class LoggedMetric(timestamp: Long, series: String, fields: JsArray, values: JsArray)
case object Kick
def log[T](metric: Metric[T]): Unit = {
println(LoggedMetric(
System.currentTimeMillis,
metric.series,
metric.jsFields,
metric.jsValues
))
}
}
And here's an example metric case class:
case class SessionCountMetric(a: Int, b: String) extends Metric[SessionCountMetric] {
val series = "sessioncount"
}
Here's what happens when I try to log it:
scala> val m = SessionCountMetric(1, "a")
m: com.confabulous.deva.SessionCountMetric = SessionCountMetric(1,a)
scala> Metrics.log(m)
LoggedMetric(1411450638296,sessioncount,[],[])
Even though the macro itself seems to work fine:
scala> m.jsFields
res1: play.api.libs.json.JsArray = ["a","b"]
scala> m.jsValues
res2: play.api.libs.json.JsArray = [1,"a"]
Here's the actual macro itself:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object MetricsMacros {
private def fieldNames[T: c.WeakTypeTag](c: Context)= {
val tpe = c.weakTypeOf[T]
tpe.decls.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asTerm.name
}
}
def jsFields[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val names = fieldNames[T](c)
Apply(
q"play.api.libs.json.Json.arr",
names.map(name => Literal(Constant(name.toString))).toList
)
}
def jsValues[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val names = fieldNames[T](c)
Apply(
q"play.api.libs.json.Json.arr",
names.map(name => q"${c.prefix.tree}.$name").toList
)
}
}
Update
I tried Eugene's second suggestion like this:
abstract class Metric[T] {
def series: String
}
trait MetricSerializer[T] {
def fields: Seq[String]
def values(metric: T): Seq[Any]
}
object MetricSerializer {
implicit def materializeSerializer[T]: MetricSerializer[T] = macro MetricsMacros.materializeSerializer[T]
}
object Metrics {
def log[T: MetricSerializer](metric: T): Unit = {
val serializer = implicitly[MetricSerializer[T]]
println(serializer.fields)
println(serializer.values(metric))
}
}
with the macro now looking like this:
object MetricsMacros {
def materializeSerializer[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val tpe = c.weakTypeOf[T]
val names = tpe.decls.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asTerm.name
}
val fields = Apply(
q"Seq",
names.map(name => Literal(Constant(name.toString))).toList
)
val values = Apply(
q"Seq",
names.map(name => q"metric.$name").toList
)
q"""
new MetricSerializer[$tpe] {
def fields = $fields
def values(metric: Metric[$tpe]) = $values
}
"""
}
}
However, when I call Metrics.log -- specifically when it calls implicitly[MetricSerializer[T]] I get the following error:
error: value a is not a member of com.confabulous.deva.Metric[com.confabulous.deva.SessionCountMetric]
Why is it trying to use Metric[com.confabulous.deva.SessionCountMetric] instead of SessionCountMetric?
Conclusion
Fixed it.
def values(metric: Metric[$tpe]) = $values
should have been
def values(metric: $tpe) = $values
You're in a situation that's very close to one described in a recent question: scala macros: defer type inference.
As things stand right now, you'll have to turn log into a macro. An alternative would also to turn Metric.jsFields and Metric.jsValues into JsFieldable and JsValuable type classes materialized by implicit macros at callsites of log (http://docs.scala-lang.org/overviews/macros/implicits.html).