Introspect argument passed to a Scala macro - scala

I would like to program a Scala macro that takes an instance of a case class as argument. All objects that can be passed to the macro have to implement a specific marker trait.
The following snippet shows the marker trait and two example case classes implementing it:
trait Domain
case class Country( id: String, name: String ) extends Domain
case class Town( id: String, longitude: Double, latitude: Double ) extends Domain
Now, I would like to write the following code using macros to avoid the heaviness of runtime reflection and its thread unsafety:
object Test extends App {
// instantiate example domain object
val myCountry = Country( "CH", "Switzerland" )
// this is a macro call
logDomain( myCountry )
}
The macro logDomain is implemented in a different project and looks similar to:
object Macros {
def logDomain( domain: Domain ): Unit = macro logDomainMacroImpl
def logDomainMacroImpl( c: Context )( domain: c.Expr[Domain] ): c.Expr[Unit] = {
// Here I would like to introspect the argument object but do not know how?
// I would like to generate code that prints out all val's with their values
}
}
The macro's purpose should be to generate code that - at runtime - outputs all values (id and name) of the given object and prints them as shown next:
id (String) : CH
name (String) : Switzerland
To achieve this, I would have to dynamically inspect the passed type argument and determine its members (vals). Then I would have to generate an AST representing the code that creates the log output. The macro should work regardless of what specific object implementing the marker trait "Domain" is passed to the macro.
At this point I am lost. I would appreciate if someone could give me a starting point or point me to some documentation? I am relatively new to Scala and have not found a solution in the Scala API docs or the Macro guide.

Listing the accessors of a case class is such a common operation when you're working with macros that I tend to keep a method like this around:
def accessors[A: u.WeakTypeTag](u: scala.reflect.api.Universe) = {
import u._
u.weakTypeOf[A].declarations.collect {
case acc: MethodSymbol if acc.isCaseAccessor => acc
}.toList
}
This will give us all the case class accessor method symbols for A, if it has any. Note that I'm using the general reflection API here—there's no need to make this macro-specific yet.
We can wrap this method up with some other convenience stuff:
trait ReflectionUtils {
import scala.reflect.api.Universe
def accessors[A: u.WeakTypeTag](u: Universe) = {
import u._
u.weakTypeOf[A].declarations.collect {
case acc: MethodSymbol if acc.isCaseAccessor => acc
}.toList
}
def printfTree(u: Universe)(format: String, trees: u.Tree*) = {
import u._
Apply(
Select(reify(Predef).tree, "printf"),
Literal(Constant(format)) :: trees.toList
)
}
}
And now we can write the actual macro code pretty concisely:
trait Domain
object Macros extends ReflectionUtils {
import scala.language.experimental.macros
import scala.reflect.macros.Context
def log[D <: Domain](domain: D): Unit = macro log_impl[D]
def log_impl[D <: Domain: c.WeakTypeTag](c: Context)(domain: c.Expr[D]) = {
import c.universe._
if (!weakTypeOf[D].typeSymbol.asClass.isCaseClass) c.abort(
c.enclosingPosition,
"Need something typed as a case class!"
) else c.Expr(
Block(
accessors[D](c.universe).map(acc =>
printfTree(c.universe)(
"%s (%s) : %%s\n".format(
acc.name.decoded,
acc.typeSignature.typeSymbol.name.decoded
),
Select(domain.tree.duplicate, acc.name)
)
),
c.literalUnit.tree
)
)
}
}
Note that we still need to keep track of the specific case class type we're dealing with, but type inference will take care of that at the call site—we won't need to specify the type parameter explicitly.
Now we can open a REPL, paste in your case class definitions, and then write the following:
scala> Macros.log(Town("Washington, D.C.", 38.89, 77.03))
id (String) : Washington, D.C.
longitude (Double) : 38.89
latitude (Double) : 77.03
Or:
scala> Macros.log(Country("CH", "Switzerland"))
id (String) : CH
name (String) : Switzerland
As desired.

From what I can see, you need to solve two problems: 1) get the necessary information from the macro argument, 2) generate trees that represent the code you need.
In Scala 2.10 these things are done with the reflection API. Follow Is there a tutorial on Scala 2.10's reflection API yet? to see what documentation is available for it.
import scala.reflect.macros.Context
import language.experimental.macros
trait Domain
case class Country(id: String, name: String) extends Domain
case class Town(id: String, longitude: Double, latitude: Double) extends Domain
object Macros {
def logDomain(domain: Domain): Unit = macro logDomainMacroImpl
def logDomainMacroImpl(c: Context)(domain: c.Expr[Domain]): c.Expr[Unit] = {
import c.universe._
// problem 1: getting the list of all declared vals and their types
// * declarations return declared, but not inherited members
// * collect filters out non-methods
// * isCaseAccessor only leaves accessors of case class vals
// * typeSignature is how you get types of members
// (for generic members you might need to use typeSignatureIn)
val vals = typeOf[Country].declarations.toList.collect{ case sym if sym.isMethod => sym.asMethod }.filter(_.isCaseAccessor)
val types = vals map (_.typeSignature)
// problem 2: generating the code which would print:
// id (String) : CH
// name (String) : Switzerland
//
// usually reify is of limited usefulness
// (see https://stackoverflow.com/questions/13795490/how-to-use-type-calculated-in-scala-macro-in-a-reify-clause)
// but here it's perfectly suitable
// a subtle detail: `domain` will be possibly used multiple times
// therefore we need to duplicate it
val stmts = vals.map(v => c.universe.reify(println(
c.literal(v.name.toString).splice +
"(" + c.literal(v.returnType.toString).splice + ")" +
" : " + c.Expr[Any](Select(domain.tree.duplicate, v)).splice)).tree)
c.Expr[Unit](Block(stmts, Literal(Constant(()))))
}
}

Related

Is it possible to call a scala macro from generic scala code?

I'm trying to use Scala macros to convert untyped, Map[String, Any]-like expressions to their corresponding typed case class expressions.
The following scala macro (almost) gets the job done:
trait ToTyped[+T] {
def apply(term: Any): T
}
object TypeConversions {
// At compile-time, "type-check" an untyped expression and convert it to
// its appropriate typed value.
def toTyped[T]: ToTyped[T] = macro toTypedImpl[T]
def toTypedImpl[T: c.WeakTypeTag](c: Context): c.Expr[ToTyped[T]] = {
import c.universe._
val tpe = weakTypeOf[T]
if (tpe <:< typeOf[Int] || tpe <:< typeOf[String]) {
c.Expr[ToTyped[T]](
q"""new ToTyped[$tpe] {
def apply(term: Any): $tpe = term.asInstanceOf[$tpe]
}""")
} else {
val companion = tpe.typeSymbol.companion
val maybeConstructor = tpe.decls.collectFirst {
case m: MethodSymbol if m.isPrimaryConstructor => m
}
val constructorFields = maybeConstructor.get.paramLists.head
val subASTs = constructorFields.map { field =>
val fieldName = field.asTerm.name
val fieldDecodedName = fieldName.toString
val fieldType = tpe.decl(fieldName).typeSignature
q"""
val subTerm = term.asInstanceOf[Map[String, Any]]($fieldDecodedName)
TypeConversions.toTyped[$fieldType](subTerm)
"""
}
c.Expr[ToTyped[T]](
q"""new ToTyped[$tpe] {
def apply(term: Any): $tpe = $companion(..$subASTs)
}""")
}
}
}
Using the above toTyped function, I can convert for example an untyped person value to its corresponding typed Person case class:
object TypeConversionTests {
case class Person(name: String, age: Int, address: Address)
case class Address(street: String, num: Int, zip: Int)
val untypedPerson = Map(
"name" -> "Max",
"age" -> 27,
"address" -> Map("street" -> "Palm Street", "num" -> 7, "zip" -> 12345))
val typedPerson = TypeConversions.toTyped[Person](untypedPerson)
typedPerson shouldEqual Person("Max", 27, Address("Palm Street", 7, 12345))
}
However, my problem arises when trying to use the toTyped macro from above in generic scala code. Suppose I have a generic function indirection that uses the toTyped macro:
object CanIUseScalaMacrosAndGenerics {
def indirection[T](value: Any): T = TypeConversions.toTyped[T](value)
import TypeConversionTests._
val indirectlyTyped = indirection[Person](untypedPerson)
indirectlyTyped shouldEqual Person("Max", 27, Address("Palm Street", 7, 12345))
Here, I get a compile-time error from the toTyped macro complaining that the type T is not yet instantiated with a concrete type. I think the reason for the error is that from the perspective of toTyped inside indirection, the type T is still generic and not inferred to be Person just yet. And therefore the macro cannot build the corresponding Person case class when called via indirection. However, from the perspective of the call-site indirection[Person](untypedPerson), we have T == Person, so I wonder if there is a way to obtain the instantiated type of T (i.e., Person) inside the macro toTyped.
Put differently: Can I combine the Scala macro toTyped with the generic function indirection and yet be able to figure out the instantiated type for type parameter T inside the toTyped macro? Or am I on a hopeless track here and there is no way to combine Scala macros and generics like this? In the latter case I would like to know if the only solution here is to push the macro usage so far "out" that I can call it instantiated as toTyped[Person] rather than as toTyped[T].
Any insights are very much appreciated. Thank you! :-)
Macros need to be expanded. Every time you use a function which body is a macro, Scala will have to generate the code and put it there. As you suspect, this is very very specific and contradict the idea of parametric polymorphism where you write code independent of specific knowledge about your type.
Type classes are one of solutions to the general problem when you want to have one generic (parametric) definition and multiple per-type implementations of certain parts of your algorithm. You basically, define something that you could consider interface which (most likely) need to follow some contract (speaking in OOP terminology) and pass this interface as as argument:
// example
trait SpecificPerType[T] {
def doSomethingSpecific(t: T): String
}
val specificForString: SpecificPerType[String] = new SpecificPerType[String] {
def doSomethingSpecific(t: String): String = s"MyString: $t"
}
val specificForInt: SpecificPerType[Int] = new SpecificPerType[Int] {
def doSomethingSpecific(t: Int): String = s"MyInt: $t"
}
def genericAlgorithm[T](values: List[T])(specific: SpecificPerType[T]): String =
values.map(specific.doSomethingSpecific).mkString("\n")
genericAlgorithm(List(1,2,3))(specificForInt)
genericAlgorithm(List("a","b","c"))(specificForString)
As you can see, it would be pretty annoying to pass this specific part around, which is one of the reasons implicits were introduced.
So you could write it using implicits like this:
implicit val specificForString: SpecificPerType[String] = new SpecificPerType[String] {
def doSomethingSpecific(t: String): String = s"MyString: $t"
}
implicit val specificForInt: SpecificPerType[Int] = new SpecificPerType[Int] {
def doSomethingSpecific(t: Int): String = s"MyInt: $t"
}
def genericAlgorithm[T](values: List[T])(implicit specific: SpecificPerType[T]): String =
values.map(specific.doSomethingSpecific).mkString("\n")
/* for implicits with one type parameter there exist a special syntax
allowing to express them as if they were type constraints e.g.:
def genericAlgorithm[T: SpecificPerType](values: List[T]): String =
values.map(implicitly[SpecificPerType[T]].doSomethingSpecific).mkString("\n")
implicitly[SpecificPerType[T]] is a summoning that let you access implicit
by type, rather than by its variable's name
*/
genericAlgorithm(List(1,2,3)) // finds specificForString using its type
genericAlgorithm(List("a","b","c")) // finds specificForInt using its type
If you generate that trait implementation using macro, you will be able to have a generic algorithm e.g.:
implicit def generate[T]: SpecificPerType[T] =
macro SpecificPerTypeMacros.impl // assuming that you defined this macro there
As far as I can tell, this (extracting macros into type classes) is one of common patterns when it comes to being
able to generate some code with macros while, still building logic on top of it
using normal, parametric code.
(Just to be clear: I do not claim that the role of type classes is limited as the carriers of macro generated code).

Scala annotation macro only works with pre-defined classes

Note: There's an EDIT below!
Note: There's another EDIT below!
I have written a Scala annotation macro that is being passed a class and creates (or rather populates) a case object. The name of the case object is the same as the name of the passed class. More importantly, for every field of the passed class, there will be a field in the case object of the same name. The fields of the case object, however, are all of type String, and their value is the name of the type of the respective field in the passed class. Example:
// Using the annotation macro to populate a case object called `String`
#RegisterClass(classOf[String]) case object String
// The class `String` defines a field called `value` of type `char[]`.
// The case object also has a field `value`, containing `"char[]"`.
println(String.value) // Prints `"char[]"` to the console
This, however, seems to only work with pre-defined classes such as String. If I define a case class A(...) and try to do #RegisterClass(classOf[A]) case object A, I get the following error:
[info] scala.tools.reflect.ToolBoxError: reflective compilation has failed:
[info]
[info] not found: type A
What have I done wrong? The code of my macro can be found below. Also, if someone notices un-idiomatic Scala or bad practices in general, I wouldn't mind a hint. Thank you very much in advance!
class RegisterClass[T](clazz: Class[T]) extends StaticAnnotation {
def macroTransform(annottees: Any*) =
macro RegisterClass.expandImpl[T]
}
object RegisterClass {
def expandImpl[T](c: blackbox.Context)(annottees: c.Expr[Any]*) = {
import c.universe._
val clazz: Class[T] = c.prefix.tree match {
case q"new RegisterClass($clazz)" => c.eval[Class[T]](c.Expr(clazz))
case _ => c.abort(c.enclosingPosition, "RegisterClass: Annotation expects a Class[T] instance as argument.")
}
annottees.map(_.tree) match {
case List(q"case object $caseObjectName") =>
if (caseObjectName.toString != clazz.getSimpleName)
c.abort(c.enclosingPosition, "RegisterClass: Annotated case object and class T of passed Class[T] instance" +
"must have the same name.")
val clazzFields = clazz.getDeclaredFields.map(field => field.getName -> field.getType.getSimpleName).toList
val caseObjectFields = clazzFields.map(field => {
val fieldName: TermName = field._1
val fieldType: String = field._2
q"val $fieldName = $fieldType"
})
c.Expr[Any](q"case object $caseObjectName { ..$caseObjectFields }")
case _ => c.abort(c.enclosingPosition, "RegisterClass: Annotation must be applied to a case object definition.")
}
}
}
EDIT: As Eugene Burmako pointed out, the error happens because class A hasn't been compiled yet, so a java.lang.Class for it doesn't exist. I have now started a bounty of 100 StackOverflow points for everyone who as an idea how one could get this to work!
EDIT 2: Some background on the use case: As part of my bachelor thesis I am working on a Scala DSL for expressing queries for event processing systems. Those queries are traditionally expressed as strings, which induces a lot of problems. A typical query would look like that: "select A.id, B.timestamp from pattern[A -> B]". Meaning: If an event of type A occurs and after that an event of type B occurs, too, give me the id of the A event and the timestamp of the B event. The types A and B usually are simple Java classes over which I have no control. id and timestamp are fields of those classes. I would like queries of my DSL to look like that: select (A.id, B.timestamp) { /* ... * / }. This means that for every class representing an event type, e.g., A, I need a companion object -- ideally of the same name. This companion object should have the same fields as the respective class, so that I can pass its fields to the select function, like so: select (A.id, B.timestamp) { /* ... * / }. This way, if I tried to pass A.idd to the select function, it would fail at compile-time if there was no such field in the original class -- because then there would not be one in the companion object either.
This isn't an answer to your macro problem, but it could be a solution to your general problem.
If you can allow a minor change to the syntax of your DSL this might be possible without using macro's (depending on other requirements not mentioned in this question).
scala> class Select[A,B]{
| def apply[R,S](fa: A => R, fb: B => S)(body: => Unit) = ???
| }
defined class Select
scala> def select[A,B] = new Select[A,B]
select: [A, B]=> Select[A,B]
scala> class MyA { def id = 42L }
defined class MyA
scala> class MyB { def timestamp = "foo" }
defined class MyB
scala> select[A,B](_.id, _.timestamp){ /* ... */ }
scala.NotImplementedError: an implementation is missing
I use the class Select here as a means to be able to specify the types of your event classes while letting the compiler infer the result types of the functions fa and fb. If your don't need those result types you could just write it as def select[A,B](fa: A => Any, fb: B => Any)(body: => Unit) = ???.
If necessary you can still implement the select or apply method as a macro. But using this syntax, you will no longer need to generate objects with macro annotations.

Scala Quasiquotes Destructuring a Type

Context:
I'm working on a library for working with JMX in Scala. One of the objectives is to have a strong typed interface to Managed Beans. I guess akin to to the Spring framework JMX library.
Objective: Macro to Deserialise TabularData to a case class:
// interface for which I'd like to generate an implementation using a macro
trait JMXTabularAssembler[T <: Product] {
def assemble(data: TabularData): T
}
object JMXAnnotations {
case class Attribute(name: String) extends StaticAnnotation
}
case class example(
#Attribute("name") name: String,
#Attribute("age") age: Int,
unmarked: String
)
Problem: There are plenty of examples of composing tree's using the q"" interpolators. But I can't figure out how to use the tq"" interpolator to
extract the fields out of a case class from a type context.
private def mkAssembler[T <: Product : c.WeakTypeTag](c: Context): c.universe.Tree = {
import c.universe._
val tt = weakTypeOf[T]
}
Question: How do I use the QuasiQuote machinery to destructure the fields of my case class so that I can loop over them and filter out fields with my annotation (my Attribute annotation is not available from the approach I am currently taking"). An implementation of the following that returns the fields with annotations in declaration order is what I am after.
private def harvestFieldsWithAnnotations[T<: Product: c.WeakTypeTag](c: Context):
List[(c.universe.Name, String, c.universe.Type, List[c.universe.Annotation])] = ???
Bonus: The objective is to get the attribute fields, generate trees for each field that extract the field from the TabularData and use these trees to create the JMXTabularAssembler Functor. If you could show me how to do this for the example above it would bootstrap my efforts :D.
What I have tried: I started solving the problem by using reflection. This does not seem the right way to do it. Snippets:
...
val dec = tt.decls.sorted
def getFields = dec.withFilter( t=> t.isTerm && ! t.isMethod)
def getCaseAccessors = dec.withFilter( t => t.isMethod && t.asMethod.isCaseAccessor)
dec.foreach { d=>
println(d.name, d.annotations)
}
getFields.foreach { f =>
println(f.annotations)
}
val types = getCaseAccessors.map { d =>
println(d.annotations)
(d.name, tt.member(d.name).asMethod.returnType)
}
...
The following method does the trick, it does not use quasi quotes. The key is to access the backing field of a symbol representing the field accessor of a case class (the accessed call).
private def harvestFieldsWithAnnotations[T <: Product : c.WeakTypeTag](c: Context) = {
import c.universe._
val tt = weakTypeOf[T]
tt.decls.sorted.filter(t => t.isMethod && t.asMethod.isCaseAccessor).map { ca =>
val asMethod = tt.member(ca.name).asMethod
(ca.name, asMethod.returnType, asMethod.accessed.annotations)
}
}
Field annotations won't get retained unless they are explicitly annotated with scala.annotation.meta.field.
So the Attribute annotation should be:
#field
case class Attribute(name: String) extends StaticAnnotation

Scala Capture Expression Name without a Macro

I'm trying to grab the name of an expression call without using a macro. I have the following example code:
case class Person(name: String, age: Int)
case class MyClass[T]() {
def doSomething(value: Any)
}
val p = Person("Bob",40)
val my = MyClass[Person]
my.doSomething(p.name)
my.doSomething(p.age)
Is there a simple way inside the method doSomething to capture the expression name being used when making the call, for example capture a String value of name and age?
I know if I turn doSomething into a macro the passed parameter is an Expr of Context and I can get the name using the Tree of the Expr, however given the complex nature of macros I'm sort of thing to avoid using one in this case.
The short answer is no. There is currently no way of getting an abstract syntax tree (AST) except with macros.
You could use reify here with some success, but that itself is a macro, and this would require the doSomething caller to have to use reify, which is strange. For example, in the following code doSomething returns the property name together with the value of the property for the instance passed to the case class constructor:
import scala.reflect.runtime.universe._
import scala.reflect.runtime._
import scala.tools.reflect.ToolBox
case class Person(name: String, age: Int)
case class MyClass[T](t: T) {
val tb = currentMirror.mkToolBox()
def doSomething[U](expr: Expr[T => U]) = Some(expr.tree) collect {
case f#Function(_, Select(_, property)) =>
val func = tb.eval(f).asInstanceOf[T => U]
property.decoded -> func(t)
}
}
Which can be called as:
val p = Person("Bob",40)
val my = MyClass(p)
//The caller needs to use reify, which is strange!
my.doSomething(reify((_:Person).name)) //Some(("name", "Bob"))
my.doSomething(reify { (p:Person) => p.age }) //Some(("age",40))
But this is fragile and error prone, and at this point you might as well write a macro!
Your other option is to pass in a String name of the property and use reflection to get that property from the instance.
You can look at the docs for Trees and macros to learn more.

Read case class object from string in Scala (something like Haskell's "read" typeclass)

I'd like to read a string as an instance of a case class. For example, if the function were named "read" it would let me do the following:
case class Person(name: String, age: Int)
val personString: String = "Person(Bob,42)"
val person: Person = read(personString)
This is the same behavior as the read typeclass in Haskell.
dflemstr answered more towards setting up the actual read method- I'll answer more for the actual parsing method.
My approach has two objects that can be used in scala's pattern matching blocks. AsInt lets you match against strings that represent Ints, and PersonString is the actual implementation for Person deserialization.
object AsInt {
def unapply(s: String) = try{ Some(s.toInt) } catch {
case e: NumberFormatException => None
}
}
val PersonRegex = "Person\\((.*),(\\d+)\\)".r
object PersonString {
def unapply(str: String): Option[Person] = str match {
case PersonRegex(name, AsInt(age)) => Some(Person(name, age))
case _ => None
}
}
The magic is in the unapply method, which scala has syntax sugar for. So using the PersonString object, you could do
val person = PersonString.unapply("Person(Bob,42)")
// person will be Some(Person("Bob", 42))
or you could use a pattern matching block to do stuff with the person:
"Person(Bob,42)" match {
case PersonString(person) => println(person.name + " " + person.age)
case _ => println("Didn't get a person")
}
Scala does not have type classes, and in this case, you cannot even simulate the type class with a trait that is inherited from, because traits only express methods on an object, meaning that they have to be "owned" by a class, so you cannot put the definition of a "constructor that takes a string as the only argument" (which is what "read" might be called in OOP languages) in a trait.
Instead, you have to simulate type classes yourself. This is done like so (equivalent Haskell code in comments):
// class Read a where read :: String -> a
trait Read[A] { def read(s: String): A }
// instance Read Person where read = ... parser for Person ...
implicit object ReadPerson extends Read[Person] {
def read(s: String): Person = ... parser for Person ...
}
Then, when you have a method that depends on the type class, you have to specify it as an implicit context:
// readList :: Read a => [String] -> [a]
// readList ss = map read ss
def readList[A: Read] (ss: List[String]): List[A] = {
val r = implicitly[Read[A]] // Get the class instance of Read for type A
ss.map(r.read _)
}
The user would probably like a polymorphic method like this for ease of use:
object read {
def apply[A: Read](s: String): A = implicitly[Read[A]].read(s)
}
Then one can just write:
val person: Person = read[Person]("Person(Bob,42)")
I am not aware of any standard implementation(s) for this type class, in particular.
Also, a disclaimer: I don't have a Scala compiler and haven't used the language for years, so I can't guarantee that this code compiles.
Starting Scala 2.13, it's possible to pattern match a Strings by unapplying a string interpolator:
// case class Person(name: String, age: Int)
"Person(Bob,42)" match { case s"Person($name,$age)" => Person(name, age.toInt) }
// Person("Bob", 42)
Note that you can also use regexes within the extractor.
Which in this case, helps for instance to match on "Person(Bob, 42)" (age with a leading space) and to force age to be an integer:
val Age = "[ ?*](\\d+)".r
"Person(Bob, 42)" match {
case s"Person($name,${Age(age)})" => Some(Person(name, age.toInt))
case _ => None
}
// Person = Some(Person(Bob,42))
The answers on this question are somewhat outdated. Scala has picked up some new features, notably typeclasses and macros, to make this more easily possible.
Using the Scala Pickling library, you can serialize/deserialize arbitrary classes to and from various serialization formats:
import scala.pickling._
import json._
case class Person(name: String, age: Int)
val person1 = Person("Bob", 42)
val str = person1.pickle.value // { tpe: "Person", name: "Bob", age: 42 }
val person2 = JSONPickle(str).unpickle[Person]
assert(person1 == person2) // Works!
The serializers/deserializers are automatically generated at compile time, so no reflection! If you need to parse case classes using a specific format (such as the case class toString format), you can extend this system with your own formats.
The uPickle library offers a solution for this problem.
Scala uses Java's serialization stuff, with no String representation.