The dsinfo library lets you access the names of values from the context of where a function is written using Scala 2 macros. The example they give is that if you have something like
val name = myFunction(x, y)
myFunction will actually be passed the name of its val in addition to the other arguments, i.e., myFunction("name", x, y).
This is very useful for DSLs where you'd like named values for error reporting or other kinds of encoding. The only other option seems to explicitly pass the name as a String, which can lead to unintentional mismatches.
Is this possible with Scala 3 macros, and if so, how do you "climb up" the tree at the macro's use location to find its id?
In Scala 3 there is no c.macroApplication. Only Position.ofMacroExpansion instead of a tree. But we can analyze Symbol.spliceOwner.maybeOwner. I presume that scalacOptions += "-Yretain-trees" is switched on.
import scala.annotation.experimental
import scala.quoted.*
object Macro {
inline def makeCallWithName[T](inline methodName: String): T =
${makeCallWithNameImpl[T]('methodName)}
#experimental
def makeCallWithNameImpl[T](methodName: Expr[String])(using Quotes, Type[T]): Expr[T] = {
import quotes.reflect.*
println(Position.ofMacroExpansion.sourceCode)//Some(twoargs(1, "one"))
val methodNameStr = methodName.valueOrAbort
val strs = methodNameStr.split('.')
val moduleName = strs.init.mkString(".")
val moduleSymbol = Symbol.requiredModule(moduleName)
val shortMethodName = strs.last
val ident = Ident(TermRef(moduleSymbol.termRef, shortMethodName))
val (ownerName, ownerRhs) = Symbol.spliceOwner.maybeOwner.tree match {
case ValDef(name, tpt, Some(rhs)) => (name, rhs)
case DefDef(name, paramss, tpt, Some(rhs)) => (name, rhs)
case t => report.errorAndAbort(s"can't find RHS of ${t.show}")
}
val treeAccumulator = new TreeAccumulator[Option[Tree]] {
override def foldTree(acc: Option[Tree], tree: Tree)(owner: Symbol): Option[Tree] = tree match {
case Apply(fun, args) if fun.symbol.fullName == "App$.twoargs" =>
Some(Apply(ident, Literal(StringConstant(ownerName)) :: args))
case _ => foldOverTree(acc, tree)(owner)
}
}
treeAccumulator.foldTree(None, ownerRhs)(ownerRhs.symbol)
.getOrElse(report.errorAndAbort(s"can't find twoargs in RHS: ${ownerRhs.show}"))
.asExprOf[T]
}
}
Usage:
package mypackage
case class TwoArgs(name : String, i : Int, s : String)
import mypackage.TwoArgs
object App {
inline def twoargs(i: Int, s: String) =
Macro.makeCallWithName[TwoArgs]("mypackage.TwoArgs.apply")
def x() = twoargs(1, "one") // TwoArgs("x", 1, "one")
def aMethod() = {
val y = twoargs(2, "two") // TwoArgs("y", 2, "two")
}
val z = Some(twoargs(3, "three")) // Some(TwoArgs("z", 3, "three"))
}
dsinfo also handles the name twoargs at call site (as template $macro) but I didn't implement this. I guess the name (if necessary) can be obtained from Position.ofMacroExpansion.sourceCode.
Update. Here is implementation handling name of inline method (e.g. twoargs) using Scalameta + Semanticdb besides Scala 3 macros.
import mypackage.TwoArgs
object App {
inline def twoargs(i: Int, s: String) =
Macro.makeCallWithName[TwoArgs]("mypackage.TwoArgs.apply")
inline def twoargs1(i: Int, s: String) =
Macro.makeCallWithName[TwoArgs]("mypackage.TwoArgs.apply")
def x() = twoargs(1, "one") // TwoArgs("x", 1, "one")
def aMethod() = {
val y = twoargs(2, "two") // TwoArgs("y", 2, "two")
}
val z = Some(twoargs1(3, "three")) // Some(TwoArgs("z", 3, "three"))
}
package mypackage
case class TwoArgs(name : String, i : Int, s : String)
import scala.annotation.experimental
import scala.quoted.*
object Macro {
inline def makeCallWithName[T](inline methodName: String): T =
${makeCallWithNameImpl[T]('methodName)}
#experimental
def makeCallWithNameImpl[T](methodName: Expr[String])(using Quotes, Type[T]): Expr[T] = {
import quotes.reflect.*
val position = Position.ofMacroExpansion
val scalaFile = position.sourceFile.getJPath.getOrElse(
report.errorAndAbort(s"maybe virtual file, can't find path to position $position")
)
val inlineMethodSymbol =
new SemanticdbInspector(scalaFile)
.getInlineMethodSymbol(position.start, position.end)
.getOrElse(report.errorAndAbort(s"can't find Scalameta symbol at position (${position.startLine},${position.startColumn})..(${position.endLine},${position.endColumn})=$position"))
val methodNameStr = methodName.valueOrAbort
val strs = methodNameStr.split('.')
val moduleName = strs.init.mkString(".")
val moduleSymbol = Symbol.requiredModule(moduleName)
val shortMethodName = strs.last
val ident = Ident(TermRef(moduleSymbol.termRef, shortMethodName))
val owner = Symbol.spliceOwner.maybeOwner
val macroApplication: Option[Tree] = {
val (ownerName, ownerRhs) = owner.tree match {
case ValDef(name, tpt, Some(rhs)) => (name, rhs)
case DefDef(name, paramss, tpt, Some(rhs)) => (name, rhs)
case t => report.errorAndAbort(s"can't find RHS of ${t.show}")
}
val treeAccumulator = new TreeAccumulator[Option[Tree]] {
override def foldTree(acc: Option[Tree], tree: Tree)(owner: Symbol): Option[Tree] = tree match {
case Apply(fun, args) if tree.pos == position /* fun.symbol.fullName == inlineMethodSymbol */ =>
Some(Apply(ident, Literal(StringConstant(ownerName)) :: args))
case _ => foldOverTree(acc, tree)(owner)
}
}
treeAccumulator.foldTree(None, ownerRhs)(ownerRhs.symbol)
}
val res = macroApplication
.getOrElse(report.errorAndAbort(s"can't find application of $inlineMethodSymbol in RHS of $owner"))
report.info(res.show)
res.asExprOf[T]
}
}
import java.nio.file.{Path, Paths}
import scala.io
import scala.io.BufferedSource
import scala.meta.*
import scala.meta.interactive.InteractiveSemanticdb
import scala.meta.internal.semanticdb.{ClassSignature, Locator, Range, SymbolInformation, SymbolOccurrence, TextDocument, TypeRef}
class SemanticdbInspector(val scalaFile: Path) {
val scalaFileStr = scalaFile.toString
var textDocuments: Seq[TextDocument] = Seq()
Locator(
Paths.get(scalaFileStr + ".semanticdb")
)((path, textDocs) => {
textDocuments ++= textDocs.documents
})
val bufferedSource: BufferedSource = io.Source.fromFile(scalaFileStr)
val source = try bufferedSource.mkString finally bufferedSource.close()
extension (tree: Tree) {
def occurence: Option[SymbolOccurrence] = {
val treeRange = Range(tree.pos.startLine, tree.pos.startColumn, tree.pos.endLine, tree.pos.endColumn)
textDocuments.flatMap(_.occurrences)
.find(_.range.exists(occurrenceRange => treeRange == occurrenceRange))
}
def info: Option[SymbolInformation] = occurence.flatMap(_.symbol.info)
}
extension (symbol: String) {
def info: Option[SymbolInformation] = textDocuments.flatMap(_.symbols).find(_.symbol == symbol)
}
def getInlineMethodSymbol(startOffset: Int, endOffset: Int): Option[String] = {
def translateScalametaToMacro3(symbol: String): String =
symbol
.stripPrefix("_empty_/")
.stripSuffix("().")
.replace(".", "$.")
.replace("/", ".")
dialects.Scala3(source).parse[Source].get.collect {
case t#Term.Apply(fun, args) if t.pos.start == startOffset && t.pos.end == endOffset =>
fun.info.map(_.symbol)
}.headOption.flatten.map(translateScalametaToMacro3)
}
}
lazy val scala3V = "3.1.3"
lazy val scala2V = "2.13.8"
lazy val scalametaV = "4.5.13"
lazy val root = project
.in(file("."))
.settings(
name := "scala3demo",
version := "0.1.0-SNAPSHOT",
scalaVersion := scala3V,
libraryDependencies ++= Seq(
"org.scalameta" %% "scalameta" % scalametaV cross CrossVersion.for3Use2_13,
"org.scalameta" % s"semanticdb-scalac_$scala2V" % scalametaV,
),
scalacOptions ++= Seq(
"-Yretain-trees",
),
semanticdbEnabled := true,
)
By the way, Semantidb can't be replaced by Tasty here because when a macro in App is being expanded, the file App.scala.semantidb already exists (it's generated early, at frontend phase of compilation) but App.tasty hasn't yet (it appears when App has been compiled i.e. after expansion of the macro, at pickler phase).
.scala.semanticdb file will appear even if .scala file doesn't compile (e.g. if there is an error in macro expansion) but .tasty file won't.
scala.meta parent of parent of Defn.Object
Is it possible to using macro to modify the generated code of structural-typing instance invocation?
Scala conditional compilation
Macro annotation to override toString of Scala function
How to merge multiple imports in scala?
How to get the type of a variable with scalameta if the decltpe is empty?
See also https://github.com/lampepfl/dotty-macro-examples/tree/main/accessEnclosingParameters
Simplified version:
import scala.quoted.*
inline def makeCallWithName[T](inline methodName: String): T =
${makeCallWithNameImpl[T]('methodName)}
def makeCallWithNameImpl[T](methodName: Expr[String])(using Quotes, Type[T]): Expr[T] = {
import quotes.reflect.*
val position = Position.ofMacroExpansion
val methodNameStr = methodName.valueOrAbort
val strs = methodNameStr.split('.')
val moduleName = strs.init.mkString(".")
val moduleSymbol = Symbol.requiredModule(moduleName)
val shortMethodName = strs.last
val ident = Ident(TermRef(moduleSymbol.termRef, shortMethodName))
val owner0 = Symbol.spliceOwner.maybeOwner
val ownerName = owner0.tree match {
case ValDef(name, _, _) => name
case DefDef(name, _, _, _) => name
case t => report.errorAndAbort(s"unexpected tree shape: ${t.show}")
}
val owner = if owner0.isLocalDummy then owner0.maybeOwner else owner0
val macroApplication: Option[Tree] = {
val treeAccumulator = new TreeAccumulator[Option[Tree]] {
override def foldTree(acc: Option[Tree], tree: Tree)(owner: Symbol): Option[Tree] = tree match {
case _ if tree.pos == position => Some(tree)
case _ => foldOverTree(acc, tree)(owner)
}
}
treeAccumulator.foldTree(None, owner.tree)(owner)
}
val res = macroApplication.getOrElse(
report.errorAndAbort("can't find macro application")
) match {
case Apply(_, args) => Apply(ident, Literal(StringConstant(ownerName)) :: args)
case t => report.errorAndAbort(s"unexpected shape of macro application: ${t.show}")
}
report.info(res.show)
res.asExprOf[T]
}
I have a List with all the methods in it example
List("method1","method2","method3")
and I have the definition for all those methods mentioned in above list
def method1(){ //definition}
def method2(){ //definition}
def method3(){ //definition}
Instead of specifying those functions to run one by one like below
method1()
method2()
method3()
I want to loop through List("method1","method2","method3") and execute those methods
If all of the target methods have the same type profile (same argument number/types, same return type) ...
def method1() = println("one")
def method2() = println("two")
def method3() = println("three")
... then all you need is a String-to-method translator.
val translate : Map[String,Function0[Unit]] =
Map("method3" -> method3
,"method1" -> method1
,"method2" -> method2)
usage:
List("method1","method2","method3")
.foreach(translate(_)())
//one
//two
//three
If method names are known at compile time you can write a macro
import scala.language.experimental.macros
import scala.reflect.macros.blackbox
def executeAll(methodNames: List[String]): Unit = macro executeAllImpl
def executeAllImpl(c: blackbox.Context)(methodNames: c.Tree): c.Tree = {
import c.universe._
c.eval(c.Expr[List[String]](c.untypecheck(methodNames)))
.foldLeft[Tree](q"()")((tree, methodName) => q"$tree; ${TermName(methodName)}()")
}
def method1(): Unit = println(1)
def method2(): Unit = println(2)
def method3(): Unit = println(3)
executeAll(List("method1","method2","method3")) //1 2 3
//Warning:scalac: {
// {
// {
// ();
// method1()
// };
// method2()
// };
// method3()
//}
or
import scala.language.experimental.macros
import scala.reflect.macros.blackbox
def executeAll(methodNames: List[String]): Unit = macro executeAllImpl
def executeAllImpl(c: blackbox.Context)(methodNames: c.Tree): c.Tree = {
import c.universe._
val calls = c.eval(c.Expr[List[String]](c.untypecheck(methodNames)))
.map(methodName => q"${TermName(methodName)}()")
q"..$calls"
}
def method1(): Unit = println(1)
def method2(): Unit = println(2)
def method3(): Unit = println(3)
executeAll(List("method1","method2","method3")) //1 2 3
//Warning:scalac: {
// method1();
// method2();
// method3()
//}
(the tree generated is slightly different but result of execution is the same).
Or if method names are known only at runtime you can use either Scala reflection
def executeAll(methodNames: List[String]): Unit = {
import scala.reflect.runtime
import scala.reflect.runtime.universe._
methodNames.foreach(methodName => {
val methodSymbol = typeOf[SomeObject.type].decl(TermName(methodName)).asMethod
runtime.currentMirror.reflect(SomeObject).reflectMethod(methodSymbol)()
})
}
object SomeObject {
def method1(): Unit = println(1)
def method2(): Unit = println(2)
def method3(): Unit = println(3)
}
val methodNames = List("method1", "method2", "method3")
executeAll(methodNames) //1 2 3
or Java reflection
def executeAll(methodNames: List[String]): Unit =
methodNames.foreach(methodName =>
SomeObject.getClass.getMethod(methodName).invoke(SomeObject)
)
object SomeObject {
def method1(): Unit = println(1)
def method2(): Unit = println(2)
def method3(): Unit = println(3)
}
val methodNames = List("method1", "method2", "method3")
executeAll(methodNames) //1 2 3
I have an ostensibly simple macro problem that I’ve been banging my head against for a few hours, with no luck. Perhaps someone with more experience can help.
I have the following macro:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object MacroObject {
def run(s: String): Unit =
macro runImpl
def runImpl(c: Context)(s: c.Tree): c.Tree = {
import c.universe._
println(s) // <-- I need the macro to know the value of s at compile time
q"()"
}
}
The problem is this: I’d like the macro to know the value s that is passed to it — not an AST of s, but the value of s itself. Specifically, I’d like it to have this behavior:
def runTheMacro(str: String): Unit = MacroObject.run(str)
final val HardCodedString1 = "Hello, world!"
runTheMacro(HardCodedString1) // the macro should print "Hello, world!"
// to the console during macro expansion
final val HardCodedString2 = "So long!"
runTheMacro(HardCodedString2) // the macro should print "So long!"
// to the console during macro expansion
It is guaranteed that the only strings that will be passed to runTheMacro are hard-coded constant values (i.e., known at compile-time).
Is this possible, and how does one do this?
--
Edit: There are also the following constraints:
It must be a blackbox macro.
The macro signature must use c.Trees, not c.Expr[_]s (legacy code; can’t change that part)
I do have a toolbox within the macro at my disposal if needed:
import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox
private val toolbox = currentMirror.mkToolBox()
/** Evaluate the given code fragment at compile time. */
private def eval[A](code: String): A = {
import scala.reflect.runtime.{universe => u}
val uTree: u.Tree = toolbox.parse(code)
toolbox.eval(uTree).asInstanceOf[A]
}
Your eval is runtime reflection's eval, compile-time macro's eval would be c.eval.
"Hello, world!" in
final val HardCodedString1 = "Hello, world!"
runTheMacro(HardCodedString1)
is a runtime value of HardCodedString1.
You can't have access to runtime value at compile time.
At compile time the tree of string HardCodedString1 just doesn't know anything about right hand side of the val tree.
Scala: what can code in Context.eval reference?
If you really need to use a runtime value inside the tree of your program you have to postpone its compilation till runtime
import scala.reflect.runtime.currentMirror
import scala.reflect.runtime.universe._
import scala.tools.reflect.ToolBox
object MacroObject {
val toolbox = currentMirror.mkToolBox()
def run(s: String): Unit = {
toolbox.eval(q"""
println($s)
()
""")
}
}
runTheMacro(HardCodedString1)//Hello, world!
runTheMacro(HardCodedString2)//So long!
Alternatively at compile time you can somehow find the tree of enclosing class and look inside it for the val tree and take its right hand side
def runImpl(c: blackbox.Context)(s: c.Tree): c.Tree = {
import c.universe._
var rhs: Tree = null
val traverser = new Traverser {
override def traverse(tree: Tree): Unit = {
tree match {
case q"$mods val $tname: $tpt = $expr" if tname == TermName("HardCodedString1") =>
rhs = expr
case _ => ()
}
super.traverse(tree)
}
}
traverser.traverse(c.enclosingClass) // deprecated
val rhsStr =
if (rhs != null) c.eval[String](c.Expr(c.untypecheck(rhs.duplicate)))
else c.abort(c.enclosingPosition, "no val HardCodedString1 defined")
println(rhsStr)
q"()"
}
runTheMacro(HardCodedString1)//Warning:scalac: Hello, world!
Or for all such variables
def runImpl(c: blackbox.Context)(s: c.Tree): c.Tree = {
import c.universe._
val sEvaluated =
try {
c.eval[String](c.Expr(c.untypecheck(s.duplicate)))
} catch {
case e: IllegalArgumentException if e.getMessage.startsWith("Could not find proxy") =>
s match {
case q"$sName" =>
var rhs: Tree = null
val traverser = new Traverser {
override def traverse(tree: Tree): Unit = {
tree match {
case q"$mods val $tname: $tpt = $expr" if tname == sName =>
rhs = expr
case _ => ()
}
super.traverse(tree)
}
}
traverser.traverse(c.enclosingClass)
if (rhs != null) c.eval[String](c.Expr(c.untypecheck(rhs.duplicate)))
else c.abort(c.enclosingPosition, s"no val $sName defined")
case _ => c.abort(c.enclosingPosition, s"unsupported tree $s")
}
}
println(sEvaluated)
q"()"
}
MacroObject.run(HardCodedString1) //Warning:scalac: Hello, world!
MacroObject.run(HardCodedString2) //Warning:scalac: So long!
runTheMacro will not work in this case: Error: no val str defined.
To make it work you can make it a macro too
def runTheMacro(str: String): Unit = macro runTheMacroImpl
def runTheMacroImpl(c: blackbox.Context)(str: c.Tree): c.Tree = {
import c.universe._
q"MacroObject.run($str)"
}
runTheMacro(HardCodedString1) //Warning:scalac: Hello, world!
runTheMacro(HardCodedString2) //Warning:scalac: So long!
I have the following code for:
#compileTimeOnly("enable macro paradise to expand macro annotations")
class replace extends StaticAnnotation {
def macroTransform(annottees: Any*) = macro replaceImpl.replace
}
object replaceImpl {
def replace(c: Context)(annottees: c.Expr[Any]*): c.Expr[Any] = {
import c.universe._
// ?
c.Expr[Any](q"")
}
}
Via this code, i want to replace variable name (x) in next example of usage:
#replace
def foo(x: Int) = x + 1
it's simple method, but if i have a big method with many expression, what are the simpest way to replace variable name (from x to y for example)?
After some investigation, i need use Transfomer#transform method, some like this:
val t = new Transformer {
override def transform(tree: Tree) = {
val nt = tree match {
case Ident(x) => Ident(someNewName)
case x => x
}
super.transform(newTree)
}
}
// and then
val result = t.transform(inputTree)
I am trying to write a proxy macro using scala macros. I want to be able to proxy a trait X and return instances of X that invoke a function for all methods of X.
Here is what I did so far. Say we want to proxy the trait TheTrait (which is defined below), we can run ProxyMacro.proxy passing a function that will be called for all invocations of the proxy methods.
trait TheTrait
{
def myMethod(x: String)(y: Int): String
}
val proxy = ProxyMacro.proxy[TheTrait] {
case ("myMethod", args) =>
"ok"
}
println(proxy.myMethod("hello")(5))
The implementation so far is this:
package macrotests
import scala.language.experimental.macros
import scala.reflect.macros.whitebox.Context
object ProxyMacro
{
type Implementor = (String, Any) => Any
def proxy[T](implementor: Implementor): T = macro impl[T]
def impl[T: c.WeakTypeTag](c: Context)(implementor: c.Expr[Implementor]): c.Expr[T] = {
import c.universe._
val tpe = weakTypeOf[T]
val decls = tpe.decls.map { decl =>
val termName = decl.name.toTermName
val method = decl.asMethod
val params = method.paramLists.map(_.map(s => internal.valDef(s)))
val paramVars = method.paramLists.flatMap(_.map { s =>
internal.captureVariable(s)
internal.referenceCapturedVariable(s)
})
q""" def $termName (...$params) = {
$implementor (${termName.toString}, List(..${paramVars}) ).asInstanceOf[${method.returnType}]
}"""
}
c.Expr[T] {
q"""
new $tpe {
..$decls
}
"""
}
}
}
But there is a problem. This doesn't compile due to List(..${paramVars}). This should just create a list with all the values of the method arguments.
But I get a compilation issue (not worth pasting it) on that line.
How can I convert the list of method arguments to their values?
showInfo is useful when you debug macro
def showInfo(s: String) =
c.info(c.enclosingPosition, s.split("\n").mkString("\n |---macro info---\n |", "\n |", ""), true)
change
val paramVars = method.paramLists.flatMap(_.map { s =>
internal.captureVariable(s)
internal.referenceCapturedVariable(s)
})(this result is List(x0$1, x1$1))
to
val paramVars = method.paramLists.flatMap(_.map { s =>
s.name
})(this result is List(x, y))