Scala Macros: Convert/parse a Tree to a Name - scala

This is a simplified example but the problem remains the same.
I want to achieve this using macros (scala based pseudocode):
(a: Int) => {
val z = "toShort"
a.z
}
If I reify it, I would obtain something similar to this:
Function(
List(
ValDef(
Modifiers(Flag.PARAM),
newTermName("a"),
Ident(scala.Int),
EmptyTree
)
),
Block(
List(
ValDef(
Modifiers(),
newTermName("z"),
TypeTree(),
Literal(Constant("toShort"))
)
),
Apply(
Select(
Ident(newTermName("a")),
newTermName("toShort")
),
List()
)
)
)
I dont know how to access to a value and then use it as a TermName.
I tried replacing newTermName("toShort") with newTermName(c.Expr[String](Select(Ident(newTermName("z")))).splice) but the compiler doesn't seem to like:
exception during macro expansion:
java.lang.UnsupportedOperationException: the function you're calling has not been spliced by > the compiler.
this means there is a cross-stage evaluation involved, and it needs to be invoked explicitly.
if you're sure this is not an oversight, add scala-compiler.jar to the classpath,
import scala.tools.reflect.Eval and call <your expr>.eval instead.
I've also tried 'eval' as suggested by compiler: newTermName(c.eval(c.Expr[String](...)) but neither worked.
How could I convert a tree like Select(Ident(newTermName("z"))) (which is a access to a value of a local val) to a Name a string which can be used as a parameter for newTermName? Is it possible?
UPDATE:
Here the real problem brought to you as a gist!
Thanks in advance,

I have a hard time understanding what you're trying to achieve, and why you are using Trees everywhere. Trees are really low level, hard to use, tricky, and it is very difficult to understand what the code does. Quasiquotes (http://docs.scala-lang.org/overviews/macros/quasiquotes.html) are the way to go indeed and you can use them on scala 2.10.x production release thanks to the macro paradise plugin (http://docs.scala-lang.org/overviews/macros/paradise.html). The you can simply write q"(a: Int) => {val z = "toShort"; a.z}" and you directly get the tree expression you just typed.
To answer your question, the first point is to remember that macros are evaluated at compile time. They therefore can not generate code which depends on a runtime value. This is why the compiler is complaining about your splice. But if you pass a value which can be computed at compile time, typically a literal, then you can use eval to get its value within your macro code. Eval does suffer a bug though, as indicated in scaladoc. It should only be called on untyped trees. So the way to call eval on an s: c.Expr[String] expression would be val s2 = c.eval(c.Expr[String](c.resetAllAttrs(c.tree.duplicate))) which gives you a String you can then use normally in your code, for instance q"(a: Int) => a.${newTermName(s2)}".
To put it all together, let's imagine you to create a macro that'll output a string value from an object and one of its String field. It'll give something like
def attr[A](a: A, field: String): String = macro attrImpl[A]
def attrImpl[A: c.WeakTypeTag](c: Context)(a: c.Expr[A], field: c.Expr[String]) = {
import c.universe._
val s = c.eval(c.Expr[String](c.resetAllAttrs(field.tree.duplicate)))
c.Expr[String](q"a.${newTermName(s)}")
}
REPL session test:
scala> object a { val field1 = "field1"; val field2 = "field2" }
defined module a
scala> attr(a, "field1")
res0: String = field1
scala> attr(a, "field2")
res1: String = field2
To understand the difference between compile time and runtime, you can meditate about the following result in REPL ;-)
scala> val s = "field1"; attr(a, s)
error: exception during macro expansion:
scala.tools.reflect.ToolBoxError: reflective compilation has failed:
$iw is not an enclosing class
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.throwIfErrors(ToolBoxFactory.scala:311)
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.compile(ToolBoxFactory.scala:244)
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl.compile(ToolBoxFactory.scala:408)
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl.eval(ToolBoxFactory.scala:411)
at scala.reflect.macros.runtime.Evals$class.eval(Evals.scala:16)
at scala.reflect.macros.runtime.Context.eval(Context.scala:6)
at .attrImpl(<console>:14)
scala> val s = "field1"
s: String = field1
scala> attr(a, s)
res3: String = field1
Hope it helps ;))

Related

Scala 3 Macros: How to invoke a method obtained as a `Symbol` in a quoted code block?

In a Scala 3 macro that takes a type parameter T, you can use TypeRepr.of[T] and the new Scala 3 reflection API to explore the companionClass of T, and find the Symbol for an arbitrary method on that companion class (eg companionClass.declarations.find(_.name == "list") to find a list() method).
Given the Symbol for a companion object method, how would you then invoke that method within a quoted code block?
I'm guessing I would need to convert that Symbol to a Expr[T], but I don't know how to do that!
In a Scala 2 macro, the invocation of a listMethod of type c.universe.Symbol in a q"..." quasiquote seems pretty simple - just say $listMethod, and then you can start mapping on the resulting list, eg:
q"""
$listMethod.map(_.toString)
"""
Trying to do a similar thing in a Scala 3 macro gets an error like this:
[error] 27 | ${listMethod}.map(_.toString)
[error] | ^^^^^^^^^^
[error] | Found: (listMethod : x$1.reflect.Symbol)
[error] | Required: quoted.Expr[Any]
What is the correct code to get this working in Scala 3?
You can see more code context in the AvroSerialisableMacro classes (Scala 2 compiles, Scala 3 currently nowhere near!) here: https://github.com/guardian/marley/pull/77/files
First, let's talk how to call a method using symbol name in general.
You might need Select. You can call obtain it in a a few different ways, e.g.:
New(TypeTree.of[YourType]).select(primaryConstructor) // when you want to create something
expression.asTerm.select(method) // when you want to call it on something
Once you selected method you can provide arguments:
select.appliedToArgs(args) // if there is only 1 argument list
select.appliedToArgss(args) // if there is more than one argument list
// (type parameter list is listed in paramSymss
// but shouldn't be used here, so filter it out!)
select.appliedToNone // if this is a method like "def method(): T"
// (single, but empty, parameter list)
select.appliedToArgss(Nil) // is this is a method like "def method: T"
// (with not even empty parameter list)
There are also other methods like appliedToType, appliedToTypeTrees, but if you have a method name as a Symbol and want to use it to call something this should be a good starting point.
And remember that source code of Quotes is your friend, so even when your IDE doesn't give you any suggestions, it can point you towards some solution.
In theory these methods are defined on Term rather than Select (<: Term) but your use case will be most likely picking an expression and calling a method on it with some parameters. So a full example could be e.g.
val expression: Expr[Input]
val method: Symbol
val args: List[Term]
// (input: Input).method(args) : Output
expression // Expr[Input]
.asTerm // Term
.select(method) // Select
.appliedToArgs(args) // Term
.asExpr // Expr[?]
.asExprOf[Output] // Expr[Output]
Obviously, proving that the expression can call method and making sure that types of Terms in args match allowed types of values that you pass to the method, is on you. It is a bit more hassle than it was in Scala 2 since quotes allow you to work with Type[T] and Expr[T] only, so anything that doesn't fall under that category has to be implemented with macros/Tasty ADT until you get to the point that you can return Expr inside ${}.
That said, the example you linked shows that these calls are rather hardcoded, so you don't have to look up Symbols and call them. Your code will most likely do away with:
// T <: ThriftEnum
// Creating companion's Expr can be done with asExprOf called on
// Ref from Dmytro Mitin's answer
def findCompanionOfThisOrParent(): Expr[ThriftEnumObject[T]] = ...
// _Expr_ with the List value you constructed instead of Symbol!
val listOfValues: Expr[List[T]] = '{
${ findCompanionOfThisOrParent() }.list
}
// once you have an Expr you don't have to do any magic
// to call a method on it, Quotes works nice
'{
...
val valueMap = Map(${ listOfValues }.map(x => x ->
org.apache.avro.generic.GenericData.get.createEnum(
com.gu.marley.enumsymbols.SnakesOnACamel.toSnake(x.name), schemaInstance)
): _*)
...
}
Difference between Scala 2 quasiquotes and Scala 3 quotations is that the former must compile during compile time of the main code using macros (i.e. during macro expansion, macro runtime) while the latter must compile earlier, at macro compile time. So Scala 3 quotations '{...}/${...} are more like Scala 2 reify{...}/.splice than Scala 2 quasiquotes q"..."/${...}.
`tq` equivalent in Scala 3 macros
You have to re-create AST. Let's see what shape AST should have:
object B:
def fff(): Unit = ()
import scala.quoted.*
inline def foo(): Unit = ${fooImpl}
def fooImpl(using Quotes): Expr[Unit] =
import quotes.reflect.*
println('{B.fff()}.asTerm.show(using Printer.TreeStructure))
'{()}
foo() // ... Apply(Select(Ident("B"), "fff"), Nil)
So in order to re-create AST try to use Apply(...) and Select.unique(..., "list"):
import scala.quoted.*
inline def foo[T](): Unit = ${fooImpl[T]}
def fooImpl[T: Type](using Quotes): Expr[Unit] =
import quotes.reflect.*
val sym = TypeRepr.of[T].typeSymbol
'{
println("aaa")
${
Apply(
Select.unique(
Ref(sym.companionModule),
"list"
),
Nil
).asExprOf[Unit]
}
}
Testing (in a different file):
class A
object A {
def list(): Unit = println("list")
}
foo[A]()
//scalac: {
// scala.Predef.println("aaa")
// A.list()
//}
// prints at runtime:
// aaa
// list
Using method symbol rather than its name and using convenience methods rather than AST nodes directly, you can rewrite fooImpl as
def fooImpl[T: Type](using Quotes): Expr[Unit] =
import quotes.reflect.*
val sym = TypeRepr.of[T].typeSymbol
val listMethod = sym.companionClass.declarations.find(_.name == "list").get
'{
println("aaa")
${
Ref(sym.companionModule)
.select(listMethod)
.appliedToArgs(Nil)
.asExprOf[Unit]
}
}
This is just an example how to create an AST. You should use your actual return type of def list() instead of Unit in .asExprOf[Unit].
How to get the list of default fields values for typed case class?
scala 3 macro how to implement generic trait

Read a tuple from a file in Scala

my Task is to read registrations from a file given like:
Keri,345246,2
Ingar,488058,2
Almeta,422016,1
and insert them into a list(Tuple of (String, Int, Int).
So far I wrote this:
The problem is that I don‘t understand why I can't try to cast value2 and value3 to Int even tho they should be Strings because they come from an Array of Strings. Could someone tell me, what my mistake is, I am relatively new to Scala
What is the point of using Scala if you are going to write Java code?
This is how you would properly read a file as a List of case classes.
import scala.io.Source
import scala.util.Using
// Use proper names for the fields.
final case class Registration(field1: String, field2: Int, field3: Int)
// You may change the error handling logic.
def readRegistrationsFromFile(fileName: String): List[Registration] =
Using(Source.fromFile(fileName)) { source =>
source.getLines().map(line => line.split(',').toList).flatMap {
case field1Raw :: field2Raw :: field3Raw :: Nil =>
for {
field2 <- field2Raw.toIntOption
field3 <- field3Raw.toIntOption
} yield Registration(field1 = field1Raw.trim, field2, field3)
case _ =>
None
}.toList
}.getOrElse(default = List.empty)
(feel free to ask any question you may have about this code)
In Scala, in order to convert a String to an Int you need explicit casting.
This can be achieved like this if you are sure the string can be parsed into a integer:
val values = values(1).toInt
If you cannot trust the input (and you probably should not), you can use .toIntOption which will give you a Option[Int] defined if the value was converted successfully or undefined if the string did not represent an integer.
The previous answers are correct. I would add a few more points.
saveContent is declared as a val. This is means it cannot be changed (assigned another value). You can use the Scala REPL (command-line) tool to check:
scala> val saveContent = Nil
val v: collection.immutable.Nil.type = List()
scala> saveContent = 3
^
error: reassignment to val
Instead, you could use a var, although it would be more idiomatic to have an overall pattern like the one provided by Luis Miguel's answer - with pattern-matching and a for-comprehension.
You can use the Scala REPL to check the types of the variables, too. Splitting a String will always lead to more Strings, not Ints, etc.
> val values = "a,2,3".split(",")
val values: Array[String] = Array(a, 2, 3)
> values(2)
val res3: String = 3
This is why a cast like Gael's is necessary.
Array-type access is done with parentheses and not square brackets, in Scala. See above, and http://scalatutorials.com/tour/interactive_tour_of_scala_lists for more details.

Why Unit is a supertype of anything else?

Here's an example:
$ scala
Welcome to Scala 2.11.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_112).
Type in expressions for evaluation. Or try :help.
scala> val a: Unit = 1
<console>:11: warning: a pure expression does nothing in statement position; you may be omitting necessary parentheses
val a: Unit = 1
^
a: Unit = ()
In Scala documentation:
There is only one value of type Unit, ()
Why is Scala compiler silently coercing values to Unit?
A bit of context: I used Future[Unit] type to describe some procedure which does not return anything. And since Future[Unit] is now effectively a subtype of Unit, I got bit by some funny bugs (someFuture.map(a => Future(a)) silently skips calling the operation instead of giving compilation warning). What am I supposed to use as a type of operation that does not return any meaningful result?
Unit is not a supertype of other types. What happens instead is called value discarding: when the expected type of an expression e is Unit, the compiler replaces it by {e; ()}. This is done to make some behavior more familiar. E.g.
val sb = new StringBuilder
val strings: List[String] = ...
for (str <- strings) { sb.append(str) }
By analogy with for loops in other languages, we would expect it to compile. But without value discarding it wouldn't: this is equivalent to strings.foreach(str => sb.append(str)), the type of str => sb.append(str) is String => StringBuilder (because all append methods on StringBuilder return the builder itself) and foreach on List[String] takes String => Unit.
You can add -Ywarn-value-discard compiler option to warn you when it happens (and write for (sb <- sbOpt) { sb.append("a"); () } explicitly).
Or you can actually go with a trick of defining your own Unit (possibly changing the name to avoid confusion for anyone reading your code):
object Unit
type Unit = Unit.type
implicit def unit2scalaunit(a: Unit): scala.Unit = ()
implicit def scalaunit2unit(a: scala.Unit): Unit = Unit
This should avoid running into the problem with Future you describe.
Unit is not a supertype of everything! Scala actually has a pretty wide variety of conversions that happen automatically and this is one of them. From section 6.26.1 Value Conversions of the Scala language spec, one of the conversions is
Value Discarding
If e has some value type and the expected type is Unit, e is converted
to the expected type by embedding it in the term { e; () }.
So when you write something like val a: Unit = 1, it gets processed into val a: Unit = { 1; () }, which is quite different. The warning is actually very helpful here - it is warning you that you probably did something wrong: the expression you are trying to put into statement position is pure (has no side-effects), so executing it has no effect (except possibly to cause the program to diverge) on the final output.

In Scala what is the correct way to fix "error: forward reference extends over definition of value"?

In Scala what is the correct way to implement something like this that produces the error "forward reference extends over definition of value b"?
object a {
def main(args: Array[String]) {
val b: Map[Int, () => Int] = Map(5 -> { () => b.size })
println(b(5)())
}
}
Making b lazy works but that doesn't seem like the correct solution.
EDIT: The other question (What does "Forward reference extends over definition of value" mean in Scala?) is about a bug in Scala itself that where this error is reported incorrectly (or that is what the accepted answer suggests). This question is about what to do when this error is reported correctly. Also I supplied sample code.
Making b lazy seems like overkill because it changes the run-time behavior, when this is a compile-time problem that I assume could be fixed with a different declaration...
As mentioned in the comments, making b a def works too, which seems closer to the solution since def works for recursive functions, but that seems to be reevaluating it at run-time every time it is accessed.
scala> var a = 5
scala> def b = a
scala> a = 2
scala> b
res0: Int = 2
scala> a = 1
scala> b
res1: Int = 1
(with lazy it can be changed before it is accessed but is stored after that)
Both of these seem like run-time solutions to a compile-time problem, I'm looking for something analogous to letrec in Scala.
When you define a non-lazy val, you cannot refer to that val inside its definition. For that you either have to use a def or a lazy val.

Scala Macros, generating type parameter calls

I'm trying to generalize setting up Squeryl (Slick poses the same problems AFAIK). I want to avoid having to name every case class explicitly for a number of general methods.
table[Person]
table[Bookmark]
etc.
This also goes for generating indexes, and creating wrapper methods around the CRUD methods for every case class.
So ideally what I want to do is have a list of classes and make them into tables, add indexes and add a wrapper method:
val listOfClasses = List(classOf[Person], classOf[Bookmark])
listOfClasses.foreach(clazz => {
val tbl = table[clazz]
tbl.id is indexed
etc.
})
I thought Scala Macros would be the thing to apply here, since I don't think you can have values as type parameters. Also I need to generate methods for every type of the form:
def insert(model: Person): Person = persons.insert(model)
I've got my mits on an example on Macros but I don't know how to generate a generic datastructure.
I got this simple example to illustrate what I want:
def makeList_impl(c: Context)(clazz: c.Expr[Class[_]]): c.Expr[Unit] = {
import c.universe._
reify {
println(List[clazz.splice]()) // ERROR: error: type splice is not a member of c.Expr[Class[_]]
}
}
def makeList(clazz: Class[_]): Unit = macro makeList_impl
How do I do this? Or is Scala Macros the wrong tool?
Unfortunately, reify is not flexible enough for your use case, but there's good news. In macro paradise (and most likely in 2.11.0) we have a better tool to construct trees, called quasiquotes: http://docs.scala-lang.org/overviews/macros/quasiquotes.html.
scala> def makeList_impl(c: Context)(clazz: c.Expr[Class[_]]): c.Expr[Any] = {
| import c.universe._
| val ConstantType(Constant(tpe: Type)) = clazz.tree.tpe
| c.Expr[Any](q"List[$tpe]()")
| }
makeList_impl: (c: scala.reflect.macros.Context)(clazz: c.Expr[Class[_]])c.Expr[Any]
scala> def makeList(clazz: Class[_]): Any = macro makeList_impl
defined term macro makeList: (clazz: Class[_])Any
scala> makeList(classOf[Int])
res2: List[Int] = List()
scala> makeList(classOf[String])
res3: List[String] = List()
Quasiquotes are even available in 2.10.x with a minor tweak to the build process (http://docs.scala-lang.org/overviews/macros/paradise.html#macro_paradise_for_210x), so you might want to give them a try.
This will probably not fill all your needs here, but it may help a bit:
The signature of table method looks like this:
protected def table[T]()(implicit manifestT: Manifest[T]): Table[T]
As you can see, it takes implicit Manifest object. That object is passed automatically by the compiler and contains information about type T. This is actually what Squeryl uses to inspect database entity type.
You can just pass these manifests explicitly like this:
val listOfManifests = List(manifest[Person], manifest[Bookmark])
listOfManifests.foreach(manifest => {
val tbl = table()(manifest)
tbl.id is indexed
etc.
})
Unfortunately tbl in this code will have type similar to Table[_ <: CommonSupertypeOfAllGivenEntities] which means that all operations on it must be agnostic of concrete type of database entity.