scala macro that refers to 'this' object - scala

I am trying to use a macro to eliminate the need for scala to construct a downward-passed function object. This code gets used in inner-loops of our system, and we don't want the inner loop to just allocate objects endlessly. This is creating performance problems for us.
Our original code was this:
dis.withBitLengthLimit(newLimit){... body ...}
And the body was a function that was passed in as a function object.
The problem I have is that the original non-macro version refers to 'this'. My workaround below is to make each place the macro is called pass the 'this' object as another argument. i.e., ugly like:
dis.withBitLengthLimit(dis, newLimit){... body ...}
It's not awful, but sure seems like passing dis should be unnecessary.
Is there a cleaner way?
Here's the macro below.
object IOMacros {
/**
* Used to temporarily vary the bit length limit.
*
* Implementing as a macro eliminates the creation of a downward function object every time this
* is called.
*
* ISSUE: this macro really wants to use a self reference to `this`. But when a macro is expanded
* the object that `this` represents changes. Until a better way to do this comes about, we have to pass
* the `this` object to the `self` argument, which makes calls look like:
* dis.withBitLengthLimit(dis, newLimit){... body ...}
* That looks redundant, and it is, but it's more important to get the allocation of this downward function
* object out of inner loops.
*/
def withBitLengthLimitMacro(c: Context)(self: c.Tree, lengthLimitInBits: c.Tree)(body: c.Tree) = {
import c.universe._
q"""{
import edu.illinois.ncsa.daffodil.util.MaybeULong
val ___dStream = $self
val ___newLengthLimit = $lengthLimitInBits
val ___savedLengthLimit = ___dStream.bitLimit0b
if (!___dStream.setBitLimit0b(MaybeULong(___dStream.bitPos0b + ___newLengthLimit))) false
else {
try {
$body
} finally {
___dStream.resetBitLimit0b(___savedLengthLimit)
}
true
}
}"""
}

The prefix method on Context provides access to the expression that the macro method is called on, which should allow you to accomplish what you're trying to do. Here's a quick example of how you can use it:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
class Foo(val i: Int) {
def bar: String = macro FooMacros.barImpl
}
object FooMacros {
def barImpl(c: Context): c.Tree = {
import c.universe._
val self = c.prefix
q"_root_.scala.List.fill($self.i + $self.i)(${ self.tree.toString }).mkString"
}
}
And then:
scala> val foo = new Foo(3)
foo: Foo = Foo#6fd7c13e
scala> foo.bar
res0: String = foofoofoofoofoofoo
Note that there are some issues you need to be aware of. prefix gives you the expression, which may not be a variable name:
scala> new Foo(2).bar
res1: String = new Foo(2)new Foo(2)new Foo(2)new Foo(2)
This means that if the expression has side effects, you have to take care not to include it in the result tree more than once (assuming you don't want them to happen multiple times):
scala> new Qux(1).bar
hey
hey
res2: String = new Qux(1)new Qux(1)
Here the constructor is called twice since we include the prefix expression in the macro's result twice. You can avoid this by defining a temporary variable in the macro:
object FooMacros {
def barImpl(c: Context): c.Tree = {
import c.universe._
val tmp = TermName(c.freshName)
val self = c.prefix
q"""
{
val $tmp = $self
_root_.scala.List.fill($tmp.i + $tmp.i)(${ self.tree.toString }).mkString
}
"""
}
}
And then:
scala> class Qux(i: Int) extends Foo(i) { println("hey") }
defined class Qux
scala> new Qux(1).bar
hey
res3: String = new Qux(1)new Qux(1)
Note that this approach (using freshName) is a lot better than just prefixing local variables in the macro with a bunch of underscores, which can cause problems if you include an expression that happens to contain a variable with the same name.
(Update about that last paragraph: actually I don't remember for sure if you can get yourself into problems with local variable names shadowing names that might be used in included trees. I avoid it myself, but I can't manufacture an example of it causing problems at the moment, so it might be fine.)

Related

Can I store a tree produced in one Scala Def-macro and then pass it into another

Say I have an expression whose tree I want to capture (without evaluating it!) and then pass it into a another macro sometime later:
// capture a tree from some arbitrary expression
val capturedTree = captureMacro( some(expression(tree())) )
// plug the tree into here
val result = processMacro(capturedTree, otherStuff)
Is there any way to do this? I've tried making capturedTree a refined type hoping that this would preserve the initial tree but that did not happen:
trait WithCapturedTree { def tree:Any }
class ReturnTreeMacro(val c:MacroContext) {
import c.universe._
def run(expression: Tree) =
q"new WithCapturedTree {val tree = ${expression}}"
}
def returnTree(expression:String):WithCapturedTree = macro ReturnTreeMacro.run
reify { returnTree("foo"+"bar") }
// returns Expr[WithCapturedTree{val tree: java.lang.String}](returnTree("foobar"))
// I.e. the result of "foo"+"bar" has already been evaluated!
Is there a way to get this approach to work? Is there a better appraoch for this problem?

In a scala macro called with an implicit def, how can I get the source statement of the caller to the implicit def as a string?

I'm trying to write an implicit def macro that captures the invoking statement (and preferably nothing else) as a string, that can hopefully be used just by having the desired output object as an implicit parameter. But I'm having some trouble getting it to work with multi line statements.
For example:
case class HowWasICalled(statement:String, lineNumber:Int)
object HowWasICalled {
implicit def generate: HowWasICalled = macro macro_impl
}
def callMe(something: Any)(implicit context: HowWasICalled)
// macro: c is the blackbox Context
def macro_impl(c: blackbox.Context): c.Expr[HowWasICalled] = { ... }
And the first attempt at implementing a method inside the macro that returns the invoking statement as a string:
def makeString:String = show(c.macroApplication)
But, this only returned "HowWasICalled.generate". I actually wanted the statement invoking callMe.
I was hoping the following would work, but it doesn't. The -Yrangepos compiler flag does not seem to result in range positions in def macros? :(
def makeString:String = {
if (c.enclosingPosition.isRange) {
new String(c.enclosingPosition.source.content.slice(
c.enclosingPosition.start,
c.enclosingPosition.end
))
}
}
The closest I got to any working implementation doesn't exactly capture just statement nor the entire statement, but at least I get the line of source invoking callMe.
def makeString:String = {
val lineNumber = c.enclosingPosition.line
val lineIndex = lineNumber-1
c.enclosingPosition.source.lineToString(lineIndex)
}
How can I improve the macro to handle the following case? It should ideally generate something like a HowIWasCalled("callMe( 1 + 2 )", 123) given the following input
0; 0; 0; val x = callMe(1 +
2)

Scala bitwise-like method argument

Let's say there is a generic method declaration that performs a set of operations based upon a designated mode that would look something like this:
def doSomethingSmart(mode: OpMode, someGoodyList: List[Any]): Boolean = { /* do foo */ }
Where OpMode is a type/enumeration consisting of:
Read
Create
Delete
Modify
Whatever
Putting the two together would obviously yield a single-mode, reusable, code block.
Now, the type/enumeration part would probably look something like this:
object OpMode extends Enumeration {
type OpMode = Value
val Read, Write, Create, Modify, Delete, Whatever = Value
}
But let's say you wanted to expand the scope of doSomethingSmart() to span what is typically done using bitwise operators, for example: Create & Modify & Whatever. Is there a "scala-way" of restricting the bit-masked argument to that limited data-set (ie, the enumeration/type). Maybe something along these lines:
def doSomethingSmarter(more: T < [BitwiseSelectionOf[OpMode]], ...
Or, is it best to simply drop back to binary-indexed value assignments - in which cases there is no "type" checking per se?
TIA.
EDIT: I guess another possibility would be to change the OpMode to be a List and then just run a series of "contains" operations.
EDIT 2: In particular, I was looking for an efficient mechanism that provides an inline construct when making a call to doSomethingSmarter()
Enumeration defines an inner type called ValueSet which gives you at least some of the functionality you are looking for. The methods on it are still Set-like (you would add a new mode using +, and check for a mode using contains), but it may serve your purposes.
EDIT: had some fun fiddling around and came up with this:
import scala.collection.BitSet
object OpMode extends Enumeration {
protected case class Val(name: String, val mask: Int) extends super.Val(nextId, name)
type OpMode = Val
val Read = Val("Read", 1)
val Write = Val("Write", 2)
val Create = Val("Create", 4)
val Modify = Val("Modify", 8)
val Delete = Val("Delete", 16)
val Whatever = Val("Whatever", 32)
case class FlagSet(bits: BitSet) {
def isSet(mode: OpMode) = bits.contains(mode.mask)
def +(mode: OpMode) = new FlagSet(bits + mode.mask)
def -(mode: OpMode) = new FlagSet(bits - mode.mask)
def &(other: FlagSet) = new FlagSet(bits & other.bits)
def &~(other: FlagSet) = new FlagSet(bits &~ other.bits)
def ^(other: FlagSet) = new FlagSet(bits ^ other.bits)
def |(other: FlagSet) = new FlagSet(bits | other.bits)
def size = bits.size
// etc.
}
object FlagSet {
def apply(flags: OpMode*): FlagSet = apply(BitSet(flags.map(_.mask):_*))
def apply(modes: ValueSet): FlagSet = apply(BitSet(modes.toSeq.map{ case m: OpMode => m.mask }:_*))
}
}
def doSomethingSmarter(modes: OpMode.FlagSet, someGoodyList: List[Any]) = modes.size
val flags = OpMode.FlagSet(OpMode.Read, OpMode.Write)
doSomethingSmarter(flags, Nil)
val modes = OpMode.ValueSet(OpMode.Read, OpMode.Write)
doSomethingSmarter(OpMode.FlagSet(modes), Nil)
Basically, I extended the Enumeration.Val type to add a suitable bit mask for each mode, and added an inner class FlagSet to interoperate between OpModes and an under-the-covers BitSet. Changing doSomethingSmarter to take in such a FlagSet enables usage that should be closer to what you are hoping for.
The above can probably be improved upon, but working with Enumeration can be tricky. As an alternative, you might find it preferable to work with a sealed trait and case classes/objects extending it - this can often give semantics closer to what is possible with Java's enum types.

Make a Scala interpreter oblivious between interpret calls

Is it possible to configure a Scala interpreter (tools.nsc.IMain) so that it "forgets" the previously executed code, whenever I run the next interpret() call?
Normally when it compiles the sources, it wraps them in nested objects, so all the previously defined variables, functions and bindings are available.
It would suffice to not generate the nested objects (or to throw them away), although I would prefer a solution which would even remove the previously compiled classes from the class loader again.
Is there a setting, or a method, or something I can overwrite, or an alternative to IMain that would accomplish this? I need to be able to still access the resulting objects / classes from the host VM.
Basically I want to isolate subsequent interpret() calls without something as heavy weight as creating a new IMain for each iteration.
Here is one possible answer. Basically there is method reset() which calls the following things (mostly private, so either you buy the whole package or not):
clearExecutionWrapper()
resetClassLoader()
resetAllCreators()
prevRequests.clear()
referencedNameMap.clear()
definedNameMap.clear()
virtualDirectory.clear()
In my case, I am using a custom execution wrapper, so that needs to be set up again, and also imports are handled through a regular interpret cycle, so either add them again, or—better—just prepend them with the execution wrapper.
I would like to keep my bindings, they are also gone:
import tools.nsc._
import interpreter.IMain
object Test {
private final class Intp(cset: nsc.Settings)
extends IMain(cset, new NewLinePrintWriter(new ConsoleWriter, autoFlush = true)) {
override protected def parentClassLoader = Test.getClass.getClassLoader
}
object Foo {
def bar() { println("BAR" )}
}
def run() {
val cset = new nsc.Settings()
cset.classpath.value += java.io.File.pathSeparator + sys.props("java.class.path")
val i = new Intp(cset)
i.initializeSynchronous()
i.bind[Foo.type]("foo", Foo)
val res0 = i.interpret("foo.bar(); val x = 33")
println(s"res0: $res0")
i.reset()
val res1 = i.interpret("println(x)")
println(s"res1: $res1")
i.reset()
val res2 = i.interpret("foo.bar()")
println(s"res2: $res2")
}
}
This will find Foo in the first iteration, correctly forget x in the second iteration, but then in the third iteration, it can be seen that the foo binding is also lost:
foo: Test.Foo.type = Test$Foo$#8bf223
BAR
x: Int = 33
res0: Success
<console>:8: error: not found: value x
println(x)
^
res1: Error
<console>:8: error: not found: value foo
foo.bar()
^
res2: Error
The following seems to be fine:
for(j <- 0 until 3) {
val user = "foo.bar()"
val synth = """import Test.{Foo => foo}
""".stripMargin + user
val res = i.interpret(synth)
println(s"res$j: $res")
i.reset()
}

Get Scala variable name at runtime

Is it possible to get the name of a scala variable at runtime?
E.g. is it possible to write a function getIntVarName(variable: Int): String behaving as follows?
val myInt = 3
assert("myInt" === getIntVarName(myInt))
For what you need to do, It seems to me that runtime is not required, since you already have your myInt variable defined at compile time. If this is the case, you just need a bit of AST manipulation via a macro.
Try
package com.natalinobusa.macros
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object Macros {
// write macros here
def getName(x: Any): String = macro impl
def impl(c: Context)(x: c.Tree): c.Tree = {
import c.universe._
val p = x match {
case Select(_, TermName(s)) => s
case _ => ""
}
q"$p"
}
}
Be aware that macro's must be compiled as a separate subproject, and cannot be part of the same project where the macro substitution has to be applied. Check this template on how to define such a macro sub-project: https://github.com/echojc/scala-macro-template
scala> import Macros._
import Macros._
scala> val myInt = 3
myInt: Int = 3
scala> "myInt" == getName(myInt)
res6: Boolean = true
You can use scala-nameof to get a variable name, function name, class member name, or type name. It happens at compile-time so there's no reflection involved and no runtime dependency needed.
val myInt = 3
assert("myInt" === nameOf(myInt))
will compile to:
val myInt = 3
assert("myInt" === "myInt")
Basically, it can't be done.
The JVM offers nothing by way of a Method handle (remember, Scala properties are encoded as methods in bytecode to support the uniform access principle). The closest you can get is to use reflection to find a list of methods defined on a particular class - which I appreciate doesn't help with your particular need.
It is possible to implement this as a Scala feature, but it would require a compiler plugin to grab the relevant symbol name from the AST and push it into code as a string literal, so not something I could demonstrate in a short code snippet :)
The other naming problem that often comes up in reflection is method parameters. That one at least I can help with. I have a work-in-progress reflection library here that's based on the compiler-generated scala signature as used by scalap. It's nowhere near being ready for serious use, but it is under active development.
Scala doesn't yet have much more than Java in terms of metadata like this. Keep an eye on the Scala Reflection project, but I doubt that will offer access to local variables anytime soon. In the meantime, consider a bytecode inspector library like ASM. Another big caveat: local variable names are lost during compilation, so you'd need to compile in "debug" mode to preserve them.
I don't think it's possible to get the name of a variable, but you can try it with objects:
object Test1 {
def main(args: Array[String]) {
object MyVar {
def value = 1
}
println(MyVar.getClass)
}
}
This prints: class Test1$MyVar$2$. So you can get 'MyVar' out of it.
This can be achieved with Scala 3 Macros (does it at compile time).
Create a Macro object (this must be in a separate file):
import scala.quoted.{Expr, Quotes}
object NameFromVariable :
def inspectCode(x: Expr[Any])(using Quotes): Expr[String] =
val name = x.show.split("""\.""").last
Expr(name)
Then you need an inline method in your class.
inline def getIntVarName(inline x: Any): Any = ${ NameFromVariable.inspectCode('x) }
And use this method, like:
val myInt = 3
assert("myInt" === getIntVarName(myInt))
See the official documentation: https://docs.scala-lang.org/scala3/guides/macros/macros.html