Is it possible for a macro implementation to maintain some form of global state (during the entire compilation run)? Specifically, I want to create a separate instance of IMain, but I don't want to create it anew in every macro expansion, so I would like to have a form of lazy val, ThreadLocal or anything where I can cache that instance. For simplicity, just imagine I want to share an object during compilation between all expansions of the same macro:
object Foo {
def next: Int = macro ???
}
trait Test {
val a = Foo.next
val b = Foo.next
val c = Foo.next
assert(a == 1 && b == 2 && c == 3)
}
Since in the actual case, the state is quite complex and not serializable, reading and writing to disk is not an option.
I can't seem to see any way to achieve that through the only context provided, scala.reflect.macros.blackbox.Context. Does that mean I have to write a full-fledged compiler plugin? Can I trick sbt into giving me some object I can write to?
Use an object and a var. As long as the file it's in doesn't use your macros, it should work. I'm not sure if it's ever guaranteed anywhere that scalac will keep the state of macros between units, but this seems to work for you.
object Foo {
def next: Int = macro next_impl
var state = ???;
def next_impl(...): ...
}
Related
I'm learning Scala and I don't really understand the following example :
object Test extends App {
def method1 = println("method 1")
val x = {
def method2 = "method 2" // method inside a block
"this is " + method2
}
method1 // statement inside an object
println(x) // same
}
I mean, it feels inconsistent to me because here I see two different concepts :
Objects/Classes/Traits, which contains members.
Blocks, which contains statements, the last statement being the value of the block.
But here we have a method part of a block, and statements part of an object. So, does it mean that blocks are objects too ? And how are handled the statements part of an object, are they members too ?
Thanks.
Does it mean that blocks are objects too?
No, blocks are not objects. Blocks are used for scoping the binding of variables. Scala enables not only defining expressions inside blocks but also to define methods. If we take your example and compile it, we can see what the compiler does:
object Test extends Object {
def method1(): Unit = scala.Predef.println("method 1");
private[this] val x: String = _;
<stable> <accessor> def x(): String = Test.this.x;
final <static> private[this] def method2$1(): String = "method 2";
def <init>(): tests.Test.type = {
Test.super.<init>();
Test.this.x = {
"this is ".+(Test.this.method2$1())
};
Test.this.method1();
scala.Predef.println(Test.this.x());
()
}
}
What the compiler did is extract method2 to an "unnamed" method on method2$1 and scoped it to private[this] which is scoped to the current instance of the type.
And how are handled the statements part of an object, are they members
too?
The compiler took method1 and println and calls them inside the constructor when the type is initialized. So you can see val x and the rest of the method calls are invoked at construction time.
method2 is actually not a method. It is a local function. Scala allows you to create named functions inside local scopes for organizing your code into functions without polluting the namespace.
It is most often used to define local tail-recursive helper functions. Often, when making a function tail-recursive, you need to add an additional parameter to carry the "state" on the call stack, but this additional parameter is a private internal implementation detail and shouldn't be exposed to clients. In languages without local functions, you would make this a private helper alongside the primary method, but then it would still be within the namespace of the class and callable by all other methods of the class, when it is really only useful for that particular method. So, in Scala, you can instead define it locally inside the method:
// non tail-recursive
def length[A](ls: List[A]) = ls match {
case Nil => 0
case x :: xs => length(xs) + 1
}
//transformation to tail-recursive, Java-style:
def length[A](ls: List[A]) = lengthRec(ls, 0)
private def lengthRec[A](ls: List[A], len: Int) = ls match {
case Nil => len
case x :: xs => lengthRec(xs, len + 1)
}
//tail-recursive, Scala-style:
def length[A](ls: List[A]) = {
//note: lengthRec is nested and thus can access `ls`, there is no need to pass it
def lengthRec(len: Int) = ls match {
case Nil => len
case x :: xs => lengthRec(xs, len + 1)
}
lengthRec(ls, 0)
}
Now you might say, well I see the value in defining local functions inside methods, but what's the value in being able to define local functions in blocks? Scala tries to as simple as possible and have as few corner cases as possible. If you can define local functions inside methods, and local functions inside local functions … then why not simplify that rule and just say that local functions behave just like local fields, you can simply define them in any block scope. Then you don't need different scope rules for local fields and local functions, and you have simplified the language.
The other thing you mentioned, being able to execute code in the bode of a template, that's actually the primary constructor (so to speak … it's technically more like an initializer). Remember: the primary constructor's signature is defined with parentheses after the class name … but where would you put the code for the constructor then? Well, you put it in the body of the class!
I understand that Scala embraces immutability fully.
Now I am thinking a scenario that I have to hold some state (via variables) in a class or such. I will need to update these variables later; then I can revisit the class later to access the updated variables.
I will try to make it simple with one very straightforward example:
class A {
var x: Int
def compute: Int = {calling some other processes or such using x as input}
}
......
def invoker() {
val a: A = new A
a.x = 1
......
val res1 = a.compute
a.x = 5
......
val res2 = a.compute
......
}
So you see, I need to keep changing x and get the results. If you argue that I can simply keep x as an argument for compute such as
def compute(x: Int)
......
That's a good idea but I cannot do it in my case as I need to separate setting value for x and computing the result completely. In other words, setting x value should not trigger "computing" to occur, rather, I need to be able to set x value anytime in the program and be able to reuse the value for computation any other time in the program when I need it.
I am using a variable (var x: Int) in this case. Is this legitimate or there is still some immutable way to handle it?
Any time you store state you will need to use mutability.
In your case, you want to store x and compute separately. Inherently, this means state is required since the results of compute depends on the state of x
If you really want the class with compute to be immutable, then some other mutable class will need to contain x and it will need to be passed to the compute method.
rather, I need to be able to set x value anytime in the program and be able to reuse the value for computation any other time in the program when I need it.
Then, by definition you want your class to be stateful. You could restructure your problem so that particular class doesn't require state, but whether that's useful and/or worth the hassle is something you'll have to figure out.
Your pattern is used in a ListBuffer for example (with size as your compute function).
So yes, there might be cases where you can use this pattern for good reasons. Example:
val l = List(1, 2, 3)
val lb = new ListBuffer[Int]
l.foreach(n => lb += n * n)
val result = lb.toList
println(result)
On the other hand a buffer is normally only used to create an immutable instance as soon as possible. If you look at this code, there are two items which might indicate that it can be changed: The mutable buffer and foreach (because foreach is only called for its side-effects)
So another option is
val l = List(1, 2, 3)
val result = l.map(n => n * n)
println(result)
which does the same in fewer lines. I prefer this style, because your are just looking at immutable instances and "functional" functions.
In your abstract example, you could try to separate the mutable state and the function:
class X(var i: Int)
class A {
def compute(x: X): Int = { ... }
}
possibly even
class X(val i: Int)
This way compute becomes functional: It's return value only depends from the parameter.
My personal favorite regarding an "unexpected" immutable class is scala.collection.immutable.Queue. With an "imperative" background, you just not expect a queue to be immutable.
So if you look at your pattern, it's likely that you can change it to being immutable.
I would create an immutable A class (here its a case class) and let an object handle the mutability. For each state change we create a new A object and change the reference in the object. This is handle concurrency bit better if you set x from a different thread, you just have to make the variable a volatile or an AtomicReference.
object A {
private[this] var a = A(0)
def setX(x: Int) { if (x != a.x) a = new A(x) }
def getA: A = a
}
case class A(x: Int) {
def compute: Int = { /*do your stuff*/ }
}
After a few more months on functional programming, here is my rethinking.
Every time a variable is modified/changed/updated/mutated, the imperative way of handling this is to record such change right with that variable. The functional way of thinking is to make the activity (that cause the change) bring the new state to you. In other words, it's like cause effect stuff. Functional way thinking focuses on the transition activity between cause and effect.
Given all that, in any given point of time in the program execution, our achievement is the intermediate result. We need somewhere to hold the result no matter how we do it. Such intermediate result is the state and yes, we need some variable to hold it. That's what I want to share with just abstract thinking.
Is it possible to get the name of a scala variable at runtime?
E.g. is it possible to write a function getIntVarName(variable: Int): String behaving as follows?
val myInt = 3
assert("myInt" === getIntVarName(myInt))
For what you need to do, It seems to me that runtime is not required, since you already have your myInt variable defined at compile time. If this is the case, you just need a bit of AST manipulation via a macro.
Try
package com.natalinobusa.macros
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object Macros {
// write macros here
def getName(x: Any): String = macro impl
def impl(c: Context)(x: c.Tree): c.Tree = {
import c.universe._
val p = x match {
case Select(_, TermName(s)) => s
case _ => ""
}
q"$p"
}
}
Be aware that macro's must be compiled as a separate subproject, and cannot be part of the same project where the macro substitution has to be applied. Check this template on how to define such a macro sub-project: https://github.com/echojc/scala-macro-template
scala> import Macros._
import Macros._
scala> val myInt = 3
myInt: Int = 3
scala> "myInt" == getName(myInt)
res6: Boolean = true
You can use scala-nameof to get a variable name, function name, class member name, or type name. It happens at compile-time so there's no reflection involved and no runtime dependency needed.
val myInt = 3
assert("myInt" === nameOf(myInt))
will compile to:
val myInt = 3
assert("myInt" === "myInt")
Basically, it can't be done.
The JVM offers nothing by way of a Method handle (remember, Scala properties are encoded as methods in bytecode to support the uniform access principle). The closest you can get is to use reflection to find a list of methods defined on a particular class - which I appreciate doesn't help with your particular need.
It is possible to implement this as a Scala feature, but it would require a compiler plugin to grab the relevant symbol name from the AST and push it into code as a string literal, so not something I could demonstrate in a short code snippet :)
The other naming problem that often comes up in reflection is method parameters. That one at least I can help with. I have a work-in-progress reflection library here that's based on the compiler-generated scala signature as used by scalap. It's nowhere near being ready for serious use, but it is under active development.
Scala doesn't yet have much more than Java in terms of metadata like this. Keep an eye on the Scala Reflection project, but I doubt that will offer access to local variables anytime soon. In the meantime, consider a bytecode inspector library like ASM. Another big caveat: local variable names are lost during compilation, so you'd need to compile in "debug" mode to preserve them.
I don't think it's possible to get the name of a variable, but you can try it with objects:
object Test1 {
def main(args: Array[String]) {
object MyVar {
def value = 1
}
println(MyVar.getClass)
}
}
This prints: class Test1$MyVar$2$. So you can get 'MyVar' out of it.
This can be achieved with Scala 3 Macros (does it at compile time).
Create a Macro object (this must be in a separate file):
import scala.quoted.{Expr, Quotes}
object NameFromVariable :
def inspectCode(x: Expr[Any])(using Quotes): Expr[String] =
val name = x.show.split("""\.""").last
Expr(name)
Then you need an inline method in your class.
inline def getIntVarName(inline x: Any): Any = ${ NameFromVariable.inspectCode('x) }
And use this method, like:
val myInt = 3
assert("myInt" === getIntVarName(myInt))
See the official documentation: https://docs.scala-lang.org/scala3/guides/macros/macros.html
OK, in the question about 'Class Variables as constants', I get the fact that the constants are not available until after the 'official' constructor has been run (i.e. until you have an instance). BUT, what if I need the companion singleton to make calls on the class:
object thing {
val someConst = 42
def apply(x: Int) = new thing(x)
}
class thing(x: Int) {
import thing.someConst
val field = x * someConst
override def toString = "val: " + field
}
If I create companion object first, the 'new thing(x)' (in the companion) causes an error. However, if I define the class first, the 'x * someConst' (in the class definition) causes an error.
I also tried placing the class definition inside the singleton.
object thing {
var someConst = 42
def apply(x: Int) = new thing(x)
class thing(x: Int) {
val field = x * someConst
override def toString = "val: " + field
}
}
However, doing this gives me a 'thing.thing' type object
val t = thing(2)
results in
t: thing.thing = val: 84
The only useful solution I've come up with is to create an abstract class, a companion and an inner class (which extends the abstract class):
abstract class thing
object thing {
val someConst = 42
def apply(x: Int) = new privThing(x)
class privThing(x: Int) extends thing {
val field = x * someConst
override def toString = "val: " + field
}
}
val t1 = thing(2)
val tArr: Array[thing] = Array(t1)
OK, 't1' still has type of 'thing.privThing', but it can now be treated as a 'thing'.
However, it's still not an elegant solution, can anyone tell me a better way to do this?
PS. I should mention, I'm using Scala 2.8.1 on Windows 7
First, the error you're seeing (you didn't tell me what it is) isn't a runtime error. The thing constructor isn't called when the thing singleton is initialized -- it's called later when you call thing.apply, so there's no circular reference at runtime.
Second, you do have a circular reference at compile time, but that doesn't cause a problem when you're compiling a scala file that you've saved on disk -- the compiler can even resolve circular references between different files. (I tested. I put your original code in a file and compiled it, and it worked fine.)
Your real problem comes from trying to run this code in the Scala REPL. Here's what the REPL does and why this is a problem in the REPL. You're entering object thing and as soon as you finish, the REPL tries to compile it, because it's reached the end of a coherent chunk of code. (Semicolon inference was able to infer a semicolon at the end of the object, and that meant the compiler could get to work on that chunk of code.) But since you haven't defined class thing it can't compile it. You have the same problem when you reverse the definitions of class thing and object thing.
The solution is to nest both class thing and object thing inside some outer object. This will defer compilation until that outer object is complete, at which point the compiler will see the definitions of class thing and object thing at the same time. You can run import thingwrapper._ right after that to make class thing and object thing available in global scope for the REPL. When you're ready to integrate your code into a file somewhere, just ditch the outer class thingwrapper.
object thingwrapper{
//you only need a wrapper object in the REPL
object thing {
val someConst = 42
def apply(x: Int) = new thing(x)
}
class thing(x: Int) {
import thing.someConst
val field = x * someConst
override def toString = "val: " + field
}
}
Scala 2.12 or more could benefit for sip 23 which just (August 2016) pass to the next iteration (considered a “good idea”, but is a work-in-process)
Literal-based singleton types
Singleton types bridge the gap between the value level and the type level and hence allow the exploration in Scala of techniques which would typically only be available in languages with support for full-spectrum dependent types.
Scala’s type system can model constants (e.g. 42, "foo", classOf[String]).
These are inferred in cases like object O { final val x = 42 }. They are used to denote and propagate compile time constants (See 6.24 Constant Expressions and discussion of “constant value definition” in 4.1 Value Declarations and Definitions).
However, there is no surface syntax to express such types. This makes people who need them, create macros that would provide workarounds to do just that (e.g. shapeless).
This can be changed in a relatively simple way, as the whole machinery to enable this is already present in the scala compiler.
type _42 = 42.type
type Unt = ().type
type _1 = 1 // .type is optional for literals
final val x = 1
type one = x.type // … but mandatory for identifiers
I'm asking a slight different question than this one. Suppose I have a code snippet:
def foo(i : Int) : List[String] = {
val s = i.toString + "!" //using val
s :: Nil
}
This is functionally equivalent to the following:
def foo(i : Int) : List[String] = {
def s = i.toString + "!" //using def
s :: Nil
}
Why would I choose one over the other? Obviously I would assume the second has a slight disadvantages in:
creating more bytecode (the inner def is lifted to a method in the class)
a runtime performance overhead of invoking a method over accessing a value
non-strict evaluation means I could easily access s twice (i.e. unnecesasarily redo a calculation)
The only advantage I can think of is:
non-strict evaluation of s means it is only called if it is used (but then I could just use a lazy val)
What are peoples' thoughts here? Is there a significant dis-benefit to me making all inner vals defs?
1)
One answer I didn't see mentioned is that the stack frame for the method you're describing could actually be smaller. Each val you declare will occupy a slot on the JVM stack, however, the whenever you use a def obtained value it will get consumed in the first expression you use it in. Even if the def references something from the environment, the compiler will pass .
The HotSpot should optimize both these things, or so some people claim. See:
http://www.ibm.com/developerworks/library/j-jtp12214/
Since the inner method gets compiled into a regular private method behind the scene and it is usually very small, the JIT compiler might choose to inline it and then optimize it. This could save time allocating smaller stack frames (?), or, by having fewer elements on the stack, make local variables access quicker.
But, take this with a (big) grain of salt - I haven't actually made extensive benchmarks to backup this claim.
2)
In addition, to expand on Kevin's valid reply, the stable val provides also means that you can use it with path dependent types - something you can't do with a def, since the compiler doesn't check its purity.
3)
For another reason you might want to use a def, see a related question asked not so long ago:
Functional processing of Scala streams without OutOfMemory errors
Essentially, using defs to produce Streams ensures that there do not exist additional references to these objects, which is important for the GC. Since Streams are lazy anyway, the overhead of creating them is probably negligible even if you have multiple defs.
The val is strict, it's given a value as soon as you define the thing.
Internally, the compiler will mark it as STABLE, equivalent to final in Java. This should allow the JVM to make all sorts of optimisations - I just don't know what they are :)
I can see an advantage in the fact that you are less bound to a location when using a def than when using a val.
This is not a technical advantage but allows for better structuring in some cases.
So, stupid example (please edit this answer, if you’ve got a better one), this is not possible with val:
def foo(i : Int) : List[String] = {
def ret = s :: Nil
def s = i.toString + "!"
ret
}
There may be cases where this is important or just convenient.
(So, basically, you can achieve the same with lazy val but, if only called at most once, it will probably be faster than a lazy val.)
For a local declaration like this (with no arguments, evaluated precisely once and with no code evaluated between the point of declaration and the point of evaluation) there is no semantic difference. I wouldn't be surprised if the "val" version compiled to simpler and more efficient code than the "def" version, but you would have to examine the bytecode and possibly profile to be sure.
In your example I would use a val. I think the val/def choice is more meaningful when declaring class members:
class A { def a0 = "a"; def a1 = "a" }
class B extends A {
var c = 0
override def a0 = { c += 1; "a" + c }
override val a1 = "b"
}
In the base class using def allows the sub class to override with possibly a def that does not return a constant. Or it could override with a val. So that gives more flexibility than a val.
Edit: one more use case of using def over val is when an abstract class has a "val" for which the value should be provided by a subclass.
abstract class C { def f: SomeObject }
new C { val f = new SomeObject(...) }