I want to be able to evaluate the expression that returns from func.
The problem is that the expression includes the variable a, which is not familiar in the scope of func but is familiar in the scope of playground.
I want to be able to send the String: s"$a + 1" when $ is not an operator and s is a part of the String. I saw that $$ should solve the problem with the $, but then the char s is not a part of the String, and the eval function needs an expression with the pattern of s"".
object playground extends App{
val a = 5.5
val expression = func()
val str: String = expression
val tb = currentMirror.mkToolBox()
val x = tb.eval(tb.parse(str)).toString
print(x)
}
object second {
def func(): String = {
s"$a + 1"
}
}
Thanks for any help :)
expression includes the variable a, which is not familiar in the scope of func but is familiar in the scope of playground.
Try to fully qualify the variable
def func(): String =
s"${playground.a} + 1"
or use import
def func(): String =
s"import playground._; $a + 1"
def func(): String =
s"""
import playground._
$a + 1
"""
or add a parameter to the function
def func(a: Double): String =
s"$a + 1"
val a = 5.5
val expression = func(a)
s"$a + 1" when $ is not an operator and s is a part of the String. I
saw that $$ should solve the problem with the $, but then the char s
is not a part of the String
I'm not sure what you're after. Try to provide some inputs and desired outputs to make it clear.
"$a + 1" is just a String (with a dollar sign), s"$a + 1" is a String where a variable a from current scope is substituted (after .toString).
Yes, you can escape $ inside s"..." as $$. So s"$$a + 1" is the same as just "$a + 1" (without prefix s).
I don't understand what "s is a part of the String", "but then the char s is not a part of the String" mean.
and the eval function needs an expression
with the pattern of s"".
Why? tb.parse (or q"...") accepts arbitrary String. tb.eval accepts arbitrary Tree.
I guess I got it. It seems you are expecting func() to expand into s"$a + 1" at a call site. Then s"$a + 1" should be not a String but a Tree itself (s"$a + 1" should be not a String s""" s"$$a + 1" """ but a piece of code) and func() should be a macro.
import scala.language.experimental.macros
import scala.reflect.macros.blackbox
object second {
def func(): String = macro funcImpl
def funcImpl(c: blackbox.Context)(): c.Tree = {
import c.universe._
q""" s"$$a + 1" """
}
}
// in a different subproject
import second.func
import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox
object playground extends App {
val a = 5.5
val expression = func()
val str: String = expression
val tb = currentMirror.mkToolBox()
val x = tb.eval(tb.parse(str)).toString
print(x) // 6.5
}
But using macros (and mixing them with runtime compilation via Toolbox) in your case seems an overkill, you should prefer easier options from the above.
Related
My code compiles with the following error: Macro expansion contains free term variable Hello ...
I have reduced it to minimal example:
class Hello(val hi: String) {
val xx = reify(hi)
var yy = q""
}
def setYYImpl(c: Context)(hExpr: c.Expr[Hello]): c.Expr[Hello] = {
import c.universe._
val hello = c.eval(c.Expr[Hello](c.untypecheck(hExpr.tree.duplicate)))
val xxVal = c.internal.createImporter(u).importTree(hello.xx.tree)
c.Expr(q"""{val h = new Hello("HO"); h.yy=$xxVal; h}""") // it should set `h.yy` to Tree:"HO"
}
def setYY(hExpr: Hello): Hello = macro setYYImpl
setYY(new Hello("HI"))
After inspecting similar question: Can this free-term-variable error (produced at macro expansion) be avoided?
I have come to conclusion that the problem is reify(hi) which refers to the compile time value Hello.hi.
Is it possible to work around this problem? reify(hi) returns Expr Hello.hi, can I somehow remove the Hello. prefix?
Try to replace
val xx = reify(hi)
with
val xx = Literal(Constant(hi))
i.e. build the tree manually (and
.importTree(hello.xx.tree)
with
.importTree(hello.xx)).
(If it's Literal(Constant... only in your example and more complex tree in actual use case, anyway try to build it manually rather than use reify.)
Then you'll have different error
Error: type mismatch;
found : String("HI")
required: reflect.runtime.universe.Tree
setYY(new Hello("HI"))
because your macro returns
Expr[Hello]({
val h = new Hello("HO");
h.yy = "HI"; // h.yy is q"" i.e. Tree, "HI" is String
h
})
I want to evaluate a function passed as a variable string in scala (sorry but i'm new to scala )
def concate(a:String,b:String): String ={
a+" "+b
}
var func="concate" //i'll get this function name from config as string
I want to perform something like
eval(func("hello","world)) //Like in Python
so output will be like
hello world
Eventually I want to execute few in built functions on a string coming from my config and I don't want to hard code the function names in the code.
EDIT
To Be More clear with my exact usecase
I have a Config file which has multiple functions defined in it that are Spark inbuilt functions on Data frame
application.conf looks like
transformations = [
{
"table" : "users",
"function" : "from_unixtime",
"column" : "epoch"
},
{
"table" : "users",
"function" : "yearofweek",
"column" : "epoch"
}
]
Now functions yearofweek and from_unixtime are Spark inbuilt functions now I want to eval my Dataframe by the functions defined in config. #all the functions are applied to a column defined.
the Obvious way is to write an if else and do string comparison calling a particular inbuilt function but that is way to much..
i am looking for a better solution.
This is indeed possible in scala, as scala is JSR 223 compliant scripting language. Here is an example (running with scala 2.11.8). Note that you need to import your method because otherwise the interpreter will not find it:
package my.example
object EvalDemo {
// evalutates scala code and returns the result as T
def evalAs[T](code: String) = {
import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox
val toolbox = currentMirror.mkToolBox()
import toolbox.{eval, parse}
eval(parse(code)).asInstanceOf[T]
}
def concate(a: String, b: String): String = a + " " + b
def main(args: Array[String]): Unit = {
var func = "concate" //i'll get this function name from config as string
val code =
s"""
|import my.example.EvalDemo._
|${func}("hello","world")
|""".stripMargin
val result: String = evalAs[String](code)
println(result) // "hello world"
}
}
Have Function to name mapping in the code
def foo(str: String) = str + ", foo"
def bar(str: String) = str + ", bar"
val fmap = Map("foo" -> foo _, "bar" -> bar _)
fmap("foo")("hello")
now based on the function name we get from the config, pass the name to the map and lookup the corresponding function and invoke the arguments on it.
Scala repl
scala> :paste
// Entering paste mode (ctrl-D to finish)
def foo(str: String) = str + ", foo"
def bar(str: String) = str + ", bar"
val fmap = Map("foo" -> foo _, "bar" -> bar _)
fmap("foo")("hello")
// Exiting paste mode, now interpreting.
foo: (str: String)String
bar: (str: String)String
fmap: scala.collection.immutable.Map[String,String => String] = Map(foo -> $$Lambda$1104/1335082762#778a1250, bar -> $$Lambda$1105/841090268#55acec99)
res0: String = hello, foo
Spark offers you a way to write your transformations or queries using SQL. So, you really don't have to worry about Scala functions, casting and evaluation in this case. You just have to parse your config to generate the SQL query.
Let's say you have registered a table users with Spark and want to do a select and transform based on provided config,
// your generated query will look like this,
val query = "SELECT from_unixtime(epoch) as time, weekofyear(epoch) FROM users"
val result = spark.sql(query)
So, all you need to do is - build that query from your config.
I am trying to dinamically interpret code given as a String.
Eg:
val myString = "def f(x:Int):Int=x+1".
Im looking for a method that will return the real function out of it:
Eg:
val myIncrementFunction = myDarkMagicFunctionThatWillBuildMyFunction(myString)
println(myIncrementFunction(3))
will print 4
Use case: I want to use some simple functions from that interpreted code later in my code. For example they can provide something like def fun(x: Int): Int = x + 1 as a string, then I use the interpreter to compile/execute that code and then I'd like to be able to use this fun(x) in a map for example.
The problem is that that function type is unknown for me, and this is one of the big problems because I need to cast back from IMain.
I've read about reflection, type system and such, and after some googling I reached this point. Also I checked twitter's util-eval but I cant see too much from the docs and the examples in their tests, it's pretty the same thing.
If I know the type I can do something like
val settings = new Settings
val imain = new IMain(settings)
val res = imain.interpret("def f(x:Int):Int=x+1; val ret=f _ ")
val myF = imain.valueOfTerm("ret").get.asInstanceOf[Function[Int,Int]]
println(myF(2))
which works correctly and prints 3 but I am blocked by the problem I said above, that I dont know the type of the function, and this example works just because I casted to the type I used when I defined the string function for testing how IMain works.
Do you know any method how I could achieve this functionality ?
I'm a newbie so please excuse me if I wrote any mistakes.
Thanks
Ok, I managed to achieve the functionality I wanted, I am still looking for improving this code, but this snippet does what I want.
I used scala toolbox and quasiquotes
import scala.reflect.runtime.universe.{Quasiquote, runtimeMirror}
import scala.tools.reflect.ToolBox
object App {
def main(args: Array[String]): Unit = {
val mirror = runtimeMirror(getClass.getClassLoader)
val tb = ToolBox(mirror).mkToolBox()
val data = Array(1, 2, 3)
println("Data before function applied on it")
println(data.mkString(","))
println("Please enter the map function you want:")
val function = scala.io.StdIn.readLine()
val functionWrapper = "object FunctionWrapper { " + function + "}"
val functionSymbol = tb.define(tb.parse(functionWrapper).asInstanceOf[tb.u.ImplDef])
// Map each element using user specified function
val dataAfterFunctionApplied = data.map(x => tb.eval(q"$functionSymbol.function($x)"))
println("Data after function applied on it")
println(dataAfterFunctionApplied.mkString(","))
}
}
And here is the result in the terminal:
Data before function applied on it
1,2,3
Please enter the map function you want:
def function(x: Int): Int = x + 2
Data after function applied on it
3,4,5
Process finished with exit code 0
I wanted to elaborate the previous answer with the comment and perform an evaluation of the solutions:
import scala.reflect.runtime.universe.{Quasiquote, runtimeMirror}
import scala.tools.reflect.ToolBox
object Runtime {
def time[R](block: => R): R = {
val t0 = System.nanoTime()
val result = block // call-by-name
val t1 = System.nanoTime()
println("Elapsed time: " + (t1 - t0) + " ns")
result
}
def main(args: Array[String]): Unit = {
val mirror = runtimeMirror(getClass.getClassLoader)
val tb = ToolBox(mirror).mkToolBox()
val data = Array(1, 2, 3)
println(s"Data before function applied on it: '${data.toList}")
val function = "def apply(x: Int): Int = x + 2"
println(s"Function: '$function'")
println("#######################")
// Function with tb.eval
println(".... with tb.eval")
val functionWrapper = "object FunctionWrapper { " + function + "}"
// This takes around 1sec!
val functionSymbol = time { tb.define(tb.parse(functionWrapper).asInstanceOf[tb.u.ImplDef])}
// This takes around 0.5 sec!
val result = time {data.map(x => tb.eval(q"$functionSymbol.apply($x)"))}
println(s"Data after function applied on it: '${result.toList}'")
println(".... without tb.eval")
val func = time {tb.eval(q"$functionSymbol.apply _").asInstanceOf[Int => Int]}
// This takes around 0.5 sec!
val result2 = time {data.map(func)}
println(s"Data after function applied on it: '${result2.toList}'")
}
}
If we execute the code above we see the following output:
Data before function applied on it: 'List(1, 2, 3)
Function: 'def apply(x: Int): Int = x + 2'
#######################
.... with tb.eval
Elapsed time: 716542980 ns
Elapsed time: 661386581 ns
Data after function applied on it: 'List(3, 4, 5)'
.... without tb.eval
Elapsed time: 394119232 ns
Elapsed time: 85713 ns
Data after function applied on it: 'List(3, 4, 5)'
Just to emphasize the importance of do the evaluation to extract a Function, and then apply to the data, without the end to evaluate again, as the comment in the answer indicates.
You can use twitter-util library to do this, check the test file:
https://github.com/twitter/util/blob/b0696d0/util-eval/src/test/scala/com/twitter/util/EvalTest.scala
If you need to use IMain, maybe because you want to use the intepreter with your own custom settings, you can do something like this:
a. First create a class meant to hold your result:
class ResHolder(var value: Any)
b. Create a container object to hold the result and interpret the code into that object:
val settings = new Settings()
val writer = new java.io.StringWriter()
val interpreter = new IMain(settings, writer)
val code = "def f(x:Int):Int=x+1"
// Create a container object to hold the result and bind in the interpreter
val holder = new ResHolder(null)
interpreter.bind("$result", holder.getClass.getName, holder) match {
case Success =>
case Error => throw new ScriptException("error in: binding '$result' value\n" + writer)
case Incomplete => throw new ScriptException("incomplete in: binding '$result' value\n" + writer)
}
val ir = interpreter.interpret("$result.value = " + code)
// Return cast value or throw an exception based on result
ir match {
case Success =>
val any = holder.value
any.asInstanceOf[(Int) => Int]
case Error => throw new ScriptException("error in: '" + code + "'\n" + writer)
case Incomplete => throw new ScriptException("incomplete in :'" + code + "'\n" + writer)
}
I am writing a macro m(expr: String), where expr is an expression in some language (not Scala):
m("SOME EXPRESSION")
m("""
SOME EXPRESSION
""")
When I am parsing the expression I would like to report error messages with proper locations in the source file. To achieve this I should know the location of the string literal itself and the number of quotes of the literal (3 or 1). Unfortunately, I did not find any method that returns the number of quotes of the literal:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object Temp {
def m(s: String) : String = macro mImpl
def mImpl(context: Context)(s: context.Expr[String]): context.universe.Tree = {
import context.universe._
s match {
case l # Literal(Constant(p: String)) =>
if (l.<TRIPLE QUOTES>) {
...
} else {
...
}
case _ =>
context.abort(context.enclosingPosition, "The argument of m must be a string literal")
}
}
}
What should I put instead of <TRIPLE QUOTES>?
The only way i can think of is to access the source file and check for triple quotes:
l.tree.pos.source.content.startsWith("\"\"\"",l.tree.pos.start)
You need also to edit your matching case:
case l # Expr(Literal(Constant(p: String))) =>
Here the version with some explanation:
val tree: context.universe.Tree = l.tree
val pos: scala.reflect.api.Position = tree.pos
val source: scala.reflect.internal.util.SourceFile = pos.source
val content: Array[Char] = source.content
val start: Int = pos.start
val isTriple: Boolean = content.startsWith("\"\"\"",start)
I would like to create a new string interpolator macro, lets call it 'X', that functions like "s" when it comes to interpolating new values in. The string I pass into my interpolator is first passed through "s" and then the resultant string is post-processed by 'X'. Optimally I'd like the string to be resolved before it even hits the macro. Is that possible at compile time? I don't want the macro to know about any referenced variables.
Thanks!
implicit class XInterp (val sc : StringContext) {
def X(args: Any*): String = macro XMacro
}
def XMacro(c: Context)(args: c.Expr[Any]*): c.Expr[String] = {
// args(0).tree == "Foo has a value of 5 and this is awesome!"?
}
val foo = 5
val bar = "this is awesome!"
val result = X"Foo has a value of $foo and $bar"
println(result)
"Foo has a value of 5 and this is awesome!"