How to invoke the Scala compiler programmatically? - scala

I want my Scala code to take a Scala class as input, compile and execute that class. How can I programmatically invoke a Scala compiler? I will be using the latest Scala version, i.e. 2.10.

ToolBox
I think the proper way of invoking the Scala compiler is doing it via Reflection API documented in Overview. Specifically, Tree Creation via parse on ToolBoxes section in 'Symbols, Trees, and Types' talks about parsing String into Tree using ToolBox. You can then invoke eval() etc.
scala.tools.nsc.Global
But as Shyamendra Solanki wrote, in reality you can drive scalac's Global to get more done. I've written CompilerMatcher so I can compile generated code with sample code to do integration tests for example.
scala.tools.ncs.IMain
You can invoke the REPL IMain to evaluate the code (this is also available in the above CompilerMatcher if you want something that works with Scala 2.10):
val main = new IMain(s) {
def lastReq = prevRequestList.last
}
main.compileSources(files.map(toSourceFile(_)): _*)
code map { c => main.interpret(c) match {
case IR.Error => sys.error("Error interpreting %s" format (c))
case _ =>
}}
val holder = allCatch opt {
main.lastReq.lineRep.call("$result")
}
This was demonstrated in Embedding the Scala Interpreter post by Josh Suereth back in 2009.

The class to be compiled and run (in file test.scala)
class Test {
println ("Hello World!")
}
// compileAndRun.scala (in same directory)
import scala.tools.nsc._
import java.io._
val g = new Global(new Settings())
val run = new g.Run
run.compile(List("test.scala")) // invoke compiler. it creates Test.class.
val classLoader = new java.net.URLClassLoader(
Array(new File(".").toURI.toURL), // Using current directory.
this.getClass.getClassLoader)
val clazz = classLoader.loadClass("Test") // load class
clazz.newInstance // create an instance, which will print Hello World.

Related

Using Scala Toolbox eval, how do I define I value I can use in later evals?

I'm using Scala toolbox to eval some Scala code for a web interpreter. It is working well, the code looks like this:
import scala.reflect.runtime.universe._
import scala.tools.reflect.ToolBox
object Eval {
val toolbox = runtimeMirror(getClass.getClassLoader).mkToolBox()
def eval[T](code: String): T = {
toolbox.eval(toolbox.parse(code)).asInstanceOf[T]
}
}
I can do something like:
Eval.eval[Long]("1 + 1")
and get 2 back. The question comes up when I want to define something:
Eval.eval[Unit]("val yellow = 5")
Eval.eval[Long]("yellow")
I get a not found: value yellow error. How do I define a value that I can use in later evaluations with Scala Toolbox?
For a persistent environment, you can use Scala's REPL directly. See the JSR-223 note at the very bottom of the 2.11 release notes.
import javax.script.ScriptEngineManager
class DummyClass
object Evaluator {
val engine = new ScriptEngineManager().getEngineByName("scala")
val settings = engine.asInstanceOf[scala.tools.nsc.interpreter.IMain].settings
settings.embeddedDefaults[DummyClass]
engine.eval("val x: Int = 5")
val thing = engine.eval("x + 9").asInstanceOf[Int]
}
The need for DummyClass (or really any class you have can substitute for DummyClass) arises from a bit of shenanigans going on here due to SBT and classloader concerns (more details here).

How to compile/eval a Scala expression at runtime?

New to Scala and looking for pointers to an idiomatic solution, if there is one.
I'd like to have arbitrary user-supplied Scala functions (which are allowed to reference functions/classes I have defined in my code) applied to some data.
For example: I have foo(s: String): String and bar(s: String): String functions defined in my myprog.scala. The user runs my program like this:
$ scala myprog data.txt --func='(s: Str) => foo(bar(s)).reverse'
This would run line by line through the data file and emit the result of applying the user-specified function to that line.
For extra points, can I ensure that there are no side-effects in the user-defined function? If not, can I restrict the function to use only a restricted subset of functions (which I can assure to be safe)?
#kenjiyoshida has a nice gist that shows how to eval Scala code. Note that when using Eval from that gist, not specifying a return value will result in a runtime failure when Scala defaults to inferring Nothing.
scala> Eval("println(\"Hello\")")
Hello
java.lang.ClassCastException: scala.runtime.BoxedUnit cannot be cast to scala.runtime.Nothing$
... 42 elided
vs
scala> Eval[Unit]("println(\"Hello\")")
Hello
It nicely handles whatever's in scope as well.
object Thing {
val thing: Int = 5
}
object Eval {
def apply[A](string: String): A = {
val toolbox = currentMirror.mkToolBox()
val tree = toolbox.parse(string)
toolbox.eval(tree).asInstanceOf[A]
}
def fromFile[A](file: File): A =
apply(scala.io.Source.fromFile(file).mkString(""))
def fromFileName[A](file: String): A =
fromFile(new File(file))
}
object Thing2 {
val thing2 = Eval[Int]("Thing.thing") // 5
}
Twitter's util package used to have util-eval, but that seems to have been deprecated now (and also triggers a compiler bug when compiled).
As for the second part of your question, the answer seems to be no. Even if you disable default Predef and imports yourself, a user can always get to those functions with the fully qualified package name. You could perhaps use Scala's scala.tools.reflect.ToolBox to first parse your string and then compare against a whitelist, before passing to eval, but at that point things could get pretty hairy since you'll be manually writing code to sanitize the Scala AST (or at the very least reject dangerous input). It definitely doesn't seem to be an "idiomatic solution."
This should be possible by using the standard Java JSR 223 Scripting Engine
see https://issues.scala-lang.org/browse/SI-874
(also mentions using scala.tools.nsc.Interpreter but not sure this is still available)
import javax.script.*;
ScriptEngine e = new ScriptEngineManager().getEngineByName("scala");
e.getContext().setAttribute("label", new Integer(4), ScriptContext.ENGINE_SCOPE);
try {
engine.eval("println(2+label)");
} catch (ScriptException ex) {
ex.printStackTrace();
}

Generating a class from string and instantiating it in Scala 2.10

In Scala 2.10 how do I generate a class from string (probably, using the Toolbox api) later to be instantiated with Scala's reflection?
W.r.t compilation toolboxes can only run expressions = return values, but not resulting classes or files/byte arrays with compilation results.
However it's still possible to achieve what you want, since in Scala it's so easy to go from type level to value level using implicit values:
Edit. In 2.10.0-RC1 some methods of ToolBox have been renamed. parseExpr is now just parse, and runExpr is now called eval.
scala> import scala.reflect.runtime._ // requires scala-reflect.jar
// in REPL it's implicitly added
// to the classpath
// but in your programs
// you need to do this on your own
import scala.reflect.runtime
scala> val cm = universe.runtimeMirror(getClass.getClassLoader)
cm # 41d0fe80: reflect.runtime.universe.Mirror = JavaMirror with scala.tools.nsc.interpreter.IMain$TranslatingClassLoader...
scala> import scala.tools.reflect.ToolBox // requires scala-compiler.jar
// in REPL it's implicitly added
// to the classpath
// but in your programs
// you need to do this on your own
import scala.tools.reflect.ToolBox
scala> val tb = cm.mkToolBox()
tb: scala.tools.reflect.ToolBox[reflect.runtime.universe.type] = scala.tools.reflect.ToolBoxFactory$ToolBoxImpl#3a962da5
scala> tb.runExpr(tb.parseExpr("class C; scala.reflect.classTag[C].runtimeClass"))
res2: Any = class __wrapper$1$f9d572ca0d884bca9333e251c64e980d$C$1
Update #1. If you don't need a java.lang.Class and just need to instantiate the compiled class, you can write new C directly in the string submitted to runExpr.
Update #2. It is also possible to have runExpr use custom mapping from variable names to runtime values. For example:
scala> val build = scala.reflect.runtime.universe.build
build: reflect.runtime.universe.BuildApi = scala.reflect.internal.BuildUtils$BuildImpl#50d5afff
scala> val x = build.setTypeSignature(build.newFreeTerm("x", 2), typeOf[Int])
x: reflect.runtime.universe.FreeTermSymbol = free term x
scala> tb.runExpr(Apply(Select(Ident(x), newTermName("$plus")), List(Literal(Constant(2)))))
res0: Any = 4
In this example I create a free term that has a value of 2 (the value doesn't have to be a primitive - it can be your custom object) and bind an identifier to it. This value is then used as-is in the code that is compiled and run by a toolbox.
The example uses manual AST assembly, but it's possible to write a function that parses a string, finds out unbound identifiers, looks up values for them in some mapping and then creates corresponding free terms. There's no such function in Scala 2.10.0 though.

Is it possible to use scalap from a scala script?

I am using scalap to read out the field names of some case classes (as discussed in this question). Both the case classes and the code that uses scalap to analyze them have been compiled and put into a jar file on the classpath.
Now I want to run a script that uses this code, so I followed the instructions and came up with something like
::#!
#echo off
call scala -classpath *;./libs/* %0 %*
goto :eof
::!#
//Code relying on pre-compiled code that uses scalap
which does not work:
java.lang.ClassCastException: scala.None$ cannot be cast to scala.Option
at scala.tools.nsc.interpreter.ByteCode$.caseParamNamesForPath(ByteCode.
scala:45)
at scala.tools.nsc.interpreter.ProductCompletion.caseNames(ProductComple
tion.scala:22)
However, the code works just fine when I compile everything. I played around with additional scala options like -savecompiled, but this did not help. Is this a bug, or can't this work in principle? (If so, could someone explain why not? As I said, the case classes that shall be analyzed by scalap are compiled.)
Note: I use Scala 2.9.1-1.
EDIT
Here is what I am essentially trying to do (providing a simple way to create multiple instances of a case class):
//This is pre-compiled:
import scala.tools.nsc.interpreter.ProductCompletion
//...
trait MyFactoryTrait[T <: MyFactoryTrait[T] with Product] {
this: T =>
private[this] val copyMethod = this.getClass.getMethods.find(x => x.getName == "copy").get
lazy val productCompletion = new ProductCompletion(this)
/** The names of all specified fields. */
lazy val fieldNames = productCompletion.caseNames //<- provokes the exception (see above)
def createSeq(...):Seq[T] = {
val x = fieldNames map { ... } // <- this method uses the fieldNames value
//[...] invoke copyMethod to create instances
}
// ...
}
//This is pre-compiled too:
case class MyCaseClass(x: Int = 0, y: Int = 0) extends MyFactoryTrait[MyCaseClass]
//This should be interpreted (but crashes):
val seq = MyCaseClass().createSeq(...)
Note: I moved on to Scala 2.9.2, the error stays the same (so probably not a bug).
This is a bug in the compiler:
If you run the program inside an ide, for example Intellij IDEA the code is executed fine, however no fields names are found.
If you run it from command line using scala, you obtain the error you mentioned.
There is no way type-safe could should ever compiler and throw a runtime ClassCastException.
Please open a bug at https://issues.scala-lang.org/secure/Dashboard.jspa

Get Scala variable name at runtime

Is it possible to get the name of a scala variable at runtime?
E.g. is it possible to write a function getIntVarName(variable: Int): String behaving as follows?
val myInt = 3
assert("myInt" === getIntVarName(myInt))
For what you need to do, It seems to me that runtime is not required, since you already have your myInt variable defined at compile time. If this is the case, you just need a bit of AST manipulation via a macro.
Try
package com.natalinobusa.macros
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object Macros {
// write macros here
def getName(x: Any): String = macro impl
def impl(c: Context)(x: c.Tree): c.Tree = {
import c.universe._
val p = x match {
case Select(_, TermName(s)) => s
case _ => ""
}
q"$p"
}
}
Be aware that macro's must be compiled as a separate subproject, and cannot be part of the same project where the macro substitution has to be applied. Check this template on how to define such a macro sub-project: https://github.com/echojc/scala-macro-template
scala> import Macros._
import Macros._
scala> val myInt = 3
myInt: Int = 3
scala> "myInt" == getName(myInt)
res6: Boolean = true
You can use scala-nameof to get a variable name, function name, class member name, or type name. It happens at compile-time so there's no reflection involved and no runtime dependency needed.
val myInt = 3
assert("myInt" === nameOf(myInt))
will compile to:
val myInt = 3
assert("myInt" === "myInt")
Basically, it can't be done.
The JVM offers nothing by way of a Method handle (remember, Scala properties are encoded as methods in bytecode to support the uniform access principle). The closest you can get is to use reflection to find a list of methods defined on a particular class - which I appreciate doesn't help with your particular need.
It is possible to implement this as a Scala feature, but it would require a compiler plugin to grab the relevant symbol name from the AST and push it into code as a string literal, so not something I could demonstrate in a short code snippet :)
The other naming problem that often comes up in reflection is method parameters. That one at least I can help with. I have a work-in-progress reflection library here that's based on the compiler-generated scala signature as used by scalap. It's nowhere near being ready for serious use, but it is under active development.
Scala doesn't yet have much more than Java in terms of metadata like this. Keep an eye on the Scala Reflection project, but I doubt that will offer access to local variables anytime soon. In the meantime, consider a bytecode inspector library like ASM. Another big caveat: local variable names are lost during compilation, so you'd need to compile in "debug" mode to preserve them.
I don't think it's possible to get the name of a variable, but you can try it with objects:
object Test1 {
def main(args: Array[String]) {
object MyVar {
def value = 1
}
println(MyVar.getClass)
}
}
This prints: class Test1$MyVar$2$. So you can get 'MyVar' out of it.
This can be achieved with Scala 3 Macros (does it at compile time).
Create a Macro object (this must be in a separate file):
import scala.quoted.{Expr, Quotes}
object NameFromVariable :
def inspectCode(x: Expr[Any])(using Quotes): Expr[String] =
val name = x.show.split("""\.""").last
Expr(name)
Then you need an inline method in your class.
inline def getIntVarName(inline x: Any): Any = ${ NameFromVariable.inspectCode('x) }
And use this method, like:
val myInt = 3
assert("myInt" === getIntVarName(myInt))
See the official documentation: https://docs.scala-lang.org/scala3/guides/macros/macros.html