I'm writing a run-time compilation module. Code is as follows:
val settings = new Settings
settings.usejavacp.value = true
val interpreter = new IMain(settings)
interpreter.interpret(""" val result = the scala code here """)
interpreter.valueOfTerm("result")//return the result var
But I have some problem.
How to let multiple threads call IMain return different results? Is assigned a IMain instance to a thread? This will be a waste of computer resources (non-stop compilation)
how to redefine has been compiled class?
interpreter generated variables (such as the "result") will always be preserved? How to remove the temporary variable? Is there a better api can be called directly, or how to improve my code?
Related
The error message
`value` can only be used within a task or setting macro, such as :=, +=, ++=, Def.task, or Def.setting.
val x = version.value
^
clearly indicates how to fix the problem, for example, using :=
val x = settingKey[String]("")
x := version.value
The explanation in sbt uses macros heavily states
The value method itself is in fact a macro, one that if you invoke it
outside of the context of another macro, will result in a compile time
error, the exact error message being...
And you can see why, since sbt settings are entirely declarative, you
can’t access the value of a task from the key, it doesn’t make sense
to do that.
however I am confused what is meant by declarative nature of sbt being the reason. For example, intuitively I would think the following vanilla Scala snippet is semantically similar to sbt's
def version: String = ???
lazy val x = s"Hello $version" // ok
trait Foo {
def version: String
val x = version // ok
}
As this is legal, clearly the Scala snippet is not semantically equivalent to the sbt one. I was wondering if someone could elaborate on why value cannot be used outside macros? Is the reason purely syntactic related to macro syntax or am I missing something fundamental about sbt's nature?
As another sentence there says
Defining sbt’s task engine is done by giving sbt a series of settings, each setting declaring a task implementation. sbt then executes those settings in order. Tasks can be declared multiple times by multiple settings, the last one to execute wins.
So at the moment when the line
val x = version.value
would be executed (if it were allowed!), that whole program is still being set up and SBT doesn't know the final definition of version.
In what sense is the program "still being set up"?
SBT's order of actions is, basically (maybe missing something):
All your Scala build code is run.
It contains some settings and tasks definitions, SBT collects those when it encounters (along with ones from core, plugins, etc.)
They are topologically sorted into the task graph, deduplicated ("the last one to execute wins"), etc.
Settings are evaluated.
Now you are allowed to actually run tasks (e.g. from SBT console).
version.value is only available after step 4, but val x = version.value runs on step 1.
Would not lazy evaluation take care of that?
Well, when you write val x = ... there is no lazy evaluation. But lazy val x = ... runs on step 1 too.
In a spark project we use Typesafe Config. We have a big config file with some (necessary) redundancy. It is quite easy to refer the wrong branch of the config json and sometimes the error slips to production code.
I need to verify that all the calls of config.getString(x) with x literal will not fail for a given config file.
I'd like to write a unit test that checks every string used in my application to obtain a config value.
A possible solution we found is to preload all the config values to a case class, so
val rawPath = config.getString("comp1.data.files.rawData")
val coresNumber = config.getLong("comp1.setup.cores")
would become
case class ConfigData(rawPath:String, coresNumber:Long)
def initConfig():ConfigData ={
val rawPath = config.getString("comp1.data.files.rawData")
val coresNumber = config.getLong("comp1.setup.cores")
ConfigData(rawPat,coresNumber)
}
val conf = initConfig()
val rawPath = conf.rawPath
val coresNumner = conf.coresNumber
and then simply call initData() to test for config load errors.
I've also thought of using scala reflection but I'd need to find all the places in my code where config.getString(x) is called and then obtain the x to test their existence in the config file, but I can't find a way to get all the instances of a method call and create a test on the parameter.
Is there anything I haven't thought of?
One solution is to provide the configuration in a Singleton, which you initialise when starting the server - so the Server starts only if the configuration is correct. Or run the configurations in Unit Tests.
In this Singleton you load your Configuration in single values or case classes depending on amount of values.
We use pureconfig which makes it pretty simple to map the configuration directly into case classes.
Here is an Example without Pure Script: DemoAdapterContext and AdaptersContext.
Let me know if you need more infos.
I'm writing an sbt plugin, and have created a TaskKey that need to get parsed arguments
lazy val getManager = TaskKey[DeployManager]("Deploy manager")
lazy val getCustomConfig = InputKey[String]("Custom config")
...
getCustomConfig := {
spaceDelimited("<arg>").parsed(0)
}
getManager := {
val conf = configResource.evaluated
...
}
but I get this error during compilation:
`parsed` can only be used within an input task macro, such as := or Def.inputTask.
I can't define getManager as InputKey since I later use it's value many times, and for an inputKey the value gets created anew on each evaluation (and I need to use the same instance)
You cannot do what you want in a reasonable way in sbt. (And the type system nicely prevents you from doing that in this case).
Imagine that getManager is a TaskKey that takes parsed arguments (a note aside, the sbt way of naming this would probably be manager, get is implied).
I now decide that, for example, compile depends on getManager. If I type compile in the shell, what arguments should getManager parse?
There is no concept of arguments inside the sbt dependency tree. They are just a shallow (and IMHO somewhat hackish) addition to make for a nicer CLI.
If you want to make getManager configurable, you can add additional settings, getManager depends on and then use set on the command line to change these where necessary.
So in you case:
lazy val configResource = SettingKey[...]("Config resource")
getManager := {
val conf = configResource.value
// ...
}
I'm trying to compile Scala files programmatically by using an instance of Global.Run:
val settings = new Settings
val reporter = new ConsoleReporter(settings)
val compiler = new Global(settings, reporter)
val run = new compiler.Run // MissingRequirementError
run compile List(path)
Unfortunately I get a MissingRequirementError saying:
object scala.runtime in compiler mirror not found
So my question is how can I compile a file programmatically by using the Run class, or what am I doing wrong here?
I tried to figure out whether I could change the settings in order to get it work. Actually I need a list of classes that are in the Scala file at path, not necessarily a fully runnable output. It would therefore be fine if symbols remained unresolved (if I could run a subset of the compiler phases).
I also at Writing Scala Compiler Plugins, but if I can run it by instanciating a Compiler Run object, I'd prefer this solution. I also stumbled across Is the Scala compiler reentrant? (similar code, different question), which makes me think it might work the way I'm thinking of.
Edit 1: Added Scala JARs to the toolcp (just sample code with absolute path!)
According to a comment I adapted scalac.bat's classpath population script to my Scala code:
// scalac.bat
// if "%_TOOL_CLASSPATH%"=="" (
// for %%f in ("!_SCALA_HOME!\lib\*") do call :add_cpath "%%f"
// for /d %%f in ("!_SCALA_HOME!\lib\*") do call :add_cpath "%%f"
// )
new File("C:\\Program Files\\scala\\lib").listFiles.foreach(f => {
settings.classpath.append(f.getAbsolutePath)
settings.toolcp.append(f.getAbsolutePath)
})
I got it running, by using bootclasspath instead of toolcp (thanks to pedrofurla's hint):
val settings = new Settings
new File("C:\\Program Files\\scala\\lib").listFiles.foreach(f => {
settings.classpath.append(f.getAbsolutePath)
settings.bootclasspath.append(f.getAbsolutePath)
})
private val reporter = new ConsoleReporter(settings)
private val compiler = new Global(settings, reporter)
val run = new compiler.Run
run compile List(path)
The compiler attempts to compile the files now. However, this seems not to be exactly what scalac.bat does. It starts it with -cp, which is the normal classpath, whereas bootclasspath is passed with -bootclasspath on the console, as visible in StandardScalaSettings trait:
val bootclasspath = PathSetting ("-bootclasspath", "Override location of bootstrap class files.", Defaults.scalaBootClassPath)
I want to create a class at run-time in Scala. For now, just consider a simple case where I want to make the equivalent of a java bean with some attributes, I only know these attributes at run time.
How can I create the scala class? I am willing to create from scala source file if there is a way to compile it and load it at run time, I may want to as I sometimes have some complex function I want to add to the class. How can I do it?
I worry that the scala interpreter which I read about is sandboxing the interpreted code that it loads so that it won't be available to the general application hosting the interpreter? If this is the case, then I wouldn't be able to use the dynamically loaded scala class.
Anyway, the question is, how can I dynamically create a scala class at run time and use it in my application, best case is to load it from a scala source file at run time, something like interpreterSource("file.scala") and its loaded into my current runtime, second best case is some creation by calling methods ie. createClass(...) to create it at runtime.
Thanks, Phil
There's not enough information to know the best answer, but do remember that you're running on the JVM, so any techniques or bytecode engineering libraries valid for Java should also be valid here.
There are hundreds of techniques you might use, but the best choice depends totally on your exact use case, as many aren't general purpose. Here's a couple of ideas though:
For a simple bean, you may as well
just use a map, or look into the
DynaBean class from apache commons.
For more advanced behaviour you could
invoke the compiler explicitly and
then grab the resulting .class file
via a classloader (this is largely
how JSPs do it)
A parser and custom DSL fit well in
some cases. As does bean shell
scripting.
Check out the ScalaDays video here: http://days2010.scala-lang.org/node/138/146
which demonstrates the use of Scala as a JSR-223 compliant scripting language.
This should cover most scenarios where you'd want to evaluate Scala at runtime.
You'll also want to look at the email thread here: http://scala-programming-language.1934581.n4.nabble.com/Compiler-API-td1992165.html#a1992165
This contains the following sample code:
// We currently call the compiler directly
// To reduce coupling, we could instead use ant and the scalac ant task
import scala.tools.nsc.{Global, Settings}
import scala.tools.nsc.reporters.ConsoleReporter
{
// called in the event of a compilation error
def error(message: String): Nothing = ...
val settings = new Settings(error)
settings.outdir.value = classesDir.getPath
settings.deprecation.value = true // enable detailed deprecation warnings
settings.unchecked.value = true // enable detailed unchecked warnings
val reporter = new ConsoleReporter(settings)
val compiler = new Global(settings, reporter)
(new compiler.Run).compile(filenames)
reporter.printSummary
if (reporter.hasErrors || reporter.WARNING.count > 0)
{
...
}
}
val mainMethod: Method = {
val urls = Array[URL]( classesDir.toURL )
val loader = new URLClassLoader(urls)
try {
val clazz: Class = loader.loadClass(...)
val method: Method = clazz.getMethod("main", Array[Class]( classOf[Array[String]] ))
if (Modifier.isStatic(method.getModifiers)) {
method
} else {
...
}
} catch {
case cnf: ClassNotFoundException => ...
case nsm: NoSuchMethodException => ...
}
}
mainMethod.invoke(null, Array[Object]( args ))