Scala REPL crashes when started using scala.tools.nsc.interpreter - scala

I'm trying to use scala.tools.nsc.interpreter to enable interactive debugging (like Python's pdb/ipdb):
val foo = 123
import scala.tools.nsc.Settings
import scala.tools.nsc.interpreter.{ ILoop, SimpleReader }
val repl = new ILoop
repl.settings = new Settings
repl.settings.usejavacp.value = true
repl.in = SimpleReader()
repl.createInterpreter()
repl.intp.bind("foo", "Int", foo)
repl.loop()
repl.closeInterpreter()
When run, this is what I get:
$ scala repl.scala
foo: Int = 123
scala> "hello"
java.lang.NullPointerException
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:95)
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:95)
at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.ready(package.scala:95)
at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:402)
at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:430)
at Main$$anon$1.<init>(repl.scala:14)
at Main$.main(repl.scala:1)
at Main.main(repl.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at scala.reflect.internal.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:70)
at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:101)
at scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:70)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:101)
at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:22)
at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:39)
at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:39)
at scala.tools.nsc.ScriptRunner.scala$tools$nsc$ScriptRunner$$runCompiled(ScriptRunner.scala:175)
at scala.tools.nsc.ScriptRunner$$anonfun$runScript$1.apply(ScriptRunner.scala:192)
at scala.tools.nsc.ScriptRunner$$anonfun$runScript$1.apply(ScriptRunner.scala:192)
at scala.tools.nsc.ScriptRunner$$anonfun$withCompiledScript$1$$anonfun$apply$mcZ$sp$1.apply(ScriptRunner.scala:161)
at scala.tools.nsc.ScriptRunner$$anonfun$withCompiledScript$1.apply$mcZ$sp(ScriptRunner.scala:161)
at scala.tools.nsc.ScriptRunner$$anonfun$withCompiledScript$1.apply(ScriptRunner.scala:129)
at scala.tools.nsc.ScriptRunner$$anonfun$withCompiledScript$1.apply(ScriptRunner.scala:129)
at scala.tools.nsc.util.package$.trackingThreads(package.scala:43)
at scala.tools.nsc.util.package$.waitingForThreads(package.scala:27)
at scala.tools.nsc.ScriptRunner.withCompiledScript(ScriptRunner.scala:128)
at scala.tools.nsc.ScriptRunner.runScript(ScriptRunner.scala:192)
at scala.tools.nsc.ScriptRunner.runScriptAndCatch(ScriptRunner.scala:205)
at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:67)
at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:87)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:98)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
Abandoning crashed session.
scala>
If I remove the repl.intp.bind("foo", "Int", foo) part, this is what I get:
$ scala repl.scala
scala> 123
null
Abandoning crashed session.
scala>
What am I doing wrong? Is there an easier way to drop into an interactive REPL during a program run for debugging purposes? breakpoints, stepping through and inspecting locals just doesn't do it sometimes.
I'm on Scala 2.11.5.

I don't know why this comment doesn't have the magic asterisks:
// start an interpreter with the given settings
def process(settings: Settings): Boolean
So you can:
scala> repl process s
Welcome to Scala version 2.11.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_25).
Type in expressions to have them evaluated.
Type :help for more information.
scala> :quit
res0: Boolean = true
Or you could:
package myrepl
import scala.tools.nsc.Settings
import scala.tools.nsc.interpreter.{ ILoop, SimpleReader }
object Test extends App {
val foo = 42
val repl = new ILoop {
override def printWelcome() = {
intp.bind("foo", foo)
super.printWelcome()
echo("Customized...")
}
}
val s = new Settings
s.usejavacp.value = true
repl.in = SimpleReader()
repl process s
/*
repl.createInterpreter()
repl.intp.bind("foo", "Int", foo)
repl.loop()
repl.closeInterpreter()
*/
}
And
$ scalac myrepl.scala && scala myrepl.Test
foo: Int = 42
Welcome to Scala version 2.11.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_20).
Type in expressions to have them evaluated.
Type :help for more information.
Customized...
scala> foo
res0: Int = 42
scala> :quit
Also,
s.Xnojline.value = true
//repl.in = SimpleReader()
I'm going to downgrade to 2.11.4 because of the repl crashing bug...
Sbt does it by overriding createInterpreter and calling bind there.
Edit:
val repl = new ILoop {
override def printWelcome() = {
//import scala.concurrent.duration._
//Await.ready(globalFuture, 10.minutes) // sorry, it's private!
super.printWelcome()
echo("Customizing...")
processLine("") // block for init to finish
intp.bind("foo", foo)
}
}
The private globalFuture is an impediment to exploding the startup:
object Solid extends App {
val foo = 42
val repl = new ILoop {
override def printWelcome() = {
super.printWelcome()
echo("Customized...")
}
}
val s = new Settings
s.Xnojline.value = true
s.usejavacp.value = true
repl.settings = s
repl.createInterpreter()
repl.in = SimpleReader()
repl.intp.initializeSynchronous()
repl.loopPostInit()
repl.globalFuture = concurrent.Future.successful(true)
repl.intp.bind("foo", "Int", foo)
try repl.loop()
finally repl.closeInterpreter()
}
The -Yrepl-sync option is no longer honored in 2.11.

Related

ILoop Tab Completion

I am creating a very simple extension of scala.tools.nsc.interpreter.ILoop with the intent of adding some additional bindings, but even in the most basic use-case the tab-completion does not seem to work. If I type in code it interprets and works as expected, but I no tab-completion. Is there something specific that needs to be defined in order for tab-completion to be enabled in the interactive interpreter (REPL)?
My use-case is as simple as the following:
val repl = new ILoop
repl.process(new Settings {
usejavacp.value = true
deprecation.value = true
})
Is there something other than ILoop I should be using?
It kind of works for me, modulo version.
$ scalacm myintp.scala && scalam myintp.Test
Welcome to Scala 2.12.0-RC2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_101).
Type in expressions for evaluation. Or try :help.
scala> 42
res0: Int = 42
scala> 42.
!= < >>> doubleValue isNaN isValidShort shortValue toDouble toShort
% << ^ floatValue isNegInfinity isWhole signum toFloat unary_+
& <= abs floor isPosInfinity longValue to toHexString unary_-
* == byteValue getClass isValidByte max toBinaryString toInt unary_~
+ > ceil intValue isValidChar min toByte toLong underlying
- >= compare isInfinite isValidInt round toChar toOctalString until
/ >> compareTo isInfinity isValidLong self toDegrees toRadians |
scala> 42.s
self shortValue signum synchronized
scala> 42.self
res1: Int = 42
scala> :quit
Source:
$ cat myintp.scala
package myintp
import scala.tools.nsc._
import scala.tools.nsc.interpreter._
/* 2.12 */
object Test extends App {
val ss = new Settings {
usejavacp.value = true
deprecation.value = true
}
def repl = new ILoop {
override def createInterpreter(): Unit = {
super.createInterpreter()
}
}
repl process ss
}
/* 2.11
object Test extends App {
def repl = new ILoop {
override def createInterpreter(): Unit = {
def binder: Unit = intp beQuietDuring {
intp directBind ("foo", "bar")
intp bind ("baz", "boo")
}
super.createInterpreter()
intp initialize binder
}
}
repl process new Settings
}
*/
/* 2.9
object Test extends App {
def repl = new ILoop {
def binder: Unit = intp beQuietDuring {
intp bind ("baz", "boo")
}
override def loop(): Unit = {
binder
super.loop()
}
}
repl process new Settings
}
*/

How can I get all (non-final) object vals and subobject vals using reflection in Scala?

Note: This question is not a duplicate of How can I get all object vals and subobject vals using reflection in Scala?
The answer provided in that question only works for final members.
For example:
scala> object Settings {
| val Host = "host"
| }
defined module Settings
deepMembers(Settings)
res0: Map[String,String] = Map()
It must be a duplicate, but I need a refresher:
$ scala
Welcome to Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_45).
Type in expressions to have them evaluated.
Type :help for more information.
scala> object Settings { val Host = "host" ; val Guest = "guest" }
defined object Settings
scala> import reflect.runtime._,universe._
import reflect.runtime._
import universe._
scala> val im = currentMirror reflect Settings
im: reflect.runtime.universe.InstanceMirror = instance mirror for Settings$#c8e4bb0
scala> im.symbol.asClass.typeSignature.members filter (s => s.isTerm && s.asTerm.isAccessor)
res0: Iterable[reflect.runtime.universe.Symbol] = SynchronizedOps(value Guest, value Host)
scala> res0 map (im reflectMethod _.asMethod) map (_.apply())
res2: Iterable[Any] = List(guest, host)
val members = Settings.getClass.getDeclaredFields.map(_.getName).filterNot(_ == "MODULE$")
members: Array[String] = Array(Host)
This works but I think there's certainly a better way of doing this.

Get same completion candidates for IMain as would appear in a REPL

I am trying to get the code completion for a Scala interpreter to work. Ideally it would work identically to the one provided by the REPL (ILoop). I am using a text document as source, so I do not want to instantiate an ILoop but just IMain.
In the following example, the completion only works for special cases:
import scala.tools.nsc.interpreter.{JLineCompletion, IMain}
import scala.tools.nsc.Settings
object CompletionTest extends App {
val settings = new Settings
settings.usejavacp.tryToSetFromPropertyValue("true")
val intp = new IMain(settings)
intp.initializeSynchronous()
assert(intp.isInitializeComplete)
val comp = new JLineCompletion(intp)
val completer = comp.completer()
val buffer = "val x = Indexe"
val choices = completer.complete(buffer, buffer.length)
println("----BEGIN COMPLETION----")
choices.candidates.foreach(println)
println("----END COMPLETION----")
intp.close()
}
The expected output would be IndexedSeq, but it is empty. If I set the buffer to just Indexe, it works. If I set the buffer to   Indexe (leading whitespace), the completion candidates are empty again.
So there must be an additional step involved in processing the buffer or invoking the completion. What exactly happens when <tab> is pressed in the REPL? It seems almost impossible to figure out which methods are called...
In JLineReader, you can see the wiring. JLineConsoleReader sets up an ArgumentCompleter with the ScalaCompleter as the underlying completer.
So the completer wants just the argument, not the line.
apm#mara:~$ scalam
Welcome to Scala version 2.11.0-M7 (OpenJDK 64-Bit Server VM, Java 1.7.0_25).
Type in expressions to have them evaluated.
Type :help for more information.
scala> :power
** Power User mode enabled - BEEP WHIR GYVE **
** :phase has been set to 'typer'. **
** scala.tools.nsc._ has been imported **
** global._, definitions._ also imported **
** Try :help, :vals, power.<tab> **
scala> val b = "Indexe"
b: String = Indexe
scala> completion.completer complete (b, b.length)
res0: scala.tools.nsc.interpreter.Completion.Candidates = Candidates(0,List(IndexedSeq))
In other keystrokes,
// paste your code
scala> val buffer = "Indexe"
buffer: String = Indexe
scala> completer.complete(buffer, buffer.length)
res6: scala.tools.nsc.interpreter.Completion.Candidates = Candidates(0,List(IndexedSeq))
scala> import tools.nsc.interpreter.Completion.Candidates
import tools.nsc.interpreter.Completion.Candidates
scala> val Candidates(_, choices) = completer.complete(buffer, buffer.length)
choices: List[String] = List(IndexedSeq)
scala> choices foreach println
IndexedSeq
To hand it the full line:
scala> val argCompletor: ArgumentCompleter =new ArgumentCompleter(new JLineDelimiter, scalaToJline(comp.completer))
argCompletor: jline.console.completer.ArgumentCompleter = jline.console.completer.ArgumentCompleter#751222c7
scala> val maybes = new java.util.ArrayList[CharSequence]
maybes: java.util.ArrayList[CharSequence] = []
scala> val buffer = "val x = Indexe"
buffer: String = val x = Indexe
scala> argCompletor.setStrict(false)
scala> argCompletor.complete(buffer, buffer.length, maybes)
res32: Int = 8
scala> maybes
res33: java.util.ArrayList[CharSequence] = [IndexedSeq]
The delimiter does the line parse.
Edit - some value-added analysis:
"Strict" mode for the completor exists because you can supply a completor for each token on the line, and require each previous argument to be completable. For n completors, all args after nth arg are handled by the last completor.
Partial answer. I managed to drill a hole in that monster by overriding scalaToJline in JLineReader. That method is invoked with a pre-massaged string, following this trace:
at CompletionTest$$anon$1$$anon$2$$anon$3.complete(CompletionTest.scala:37)
at scala.tools.jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:150)
at scala.tools.jline.console.ConsoleReader.complete(ConsoleReader.java:1543)
at scala.tools.jline.console.ConsoleReader.readLine(ConsoleReader.java:1312)
at scala.tools.jline.console.ConsoleReader.readLine(ConsoleReader.java:1170)
at scala.tools.nsc.interpreter.JLineReader.readOneLine(JLineReader.scala:74)
at scala.tools.nsc.interpreter.InteractiveReader$$anonfun$readLine$2.apply(InteractiveReader.scala:42)
at scala.tools.nsc.interpreter.InteractiveReader$$anonfun$readLine$2.apply(InteractiveReader.scala:42)
at scala.tools.nsc.interpreter.InteractiveReader$.restartSysCalls(InteractiveReader.scala:49)
at scala.tools.nsc.interpreter.InteractiveReader$class.readLine(InteractiveReader.scala:42)
at scala.tools.nsc.interpreter.JLineReader.readLine(JLineReader.scala:19)
at scala.tools.nsc.interpreter.ILoop.readOneLine$1(ILoop.scala:568)
at scala.tools.nsc.interpreter.ILoop.innerLoop$1(ILoop.scala:584)
at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:587)
So this my "unwiring" which seems to be working as expected:
import scala.tools.nsc.interpreter._
import scala.tools.jline.console.completer.{Completer, ArgumentCompleter}
import scala.tools.nsc.interpreter.Completion.{Candidates, ScalaCompleter}
import scala.tools.nsc.Settings
import collection.JavaConverters._
 
object Completion2 extends App {
val settings = new Settings
settings.usejavacp.tryToSetFromPropertyValue("true")
val intp = new IMain(settings)
intp.initializeSynchronous()
val completion = new JLineCompletion(intp)
def scalaToJline(tc: ScalaCompleter): Completer = new Completer {
def complete(_buf: String, cursor: Int, candidates: JList[CharSequence]): Int = {
val buf = if (_buf == null) "" else _buf
val Candidates(newCursor, newCandidates) = tc.complete(buf, cursor)
newCandidates foreach (candidates add _)
newCursor
}
}
val argCompletor: ArgumentCompleter =
new ArgumentCompleter(new JLineDelimiter, scalaToJline(completion.completer()))
argCompletor.setStrict(false)
val jlist: java.util.List[CharSequence] = new java.util.ArrayList
val buffer = "val x = Indexe"
argCompletor.complete(buffer, buffer.length, jlist)
val list = jlist.asScala
println("----BEGIN COMPLETION----")
list.foreach(println)
println("----END COMPLETION----")
intp.close()
}
Edit: This has problems with wildcard imports for some reason. Like if I execute
import mypackage.MySymbol
Then MySymbol is found by the completer. But if I execute instead
import mypackage._
Then none of the contents of mypackage are found. Any ideas?

Scala - Initialize REPL environment

-Hi. I'd like to embed Scala REPL with initialized environment into my app. I've looked at IMain class and it seems I could do it via instance of it. The instance is created and then stored into intp public var in process() of ILoop.
How can I bind some names and/or add some imports before process() (e.g. before REPL)?
Following code fails on line 3 because intp is not yet created (=> NPE):
val x = 3
val interp = new ILoop
interp.bind("x", x) // -> interp.intp.bind("x", x)
val settings = new Settings
settings.usejavacp.value = true
interp.process(settings)
Thank you-.
UPDATE: Overriding createInterpreter() unfortunately doesn't work:
val x = 3
val interp = new ILoop {
override def createInterpreter() {
super.createInterpreter()
intp.bind("x", x) // -> interp.intp.bind("x", x)
}
}
val settings = new Settings
settings.usejavacp.value = true
interp.process(settings)
Interpreter is stuck on input (looks like deadlock, happens only with code above):
x: Int = 3
Failed to created JLineReader: java.lang.NoClassDefFoundError: scala/tools/jline/console/completer/Completer
Falling back to SimpleReader.
Welcome to Scala version 2.9.2 (OpenJDK 64-Bit Server VM, Java 1.7.0_06-icedtea).
Type in expressions to have them evaluated.
Type :help for more information.
scala> println
<infinite_sleep>
Thanks dvigal for suggestion.
There is a github project called scala-ssh-shell which may do what you want, or at least get you closer.
-Hi, sorry I not Scala REPL hacker but i think you can do something like:
class YourILoop(in0: Option[BufferedReader], protected override val out: JPrintWriter)
extends ILoop(in0, out) {
override def createInterpreter() {
if (addedClasspath != "")
settings.classpath append addedClasspath
intp = new ILoopInterpreter
val x = 3;
intp.bind("x", x)
}
}
object Run {
def errorFn(str: String): Boolean = {
Console.err println str
false
}
def process(args: Array[String]): Boolean = {
val command = new GenericRunnerCommand(args.toList, (x: String) => errorFn(x))
import command.{ settings, howToRun, thingToRun }
new YourILoop process settings
}
def main(args: Array[String]) {
process(args)
}
}

Hotkey to stop REPL?

Is there a hotkey to stop/exit the Scala REPL ?
Example: I start the Scala REPL from SBT with command console, then do something stupid like an endless loop and want to end the REPL without closing the shell at all. Something like Ctrl+C, Ctrl+D or Ctrl+Z (which all do not work).
Update:
Used OS: Windows 7 64 Bit.
Ctrl+D exists SBT and the REPL, but Ctrl+D does NOT exit the REPL when I am in an endless loop like
while(true) prinln("test")
Is there a way to get out of the endless loop with a hotkey without closing the shell ? Or is this not possible because the REPL will not react to hotkeys until the loop is through (which will, of course, not happen in this case) ?
The following works with Scala 2.10.0-M6, but in 2.9.2 you can probably achieve something similar using :wrap in REPL power mode.
Assume that the REPL is started from sbt via sbt console—without loss of generality (you may otherwise just put the ReplUtil class on scala's class path). Assume the following class is on the class path, e.g. its source is at src/main/scala/ReplUtil.scala:
import java.util.concurrent.{Executors, ExecutorService, TimeoutException, TimeUnit}
import concurrent._
object ReplUtil {
#volatile private var exec: ExecutorService = _
#volatile private var threads = Set.empty[ Thread ]
private def freshPool() { exec = Executors.newCachedThreadPool() }
freshPool()
private implicit def context = ExecutionContext.fromExecutorService( exec )
def panic() {
(new Thread {
override def run() {
try {
exec.shutdownNow()
exec.awaitTermination( 1, TimeUnit.SECONDS )
} finally {
val th = threads
threads = Set.empty
th.foreach( _.stop )
freshPool()
}
}
}).start()
}
def spawn[T](t: => T) = {
var futPrint = false
val fut = future {
val th = Thread.currentThread()
threads += th
val res = try { t } finally { threads -= th }
if( futPrint ) println( "<calculation done>\n" + res )
t
}
try {
Await.result( fut, util.Duration( 4, TimeUnit.SECONDS )).toString
} catch {
case e: TimeoutException =>
futPrint = true
"<ongoing calculation>"
}
}
}
Then the following will activate the semi-asynchronous REPL:
$ sbt console
...
Welcome to Scala version 2.10.0-M6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_33).
...
scala> import ReplUtil.panic
import ReplUtil.panic
scala> :power
** Power User mode enabled - BEEP WHIR GYVE **
...
scala> power.intp.setExecutionWrapper("ReplUtil.spawn")
scala> 2+2
res1: Int = 4
scala> Thread.sleep(6000); 33
<ongoing calculation>
scala> <calculation done>
res2: Int = 33
scala> while(true) { Thread.sleep(2000); println( "Zzz" )}
Zzz
Zzz
<ongoing calculation>
scala> panic
scala> [error] (pool-5-thread-1) java.lang.ExceptionInInitializerError
java.lang.ExceptionInInitializerError
...
Caused by: java.lang.InterruptedException: sleep interrupted
...
related to the topic is a useful Shift+D key binding for exiting non terminating evaluations of a program when inside an eclipse scala worksheet.