ILoop Tab Completion - scala

I am creating a very simple extension of scala.tools.nsc.interpreter.ILoop with the intent of adding some additional bindings, but even in the most basic use-case the tab-completion does not seem to work. If I type in code it interprets and works as expected, but I no tab-completion. Is there something specific that needs to be defined in order for tab-completion to be enabled in the interactive interpreter (REPL)?
My use-case is as simple as the following:
val repl = new ILoop
repl.process(new Settings {
usejavacp.value = true
deprecation.value = true
})
Is there something other than ILoop I should be using?

It kind of works for me, modulo version.
$ scalacm myintp.scala && scalam myintp.Test
Welcome to Scala 2.12.0-RC2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_101).
Type in expressions for evaluation. Or try :help.
scala> 42
res0: Int = 42
scala> 42.
!= < >>> doubleValue isNaN isValidShort shortValue toDouble toShort
% << ^ floatValue isNegInfinity isWhole signum toFloat unary_+
& <= abs floor isPosInfinity longValue to toHexString unary_-
* == byteValue getClass isValidByte max toBinaryString toInt unary_~
+ > ceil intValue isValidChar min toByte toLong underlying
- >= compare isInfinite isValidInt round toChar toOctalString until
/ >> compareTo isInfinity isValidLong self toDegrees toRadians |
scala> 42.s
self shortValue signum synchronized
scala> 42.self
res1: Int = 42
scala> :quit
Source:
$ cat myintp.scala
package myintp
import scala.tools.nsc._
import scala.tools.nsc.interpreter._
/* 2.12 */
object Test extends App {
val ss = new Settings {
usejavacp.value = true
deprecation.value = true
}
def repl = new ILoop {
override def createInterpreter(): Unit = {
super.createInterpreter()
}
}
repl process ss
}
/* 2.11
object Test extends App {
def repl = new ILoop {
override def createInterpreter(): Unit = {
def binder: Unit = intp beQuietDuring {
intp directBind ("foo", "bar")
intp bind ("baz", "boo")
}
super.createInterpreter()
intp initialize binder
}
}
repl process new Settings
}
*/
/* 2.9
object Test extends App {
def repl = new ILoop {
def binder: Unit = intp beQuietDuring {
intp bind ("baz", "boo")
}
override def loop(): Unit = {
binder
super.loop()
}
}
repl process new Settings
}
*/

Related

Need workaround for scala breeze matrix slicing and vector indexing

Because of the odd behaviour in method foo I cannot write methods like bar,
which I need:
import breeze.linalg.DenseMatrix
import breeze.linalg.DenseVector
class Test {
val dim = 3
val X:DenseMatrix[Double] = DenseMatrix.rand(dim,dim)
val u:DenseVector[Double] = DenseVector.fill(dim){1.0}
def foo:Unit = {
val i=0;
val row_i:DenseVector[Double] = X(i,::).t // OK
val s = u(i)+u(i) // OK
val j:Integer = 0
val row_j:DenseVector[Double] = X(j,::).t // does not compile (A)
val a = u(j)+u(j) // does not compile (B)
}
def bar(i:Integer):Double = u(i)+u(i) // does not compile (C)
}
Is there a workaround?
Thanks in advance for all replies.
Compilation errors:
(A) could not find implicit value for parameter canSlice:
breeze.linalg.support.CanSlice2[breeze.linalg.DenseMatrix[Double],Integer,collection.immutable.::.type,Result]
not enough arguments for method apply: (implicit canSlice:
breeze.linalg.support.CanSlice2[breeze.linalg.DenseMatrix[Double],Integer,collection.immutable.::.type,Result])
Result in trait TensorLike. Unspecified value parameter canSlice.
(B), (C)
could not find implicit value for parameter canSlice:
breeze.linalg.support.CanSlice[breeze.linalg.DenseVector[Double],Integer,Result]
not enough arguments for method apply: (implicit canSlice: breeze.linalg.support.CanSlice[breeze.linalg.DenseVector[Double],Integer,Result])Result
in trait TensorLike. Unspecified value parameter canSlice.
First off: convert your Integer to Int . That would take care of at least the first compilation error. Following update to your code does compile
import breeze.linalg.DenseMatrix
import breeze.linalg.DenseVector
import breeze.linalg.DenseMatrix
import breeze.linalg.DenseVector
class Test {
val dim = 3
val X:DenseMatrix[Double] = DenseMatrix.rand(dim,dim)
val u:DenseVector[Double] = DenseVector.fill(dim){1.0}
def foo:Unit = {
val i=0;
val row_i:DenseVector[Double] = X(i,::).t // OK
val s = u(i)+u(i) // OK
val j:Int = 0
val row_j:DenseVector[Double] = X(j,::).t // does not compile (A)
val a = u(j)+u(j) // does not compile (B)
}
def bar(i:Int):Double = u(i)+u(i) // does not compile (C)
}
From the repl:
// Exiting paste mode, now interpreting.
import breeze.linalg.DenseMatrix
import breeze.linalg.DenseVector
defined class Test
So your other errors also disappear for me (scala 2.11.8). Let me know if you have further issues.

How to wait for N seconds between statements in Scala?

I have two statements like this:
val a = 1
val b = 2
In between the 2 statements, I want to pause for N seconds like I can in bash with sleep command.
You can try:
val a = 1
Thread.sleep(1000) // wait for 1000 millisecond
val b = 2
You can change 1000 to other values to accommodate to your needs.
Given:
package object wrap {
import java.time._
def delayed[A](a: => A): A = {
Console println Instant.now
Thread.sleep(1000L)
val x = a
Console println Instant.now
x
}
}
You can:
Welcome to Scala 2.12.0-M3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60).
Type in expressions for evaluation. Or try :help.
scala> $intp.setExecutionWrapper("wrap.delayed")
scala> { println("running"); 42 }
2016-02-20T06:28:17.372Z
running
2016-02-20T06:28:18.388Z
res1: Int = 42
scala> :quit

Scala REPL crashes when started using scala.tools.nsc.interpreter

I'm trying to use scala.tools.nsc.interpreter to enable interactive debugging (like Python's pdb/ipdb):
val foo = 123
import scala.tools.nsc.Settings
import scala.tools.nsc.interpreter.{ ILoop, SimpleReader }
val repl = new ILoop
repl.settings = new Settings
repl.settings.usejavacp.value = true
repl.in = SimpleReader()
repl.createInterpreter()
repl.intp.bind("foo", "Int", foo)
repl.loop()
repl.closeInterpreter()
When run, this is what I get:
$ scala repl.scala
foo: Int = 123
scala> "hello"
java.lang.NullPointerException
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:95)
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:95)
at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.ready(package.scala:95)
at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:402)
at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:430)
at Main$$anon$1.<init>(repl.scala:14)
at Main$.main(repl.scala:1)
at Main.main(repl.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at scala.reflect.internal.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:70)
at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:101)
at scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:70)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:101)
at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:22)
at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:39)
at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:39)
at scala.tools.nsc.ScriptRunner.scala$tools$nsc$ScriptRunner$$runCompiled(ScriptRunner.scala:175)
at scala.tools.nsc.ScriptRunner$$anonfun$runScript$1.apply(ScriptRunner.scala:192)
at scala.tools.nsc.ScriptRunner$$anonfun$runScript$1.apply(ScriptRunner.scala:192)
at scala.tools.nsc.ScriptRunner$$anonfun$withCompiledScript$1$$anonfun$apply$mcZ$sp$1.apply(ScriptRunner.scala:161)
at scala.tools.nsc.ScriptRunner$$anonfun$withCompiledScript$1.apply$mcZ$sp(ScriptRunner.scala:161)
at scala.tools.nsc.ScriptRunner$$anonfun$withCompiledScript$1.apply(ScriptRunner.scala:129)
at scala.tools.nsc.ScriptRunner$$anonfun$withCompiledScript$1.apply(ScriptRunner.scala:129)
at scala.tools.nsc.util.package$.trackingThreads(package.scala:43)
at scala.tools.nsc.util.package$.waitingForThreads(package.scala:27)
at scala.tools.nsc.ScriptRunner.withCompiledScript(ScriptRunner.scala:128)
at scala.tools.nsc.ScriptRunner.runScript(ScriptRunner.scala:192)
at scala.tools.nsc.ScriptRunner.runScriptAndCatch(ScriptRunner.scala:205)
at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:67)
at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:87)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:98)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
Abandoning crashed session.
scala>
If I remove the repl.intp.bind("foo", "Int", foo) part, this is what I get:
$ scala repl.scala
scala> 123
null
Abandoning crashed session.
scala>
What am I doing wrong? Is there an easier way to drop into an interactive REPL during a program run for debugging purposes? breakpoints, stepping through and inspecting locals just doesn't do it sometimes.
I'm on Scala 2.11.5.
I don't know why this comment doesn't have the magic asterisks:
// start an interpreter with the given settings
def process(settings: Settings): Boolean
So you can:
scala> repl process s
Welcome to Scala version 2.11.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_25).
Type in expressions to have them evaluated.
Type :help for more information.
scala> :quit
res0: Boolean = true
Or you could:
package myrepl
import scala.tools.nsc.Settings
import scala.tools.nsc.interpreter.{ ILoop, SimpleReader }
object Test extends App {
val foo = 42
val repl = new ILoop {
override def printWelcome() = {
intp.bind("foo", foo)
super.printWelcome()
echo("Customized...")
}
}
val s = new Settings
s.usejavacp.value = true
repl.in = SimpleReader()
repl process s
/*
repl.createInterpreter()
repl.intp.bind("foo", "Int", foo)
repl.loop()
repl.closeInterpreter()
*/
}
And
$ scalac myrepl.scala && scala myrepl.Test
foo: Int = 42
Welcome to Scala version 2.11.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_20).
Type in expressions to have them evaluated.
Type :help for more information.
Customized...
scala> foo
res0: Int = 42
scala> :quit
Also,
s.Xnojline.value = true
//repl.in = SimpleReader()
I'm going to downgrade to 2.11.4 because of the repl crashing bug...
Sbt does it by overriding createInterpreter and calling bind there.
Edit:
val repl = new ILoop {
override def printWelcome() = {
//import scala.concurrent.duration._
//Await.ready(globalFuture, 10.minutes) // sorry, it's private!
super.printWelcome()
echo("Customizing...")
processLine("") // block for init to finish
intp.bind("foo", foo)
}
}
The private globalFuture is an impediment to exploding the startup:
object Solid extends App {
val foo = 42
val repl = new ILoop {
override def printWelcome() = {
super.printWelcome()
echo("Customized...")
}
}
val s = new Settings
s.Xnojline.value = true
s.usejavacp.value = true
repl.settings = s
repl.createInterpreter()
repl.in = SimpleReader()
repl.intp.initializeSynchronous()
repl.loopPostInit()
repl.globalFuture = concurrent.Future.successful(true)
repl.intp.bind("foo", "Int", foo)
try repl.loop()
finally repl.closeInterpreter()
}
The -Yrepl-sync option is no longer honored in 2.11.

Creating serializable objects from Scala source code at runtime

To embed Scala as a "scripting language", I need to be able to compile text fragments to simple objects, such as Function0[Unit] that can be serialised to and deserialised from disk and which can be loaded into the current runtime and executed.
How would I go about this?
Say for example, my text fragment is (purely hypothetical):
Document.current.elements.headOption.foreach(_.open())
This might be wrapped into the following complete text:
package myapp.userscripts
import myapp.DSL._
object UserFunction1234 extends Function0[Unit] {
def apply(): Unit = {
Document.current.elements.headOption.foreach(_.open())
}
}
What comes next? Should I use IMain to compile this code? I don't want to use the normal interpreter mode, because the compilation should be "context-free" and not accumulate requests.
What I need to get hold off from the compilation is I guess the binary class file? In that case, serialisation is straight forward (byte array). How would I then load that class into the runtime and invoke the apply method?
What happens if the code compiles to multiple auxiliary classes? The example above contains a closure _.open(). How do I make sure I "package" all those auxiliary things into one object to serialize and class-load?
Note: Given that Scala 2.11 is imminent and the compiler API probably changed, I am happy to receive hints as how to approach this problem on Scala 2.11
Here is one idea: use a regular Scala compiler instance. Unfortunately it seems to require the use of hard disk files both for input and output. So we use temporary files for that. The output will be zipped up in a JAR which will be stored as a byte array (that would go into the hypothetical serialization process). We need a special class loader to retrieve the class again from the extracted JAR.
The following assumes Scala 2.10.3 with the scala-compiler library on the class path:
import scala.tools.nsc
import java.io._
import scala.annotation.tailrec
Wrapping user provided code in a function class with a synthetic name that will be incremented for each new fragment:
val packageName = "myapp"
var userCount = 0
def mkFunName(): String = {
val c = userCount
userCount += 1
s"Fun$c"
}
def wrapSource(source: String): (String, String) = {
val fun = mkFunName()
val code = s"""package $packageName
|
|class $fun extends Function0[Unit] {
| def apply(): Unit = {
| $source
| }
|}
|""".stripMargin
(fun, code)
}
A function to compile a source fragment and return the byte array of the resulting jar:
/** Compiles a source code consisting of a body which is wrapped in a `Function0`
* apply method, and returns the function's class name (without package) and the
* raw jar file produced in the compilation.
*/
def compile(source: String): (String, Array[Byte]) = {
val set = new nsc.Settings
val d = File.createTempFile("temp", ".out")
d.delete(); d.mkdir()
set.d.value = d.getPath
set.usejavacp.value = true
val compiler = new nsc.Global(set)
val f = File.createTempFile("temp", ".scala")
val out = new BufferedOutputStream(new FileOutputStream(f))
val (fun, code) = wrapSource(source)
out.write(code.getBytes("UTF-8"))
out.flush(); out.close()
val run = new compiler.Run()
run.compile(List(f.getPath))
f.delete()
val bytes = packJar(d)
deleteDir(d)
(fun, bytes)
}
def deleteDir(base: File): Unit = {
base.listFiles().foreach { f =>
if (f.isFile) f.delete()
else deleteDir(f)
}
base.delete()
}
Note: Doesn't handle compiler errors yet!
The packJar method uses the compiler output directory and produces an in-memory jar file from it:
// cf. http://stackoverflow.com/questions/1281229
def packJar(base: File): Array[Byte] = {
import java.util.jar._
val mf = new Manifest
mf.getMainAttributes.put(Attributes.Name.MANIFEST_VERSION, "1.0")
val bs = new java.io.ByteArrayOutputStream
val out = new JarOutputStream(bs, mf)
def add(prefix: String, f: File): Unit = {
val name0 = prefix + f.getName
val name = if (f.isDirectory) name0 + "/" else name0
val entry = new JarEntry(name)
entry.setTime(f.lastModified())
out.putNextEntry(entry)
if (f.isFile) {
val in = new BufferedInputStream(new FileInputStream(f))
try {
val buf = new Array[Byte](1024)
#tailrec def loop(): Unit = {
val count = in.read(buf)
if (count >= 0) {
out.write(buf, 0, count)
loop()
}
}
loop()
} finally {
in.close()
}
}
out.closeEntry()
if (f.isDirectory) f.listFiles.foreach(add(name, _))
}
base.listFiles().foreach(add("", _))
out.close()
bs.toByteArray
}
A utility function that takes the byte array found in deserialization and creates a map from class names to class byte code:
def unpackJar(bytes: Array[Byte]): Map[String, Array[Byte]] = {
import java.util.jar._
import scala.annotation.tailrec
val in = new JarInputStream(new ByteArrayInputStream(bytes))
val b = Map.newBuilder[String, Array[Byte]]
#tailrec def loop(): Unit = {
val entry = in.getNextJarEntry
if (entry != null) {
if (!entry.isDirectory) {
val name = entry.getName
// cf. http://stackoverflow.com/questions/8909743
val bs = new ByteArrayOutputStream
var i = 0
while (i >= 0) {
i = in.read()
if (i >= 0) bs.write(i)
}
val bytes = bs.toByteArray
b += mkClassName(name) -> bytes
}
loop()
}
}
loop()
in.close()
b.result()
}
def mkClassName(path: String): String = {
require(path.endsWith(".class"))
path.substring(0, path.length - 6).replace("/", ".")
}
A suitable class loader:
class MemoryClassLoader(map: Map[String, Array[Byte]]) extends ClassLoader {
override protected def findClass(name: String): Class[_] =
map.get(name).map { bytes =>
println(s"defineClass($name, ...)")
defineClass(name, bytes, 0, bytes.length)
} .getOrElse(super.findClass(name)) // throws exception
}
And a test case which contains additional classes (closures):
val exampleSource =
"""val xs = List("hello", "world")
|println(xs.map(_.capitalize).mkString(" "))
|""".stripMargin
def test(fun: String, cl: ClassLoader): Unit = {
val clName = s"$packageName.$fun"
println(s"Resolving class '$clName'...")
val clazz = Class.forName(clName, true, cl)
println("Instantiating...")
val x = clazz.newInstance().asInstanceOf[() => Unit]
println("Invoking 'apply':")
x()
}
locally {
println("Compiling...")
val (fun, bytes) = compile(exampleSource)
val map = unpackJar(bytes)
println("Classes found:")
map.keys.foreach(k => println(s" '$k'"))
val cl = new MemoryClassLoader(map)
test(fun, cl) // should call `defineClass`
test(fun, cl) // should find cached class
}

Scala - Initialize REPL environment

-Hi. I'd like to embed Scala REPL with initialized environment into my app. I've looked at IMain class and it seems I could do it via instance of it. The instance is created and then stored into intp public var in process() of ILoop.
How can I bind some names and/or add some imports before process() (e.g. before REPL)?
Following code fails on line 3 because intp is not yet created (=> NPE):
val x = 3
val interp = new ILoop
interp.bind("x", x) // -> interp.intp.bind("x", x)
val settings = new Settings
settings.usejavacp.value = true
interp.process(settings)
Thank you-.
UPDATE: Overriding createInterpreter() unfortunately doesn't work:
val x = 3
val interp = new ILoop {
override def createInterpreter() {
super.createInterpreter()
intp.bind("x", x) // -> interp.intp.bind("x", x)
}
}
val settings = new Settings
settings.usejavacp.value = true
interp.process(settings)
Interpreter is stuck on input (looks like deadlock, happens only with code above):
x: Int = 3
Failed to created JLineReader: java.lang.NoClassDefFoundError: scala/tools/jline/console/completer/Completer
Falling back to SimpleReader.
Welcome to Scala version 2.9.2 (OpenJDK 64-Bit Server VM, Java 1.7.0_06-icedtea).
Type in expressions to have them evaluated.
Type :help for more information.
scala> println
<infinite_sleep>
Thanks dvigal for suggestion.
There is a github project called scala-ssh-shell which may do what you want, or at least get you closer.
-Hi, sorry I not Scala REPL hacker but i think you can do something like:
class YourILoop(in0: Option[BufferedReader], protected override val out: JPrintWriter)
extends ILoop(in0, out) {
override def createInterpreter() {
if (addedClasspath != "")
settings.classpath append addedClasspath
intp = new ILoopInterpreter
val x = 3;
intp.bind("x", x)
}
}
object Run {
def errorFn(str: String): Boolean = {
Console.err println str
false
}
def process(args: Array[String]): Boolean = {
val command = new GenericRunnerCommand(args.toList, (x: String) => errorFn(x))
import command.{ settings, howToRun, thingToRun }
new YourILoop process settings
}
def main(args: Array[String]) {
process(args)
}
}