Ammonite: how to use another script from an Ivy dependency? - scala

I have an Ammonite Script that I want to deliver in a JAR.
In another project I want to use this Script - but so far with no success.
I tried according to the documentation (sol_local_build.sc):
import $ivy.`mycompany:myproject_2.12:2.1.0-SNAPSHOT`, local_build
#main
def doit():Unit =
println(local_build.curl("http://localhost:8080"))
local_build.sc is in the Script I want to use.
This is the exception I get:
sol_local_build.sc:2: '.' expected but eof found.
^

The script must be compiled on the fly.
Put your script in a standard sbt project
inside a directory, example directory name: "test1"
Put your external script (example name: "script.sc")
// script.sc
println("Hello world!")
into the resource directory ("test1\src\main\resources\script.sc") of the test1 project
Publish the projekt local, i.e. sbt publishLocal
It is published to ".ivy2\local\default\test1_2.12\0.1-SNAPSHOT\ ... " directory.
Now you can use the following ammonite script "test.sc".
It reads the "script.sc" from the jar in the local ivy repository
and writes it to the local directory (must have read/write access) and then executes an external process,
which calls the scala "interpreter" and executes the script.
// test.sc
import $ivy.`default:test1_2.12:0.1-SNAPSHOT`
val scriptCode = scala.util.Try {scala.io.Source.fromResource("script.sc").mkString} getOrElse """Println("Script-file not found!")"""
println("*" * 30)
println(scriptCode)
println("*" * 30)
println()
java.nio.file.Files.write(java.nio.file.Paths.get("script.sc"), scriptCode.getBytes(java.nio.charset.StandardCharsets.UTF_8))
val cmd = Seq("cmd.exe", "/c", "scala", "script.sc")
val output = sys.process.Process(cmd).!!
println(output)
Executing the script the Ammonite REPL, you get:
******************************
// script.sc
println("Hello world!")
******************************
Hello world!
The script has no error handling and leaves the file in the running directory.
You can speed up the execution with the "-savecompiled" compiler switch, i.e
val cmd = Seq("cmd.exe", "/c", "scala", "-savecompiled", "script.sc")
An additional .jar file is created then in the running directory.

Scala Scripts are not really interpreted, but are compiled "under the hood"
as every normal Scala programm.
Therefor all code must be reachable during compile time
and you cannot call a function inside the other script from the jar-file!
But Ammonite has a buid in multi-stage feature.
It compiles one part, executes it and then compiles the next part!
Little improved ammonite-script.
It's not error free but runs.
Maybe there is better way to get the script out of the jar.
You should ask Li Haoyi!
// test_ammo.sc
// using ammonite ops
// in subdirectoy /test1
// Ammonite REPL:
// import $exec.test1.test_ammo
// # Ammonite-multi-stage
import $ivy.`default::test1:0.1-SNAPSHOT`
//import scala.util.Properties
import scala.sys.process.Process
val scriptFileName = "script.sc"
write.over(pwd/"test1"/scriptFileName, read(resource(getClass.getClassLoader)/scriptFileName))
val cmd = Seq("cmd.exe", "/c", "scala", scriptFileName)
val output = Process(cmd).!!
println(output)
#
import $exec.script // no .sc suffix
ppp() // is a function inside script.sc
script.sc inside resources folder of project
published local with "sbt publishLocal":
// script.sc
println("Hello world!")
def ppp() = println("Hello world from ppp!")

For completeness, I could solve my problem as follows:
Just create a Scala File in this project.
Copy the Script content in
an Object.
package mycompany.myproject
object LocalBuild {
def curl(..)...
}
Add the dependencies to your sbt file (e.g. ammonite.ops)
Use it like:
$ivy.`mycompany:myproject_2.12:2.1.0-SNAPSHOT`, mycompany.myproject.LocalBuild
#main
def doit():Unit =
println(LocalBuild.curl("http://localhost:8080"))

Related

Different output depending on compilation with intellij or sbt

I have the following scala program that simply counts the words in a file:
package com.impatient
import java.util.Scanner
object Main extends App {
countWords()
def countWords(): Unit = {
val in = new Scanner(new java.io.File("C:\\tmp\\SampleText.txt"))
var wc = 0
while (in.hasNext()) {
var word = in.next()
wc += 1
println( wc + ". " + word)
}
in.close()
println("======================")
println(s"Total words: $wc")
}
def TestFunction(): Unit ={
println("Hello from test function!")
}
TestFunction()
}
When I execute this in intellij, either directly or via the sbt shell inside intellij, I get the following output:
mostly
positive:
======================
Total words: 309
Hello from test function!
So it correctly counts 309 words in the text file.
However if I try to execute the jar that I just compiled with intellij, directly, either via scala or sbt I get the following output:
λ scala .\impatientscala2_2.12-1.0.jar
======================
Total words: 0
Hello from test function!
So in this case, although the methods in Main are executed the wordcount is 0.
Why would be this the case? Is intellij referencing some other java libraries that standalone sbt or scala isn't? In that case why doesn't the Scanner fail?
Edit 1
I have cloned this repo to another PC to see what happens. And in that case I can observe the exact opposite, i.e.:
The program returns 0 word count when run from Intellij
When the jar is run directly with scala, it gives the current wordcount.
The 0 wordcount is due to not processing the while loop, as in.hasNext() returns nothing.
I found a workaround by using Paths.get from java.nio.file._.
//val in = new Scanner(new java.io.File("C:/tmp/SampleText.txt"))
val in: Scanner = new Scanner(Paths.get("C/tmp/SampleText.txt"))
This produces the correct jars on both PCs, when using Run | Sbt Task in Intellij, using SBT Shell in intellij, and using standalone SBT shell. If anyone can provide an explanation of why would java.io.File fail in certain setups that would be great.

Test initialCommands in SBT

I have a subproject in my build.sbt with a rather long setting for initialCommands, comprising a list of imports and some definitions. I'd like to test this as part of regular CI, because otherwise I won't notice breaking changes after refactoring code. It is not clear to me how to do so.
Just running sbt console doesn't seem to cut it, because there is always a "successful" exit code even when code doesn't compile.
Moving the code out into an object defined in a special source file won't help because I need the list of imports to be present (and I don't want to cakeify my whole code base).
Moving the code out into a source file and then loading that with :load also always gives a successful exit code.
I found out about scala -e but that does strange things on my machine (see the error log below).
This is Scala 2.12.
$ scala -e '1'
cat: /release: No such file or directory
Exception in thread "main" java.net.UnknownHostException: <my-host-name-here>: <my-host-name-here>: Name or service not known
You could generate a file and run it like any other test file:
(sourceGenerators in Test) += Def.task {
val contents = """object TestRepl {
{{}}
}""".replace("{{}}", (initialCommands in console).value)
val file = (sourceManaged in Test).value / "repltest.scala"
IO.write(file, contents)
Seq(file)
}.taskValue

How can I change Scala script's directory?

I want use Scala like Python, so I install REPL in Sublime Text(Os is win8)
Everytime in REPL, I have to
scala> :load <my file>
, so I think it's inconvenient.
And I can't change
scala> :settings -d <路径名>
in Chinese directory.
I'm confused whether I can't change Scala script's directory with non-english language.
Thanks a lot!
If you use sbt then you can define initial commands when you launch the console.
yourproject/build.sbt:
// build.sbt
name := "initial-commands-example"
initialCommands := "import Foo._"
yourproject/script.scala:
// script.scala
object Foo {
def hello(name: String) = s"hello $name"
val msg = hello("world")
}
Inside yourproject, run sbt console, and you will have everything in Foo available inside that repl. See sbt initialCommands docs for more information.

How to access static resources in jar (that correspond to src/main/resources folder)?

I have a Spark Streaming application built with Maven (as jar) and deployed with the spark-submit script. The application project layout follows the standard directory layout:
myApp
src
main
scala
com.mycompany.package
MyApp.scala
DoSomething.scala
...
resources
aPerlScript.pl
...
test
scala
com.mycompany.package
MyAppTest.scala
...
target
...
pom.xml
In the DoSomething.scala object I have a method (let's call it doSomething()) that tries to execute a Perl script -- aPerlScript.pl (from the resources folder) -- using scala.sys.process.Process and passing two arguments to the script (the first one is the absolute path to a binary file used as input, the second one is the path/name of the produced output file). I call then DoSomething.doSomething().
The issue is that I was not able to access the script, not with absolute paths, relative paths, getClass.getClassLoader.getResource, getClass.getResource, I have specified the resources folder in my pom.xml. None of my attempts succeeded. I don't know how to find the stuff I put in src/main/resources.
I will appreciate any help.
SIDE NOTES:
I use an external Process instead of a Spark pipe because, at this step of my workflow, I must handle binary files as input and output.
I'm using Spark-streaming 1.1.0, Scala 2.10.4 and Java 7. I build the jar with "Maven install" from within Eclipse (Kepler)
When I use the getClass.getClassLoader.getResource "standard" method to access resources I find that the actual classpath is the spark-submit script's one.
There are a few solutions. The simplest is to use Scala's process infrastructure:
import scala.sys.process._
object RunScript {
val arg = "some argument"
val stream = RunScript.getClass.getClassLoader.getResourceAsStream("aPerlScript.pl")
val ret: Int = (s"/usr/bin/perl - $arg" #< stream).!
}
In this case, ret is the return code for the process and any output from the process is directed to stdout.
A second (longer) solution is to copy the file aPerlScript.pl from the jar file to some temporary location and execute it from there. This code snippet should have most of what you need.
object RunScript {
// Set up copy destination from the Java temporary directory. This is /tmp on Linux
val destDir = System.getProperty("java.io.tmpdir") + "/"
// Get a stream to the script in the resources dir
val source = Channels.newChannel(RunScript.getClass.getClassLoader.getResourceAsStream("aPerlScript.pl"))
val fileOut = new File(destDir, "aPerlScript.pl")
val dest = new FileOutputStream(fileOut)
// Copy file to temporary directory
dest.getChannel.transferFrom(source, 0, Long.MaxValue)
source.close()
dest.close()
}
// Schedule the file for deletion for when the JVM quits
sys.addShutdownHook {
new File(destDir, "aPerlScript.pl").delete
}
// Now you can execute the script.
This approach allows you to bundle native libraries in JAR files. Copying them out allows the libraries to be loaded at runtime for whatever JNI mischief you have planned.

How do you write a Scala script that will react to file changes

I would like to change the following batch script to Scala (just for fun), however, the script must keep running and listen for changes to the *.mkd files. If any file is changed, then the script should re-generate the affected doc. File IO has always been my Achilles heel...
#!/bin/sh
for file in *.mkd
do
pandoc --number-sections $file -o "${file%%.*}.pdf"
done
Any ideas around a good approach to this will be appreciated.
The following code, taken from my answer on: Watch for project files also can watch a directory and execute a specific command:
#!/usr/bin/env scala
import java.nio.file._
import scala.collection.JavaConversions._
import scala.sys.process._
val file = Paths.get(args(0))
val cmd = args(1)
val watcher = FileSystems.getDefault.newWatchService
file.register(
watcher,
StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_MODIFY,
StandardWatchEventKinds.ENTRY_DELETE
)
def exec = cmd run true
#scala.annotation.tailrec
def watch(proc: Process): Unit = {
val key = watcher.take
val events = key.pollEvents
val newProc =
if (!events.isEmpty) {
proc.destroy()
exec
} else proc
if (key.reset) watch(newProc)
else println("aborted")
}
watch(exec)
Usage:
watchr.scala markdownFolder/ "echo \"Something changed!\""
Extensions have to be made to the script to inject file names into the command. As of now this snippet should just be regarded as a building block for the actual answer.
Modifying the script to incorporate the *.mkd wildcards would be non-trivial as you'd have to manually search for the files and register a watch on all of them. Re-using the script above and placing all files in a directory has the added advantage of picking up new files when they are created.
As you can see it gets pretty big and messy pretty quick just relying on Scala & Java APIs, you would be better of relying on alternative libraries or just sticking to bash while using INotify.