How can I get automatic dependency resolution in my scala scripts? - scala

Question

I'm just learning scala coming out of the groovy/java world. My first script requires a 3rd party library TagSoup for XML/HTML parsing, and I'm loath to have to add it the old school way: that is, downloading TagSoup from its developer website, and then adding it to the class path.
Is there a way to resolve third party libraries in my scala scripts? I'm thinking Ivy, I'm thinking Grape.
Ideas?
The answer that worked best for me was to install n8:
curl https://raw.github.com/n8han/conscript/master/setup.sh | sh
cs harrah/xsbt --branch v0.11.0
Then I could import tagsoup fairly easily example.scala
/***
libraryDependencies ++= Seq(
"org.ccil.cowan.tagsoup" % "tagsoup" % "1.2.1"
)
*/
def getLocation(address:String) = {
...
}
And run using scalas:
scalas example.scala
Thanks for the help!

Answer 1

While the answer is SBT, it could have been more helpful where scripts are regarded. See, SBT has a special thing for scripts, as described here. Once you get scalas installed, either by installing conscript and then running cs harrah/xsbt --branch v0.11.0, or simply by writing it yourself more or less like this:
#!/bin/sh
java -Dsbt.main.class=sbt.ScriptMain \
-Dsbt.boot.directory=/home/user/.sbt/boot \
-jar sbt-launch.jar "$#"
Then you can write your script like this:
#!/usr/bin/env scalas
!#
/***
scalaVersion := "2.9.1"
libraryDependencies ++= Seq(
"net.databinder" %% "dispatch-twitter" % "0.8.3",
"net.databinder" %% "dispatch-http" % "0.8.3"
)
*/
import dispatch.{ json, Http, Request }
import dispatch.twitter.Search
import json.{ Js, JsObject }
def process(param: JsObject) = {
val Search.text(txt) = param
val Search.from_user(usr) = param
val Search.created_at(time) = param
"(" + time + ")" + usr + ": " + txt
}
Http.x((Search("#scala") lang "en") ~> (_ map process foreach println))
You may also be interested in paulp's xsbtscript, which creates an xsbtscript shell that has the same thing as scalas (I guess the latter was based on the former), with the advantage that, without either conscript or sbt installed, you can get it ready with this:
curl https://raw.github.com/paulp/xsbtscript/master/setup.sh | sh
Note that it installs sbt and conscript.
And there's also paulp's sbt-extras, which is an alternative "sbt" command line, with more options. Note that it's still sbt, just the shell script that starts it is more intelligent.

Answer 2

SBT (Simple Build Tool) seems to be the build tool of choice in the Scala world. It supports a number of different dependency resolution mechanisms: https://github.com/harrah/xsbt/wiki/Library-Management

Answer 3

Placed as an answer cause it doesn't fit in comment length constraint.
In addition to #Chris answer, I would like to recommend you some commons for sbt (which I personally think is absolutely superb). Although sbt denote Simple Build Tool, sometimes it is not so easy for first-timers to setup project with sbt (all this things with layouts, configs, and so on).
Use giter (g8) to create new project with predefined template (which g8 fetches from github.com). There are templates for Android app, unfiltered and more. Sometimes they are include some of the dependencies by default.
To create layout just type:
g8 gseitz/android-sbt-project
(An example for Android app)
Alternatively, use np pluggin for sbt, which provides interactive type-through way to create new project and basic layout.

Answer 4

A corrected and simplified version of the current main answer: use scalas.
You have to compose your script of 3 parts. One would be sbt, another would be a very simple wrapper around SBT called scalas, the last one is your custom script. Note that the first two scripts can be installed either globally (/usr/bin/, ~/bin/) or locally (in the same directory).
the first part is sbt. If you already have it installed then good. If not, you can either install it, or use a very cool script from paulp: https://github.com/paulp/sbt-extras/blob/master/sbt BTW, that thing is a charming way to use sbt on Unix. Although not available on windows. Anyways...
the second part is scalas. It's just an entrypoint to SBT.
#!/bin/sh
exec /path/to/sbt -Dsbt.main.class=sbt.ScriptMain -sbt-create \
-Dsbt.boot.directory=$HOME/.sbt/boot \
"$#"
the last part is your custom script. Example:
#!/usr/bin/env scalas
/***
scalaVersion := "2.11.0"
libraryDependencies ++= Seq(
"org.joda" % "joda-convert" % "1.5",
"joda-time" % "joda-time" % "2.3"
)
*/
import org.joda.time._
println(DateTime.now())
//println(DateTime.now().minusHours(12).dayOfMonth())

Answer 5

What Daniel said. Although it's worth mentioning that the sbt docs carefully label this functionality "experimental".
Indeed, if you try to run the embedded script with scalaVersion := "2.10.3", you'll get not found: value !#
Luckily, the !# script header-closer is unnecessary here, so you can leave it out.
Under scalaVersion := "2.10.3", the script will need to have the file extension ".scala"; using the bash shell script file extension, ".sh", won't work.
Also, it isn't clear to me that the latest version of Dispatch (0.11.0) supports dispatch-twitter, which is used in the example.
For more about header-closers in this context, see Alvin Alexander's blog post on Scala scripting, or section 14.10 of his Scala Cookbook.

Answer 6

I have a build.gradle file with the following task:
task classpath(dependsOn: jar) << {
println "CLASSPATH=${tasks.jar.archivePath}:${configurations.runtime.asPath}"
}
Then, in my Scala script:
#!
script_dir=$(cd $(dirname "$0") >/dev/null; pwd -P)
classpath=$(cd ${script_dir} && ./gradlew classpath | grep '^CLASSPATH=' | sed -e 's|^CLASSPATH=||')
PATH=${SCALA_HOME}/bin:${PATH}
JAVA_OPTS="-Xmx4g -XX:MaxPermSize=1g" exec scala -classpath ${classpath} "$0" "$0" "$#"
!#

Answer 7

Note that we don't need a separate scalas executable in our PATH, since we can use the self-executing shell script trick.
Here's an example script, which reads its own content (via the $0 variable), chops off everything before an arbitrary marker (__BEGIN_SCRIPT__) and runs sbt on the result. We use process substitution to pretend this calculated content is a real file. One problem with this approach is that sbt will seek within the given file, i.e. it doesn't read it sequentially. That stops it working with the <(foo) form of process substitution, as found in bash; however zsh has a =(foo) form which is seekable.
#!/usr/bin/env zsh
set -e
# Find the line # in this file ($0) after the line beginning __BEGIN_SCRIPT__
LINENUM=$(awk '/^__BEGIN_SCRIPT__/ {print NR + 1; exit 0; }' "$0")
sbtRun() {
# Run the sbt command, such that it will fetch dependencies and execute a
# script
sbt -Dsbt.main.class=sbt.ScriptMain \
-sbt-create \
-Dsbt.boot.directory="$HOME/.sbt/boot" \
"$#"
}
# Run SBT on the contents of this file, starting at LINENUM
sbtRun =(tail -n+"$LINENUM" "$0")
exit 0
__BEGIN_SCRIPT__
/***
scalaVersion := "2.11.0"
libraryDependencies ++= Seq(
"org.joda" % "joda-convert" % "1.5",
"joda-time" % "joda-time" % "2.3"
)
*/
import org.joda.time._
println(DateTime.now())