I wanted to print the dependency jars so that I can put them in a classpath variable.
I can fetch a classpath string for dependency jar files.
Curenttly, I can use compile:dependencyClasspath but it returns a format which needs to be manually transformed into a classpath string.
$ sbt "show compile:dependencyClasspath"
[info] Set current project to scala-snippets (in build file:/home/tuxdna/work/learn/mine/scala-snippets/)
[info] List(Attributed(/home/tuxdna/.ivy2/cache/org.apache.lucene/lucene-core/jars/lucene-core-3.0.1.jar), Attributed(/home/tuxdna/.ivy2/cache/com.netflix.rxjava/rxjava-scala/jars/rxjava-scala-0.15.0.jar), Attributed(/home/tuxdna/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.10.4-RC1.jar), Attributed(/home/tuxdna/.ivy2/cache/com.netflix.rxjava/rxjava-core/jars/rxjava-core-0.15.0.jar), Attributed(/home/tuxdna/.ivy2/cache/com.typesafe.akka/akka-actor_2.10/jars/akka-actor_2.10-2.2.3.jar), Attributed(/home/tuxdna/.ivy2/cache/com.typesafe/config/bundles/config-1.0.2.jar), Attributed(/home/tuxdna/.ivy2/cache/org.scala-lang.modules/scala-async_2.10/bundles/scala-async_2.10-0.9.0-M2.jar), Attributed(/home/tuxdna/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.10.3.jar), Attributed(/home/tuxdna/.ivy2/cache/org.scala-lang/scala-actors/jars/scala-actors-2.10.2.jar))
[success] Total time: 0 s, completed Dec 17, 2013 4:00:05 PM
This is how I am transforming it into a classpath string
$ sbt "show compile:dependencyClasspath" | grep -B1 'success' | head -1 | ruby -e 's = STDIN.read; puts s[(s =~ /Attributed(.*?,)/)..-1].split(", ").map { |x| x.sub("Attributed(","").gsub(")", "")}.join(":")'
/home/tuxdna/.ivy2/cache/org.apache.lucene/lucene-core/jars/lucene-core-3.0.1.jar:/home/tuxdna/.ivy2/cache/com.netflix.rxjava/rxjava-scala/jars/rxjava-scala-0.15.0.jar:/home/tuxdna/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.10.4-RC1.jar:/home/tuxdna/.ivy2/cache/com.netflix.rxjava/rxjava-core/jars/rxjava-core-0.15.0.jar:/home/tuxdna/.ivy2/cache/com.typesafe.akka/akka-actor_2.10/jars/akka-actor_2.10-2.2.3.jar:/home/tuxdna/.ivy2/cache/com.typesafe/config/bundles/config-1.0.2.jar:/home/tuxdna/.ivy2/cache/org.scala-lang.modules/scala-async_2.10/bundles/scala-async_2.10-0.9.0-M2.jar:/home/tuxdna/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.10.3.jar:/home/tuxdna/.ivy2/cache/org.scala-lang/scala-actors/jars/scala-actors-2.10.2.jar
How could similar functionality be achieved using SBT?
Also I tried to use sbt consoleProject to try and create a SBT Task.
Here is how:
$ sbt
sbt> consoleProject
scala> val dcc = (dependencyClasspath in Compile)
dcc: sbt.TaskKey[sbt.Keys.Classpath] = sbt.Scoped$$anon$2#3fa1115c
Now I can't figure out how to use SBT API to fetch the list of files. I have already tried to understand the Classpath from this page.
Any idea how to fetch the list of files?
Any task can define a separate log stream (named export) and dump information to it. All the classpath tasks do so.
sbt "export compile:dependencyClasspath"
sbt "export <...>" > foo should work, unless you have plugins writing to stdout directly rather than a logger.
This was broken prior to sbt 0.13.2 or 0.13.5 IIRC, but it should work now.
Related
I have a jar that was created by running sbt package
I've already set the main file in the jar in build.sbt
mainClass in (Compile, packageBin) := Some("com.company.mysql.Main")
addCommandAlias("updatemysql", "runMain com.company.mysql.Main")
I've tried
sbt "runMain target/scala-2.12/update-mysql_2.12-0.1-SNAPSHOT.jar"
sbt target/scala-2.12/update-mysql_2.12-0.1-SNAPSHOT.jar com.company.mysql.Main
sbt target/scala-2.12/update-mysql_2.12-0.1-SNAPSHOT.jar:com.company.mysql.Main
sbt update-mysql-assembly-0.1-SNAPSHOT.jar/run
sbt run update-mysql-assembly-0.1-SNAPSHOT.jar
^ this gives No main class detected even though main class is set in build.sbt as shown a few lines above.
I need to run the jar through sbt because it's the only way I know how to overwrite config file that is contained in the jar using -Dpath.to.config.param=new_value
In sbt run and runMain use classpath containing all the dependencies as well as folders with outputs of compilation tasks - which means that none of them takes JAR as an argument.
I think it would be possible to run this particular JAR from sbt by writing a custom task that would depend on output of package task (that is JAR filepath value) and run it as external process... though from the question it seems that this is not the actual problem.
The actual problem is running the JAR with flags passed into JVM instead of program itself which can be achieved by something like:
# clean assembly ensures that there is only 1 JAR in target
# update-mysql_2.12-*.jar picks the only JAR no matter what is its version
# -D arguments NEED to be passed before -jar to pass it to JVM and not the JAR
sbt clean assembly && \
java -Dpath.to.config.param=new_value -jar target/scala-2.12/update-mysql_2.12-*.jar
I am running my fat jar with command java -Djava.security.krb5.conf=/krb5.conf -jar my.jar.
How to run my app with this option via sbt?
$ sbt -Djava.security.krb5.conf="module\\src\\main\\resources\\krb5.conf" run doesn't work. Error:
ctl-scala>sbt -Djava.security.krb5.conf="ctl-core\src\main\resources\krb5.conf" ctl-ui-backend/run
Warning: invalid system property 'java.security.krb5.conf'
[info] Loading global plugins from C:\Users\User\.sbt\0.13\plugins
[info] Loading project definition from C:\Users\User\IdeaProjects\ctl-scala\project
[info] Set current project to ctl (in build file:/C:/Users/User/IdeaProjects/ctl-scala/)
[error] No valid parser available.
[error] ctl-core\\src\\main\\resources\\krb5.conf
[error] ^
Can you try sbt -J-Djava.security.krb5.conf="module/src/main/resources/krb5.conf" run
The -J causes the sbt launcher to pass those as options to the JVM.
As sbt manual it will pass JAVA_OPTS environment variable to the java and if you can not set the variable .jvmopts will hold the java commandline arguments. so if you are in bash :
export JAVA_OPTS="-Djava.security.krb5.conf=/krb5.conf"
before sbt command will pass the argument to java runtime.
You can force sbt to fork a new JVM when running the application, and set the desired java options with the following settings in the build.sbt file:
fork := true,
javaOptions ++= Seq(
"-Djava.security.krb5.conf=/krb5.conf"
)
Simply run the run task and it should start the application in its own JVM with the required java options.
Another option, is to use .sbtopts file. It should be in the root folder, next to sbt.build. Its content should be the java options, prefixed with -J, as written in previously answers, to tell sbt to pass those options to the JVM.
For example its content can be:
-J-Djava.security.krb5.conf=/krb5.conf
I'm using sbt for building my Scala project and I was looking for a way to filter output of any command (like compile) by sub-string. In particular, I want to use grep in combination with sbt commands. For example > compile | grep MyFile.scala, should print only lines where MyFile.scala was mentioned.
Is there any way to do that?
$ sbt --version
sbt launcher version 0.13.5
The best way to grep the output of SBT in interactive mode (tested with sbt 0.13.11) is to use the last-grep.
This will re-print the output of the last command and it will take different grep-like arguments.
I'm not sure about doing it from within the sbt console, but how about from your shell?
$ sbt "tasks" | grep 'clean'
clean Deletes files produced by the build, such as generated sources, compiled classes, and task caches.
Or for substrings as you mention:
$ sbt "tasks" | grep 'class'
clean Deletes files produced by the build, such as generated sources, compiled classes, and task caches.
console Starts the Scala interpreter with the project classes on the classpath.
consoleProject Starts the Scala interpreter with the sbt and the build definition on the classpath and useful imports.
consoleQuick Starts the Scala interpreter with the project dependencies on the classpath.
run Runs a main class, passing along arguments provided on the command line.
runMain Runs the main class selected by the first argument, passing the remaining arguments to the main method.
My question is: Why can't I publish from SBT to my server via SSH?
Context:
I am developing a scala library and I want to publish it to a remote repository with SBT v0.12.3 over SSH (using an SFTP resolver). The relevant portion of my project/Build.scala SBT settings file is configured as prescribed by https://github.com/harrah/xsbt/wiki/Resolvers:
publishTo <<= version { v =>
Some(Resolver.sftp(
"My Repository",
"example.com",
"/var/www/public_html/repositories/" + (
if (v.trim.endsWith("SNAPSHOT")) { "snapshots" } else { "releases" }
)
))
},
resolvers ++= Seq(
{
import java.io.File
val privateKeyFile: File = new File(sys.env("HOME") + "/.ssh/id_rsa")
Resolver.ssh("scala-sh", "example.com") as("my-username", privateKeyFile) withPermissions("0644")
},
...
),
When I run sbt publish, things go fine until the authorization, where it still attempts to prompt me for login/password. When I run it locally, it brings up the username/password prompt, and when I try to publish remotely while SSH'd in to a machine it fails with a java.awt.HeadlessException. The result appears to be that the desired private-key type of authentication is not being attempted.
Here is a log of the remote session publish attempt:
> sbt-version
[info] 0.12.3
> publish
[info] Packaging /home/me/my-lib/target/scala-2.10.1/my-lib_2.10.1-SNAPSHOT-sources.jar ...
[info] Done packaging.
[info] Wrote /home/me/my-lib/target/scala-2.10.1/my-lib_2.10.1-SNAPSHOT.pom
[info] :: delivering :: org.example#my-lib_2.10.1;SNAPSHOT :: SNAPSHOT :: release :: Sun Apr 21 12:48:59 PDT 2013
[info] delivering ivy file to /home/me/my-lib/target/scala-2.10.1/ivy-SNAPSHOT.xml
[info] Generating API documentation for main sources...
model contains 75 documentable templates
[info] API documentation generation successful.
[info] Packaging /home/me/my-lib/target/scala-2.10.1/my-lib_2.10.1-SNAPSHOT-javadoc.jar ...
[info] Done packaging.
[info] Packaging my-lib-SNAPSHOT.jar ...
[info] Done packaging.
[trace] Stack trace suppressed: run last *:publish for the full output.
[error] (*:publish) java.awt.HeadlessException:
[error] No X11 DISPLAY variable was set, but this program performed an operation which requires it.
[error] Total time: 35 s, completed Apr 21, 2013 12:49:33 PM
It fails because there is no X11 display. This is unexpected behavior because the SBT project configuration is set to use private key authentication (see resolvers above).
So far I can think of 2 possible causes for the problem, detailed below.
Possible cause #1: Misconfiguration of SBT
Is there a problem in my configuration above?
Possible cause #2: Hitting an Ivy bug from an old version
At time of writing, I am using the latest SBT, 0.12.3. Maybe the version of Ivy being used by SBT is old. The more I think about it, the less likely this seems, but I haven't been able to rule it out yet.
How can I find out what version of Ivy SBT is using?
and then..
IF it is old, is there a way to get SBT to use a newer version of ivy?
There is another relevant question, see ivy ssh publisher, which references[0] an Old Ivy bug which caused java.awt.HeadlessExceptions.
[0] ivy ssh publisher
"Which version of Ivy are you using? There is a Jira Bug for Version 2.0 : issues.apache.org/jira/browse/IVY-783 which should be fixed now."
"Seems like if I upgrade to ivy 2.3 rc-2. SSH publish works."
It may after all be an Ivy version related bug. I am using SBT 0.12.2, which AFAIK is using Ivy 2.0. I am looking at the Ivy cache, located in ~/.ivy2/cache/, where Ivy creates a bunch of XML log files for the dependencies it has resolved, and I can see Ivy module version 2.0 in every XML file generated.
I don't know of a way to update the Ivy version used by SBT, but judging by the default SBT documentation, a possible solution is to manually upgrade Ivy and make sure the default machine wide path points to the right Ivy version.
Then run sbt clean or sbt update to re-fetch dependencies and allow Ivy to regenerate XML config files, etc. for the new Ivy version. More on SBT dependency management HERE:
Ivy Home Directory
By default, sbt uses the standard Ivy home directory location
${user.home}/.ivy2/. This can be configured machine-wide, for use by
both the sbt launcher and by projects, by setting the system property
sbt.ivy.home in the sbt startup script (described in Setup).
For example:
java -Dsbt.ivy.home=/tmp/.ivy2/ ...
Update
By checking the SBT Scala source code for the latest version, the version issue is confirmed again. Even SBT 0.13 appears to be using Ivy version 2.0.0, not 2.3. Have a look at the SBT source code, specifically the last few lines of THIS file.
How can I run an sbt app from the shell, so that I can run my app as a normal command-line program (as if run directly via scala but without having to set up an enormous classpath)?
I know I can do:
echo hello | sbt 'run-main com.foo.MyMain3 arg1 arg2' > out.txt
But this (1) takes forever to start because it starts sbt, (2) causes all stdout and stderr to go to stdout, and (3) causes all output to be decorated with a logger [info] or [error].
I looked at https://github.com/harrah/xsbt/wiki/Launcher but it seems too heavyweight, since it downloads dependencies and sets up a new environment and whatnot. I just want to run this app within my existing development environment.
Thus far I've cobbled together my own script to build up a classpath, and you can also do some other things like modify your project file to get sbt to print the raw classpath, but I feel like there must be a better way.
Here's what I have in my SBT (version 0.10) project definition,
val Mklauncher = config("mklauncher") extend(Compile)
val mklauncher = TaskKey[Unit]("mklauncher")
val mklauncherTask = mklauncher <<= (target, fullClasspath in Runtime) map { (target, cp) =>
def writeFile(file: File, str: String) {
val writer = new PrintWriter(file)
writer.println(str)
writer.close()
}
val cpString = cp.map(_.data).mkString(":")
val launchString = """
CLASSPATH="%s"
scala -usejavacp -Djava.class.path="${CLASSPATH}" "$#"
""".format(cpString)
val targetFile = (target / "scala-sbt").asFile
writeFile(targetFile, launchString)
targetFile.setExecutable(true)
}
... // remember to add mklauncherTask to Project Settings
The mklauncher task creates a script target/scala-sbt that executes scala with the project classpath already set. It would be nice to have mklauncher executed automatically whenever the classpath changes, but I haven't looked into doing this yet.
(I use the Java classpath, rather than Scala's, for ease of creating embedded interpreters.)
The start-script SBT plugin is now at:
https://github.com/sbt/sbt-start-script
It requires a few steps to set up and generates scripts that do not work on OS X, but that can be easily fixed if you're on that platform (see below).
Setup
Install greadlink (OS X only):
a) brew install coreutils
b) map readlink to the new function (greadlink) by adding these lines to ~/.bashrc:
function readlink() { greadlink "$#"; }
export -f readlink`
Add start-script plugin to ~/.sbt/plugins/build.sbt:
addSbtPlugin("com.typesafe.sbt" % "sbt-start-script" % "0.8.0")
Add start-script task to current project:
$ sbt add-start-script-tasks # execute from directory where build.sbt resides
Add start-script support to current build.sbt:
import com.typesafe.sbt.SbtStartScript
seq(SbtStartScript.startScriptForClassesSettings: _*)
Note the blank line in between statements (de rigueur for SBT build files).
Generate Start Script
Then, whenever you want to create a script to start your app like sbt run-main, but without sbt, execute:
$ sbt start-script
Run
target/start mypackage.MyMainClass
Time flies by and a lot have changed since the other answers. It's currently SBT 0.13.6 time.
I think what you may need is the sbt-onejar plugin or the SBT Native Packager plugin.
sbt-onejar "is a simple-build-tool plugin for building a single executable JAR containing all your code and dependencies as nested JARs."
SBT Native Packager's "goal is to be able to bundle up Scala software built with SBT for native packaging systems, like deb, rpm, homebrew, msi."
Just discovered the sbt start script plugin: https://github.com/typesafehub/xsbt-start-script-plugin:
This plugin allows you to generate a script target/start for a
project. The script will run the project "in-place" (without having to
build a package first).
The target/start script is similar to sbt run but it doesn't rely on
SBT. sbt run is not recommended for production use because it keeps
SBT itself in-memory. target/start is intended to run an app in
production.
The plugin adds a task start-script which generates target/start. It
also adds a stage task, aliased to the start-script task.