I am creating a specialized build task in my project that would require user input in STDIN which will let users select which version of hadoop,spark etc libraries will be used to build the application.
so far I have created a that task looks like the below.
lazy val build = SettingKey[Unit]("build", "build the app with all dependencies") in all // all is a project key
build := {
println(s"input hadoop client version")
val hadoop = scala.io.Source.fromInputStream(System.in).bufferedReader().readLine
println(s"input hiveserver jdbc version")
val hiveserver = scala.io.Source.fromInputStream(System.in).bufferedReader().readLine
// bunch of code to customize the build
}
The problem with the above code is that the body of build task key is run whenever the project is initialized in sbt or whenever reload task is run. so whenever I open the project in sbt, it hangs waiting for me to enter the version inputs in the standard input. How the task be made such that its body is executed only when the task is run and not when the build is initialized
I would consider defining a new custom task [http://www.scala-sbt.org/0.12.2/docs/Getting-Started/Custom-Settings.html#implementing-a-task] rather than extending build with blocking commands such as user input. As you experienced build is executed quite often.
You could for example extend build to use predefined values, read from environment settings. And have a interactiveBuild task where you prompt the user, showing the defaults retrieved from the env vars.
For more info about defining custom tasks check these out:
https://github.com/earldouglas/sbt-custom-task/tree/master/project
SBT plugin - User defined configuration for Command via their build.sbt
Related
I need to create directories and files for some tests. My project uses sbt as the build tool, and common practice is to use File.createTempFile or similar APIs, but I abhor that practice. I want all files created by my tests to reside somewhere inside the output directory (<module>/target/), so that they'll be removed when I run clean, but otherwise preserved if I have need of them to figure out test failures.
The test framework is not relevant: if your solution requires a particular framework, I'll happily adopt it or figure out how it does the trick and use that.
In short, I need the answer to one of these two questions:
How can I create a file inside the build output directory from a test run by sbt?
How can I find out what is the build output directory for the current project from a test run by sbt?
In ScalaTest, try passing target
settingKey[File]("Main directory for files generated by the build.")
to config map as -Dkey=value. For example, in build.sbt specify
Test / testOptions += Tests.Argument(s"-DtargetDir=${target.value}")
and then define test like so
import org.scalatest._
class ExampleSpec extends fixture.FlatSpec with fixture.ConfigMapFixture with Matchers {
"The config map" should "contain target directory used by sbt" in { configMap =>
configMap should contain key "targetDir"
}
I'm trying to run certain tasks and startup servers after running sbt. I want to be able to run commands in terminal to do this. How can I define them? Are plugins the right way to do this:
I see some code like this:
object DoThing extends AutoPlugin {
object autoImport {
val vpnCheck = taskKey[Boolean]("Check for a VPN connection.")
}
import autoImport._
override lazy val projectSettings = Seq(
vpnCheck := {
doVpnCheck()
}
)
What is the projectSettings method doing? Are plugins the way?
From the plugins page:
A plugin is a way to use external code in a build definition. A plugin can be a library used to implement a task (you might use Knockoff to write a markdown processing task). A plugin can define a sequence of sbt settings that are automatically added to all projects or that are explicitly declared for selected projects. For example, a plugin might add a proguard task and associated (overridable) settings. Finally, a plugin can define new commands (via the commands setting).
But I can't seem to figure this out.
For your scenario, maybe you can just a create Task in sbt file to do this, like:
val hello = taskKey[Unit]("hello world")
hello := {
println("hello")
}
and if you run it automatically in startup time, you can create .sbtrc file in project directory, and it like:
alias boot = ;reload ;hello ;iflast shell
I'm currently using sbt-native-packager to generate a start script for my scala application. I'm using packageArchetype.java_application. I create the script in sbt:
sbt clean myproject/stage
and then "install" the application by copying the created lib and bin directories to the installation directory. I'm not distributing it to anyone, so I'm not creating an executable jar or tarball or anything like that. I'm just compiling my classes, and putting my jar and all the library dependency jars in one place so the start script can execute.
Now I want to add a second main class to my application, so I want a second start script to appear in target/universal/stage/bin when I run sbt stage. I expect it will be the same script but with a different name and app_mainclass set to the different class. How would I do this?
The sbt-native-packager generated script allows you to pass in a -main argument to specify the main class you want to run. Here's what I do for a project named foo:
Create a run.sh script with whatever common options you want that calls the sbt-native-packager generated script:
#!/bin/bash
./target/universal/stage/bin/foo -main "$#"
Then I create a separate script for each main class I want to run. For example first.sh:
#!/bin/bash
export JAVA_OPTS="-Xms512m -Xmx512m"
./run.sh com.example.FirstApp -- "$#"
and second.sh:
#!/bin/bash
export JAVA_OPTS="-Xms2048m -Xmx2048m -XX:+UseConcMarkSweepGC -XX:+UseParNewGC"
./run.sh com.example.SecondApp -- "$#"
Having multiple main classes is... supported now (Q4 2016, native package 1.2.0)
See "SBT Native Packager 1.2.0" by Muki Seiler
Single Project — Multiple Apps
A major pain point for beginners is the start script creation. The bash and bat start scripts are only generated when there is a either
Exactly one main class
Explicitly set main class with
mainClass in Compile := Some(“com.example.MainClass”)
For 1.2.x we will extends the implementation and support multiple main classes by default.
Native packager will generate a start script for each main class found on the classpath.
SBT provides them via the discoveredMainClasses in Compile task.
If there is only one main class, SBT will assign it to the mainClass in Compile setting. This leads to three cases:
Exactly one main class.
In this case native-packager will behave like previous versions and just generate a single start script, using the executableScriptName setting for the script name.
Multiple main classes and mainClass in Compile := None.
This is the default behaviour defined by SBT. In this case native-packager will generate the same start script for each main class.
Multiple main classes and mainClass in Compile := Some(…).
The user has set a specific main class, which will lead to a main start script being generated using the executableScriptName setting. For all other main classes native-packager generates forwarder scripts.
Having multiple main classes not supported now. As a workaround you could use single main class and check command line args.
Starting your app:
myApp prog1
In your main class:
def main(args: Array[String]): Unit = {
if(args[0] == "prog1")
Programm1.start()
else
Programm2.start()
}
I use sbt in the following fashion: I run ~ test:compile in sbt and then work in IDE, watching occasionaly if the project still compiles, because the IDE's presentation compiler tends to be buggy. When I git pull some code, there might be changes in the project/ files, so I want to have reload. Is there a way, how to watch both source files and project files, so when there is change in project files, I actually get the update?
As jsuereth explained this isn't a task SBT can handle in 1 instance. What's required is a reboot of SBT to abort the watching process and reload it's own configuration.
The following Scala script does exactly this, it uses Java NIO WatchService and Scala Process to monitor a path and restart a process. The code should be fairly simple to understand:
#!/usr/bin/env scala
import java.nio.file._
import scala.collection.JavaConversions._
import scala.sys.process._
val file = Paths.get(args(0))
val cmd = args(1)
val watcher = FileSystems.getDefault.newWatchService
file.register(
watcher,
StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_MODIFY,
StandardWatchEventKinds.ENTRY_DELETE
)
def exec = cmd run true
#scala.annotation.tailrec
def watch(proc: Process): Unit = {
val key = watcher.take
val events = key.pollEvents
val newProc =
if (!events.isEmpty) {
proc.destroy()
exec
} else proc
if (key.reset) watch(newProc)
else println("aborted")
}
watch(exec)
Usage in your sbt dir would be:
watchr.scala project/ "sbt ~ test:compile"
If anything is unclear don't hesitate to ask, of course any scripting language could be used to implement this behavior.
You actually can't use ~ <task> and have it rebuild the project itself right now, because ~ <task> needs to read the build definition itself to determine:
What source files to watch
How to run the task.
What you're doing is altering the config whe project/ changes. This requires a full reload or reboot of sbt to pull in the new configuration.
So, as of sbt 0.13, this isn't possible. You can have it so it will rebuild your source code when project/ changes, but without rebuilding the build definition, not much help.
You could create a new sbt prompt, or task, that when run could check to see if source files in project/ are updated and issue a warning/error so you know to reboot. That's probably the best option right now.
I'm trying to get sbt to compile and build some benchmarks. I've told it to add the benchmarks to the test path so they're recompiled along with tests, but I can't figure out how to write an action to let me actually run them. Is it possible to invoke classes from the Project definition class, or even just from the command line?
Yes, it is.
If you'd like to run them in the same VM the SBT is run in, then write a custom task similar to the following in your project definition file:
lazy val benchmark = task {
// code to run benchmarks
None // Some("will return an error message")
}
Typing benchmark in SBT console will run the task above. To actually run the benchmarks, or, for that matter, any other class you've compiled, you can reuse some of the existing infrastructure of SBT, namely the method runTask which will create a task that runs something for you. It has the following signature:
def runTask(mainClass: => Option[String], classpath: PathFinder, options: String*): Task
Simply add the following to your file:
lazy val benchmark = task { args =>
runTask(Some("whatever.your.mainclass.is"), testClasspath, args)
}
When running benchmarks, it is sometimes recommended that you run them in a separate jvm invocation, to get more reliable results. SBT allows you to run separate processes by invoking a method ! on a string command. Say you have a command java -jar path-to-artifact.jar you want to run. Then:
"java -jar path-to-artifact.jar" !
runs the command in SBT. You want to put the snippet above in a separate task, same as earlier.
And don't forget to reload when you change your project definition.
Couldn't you simply write the benchmarks as tests, so they will be run when you call 'test' in SBT?
You could also run a specific test with 'test-only', or run a main with 'run' or 'exec' (see http://code.google.com/p/simple-build-tool/wiki/RunningSbt for details).