I'm trying to get sbt to compile and build some benchmarks. I've told it to add the benchmarks to the test path so they're recompiled along with tests, but I can't figure out how to write an action to let me actually run them. Is it possible to invoke classes from the Project definition class, or even just from the command line?
Yes, it is.
If you'd like to run them in the same VM the SBT is run in, then write a custom task similar to the following in your project definition file:
lazy val benchmark = task {
// code to run benchmarks
None // Some("will return an error message")
}
Typing benchmark in SBT console will run the task above. To actually run the benchmarks, or, for that matter, any other class you've compiled, you can reuse some of the existing infrastructure of SBT, namely the method runTask which will create a task that runs something for you. It has the following signature:
def runTask(mainClass: => Option[String], classpath: PathFinder, options: String*): Task
Simply add the following to your file:
lazy val benchmark = task { args =>
runTask(Some("whatever.your.mainclass.is"), testClasspath, args)
}
When running benchmarks, it is sometimes recommended that you run them in a separate jvm invocation, to get more reliable results. SBT allows you to run separate processes by invoking a method ! on a string command. Say you have a command java -jar path-to-artifact.jar you want to run. Then:
"java -jar path-to-artifact.jar" !
runs the command in SBT. You want to put the snippet above in a separate task, same as earlier.
And don't forget to reload when you change your project definition.
Couldn't you simply write the benchmarks as tests, so they will be run when you call 'test' in SBT?
You could also run a specific test with 'test-only', or run a main with 'run' or 'exec' (see http://code.google.com/p/simple-build-tool/wiki/RunningSbt for details).
Related
I have an sbt subproject that includes end to end tests. These are run as e2e:test. I have defined my config as
I have defined a tag in the same subproject.
object HealthCheckTest extends Tag("HealthCheckTest")
I am tagging some of my end to end tests with HealthCheckTest as follows:
it("should be able to verify the data", HealthCheckTest)
I want to run only the health check tests from command line. I am trying to do this via:
sbt 'project e2e' e2e:testOnly -- -n HealthCheckTest
but this leads to all of the e2e tests being run. I have tried giving the full path to the tag (com.s.p.e2etests.HealthCheckTest), but that does not work either.
Occasionally I get warnings about -- and - being deprecated; however, all documentation online says to use this syntax including scalatest docs.
How can I run just my tagged e2e tests?
I have also tried to create a separate task for health check tests but could only figure out how to filter based on test class name, not by tag.
The commandline you posted was missing quotes. Try it like this:
sbt 'project e2e' 'testOnly -- -n HealthCheckTest'
The sbt executable treats each argument as a full line to run in the sbt console, so you must place quotes around each full line.
Note that while this won't run the test cases, it will still instantiate test class and print out test case names.
I am using ScalaTest and have a test suite that contains many tests, and each test can take about a minute to run.
class LargeSuite extends FunSuite {
test("Name of test 1") { ... }
...
test("Name of test n") { ... }
}
As is usual with running ScalaTest tests from the SBT console, nothing is reported to the screen until each test in the FunSuite has been run. The problem is that when n is large and each test is slow to run, you do not know what is happening. (Running top or Windows Task Manager and looking for the CPU usage of Java is not really satisfactory.)
But the real problem is when the build is run by Travis CI: Travis assumes that the build has gone wrong and kills it if 10 minutes pass and nothing is printed to the screen. In my case, Travis is killing my build, even though the tests are still running, and each individual test in a FunSuite does not require 10 minutes (although the entire suite does require more than 10 minutes because of the number of tests).
My first question is therefore this: how can I get ScalaTest to report on progress to the console after each test in FunSuite, in an idiomatic way?
My partial solution is to use the following trait as a mixin, which solves the problem with Travis:
trait ProgressConsolePrinter extends BeforeAndAfterEach with BeforeAndAfterAll {
self: Suite =>
override def beforeAll: Unit = {
Console.print(s"$suiteName running")
Console.flush
}
override def afterEach: Unit = {
Console.print(".")
Console.flush
}
override def afterAll: Unit = {
Console.println
Console.flush
}
}
But I understand that using Console to print to the SBT console is not entirely reliable (Googling this seems to somewhat confirm this from the experiences of others).
Also (i) anything you print from Console does not go via SBT's logger, and is therefore not prefixed with [info], and (ii) when trying the above, the messages printed from Console are jumbled up with other messages from SBT. If I was able to use the proper logger, then neither of these things would happen.
My second question is therefore this: How can I print to an SBT logger from within a test in ScalaTest?
To get ScalaTest to report to the console after each test within SBT, add
logBuffered in Test := false
to your build configuration. (See the SBT docs.)
For general logging purposes within SBT, you can use an instance of sbt.util.Logger obtained from streams.value.log within an SBT task as described here. Wilson's answer below can be used if logging is required from the test code.
(I'm answering my own question two years later, but this is currently the first hit on Google for "ScalaTest sbt progress".)
This may be helpful for your second question.
To log like that you must use scala logging. You must add scala logging as a test dependency, extend some of its logging classes (suggest LazyLogging) and then call logger.info("Your message").
I'm currently using sbt-native-packager to generate a start script for my scala application. I'm using packageArchetype.java_application. I create the script in sbt:
sbt clean myproject/stage
and then "install" the application by copying the created lib and bin directories to the installation directory. I'm not distributing it to anyone, so I'm not creating an executable jar or tarball or anything like that. I'm just compiling my classes, and putting my jar and all the library dependency jars in one place so the start script can execute.
Now I want to add a second main class to my application, so I want a second start script to appear in target/universal/stage/bin when I run sbt stage. I expect it will be the same script but with a different name and app_mainclass set to the different class. How would I do this?
The sbt-native-packager generated script allows you to pass in a -main argument to specify the main class you want to run. Here's what I do for a project named foo:
Create a run.sh script with whatever common options you want that calls the sbt-native-packager generated script:
#!/bin/bash
./target/universal/stage/bin/foo -main "$#"
Then I create a separate script for each main class I want to run. For example first.sh:
#!/bin/bash
export JAVA_OPTS="-Xms512m -Xmx512m"
./run.sh com.example.FirstApp -- "$#"
and second.sh:
#!/bin/bash
export JAVA_OPTS="-Xms2048m -Xmx2048m -XX:+UseConcMarkSweepGC -XX:+UseParNewGC"
./run.sh com.example.SecondApp -- "$#"
Having multiple main classes is... supported now (Q4 2016, native package 1.2.0)
See "SBT Native Packager 1.2.0" by Muki Seiler
Single Project — Multiple Apps
A major pain point for beginners is the start script creation. The bash and bat start scripts are only generated when there is a either
Exactly one main class
Explicitly set main class with
mainClass in Compile := Some(“com.example.MainClass”)
For 1.2.x we will extends the implementation and support multiple main classes by default.
Native packager will generate a start script for each main class found on the classpath.
SBT provides them via the discoveredMainClasses in Compile task.
If there is only one main class, SBT will assign it to the mainClass in Compile setting. This leads to three cases:
Exactly one main class.
In this case native-packager will behave like previous versions and just generate a single start script, using the executableScriptName setting for the script name.
Multiple main classes and mainClass in Compile := None.
This is the default behaviour defined by SBT. In this case native-packager will generate the same start script for each main class.
Multiple main classes and mainClass in Compile := Some(…).
The user has set a specific main class, which will lead to a main start script being generated using the executableScriptName setting. For all other main classes native-packager generates forwarder scripts.
Having multiple main classes not supported now. As a workaround you could use single main class and check command line args.
Starting your app:
myApp prog1
In your main class:
def main(args: Array[String]): Unit = {
if(args[0] == "prog1")
Programm1.start()
else
Programm2.start()
}
In my specs2 tests, I frequently use helper functions to test groups of conditions at once. Unfortunately, that makes the line number output of failed tests useless, since all failures are on the same line.
Google turned up that there's a "failtrace" option that will output the stack trace of failure. However, I can't find an example of how to actually use that. Is it in build.sbt? Is it used on the SBT command line? Is it set somehow in the constructor of the Specification class?
You can set the failtrace option at the specification level:
class MySpec extends org.specs2.mutable.Specification {
args.report(failtrace = true)
...
}
You can also pass failtrace on the sbt command line:
sbt> test-only *MySpec* -- failtrace
Eric's solution can also be applied to all tests run by SBT by including the following in your build.sbt file:
testOptions += Tests.Argument(TestFrameworks.Specs2, "failtrace")
I use sbt in the following fashion: I run ~ test:compile in sbt and then work in IDE, watching occasionaly if the project still compiles, because the IDE's presentation compiler tends to be buggy. When I git pull some code, there might be changes in the project/ files, so I want to have reload. Is there a way, how to watch both source files and project files, so when there is change in project files, I actually get the update?
As jsuereth explained this isn't a task SBT can handle in 1 instance. What's required is a reboot of SBT to abort the watching process and reload it's own configuration.
The following Scala script does exactly this, it uses Java NIO WatchService and Scala Process to monitor a path and restart a process. The code should be fairly simple to understand:
#!/usr/bin/env scala
import java.nio.file._
import scala.collection.JavaConversions._
import scala.sys.process._
val file = Paths.get(args(0))
val cmd = args(1)
val watcher = FileSystems.getDefault.newWatchService
file.register(
watcher,
StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_MODIFY,
StandardWatchEventKinds.ENTRY_DELETE
)
def exec = cmd run true
#scala.annotation.tailrec
def watch(proc: Process): Unit = {
val key = watcher.take
val events = key.pollEvents
val newProc =
if (!events.isEmpty) {
proc.destroy()
exec
} else proc
if (key.reset) watch(newProc)
else println("aborted")
}
watch(exec)
Usage in your sbt dir would be:
watchr.scala project/ "sbt ~ test:compile"
If anything is unclear don't hesitate to ask, of course any scripting language could be used to implement this behavior.
You actually can't use ~ <task> and have it rebuild the project itself right now, because ~ <task> needs to read the build definition itself to determine:
What source files to watch
How to run the task.
What you're doing is altering the config whe project/ changes. This requires a full reload or reboot of sbt to pull in the new configuration.
So, as of sbt 0.13, this isn't possible. You can have it so it will rebuild your source code when project/ changes, but without rebuilding the build definition, not much help.
You could create a new sbt prompt, or task, that when run could check to see if source files in project/ are updated and issue a warning/error so you know to reboot. That's probably the best option right now.