Make tests in multi project sbt projects run only if tests pass in dependent projects - scala

Assuming a multiproject SBT project with a foo-project and bar-project, such that foo-project depends on bar-project for code etc.
I would like tests in foo-project to run iff the tests in bar-project pass.
How?

You may provide explicit dependencies between projects. For example root -> A -> B
Test case on GitHub. Project definition:
val commonSettings = Seq(libraryDependencies += "org.scalatest" %% "scalatest" % "1.9.1")
lazy val a: Project = (project in file("a")) settings(commonSettings: _*) settings(
name := "a",
test in Test <<= test in Test dependsOn (test in Test in b)
)
lazy val b: Project = (project in file("b")) settings(commonSettings: _*) settings(
name := "b"
)
lazy val root: Project = (project in file(".")) settings(commonSettings: _*) settings(
name := "root",
test in Test <<= test in Test dependsOn (test in Test in a)
)
Beginning from B and complete them successful:
ezh#mobile ZZZZZZ % sbt-0.13
[info] Set current project to root (in build file:/home/ezh/ZZZZZZ/)
> root/test
[info] Compiling 1 Scala source to /home/ezh/ZZZZZZ/b/target/scala-2.10/test-classes...
[info] TestSpecB:
[info] This test
[info] - should fail
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[info] Compiling 1 Scala source to /home/ezh/ZZZZZZ/a/target/scala-2.10/test-classes...
[info] TestSpecA:
[info] This test
[info] - should succeed
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for root/test:test
[success] Total time: 5 s, completed 28.11.2013 16:20:12
Beginning from B but fail:
ezh#mobile ZZZZZZ % sbt-0.13
[info] Set current project to root (in build file:/home/ezh/ZZZZZZ/)
> test
[info] Compiling 1 Scala source to /home/ezh/ZZZZZZ/b/target/scala-2.10/test-classes...
[info] TestSpecB:
[info] This test
[info] - should fail *** FAILED ***
[info] 2 did not equal 3 (Test.scala:5)
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] TestSpecB
[error] (b/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 3 s, completed 28.11.2013 16:20:35
>

as already noted this is probably evil, however this should probably work:
import sbt._
import sbt.Keys._
object Build extends Build {
lazy val projectA = project
lazy val myTest = taskKey[Seq[Option[Tests.Output]]]("my test")
lazy val root: Project = project in file(".") settings (myTest <<= myTestTask) dependsOn projectA
def myTestTask = Def.task {
val state: State = Keys.state.value
val log: Logger = streams.value.log
val extracted = Project.extract(state)
import extracted._
def noTestsMessage(scoped: ScopedKey[_])(implicit display: Show[ScopedKey[_]]): String =
"No tests to run for " + display(scoped)
def f(ref: ProjectReference) = for {
state Pair Value(r) <- Project.runTask(executeTests in(ref, Test), state)
_ = Tests.showResults(log, r, noTestsMessage(test in ref))
} yield r
val depsTests = currentProject.dependencies.map(_.project).map(f)
val passed = depsTests.forall(_.forall(_.overall == TestResult.Passed))
if (passed) depsTests :+ f(ThisProject) else depsTests
}
}
http://scastie.org/3319

Related

How do I import a package defined in src/test directory from inside src/it [duplicate]

I would like to share a helper trait between my "test" and "it" configurations in SBT, but I have not figured out how.
Here is a minimal example:
project/Build.scala
import sbt._
import Keys._
object MyBuild extends Build {
val scalaTest = "org.scalatest" %% "scalatest" % "2.0" % "test,it"
lazy val myProject =
Project(id = "my-project", base = file("."))
.configs(IntegrationTest)
.settings(Defaults.itSettings: _*)
.settings(
scalaVersion := "2.10.3",
libraryDependencies ++= Seq(
scalaTest
)
)
}
src/test/scala/Helpers.scala
trait Helper {
def help() { println("helping.") }
}
src/test/scala/TestSuite.scala
import org.scalatest._
class TestSuite extends FlatSpec with Matchers with Helper {
"My code" should "work" in {
help()
true should be(true)
}
}
src/it/scala/ItSuite.scala
import org.scalatest._
class ItSuite extends FlatSpec with Matchers with Helper {
"My code" should "work" in {
help()
true should be(true)
}
}
then, in sbt, "test" works:
sbt> test
helping.
[info] TestSuite:
[info] My code
[info] - should work
[info] Run completed in 223 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 0 s, completed Dec 17, 2013 1:54:56 AM
but "it:test" doesn't compile:
sbt> it:test
[info] Compiling 1 Scala source to ./target/scala-2.10/it-classes...
[error] ./src/it/scala/ItSuite.scala:3: not found: type Helper
[error] class ItSuite extends FlatSpec with Matchers with Helper {
[error] ^
[error] ./src/it/scala/ItSuite.scala:5: not found: value help
[error] help()
[error] ^
[error] two errors found
[error] (it:compile) Compilation failed
[error] Total time: 1 s, completed Dec 17, 2013 1:55:00 AM
If you want to share code from Test configuration, it's probably better to create a custom test configuration from Test. See Custom test configuration.
Your project/Build.scala becomes:
import sbt._
import Keys._
object MyBuild extends Build {
lazy val FunTest = config("fun") extend(Test)
val scalaTest = "org.scalatest" %% "scalatest" % "2.0" % "test"
lazy val myProject =
Project(id = "my-project", base = file("."))
.configs(FunTest)
.settings(inConfig(FunTest)(Defaults.testSettings) : _*)
.settings(
scalaVersion := "2.10.3",
libraryDependencies ++= Seq(
scalaTest
)
)
}
Also rename src/it/ to src/fun/. Now fun:test works:
> fun:test
helping.
[info] ItSuite:
[info] My code
[info] - should work
[info] Run completed in 245 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 1 s, completed Dec 17, 2013 8:43:17 AM
You can redefine the IntegrationTest Configuration in your project to extend the Test configuration instead of the Runtime Configuration (the default). This will make everything in your test configuration available to your IntegrationTest configuration.
import sbt._
import Keys._
object MyBuild extends Build {
val scalaTest = "org.scalatest" %% "scalatest" % "2.0" % "test,it"
lazy val IntegrationTest = config("it") extend(Test)
lazy val myProject =
Project(id = "my-project", base = file("."))
.configs(IntegrationTest)
.settings(Defaults.itSettings: _*)
.settings(
scalaVersion := "2.10.3",
libraryDependencies ++= Seq(
scalaTest
)
)
}

Running setup/teardown tasks during stb test task

I try to configure sbt to run a setup task at the beginning and a tear down task at the end of myProject/test command.
My build.sbt is here:
name := "ch-2"
version := "1.0"
libraryDependencies += "org.specs2" % "specs2_2.10" % "1.14" % "test"
lazy val common = (
Project("common", file("common")).
settings()
)
lazy val subPro = (
Project("sub", file("subA")).settings(
).dependsOn(common)
settings(libraryDependencies += "org.specs2" % "specs2_2.10" % "1.14" % "test" )
)
val startS = taskKey[Unit]("Start")
val stopS = taskKey[Unit]("Stop")
startS := { println("Running start")}
stopS := { println("Running stop")}
testOptions in Test in subPro += Tests.Setup { () => startS.value }
testOptions in Test in subPro += Tests.Cleanup { () => stopS.value }
The actual dummy test class is here:
import org.specs2.mutable.Specification
/**
* Created by jk on 26.3.2017.
*/
object FooSpec extends Specification {
"The TEST method" should {
"blaa blaa 1" in {
println("test 1 running...")
true
}
"blaa blaa 2" in {
println("test 2 running...")
true
}
}
}
When I run the tests for project sub, I get following output:
> sub/test
Running stop
Running start
[info] Updating {file:/home/jk/workspace/sbt-in-action/ch2/}sub...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 1 Scala source to /home/jk/workspace/sbt-in-action/ch2/subA/target/scala-2.10/test-classes...
test 1 running...
test 2 running...
[info] FooSpec
[info]
[info] The TEST method should
[info] + blaa blaa 1
[info] + blaa blaa 2
[info]
[info]
[info] Total for specification FooSpec
[info] Finished in 18 ms
[info] 2 examples, 0 failure, 0 error
[info]
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 3 s, completed Mar 26, 2017 7:09:34 PM
Why is stop task run even before compilation is finished and how to fix it so that it is run after all test cases are run (despite the result of the test cases)?
Also the start task should run after successfull compilation but before first test case. How to fix these?
How about doing it like this instead?
def startS(): Unit = { println("Running start")}
def stopS(): Unit = { println("Running stop")}
testOptions in Test in subPro += Tests.Setup { () => startS() }
testOptions in Test in subPro += Tests.Cleanup { () => stopS() }

ScalaTest on sbt not running any tests

I am trying to run my tests with:
sbt and then test.
My build.sbt looks like this
lazy val scalatest = "org.scalatest" % "scalatest_2.11" % "2.2.4" % "test"
lazy val root = (project in file(".")).
settings(
name := "highlight2pdf",
version := "0.1",
scalaVersion := "2.11.6",
libraryDependencies += scalatest
)
And i just put the example test on test/scala
import collection.mutable.Stack
import org.scalatest._
class ExampleSpec extends FlatSpec with Matchers {
"A Stack" should "pop values in last-in-first-out order" in {
val stack = new Stack[Int]
stack.push(1)
stack.push(2)
stack.pop() should be (2)
stack.pop() should be (1)
}
it should "throw NoSuchElementException if an empty stack is popped" in {
val emptyStack = new Stack[Int]
a [NoSuchElementException] should be thrownBy {
emptyStack.pop()
}
}
}
Still it always show:
[info] No tests were executed.
Any thoughts on why it's not working?
The directory structure isn't the correct convention for sbt to find ExampleSpec.scala by default.
├── built.sbt
├── src
│   └── test
│   └── scala
│   ├── ExampleSpec.scala
Change it to the structure above and run sbt test in the top level directory and it should work. Likewise, scala source would go in src/main/scala and would get compiled.
> test
[info] Compiling 1 Scala source to /tmp/TestsWontRun/target/scala-2.11/test-classes...
[info] ExampleSpec:
[info] A Stack
[info] - should pop values in last-in-first-out order
[info] - should throw NoSuchElementException if an empty stack is popped
[info] Run completed in 289 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 7 s, completed Apr 30, 2015 8:54:30 AM

How to add examples to an SBT project?

I want to add some examples to my Scala library. I prefer a directory layout like this:
examples/
example1/
Main.scala
...
example2/
Main.scala
...
src/
main/
scala/
...
test/
scala/
...
build.sbt
The examples use the packages in src/main/scala/. Is it possible to build and run the examples using SBT? Is it also possible to have dependencies specific to the examples?
It is possible, but you likely have to resort to complete .scala definition. I've put example project on bitbucket.
Here is the showcase:
// [info] Loading global plugins from /Users/omnomnom/.sbt/0.13/plugins
// [info] Loading project definition from /Users/omnomnom/projects/example-of-examples/project
// [info] Set current project to root-project (in build file:/Users/omnomnom/projects/example-of-examples/)
> projects
// [info] In file:/Users/omnomnom/projects/example-of-examples/
// [info] example1
// [info] example2
// [info] * root-project
> project example1
// [info] Set current project to example1 (in build file:/Users/omnomnom/projects/example-of-examples/)
> run
// [info] Updating {file:/Users/omnomnom/projects/example-of-examples/}root-project...
// [info] Resolving org.fusesource.jansi#jansi;1.4 ...
// [info] Done updating.
// [info] Updating {file:/Users/omnomnom/projects/example-of-examples/}example1...
// [info] Resolving org.fusesource.jansi#jansi;1.4 ...
// [info] Done updating.
// [info] Compiling 1 Scala source to /Users/omnomnom/projects/example-of-examples/target/scala-2.10/classes...
// [info] Running Main
I'm example #1
// [success] Total time: 4 s, completed Jul 4, 2014 10:17:54 PM
> project example2
// [info] Set current project to example2 (in build file:/Users/omnomnom/projects/example-of-examples/)
> run
// [info] Updating {file:/Users/omnomnom/projects/example-of-examples/}example2...
// [info] Resolving org.fusesource.jansi#jansi;1.4 ...
// [info] Done updating.
// [info] Compiling 1 Scala source to /Users/omnomnom/projects/example-of-examples/examples/example2/target/scala-2.10/classes...
// [info] Running Main
I'm example #2
// [success] Total time: 3 s, completed Jul 4, 2014 10:18:04 PM
And Build.scala, just for the case:
import sbt._
import Keys._
object Build extends Build {
override lazy val settings = super.settings ++ Seq(
scalacOptions ++= Seq("-unchecked", "-deprecation", "-Xlint"),
organization := "me.lazyval",
scalaVersion := "2.10.4",
initialCommands in console := "import me.lazyval._"
)
// altering source path, since you don't want to replicate usual src/main ... stuff
val exampleSourcePath = scalaSource in Compile := baseDirectory.value / "."
lazy val root = Project(id = "root-project", base = file("."), settings = Project.defaultSettings ++ settings)
lazy val example1 = Project(id = "example1", base = file("./examples/example1"), settings = Project.defaultSettings ++ settings ++ Seq(exampleSourcePath)) dependsOn root
// in `settings= ...` section you can set whatever dependencies you like
lazy val example2 = Project(id = "example2", base = file("./examples/example2"), settings = Project.defaultSettings ++ settings ++ Seq(exampleSourcePath)) dependsOn root
}

Play test-only doesn't ignore tests

I have a play application and I need to ignore my functional tests when I compile and build, then later run only the integration tests.
This is my test code:
ApplicationSpec.scala
#RunWith(classOf[JUnitRunner])
class ApplicationSpec extends Specification {
"Application" should {
"send 404 on a bad request" in new WithApplication {
route(FakeRequest(GET, "/boum")) must beNone
}
}
IntegrationSpec.scala
#RunWith(classOf[JUnitRunner])
class IntegrationSpec extends Specification {
"Application" should {
"work from within a browser" in {
running(TestServer(9000), FIREFOX) { browser =>
browser.goTo("http://localhost:9000/app")
browser.pageSource must contain("Your new application is ready.")
}
}
} section "integration"
}
The docs tells me I can use something like this from the command line:
play "test-only -- exclude integration"
The only problem is that this doesn't actually exclude any tests and my integration tests invoke firefox and start running. What am I doing wrong? How can I exclude the integration tests and then later run them by themselves?
sbt's Testing documentation describes several ways to define separate test configurations.
For instance, the Additional test configurations with shared sources example could be used in this situation:
In this approach, the sources are compiled together using the same classpath and are packaged together. However, different tests are run depending on the configuration.
In a default Play Framework 2.2.2 application created with play new, add a file named project/Build.scala with this content:
import sbt._
import Keys._
object B extends Build {
lazy val root =
Project("root", file("."))
.configs(FunTest)
.settings(inConfig(FunTest)(Defaults.testTasks) : _*)
.settings(
libraryDependencies += specs,
testOptions in Test := Seq(Tests.Filter(unitFilter)),
testOptions in FunTest := Seq(Tests.Filter(itFilter))
)
def itFilter(name: String): Boolean = (name endsWith "IntegrationSpec")
def unitFilter(name: String): Boolean = (name endsWith "Spec") && !itFilter(name)
lazy val FunTest = config("fun") extend(Test)
lazy val specs = "org.specs2" %% "specs2" % "2.0" % "test"
}
To run standard unit tests:
$ sbt test
Which produces:
[info] Loading project definition from /home/fernando/work/scratch/so23160453/project
[info] Set current project to so23160453 (in build file:/home/fernando/work/scratch/so23160453/)
[info] ApplicationSpec
[info] Application should
[info] + send 404 on a bad request
[info] + render the index page
[info] Total for specification ApplicationSpec
[info] Finished in 974 ms
[info] 2 examples, 0 failure, 0 error
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 3 s, completed Apr 22, 2014 12:42:37 PM
To run tests for the added configuration (here, "fun"), prefix it with the configuration name:
$ sbt fun:test
Which produces:
[info] Loading project definition from /home/fernando/work/scratch/so23160453/project
[info] Set current project to so23160453 (in build file:/home/fernando/work/scratch/so23160453/)
[info] IntegrationSpec
[info] Application should
[info] + work from within a browser
[info] Total for specification IntegrationSpec
[info] Finished in 0 ms
[info] 1 example, 0 failure, 0 error
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 6 s, completed Apr 22, 2014 12:43:17 PM
You can also test only a particular class, like this:
$ sbt "fun:testOnly IntegrationSpec"
See this example in GitHub.