Running setup/teardown tasks during stb test task - scala

I try to configure sbt to run a setup task at the beginning and a tear down task at the end of myProject/test command.
My build.sbt is here:
name := "ch-2"
version := "1.0"
libraryDependencies += "org.specs2" % "specs2_2.10" % "1.14" % "test"
lazy val common = (
Project("common", file("common")).
settings()
)
lazy val subPro = (
Project("sub", file("subA")).settings(
).dependsOn(common)
settings(libraryDependencies += "org.specs2" % "specs2_2.10" % "1.14" % "test" )
)
val startS = taskKey[Unit]("Start")
val stopS = taskKey[Unit]("Stop")
startS := { println("Running start")}
stopS := { println("Running stop")}
testOptions in Test in subPro += Tests.Setup { () => startS.value }
testOptions in Test in subPro += Tests.Cleanup { () => stopS.value }
The actual dummy test class is here:
import org.specs2.mutable.Specification
/**
* Created by jk on 26.3.2017.
*/
object FooSpec extends Specification {
"The TEST method" should {
"blaa blaa 1" in {
println("test 1 running...")
true
}
"blaa blaa 2" in {
println("test 2 running...")
true
}
}
}
When I run the tests for project sub, I get following output:
> sub/test
Running stop
Running start
[info] Updating {file:/home/jk/workspace/sbt-in-action/ch2/}sub...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 1 Scala source to /home/jk/workspace/sbt-in-action/ch2/subA/target/scala-2.10/test-classes...
test 1 running...
test 2 running...
[info] FooSpec
[info]
[info] The TEST method should
[info] + blaa blaa 1
[info] + blaa blaa 2
[info]
[info]
[info] Total for specification FooSpec
[info] Finished in 18 ms
[info] 2 examples, 0 failure, 0 error
[info]
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 3 s, completed Mar 26, 2017 7:09:34 PM
Why is stop task run even before compilation is finished and how to fix it so that it is run after all test cases are run (despite the result of the test cases)?
Also the start task should run after successfull compilation but before first test case. How to fix these?

How about doing it like this instead?
def startS(): Unit = { println("Running start")}
def stopS(): Unit = { println("Running stop")}
testOptions in Test in subPro += Tests.Setup { () => startS() }
testOptions in Test in subPro += Tests.Cleanup { () => stopS() }

Related

How do I import a package defined in src/test directory from inside src/it [duplicate]

I would like to share a helper trait between my "test" and "it" configurations in SBT, but I have not figured out how.
Here is a minimal example:
project/Build.scala
import sbt._
import Keys._
object MyBuild extends Build {
val scalaTest = "org.scalatest" %% "scalatest" % "2.0" % "test,it"
lazy val myProject =
Project(id = "my-project", base = file("."))
.configs(IntegrationTest)
.settings(Defaults.itSettings: _*)
.settings(
scalaVersion := "2.10.3",
libraryDependencies ++= Seq(
scalaTest
)
)
}
src/test/scala/Helpers.scala
trait Helper {
def help() { println("helping.") }
}
src/test/scala/TestSuite.scala
import org.scalatest._
class TestSuite extends FlatSpec with Matchers with Helper {
"My code" should "work" in {
help()
true should be(true)
}
}
src/it/scala/ItSuite.scala
import org.scalatest._
class ItSuite extends FlatSpec with Matchers with Helper {
"My code" should "work" in {
help()
true should be(true)
}
}
then, in sbt, "test" works:
sbt> test
helping.
[info] TestSuite:
[info] My code
[info] - should work
[info] Run completed in 223 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 0 s, completed Dec 17, 2013 1:54:56 AM
but "it:test" doesn't compile:
sbt> it:test
[info] Compiling 1 Scala source to ./target/scala-2.10/it-classes...
[error] ./src/it/scala/ItSuite.scala:3: not found: type Helper
[error] class ItSuite extends FlatSpec with Matchers with Helper {
[error] ^
[error] ./src/it/scala/ItSuite.scala:5: not found: value help
[error] help()
[error] ^
[error] two errors found
[error] (it:compile) Compilation failed
[error] Total time: 1 s, completed Dec 17, 2013 1:55:00 AM
If you want to share code from Test configuration, it's probably better to create a custom test configuration from Test. See Custom test configuration.
Your project/Build.scala becomes:
import sbt._
import Keys._
object MyBuild extends Build {
lazy val FunTest = config("fun") extend(Test)
val scalaTest = "org.scalatest" %% "scalatest" % "2.0" % "test"
lazy val myProject =
Project(id = "my-project", base = file("."))
.configs(FunTest)
.settings(inConfig(FunTest)(Defaults.testSettings) : _*)
.settings(
scalaVersion := "2.10.3",
libraryDependencies ++= Seq(
scalaTest
)
)
}
Also rename src/it/ to src/fun/. Now fun:test works:
> fun:test
helping.
[info] ItSuite:
[info] My code
[info] - should work
[info] Run completed in 245 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 1 s, completed Dec 17, 2013 8:43:17 AM
You can redefine the IntegrationTest Configuration in your project to extend the Test configuration instead of the Runtime Configuration (the default). This will make everything in your test configuration available to your IntegrationTest configuration.
import sbt._
import Keys._
object MyBuild extends Build {
val scalaTest = "org.scalatest" %% "scalatest" % "2.0" % "test,it"
lazy val IntegrationTest = config("it") extend(Test)
lazy val myProject =
Project(id = "my-project", base = file("."))
.configs(IntegrationTest)
.settings(Defaults.itSettings: _*)
.settings(
scalaVersion := "2.10.3",
libraryDependencies ++= Seq(
scalaTest
)
)
}

ScalaTest: no tests are run for GeneratorDrivenPropertyChecks

I am trying to set-up a property based testing with ScalaTest and ScalaCheck ... and based on the ouput it seems that I am succeeding, but it takes too fast and from what I understand normally ScalaCheck should inform you about how may tests were run, in my case this information is absent:
[IJ]sbt:algorithms2_1> testOnly *MedianOf3PartitioningProps
[info] Compiling 1 Scala source to /Users/vasile.gorcinschi/gitPerso/Algorithms/Chapter 2 Sorting/algorithms2_1/target/scala-2.12/test-classes ...
[warn] there was one deprecation warning; re-run with -deprecation for details
[warn] one warning found
[info] Done compiling.
[info] MedianOf3PartitioningProps:
[info] sort
[info] - should sort array of ints from 0 to 100
[info] +
[info] ScalaTest
[info] Run completed in 412 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
Here is the test class:
class MedianOf3PartitioningProps extends FlatSpec with Matchers with GeneratorDrivenPropertyChecks with Gens {
private val medianOf3Partitioning = new MedianOf3Partitioning[Int]
implicit override val generatorDrivenConfig: PropertyCheckConfiguration = PropertyCheckConfig(minSuccessful = 1, maxDiscarded = 500, workers = 1)
behavior of "sort"
it should "sort array of ints from 0 to 100" in {
forAll(arraysGen){ a: Array[Int] =>
info(s"${a.mkString(",")}")
medianOf3Partitioning.sort(a) shouldEqual a.sorted }
}
}
The Gens trait is mine - it only comprises definition for the Gen[Array[Int]]:
trait Gens {
val arraysGen: Gen[Array[Int]] = containerOf[Array, Int](
chooseNum(Int.MinValue, Int.MaxValue) suchThat { _ < 100 }
).suchThat(_.length < 50)
}
I used this source for the test set-up. Just in case, I'm providing versions of scalacheck and scalatest (from Dependencies.scala and build.sbt):
lazy val scalaTest = "org.scalatest" %% "scalatest" % "3.0.5"
...
libraryDependencies ++= {
val scalaTestVersion = "3.0.5"
val scalaCheckVersion = "1.14.0"
Seq(scalaTest % Test,
"org.scalatest" %% "scalatest" % scalaTestVersion % "test",
"org.scalacheck" %% "scalacheck" % scalaCheckVersion % "test",
"com.storm-enroute" %% "scalameter" % "0.9"
)
}
Based on the small example from M. Odersky's "Programming in Scala", I switched from GeneratorDrivenPropertyChecks to more general PropertyChecks. I've also discovered issues with my Gen[Array[Int]] so I had to tweek that too. Posting a solution that worked (discovered cases that fail) in case this will help anyone else:
Gens trait:
trait Gens {
val minIntArraysGen: Gen[Array[Int]] = containerOf[Array, Int](Gen.chooseNum(0, 100))
}
The property based test:
import ca.vgorcinschi.Gens
import org.scalatest.MustMatchers._
import org.scalatest.WordSpec
import org.scalatest.prop.PropertyChecks
class MedianOf3PartitioningProps extends WordSpec with PropertyChecks with Gens {
"sort method" must {
"sort any Int array" in {
forAll (minIntArraysGen){ (a: Array[Int]) =>
whenever(a.nonEmpty) {
val maybeSorted = new MedianOf3Partitioning[Int].sort(a)
maybeSorted must equal (a.sorted)
}
}
}
}
}

initializate value only once before all tests runs

I want to initializate value only once before all tests runs
Now the value insertDbScheme create scheme in every test, if it possible to run it only once. Insert scheme from sbt task not suitable
I have two tests
test1:
class Test1 extends Specification with InsertDbScheme {
"test" in {
insertDbScheme
// some code
ok
}
}
test2:
class Test2 extends Specification with InsertDbScheme {
"test" in {
insertDbScheme
// some code
ok
}
}
and a base trait that insert scheme to database
trait InsertDbScheme {
lazy val insertDbScheme = {
println("insert scheme to database")
}
}
result of running tests
insert scheme to database
insert scheme to database
[info] Test1
[info]
[info] + test
[info]
[info]
[info] Total for specification Test1
[info] Finished in 152 ms
[info] 1 example, 0 failure, 0 error
[info]
[info] Test2
[info]
[info] + test
[info]
[info]
[info] Total for specification Test2
[info] Finished in 152 ms
[info] 1 example, 0 failure, 0 error
[info]
[info] ScalaCheck
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 12 s, completed May 18, 2017 6:35:21 PM
You can use an object to make sure it's lazy val is only instantiated once:
trait InsertDbScheme {
import InsertDbScheme._
lazy val insertDbScheme = insertDbSchemeOnce
}
object InsertDbScheme {
lazy val insertDbSchemeOnce = {
println("insert scheme to database")
}
}
The easiest thing is to use a variable in a InsertDbScheme object:
trait InsertDbScheme {
def insertDbScheme = synchronized {
if (!InsertDbScheme.done) {
println("insert scheme to database")
InsertDbScheme.done = true
}
}
}
object InsertDbScheme {
var done = false
}

ScalaCheck specificy minimum successful tests for property

I'm trying to make sure that my ScalaCheck property runs 500 times instead of the default 100 times. I'm having trouble configuring this though.
class BlockSpec extends Properties("BlockSpec") with BitcoinSLogger {
val myParams = Parameters.default.withMinSuccessfulTests(500)
override def overrideParameters(p: Test.Parameters) = myParams
property("Serialization symmetry") =
Prop.forAll(BlockchainElementsGenerator.block) { block =>
logger.warn("Hex:" + block.hex)
Block(block.hex) == block
}
}
However when I actually run this test it only says 100 tests passed successfully
EDIT:
$ sbt
[info] Loading project definition from /home/chris/dev/bitcoins-core/project
[info] Set current project to bitcoin-s-core (in build file:/home/chris/dev/bitcoins-core/)
> test-only *BlockSpec*
[info] + BlockSpec.Serialization symmetry: OK, passed 100 tests.
[info] Elapsed time: 1 min 59.775 sec
[info] ScalaCheck
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[info] ScalaTest
[info] Run completed in 2 minutes.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 123 s, completed Aug 1, 2016 11:36:17 AM
>
How do I actually pass this to my property?
As far as I understand you can specify the test parameters at two levels and they don't seem to communicate.
The first option is within the property as you're trying to do:
import org.scalacheck.Properties
import org.scalacheck.Test.{ TestCallback, Parameters }
import org.scalacheck.Prop.{ forAll, BooleanOperators }
import org.scalacheck.Test
class TestFoo extends Properties("BlockSpec") {
override def overrideParameters(p: Parameters) =
p.withMinSuccessfulTests(1000000)
property("Serialization symmetry") = forAll { n: Int =>
(n > 0) ==> (math.abs(n) == n)
}
}
This will have no impact as long as you don't call .check on the property.
Can be from the sbt shell or directly within the class.
Now if you want to impact the number of tests run when calling the sbt:test target, it seems you have to play with options build.sbt (taken from here):
name := "scalacheck-demo"
scalaVersion := "2.11.5"
libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.2" % "test"
testOptions in Test += Tests.Argument(TestFrameworks.ScalaCheck, "-maxSize", "5", "-minSuccessfulTests", "33", "-workers", "1", "-verbosity", "1")
There's definitely an easier way to achieve that than overriding any kind of global test config:
class SampleTest extends FlatSpec
with Matchers with GeneratorDrivenPropertyChecks {
it should "work for a basic scenario" in {
// This will require 500 successful tests to succeed
forAll(minSuccessful(500)) { (d: String) =>
whenever (d.nonEmpty) {
d.length shouldBe > 0
}
}
}
}

Make tests in multi project sbt projects run only if tests pass in dependent projects

Assuming a multiproject SBT project with a foo-project and bar-project, such that foo-project depends on bar-project for code etc.
I would like tests in foo-project to run iff the tests in bar-project pass.
How?
You may provide explicit dependencies between projects. For example root -> A -> B
Test case on GitHub. Project definition:
val commonSettings = Seq(libraryDependencies += "org.scalatest" %% "scalatest" % "1.9.1")
lazy val a: Project = (project in file("a")) settings(commonSettings: _*) settings(
name := "a",
test in Test <<= test in Test dependsOn (test in Test in b)
)
lazy val b: Project = (project in file("b")) settings(commonSettings: _*) settings(
name := "b"
)
lazy val root: Project = (project in file(".")) settings(commonSettings: _*) settings(
name := "root",
test in Test <<= test in Test dependsOn (test in Test in a)
)
Beginning from B and complete them successful:
ezh#mobile ZZZZZZ % sbt-0.13
[info] Set current project to root (in build file:/home/ezh/ZZZZZZ/)
> root/test
[info] Compiling 1 Scala source to /home/ezh/ZZZZZZ/b/target/scala-2.10/test-classes...
[info] TestSpecB:
[info] This test
[info] - should fail
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[info] Compiling 1 Scala source to /home/ezh/ZZZZZZ/a/target/scala-2.10/test-classes...
[info] TestSpecA:
[info] This test
[info] - should succeed
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for root/test:test
[success] Total time: 5 s, completed 28.11.2013 16:20:12
Beginning from B but fail:
ezh#mobile ZZZZZZ % sbt-0.13
[info] Set current project to root (in build file:/home/ezh/ZZZZZZ/)
> test
[info] Compiling 1 Scala source to /home/ezh/ZZZZZZ/b/target/scala-2.10/test-classes...
[info] TestSpecB:
[info] This test
[info] - should fail *** FAILED ***
[info] 2 did not equal 3 (Test.scala:5)
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] TestSpecB
[error] (b/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 3 s, completed 28.11.2013 16:20:35
>
as already noted this is probably evil, however this should probably work:
import sbt._
import sbt.Keys._
object Build extends Build {
lazy val projectA = project
lazy val myTest = taskKey[Seq[Option[Tests.Output]]]("my test")
lazy val root: Project = project in file(".") settings (myTest <<= myTestTask) dependsOn projectA
def myTestTask = Def.task {
val state: State = Keys.state.value
val log: Logger = streams.value.log
val extracted = Project.extract(state)
import extracted._
def noTestsMessage(scoped: ScopedKey[_])(implicit display: Show[ScopedKey[_]]): String =
"No tests to run for " + display(scoped)
def f(ref: ProjectReference) = for {
state Pair Value(r) <- Project.runTask(executeTests in(ref, Test), state)
_ = Tests.showResults(log, r, noTestsMessage(test in ref))
} yield r
val depsTests = currentProject.dependencies.map(_.project).map(f)
val passed = depsTests.forall(_.forall(_.overall == TestResult.Passed))
if (passed) depsTests :+ f(ThisProject) else depsTests
}
}
http://scastie.org/3319