I want to write a test that calls a remote server and validates the response because the server may change (it's not under my control). To do this I figure I'd give it a tag (RemoteTest) and then exclude it when calling the runner:
sbt> test-only * -- -l RemoteTest
However, when doing this all my tests are run, including RemoteTest. How do I call the runner from within sbt so that it is excluded?
If you have the following:-
package com.test
import org.scalatest.FlatSpec
import org.scalatest.Tag
object SlowTest extends Tag("com.mycompany.tags.SlowTest")
object DbTest extends Tag("com.mycompany.tags.DbTest")
class TestSuite extends FlatSpec {
"The Scala language" must "add correctly" taggedAs(SlowTest) in {
val sum = 1 + 1
assert(sum === 2)
}
it must "subtract correctly" taggedAs(SlowTest, DbTest) in {
val diff = 4 - 1
assert(diff === 3)
}
}
To exclude DbTest tag, you would do:-
test-only * -- -l com.mycompany.tags.DbTest
Note that you'll need to include the full tag name. If it's still not working for you, would you mind sharing part of source code that's not working?
Related
I have a legacy Scala project to work with using maven as build tool.
Initially there is core code only under folder structure src/main/scala/ with package x.y.z and a file Main.scala with code as:
object Main {
def cube(x: Int) = {
x * x * x
}
}
There are no tests for the code. So I manually added a folder structure as test/scala under src.
Then I copied the package as for core code, ie x.y.z and added MainTest.scala with test code as:
class MainTest extends FunSuite {
test("Main.cube") {
assert(Main.cube(3) === 27)
}
}
Running test gives error as:
Error:(8, 12) not found: value Main
assert(Main.cube(3) === 27)
Why do I get this error?
For below project structure MainTest.scala and Main.scala are in the same package x.y.z therefore no package import is required, However MainTempTest.scala and Main.scala are in different package therefore an explicit import has to be done import x.y.z.Main
src
-main
--scala
---x.y.z
----Main.scala
-test
--scala
---MainTempTest.scala
---x.y.z
----MainTest.scala
Below is the main class :
object HelloWorld {
def hello() = "Hello "
}
Below is test suite:
import org.scalatest.{Matchers, FunSuite}
class HelloWorldTest extends FunSuite with Matchers {
test("Say Hi!") {
HelloWorld.hello() should be ("Hello, World!")
}
}
I wanted to run the test cases from Intellij. I tried configuration type ScalaTest and Application and the configuration package does not appear to select.
How to run the test cases from Intellij
Not sure if I got you right but you can do it in multiple ways:
Right click on HelloWorldTest and choose Run
Right click on the file name on the project file tree and click Run
You can select Run in toolbar => Edit Configurations => + on left up side => ScalaTest => give it a name and write HelloWorldTest in "Test class:" => Ok and then Run and run again and choose the name you gave it
I am trying to specify the order in which Spec2 Specifications are run, I know about the sequential keyword that makes sure they run one after another but this corresponds to the tests within a Specification (and doesn't actually guarantee any order)
I found this SO question: https://stackoverflow.com/a/15832297/1757402 which looked promising but again, seems to just sort the tests within a Specification
I am assuming SBT/Specs runs the Specifications in the order in which the JVM returns the classes, is there any way to change this? Or any way to guarantee an order?
So say I have the following Specifications:
CApplicationSpec.scala
#RunWith(classOf[JUnitRunner])
class CApplicationSpec extends Specification {
"CApplicationSpec" should {
"be OK" in new WithApplication{
OK must equalTo(OK)
}
}
}
BApplicationSpec
#RunWith(classOf[JUnitRunner])
class BApplicationSpec extends Specification {
"BApplicationSpec" should {
"be OK" in new WithApplication{
OK must equalTo(OK)
}
}
}
At the moment if I test these, the order of execution can change each time, I want a way to be able to guarantee that BApplication (or any other Spec) will always run first, maybe by sorting them alphabetically?
You can create a specification which will "link" other specifications and run them in a specified order:
object AllSpecifications extends Specification { def is = s2"""
${"run BSpecification" ~ BSpecification}
${"run CSpecification" ~ CSpecification}
"""
}
I ended up doing it via SBT with testGrouping
//Put all tests into a single Group so that SBT reports correct number of tests, but order them by test name
testGrouping in Test <<= definedTests in Test map { tests => {
val sortedTests = tests map {
test => new Tests.Group(test.name, Seq(test), Tests.InProcess)
} sortBy (_.name.toLowerCase) flatMap {
_.tests
}
Seq(new Tests.Group("Tests", sortedTests, Tests.InProcess))
}
}
Which orders all the tests alphabetically (including package name) I then have all specs I want to run first in a specific package
I am running a multi-project SBT (v0.13) build and would like it to fail fast (stop) on the first error (compile) encountered in a sub-project.
The current behavior is that when something fails to compile in a sub-project, the build continues (to compile all the other sub-projects).
Is there a way/setting to make SBT stop and exit as soon as the first sub-project with a compile error is encountered?
In short, to my knowledge, no, SBT can not "fail fast" on a compiler or test error.
To my knowledge SBT doesn't control this. SBT is just going to invoke the appropriate test framework when it inspects your unit tests. The test framework can then decide what order to run the tests in, to run them concurrently, how to report issues, etc. Of course each test framework has its own features, configuration, and conventions. The two most widely used test frameworks for Scala are ScalaTest and Specs2.
Fortunately you can get the behavior you've requested in either Specs2 or ScalaTest. I've provided simple unit test examples below that fail early.
ScalaTest
You can get fail-fast behavior in ScalaTest for single test Suites by mixing in the CancelAfterFailure trait. For example, this test would execute the first test, fail the second, and show the third as cancelled.
class SomeSuite extends FunSuite with CancelAfterFailure with Assertions {
test("first") {
println("first")
}
test("second") {
assert(false)
}
test("third") {
println("third")
}
}
Specs2
Similar to ScalaTest you can control behavior in Specs2 on a per-Specification basis. To get fail-fast-like behavior you need to add two Arguments to your Specification: sequential and stopOnFail. See the docs for a full list of arguments you can set. (You do need both if you want an obvious linear ordering since Specs2 by default will execute your tests concurrently!)
class SomeSpec extends Specification {
sequential
stopOnFail
"specs2" should {
"first" in {
println("first")
ok
}
"second" in {
"foo" must equalTo ("bar")
}
"third" in {
println("third")
}
}
}
I've discovered that throwing a new java.lang.VirtualMachineError() {} halts the task immediately, while all other kinds of exceptions I have tried so far were swallowed.
I've tried ThreadDeath, InterruptedException, LinkageError, and ControlThrowable, and of course, the usual RuntimeException and such (the list came from scala.util.control.NonFatal)
<project root>/project/BuildUtils.scala
import sbt._
import sbt.Def._
object BuildUtils {
private def details(inc: Incomplete): Seq[Throwable] =
inc.directCause.toSeq ++ inc.causes.flatMap(details)
implicit class TaskSyntax[A](private val task: Initialize[Task[A]]) extends AnyVal {
def haltWhenFailed: Initialize[Task[A]] = task.result.map {
case Inc(cause) =>
throw new VirtualMachineError({
s"""Task has failed during execution.
|TaskNode: ${cause.node}
|DirectCause: ${details(cause).map(_.getMessage).distinct}
|""".stripMargin
}) {}
case Value(value) => value
}
}
}
<project root>/build.sbt
import BuildUtils._
lazy val ciBuild = taskKey[Unit]("Compile and run test")
ciBuild := {
val t1 # _ = (Test / compile).haltWhenFailed.value
val t2 # _ = (Test / test).haltWhenFailed.value
}
ciBuild will halt at first compilation error with rather verbose stacktrace and the specified message.
ScalaTest allows the setting of tags to be excluded via a filter called tagsToExclude.
How can I configure my SBT build to set this value?
Attempt 1
The CLI of ScalaTest specifies the -l flag for tags to exclude.
SBT allows the setting of CLI params like so:
testOptions in Test += Tests.Argument(
TestFrameworks.ScalaTest, "-l", "DataFileTest")`
But this seems to have no effect (i.e. the test still executes).
For reference the test looks like:
object DataFileTest extends org.scalatest.Tag("com.mydomain.DataFileTest")
class MyDataFileDependantSpec extends FunSpec
with Matchers
with BeforeAndAfter
with BeforeAndAfterAll {
describe("Something") {
it("reads a file and does things", DataFileTest) {
...
}
}
testOptions in Test ++= Seq(Tests.Argument(TestFrameworks.ScalaTest,
"-l", "org.scalatest.tags.Slow")
This works.
See if the problem is to do with the full path name of DataFileTest.