ScalaTest allows the setting of tags to be excluded via a filter called tagsToExclude.
How can I configure my SBT build to set this value?
Attempt 1
The CLI of ScalaTest specifies the -l flag for tags to exclude.
SBT allows the setting of CLI params like so:
testOptions in Test += Tests.Argument(
TestFrameworks.ScalaTest, "-l", "DataFileTest")`
But this seems to have no effect (i.e. the test still executes).
For reference the test looks like:
object DataFileTest extends org.scalatest.Tag("com.mydomain.DataFileTest")
class MyDataFileDependantSpec extends FunSpec
with Matchers
with BeforeAndAfter
with BeforeAndAfterAll {
describe("Something") {
it("reads a file and does things", DataFileTest) {
...
}
}
testOptions in Test ++= Seq(Tests.Argument(TestFrameworks.ScalaTest,
"-l", "org.scalatest.tags.Slow")
This works.
See if the problem is to do with the full path name of DataFileTest.
Related
I have a Scala- / Mill-Project that I wan't to import into VSCode.
The Metals Doctor gives me the following warnings:
Is there something missing in my project?
Here is my Mill configuration (build.sc):
import mill._
import mill.define.Target
import scalalib._
trait MyModule extends ScalaModule {
def scalaVersion = "2.13.1"
object version {
val cats = "2.0.0"
..
}
object libs {
val cats = ivy"org.typelevel::cats-core:${version.cats}"
..
}
object test extends Tests {
...
}
}
object entity extends MyModule {
override def ivyDeps = {
Agg(
libs.cats,
..
)
}
}
object macros extends MyModule {
..
}
Here is the whole project: https://github.com/pme123/zio-examples
The console output shows no warnings, here is the end:
...
time: connected to build server in 0.33s
time: imported build in 0.41s
time: indexed workspace in 3.85s
no build target: /Users/mpa/dev/Github/pme123/zio-examples/build.sc
This error/help message is misleading, as it presumes you are using Bloop, in which case re-importing solves the isssue (from experience). When you are using Mill's built-in BSP server, this won't change much.
Mill's recent 0.10.0-M4 release got a major BSP revamp. Also Metals improved it's Mill support. So you should have a better overall experience.
Here is some documentation: https://com-lihaoyi.github.io/mill/mill/Intro_to_Mill.html#_build_server_protocol_bsp
Junit5 has introduced #BeforeAll to signal that the annotated method should be executed before all tests in the current test class. However it has requirement of method being static.
I have following structure in my Scala test suite:
class BaseTest extends MockitoSugar {
#BeforeAll
def codeNeedeForAllTests() = { <== this does not work as as it is not static
println("calling codeNeedeForAllTests")
// other code
}
}
and other test classes:
class IndexerTest extends BaseTest {
#Test
def add() = {
//test code
}
}
So I want that codeNeedeForAllTests get called before all tests, however the catch is #BeforeAll's requirement of method being static, for which I need to make codeNeedeForAllTests as object to have static method in Scala.
Now in Scala, a class can not extend an object and and object also can not extend object.
I also tried creating companion object of BaseTest, but that also did not work, any clean approach to do this?
#BeforeAll methods in JUnit Jupiter (a.k.a., JUnit 5) do not have to be static.
The link you provided is to JUnit 4.
This link is to JUnit Jupiter.
Excerpt:
Denotes that the annotated method should be executed before all #Test, #RepeatedTest, #ParameterizedTest, and #TestFactory methods in the current class; analogous to JUnit 4’s #BeforeClass. Such methods are inherited (unless they are hidden or overridden) and must be static (unless the "per-class" test instance lifecycle is used).
And that links to the documentation on Test Instance Lifecycle.
Thus, you should be able to annotate your test class with #TestInstance(Lifecycle.PER_CLASS) and declare your #BeforeAll method as non-static.
You can do this with FunCpec.
package net
import org.scalatest._
class Test extends FunSpec with BeforeAndAfterAll {
override def beforeAll: Unit = {
println("beforeAll")
}
describe("running a test") {
println("in describe")
it("should do something") {
println("in my test")
assert(true)
}
}
}
If u'll test it in REPL:
scala> test
[info] Compiling 1 Scala source to /Users/kmmere/Workspace/Work/scalatest_before/target/scala-2.11/test-classes...
in describe
beforeAll
in my test
[info] Test:
[info] running a test
[info] - should do something
[info] Run completed in 99 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 1 s, completed Oct 19, 2015 4:32:56 PM
Don't forget to include dependency of scalatest in yor build.sbt file:
"org.scalatest" %% "scalatest" % "3.2.0-SNAP10" % Test
class FunSpec - facilitates a “behavior-driven” style of development (BDD), in which tests are combined with text that specifies the behavior the tests verify.
Recommended Usage: For teams coming from Ruby's RSpec tool, FunSpec will feel familiar and comfortable; More generally, for any team that prefers BDD, FunSpec's nesting and gentle guide to structuring text (with describe and it) provide an excellent general-purpose choice for writing specification-style tests.
Currently I'm using IntelliJ Idea 15 and Scalatest framework to make some unit tests. And I need to pass my own arguments into test and somehow read them from code. For instance:
Suppose I have such class
class Test extends FunSuite {
test("Some test") {
val arg = // here I want to get an argument which I want to pass. Something like args("arg_name")
println(arg)
assert(2 == 2)
}
}
And to run the test with argument I want to do something like
test -arg_name=blabla
So, the question is how to pass this an argument and how to obtain it.
I found interesting trait BeforeAndAfterAllConfigMap. This one has beforeAll method with a parameter configMap: ConfigMap. So here is my solution:
class Test extends FunSuite with BeforeAndAfterAllConfigMap {
override def beforeAll(configMap: ConfigMap) = {
//here will be some stuff and all args are available in configMap
}
test("Some test") {
val arg = // here I want to get an argument which I want to pass. Something like args("arg_name")
println(arg)
assert(2 == 2)
}
}
In scalatest, we can use configMap to pass command parameters.
There is an example with using configMap:
import org.scalatest.{ConfigMap, fixture}
class TestSuite extends fixture.Suite with fixture.ConfigMapFixture{
def testConfigMap(configMap: Map[String, Any]) {
println(configMap.get("foo"))
assert(configMap.get("foo").isDefined)
}
}
object Test {
def main(args: Array[String]): Unit = {
(new TestSuite).execute(configMap = ConfigMap.apply(("foo","bar")))
}
}
also we can run test with command line parameters:
scala -classpath scalatest-<version>.jar org.scalatest.tools.Runner -R compiled_tests -Dfoo=bar
scalatest runner
ConfigMapFixture Test
I found BeforeAndAfterAllConfigMap incorporates into existing scalatest easily if you have already using FunSpec, FunSuite or FlatSpec. all you have to do is import & extend your test with BeforeAndAfterAllConfigMap.
One of the ways you can pass argument to scalatest is:
Pass the argument from command line as either
system or project property
In build.gradle file retrieve the [system|project] property
Pass property retrieved in step 2 to
test runtime.
Here is a complete example in FlatSpec but easily applicable in FunSuite as well:
package com.passarg.test
import org.scalatest.{BeforeAndAfterAllConfigMap, ConfigMap, FlatSpec}
class PassArgsToScala extends FlatSpec with BeforeAndAfterAllConfigMap {
var foo = ""
override def beforeAll(configMap: ConfigMap) = {
// foo=bar is expected to be passed as argument
if (configMap.get("foo").isDefined) {
foo = configMap.get("foo").fold("")(_.toString)
}
println("foo=" + foo)
}
"Arg passed" must "be bar" in {
assert(foo === "bar")
info("passing arg seem to work ==> " + "foo=" + foo)
}
}
build.gradle
apply plugin: 'scala'
repositories {
mavenCentral()
}
dependencies {
compile 'org.scala-lang:scala-library:2.11.8'
testCompile 'org.scalatest:scalatest_2.11:3.0.0'
}
task spec(dependsOn: ['testClasses']) {
javaexec {
main = 'org.scalatest.tools.Runner'
args = ['-R', 'build/classes/test', '-o']
args += '-m'
args += 'com.passarg.test'
if (project.hasProperty("foo")) {
args += "-Dfoo=" + project.foo
}
classpath = sourceSets.test.runtimeClasspath
}
}
Running the test from command-line:
./gradlew -q spec -Pfoo=bar
Result:
$ ./gradlew -q spec -Pfoo=bar
...
Run starting. Expected test count is: 1
PassArgsToScala:
Arg passed
- must be bar
+ passing arg seem to work ==> foo=bar
I recently had to pass a feature flag to scalatest tests so I could test two implementations of the same API. I used an environment variable for this. It works for sbt & maven. The content of the variable will end up in the configMap. How to deal with the configMap you can check at other posts (or at the post I base this answer on).
You can pass the argument "ABC" as follows:
FEATURE_FLAG="ABC" sbt test
# or
FEATURE_FLAG="ABC" mvn test
sbt setup:
// build.sbt
val featureFlagArg = sys.env.get("FEATURE_FLAG")
.map(ff => s"-DfeatureFlag=$ff")
.map(Tests.Argument(TestFrameworks.ScalaTest, _))
Test / testOptions ++= featureFlagArg
maven setup (using the scalatest maven plugin):
<!-- pom.xml -->
<!-- ... -->
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<config>featureFlag=${env.FEATURE_FLAG}</config>
</configuration>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- ... -->
I am running a multi-project SBT (v0.13) build and would like it to fail fast (stop) on the first error (compile) encountered in a sub-project.
The current behavior is that when something fails to compile in a sub-project, the build continues (to compile all the other sub-projects).
Is there a way/setting to make SBT stop and exit as soon as the first sub-project with a compile error is encountered?
In short, to my knowledge, no, SBT can not "fail fast" on a compiler or test error.
To my knowledge SBT doesn't control this. SBT is just going to invoke the appropriate test framework when it inspects your unit tests. The test framework can then decide what order to run the tests in, to run them concurrently, how to report issues, etc. Of course each test framework has its own features, configuration, and conventions. The two most widely used test frameworks for Scala are ScalaTest and Specs2.
Fortunately you can get the behavior you've requested in either Specs2 or ScalaTest. I've provided simple unit test examples below that fail early.
ScalaTest
You can get fail-fast behavior in ScalaTest for single test Suites by mixing in the CancelAfterFailure trait. For example, this test would execute the first test, fail the second, and show the third as cancelled.
class SomeSuite extends FunSuite with CancelAfterFailure with Assertions {
test("first") {
println("first")
}
test("second") {
assert(false)
}
test("third") {
println("third")
}
}
Specs2
Similar to ScalaTest you can control behavior in Specs2 on a per-Specification basis. To get fail-fast-like behavior you need to add two Arguments to your Specification: sequential and stopOnFail. See the docs for a full list of arguments you can set. (You do need both if you want an obvious linear ordering since Specs2 by default will execute your tests concurrently!)
class SomeSpec extends Specification {
sequential
stopOnFail
"specs2" should {
"first" in {
println("first")
ok
}
"second" in {
"foo" must equalTo ("bar")
}
"third" in {
println("third")
}
}
}
I've discovered that throwing a new java.lang.VirtualMachineError() {} halts the task immediately, while all other kinds of exceptions I have tried so far were swallowed.
I've tried ThreadDeath, InterruptedException, LinkageError, and ControlThrowable, and of course, the usual RuntimeException and such (the list came from scala.util.control.NonFatal)
<project root>/project/BuildUtils.scala
import sbt._
import sbt.Def._
object BuildUtils {
private def details(inc: Incomplete): Seq[Throwable] =
inc.directCause.toSeq ++ inc.causes.flatMap(details)
implicit class TaskSyntax[A](private val task: Initialize[Task[A]]) extends AnyVal {
def haltWhenFailed: Initialize[Task[A]] = task.result.map {
case Inc(cause) =>
throw new VirtualMachineError({
s"""Task has failed during execution.
|TaskNode: ${cause.node}
|DirectCause: ${details(cause).map(_.getMessage).distinct}
|""".stripMargin
}) {}
case Value(value) => value
}
}
}
<project root>/build.sbt
import BuildUtils._
lazy val ciBuild = taskKey[Unit]("Compile and run test")
ciBuild := {
val t1 # _ = (Test / compile).haltWhenFailed.value
val t2 # _ = (Test / test).haltWhenFailed.value
}
ciBuild will halt at first compilation error with rather verbose stacktrace and the specified message.
I want to write a test that calls a remote server and validates the response because the server may change (it's not under my control). To do this I figure I'd give it a tag (RemoteTest) and then exclude it when calling the runner:
sbt> test-only * -- -l RemoteTest
However, when doing this all my tests are run, including RemoteTest. How do I call the runner from within sbt so that it is excluded?
If you have the following:-
package com.test
import org.scalatest.FlatSpec
import org.scalatest.Tag
object SlowTest extends Tag("com.mycompany.tags.SlowTest")
object DbTest extends Tag("com.mycompany.tags.DbTest")
class TestSuite extends FlatSpec {
"The Scala language" must "add correctly" taggedAs(SlowTest) in {
val sum = 1 + 1
assert(sum === 2)
}
it must "subtract correctly" taggedAs(SlowTest, DbTest) in {
val diff = 4 - 1
assert(diff === 3)
}
}
To exclude DbTest tag, you would do:-
test-only * -- -l com.mycompany.tags.DbTest
Note that you'll need to include the full tag name. If it's still not working for you, would you mind sharing part of source code that's not working?