Passing additional arguments to tests with ScalaTest - scala

Currently I'm using IntelliJ Idea 15 and Scalatest framework to make some unit tests. And I need to pass my own arguments into test and somehow read them from code. For instance:
Suppose I have such class
class Test extends FunSuite {
test("Some test") {
val arg = // here I want to get an argument which I want to pass. Something like args("arg_name")
println(arg)
assert(2 == 2)
}
}
And to run the test with argument I want to do something like
test -arg_name=blabla
So, the question is how to pass this an argument and how to obtain it.

I found interesting trait BeforeAndAfterAllConfigMap. This one has beforeAll method with a parameter configMap: ConfigMap. So here is my solution:
class Test extends FunSuite with BeforeAndAfterAllConfigMap {
override def beforeAll(configMap: ConfigMap) = {
//here will be some stuff and all args are available in configMap
}
test("Some test") {
val arg = // here I want to get an argument which I want to pass. Something like args("arg_name")
println(arg)
assert(2 == 2)
}
}

In scalatest, we can use configMap to pass command parameters.
There is an example with using configMap:
import org.scalatest.{ConfigMap, fixture}
class TestSuite extends fixture.Suite with fixture.ConfigMapFixture{
def testConfigMap(configMap: Map[String, Any]) {
println(configMap.get("foo"))
assert(configMap.get("foo").isDefined)
}
}
object Test {
def main(args: Array[String]): Unit = {
(new TestSuite).execute(configMap = ConfigMap.apply(("foo","bar")))
}
}
also we can run test with command line parameters:
scala -classpath scalatest-<version>.jar org.scalatest.tools.Runner -R compiled_tests -Dfoo=bar
scalatest runner
ConfigMapFixture Test

I found BeforeAndAfterAllConfigMap incorporates into existing scalatest easily if you have already using FunSpec, FunSuite or FlatSpec. all you have to do is import & extend your test with BeforeAndAfterAllConfigMap.
One of the ways you can pass argument to scalatest is:
Pass the argument from command line as either
system or project property
In build.gradle file retrieve the [system|project] property
Pass property retrieved in step 2 to
test runtime.
Here is a complete example in FlatSpec but easily applicable in FunSuite as well:
package com.passarg.test
import org.scalatest.{BeforeAndAfterAllConfigMap, ConfigMap, FlatSpec}
class PassArgsToScala extends FlatSpec with BeforeAndAfterAllConfigMap {
var foo = ""
override def beforeAll(configMap: ConfigMap) = {
// foo=bar is expected to be passed as argument
if (configMap.get("foo").isDefined) {
foo = configMap.get("foo").fold("")(_.toString)
}
println("foo=" + foo)
}
"Arg passed" must "be bar" in {
assert(foo === "bar")
info("passing arg seem to work ==> " + "foo=" + foo)
}
}
build.gradle
apply plugin: 'scala'
repositories {
mavenCentral()
}
dependencies {
compile 'org.scala-lang:scala-library:2.11.8'
testCompile 'org.scalatest:scalatest_2.11:3.0.0'
}
task spec(dependsOn: ['testClasses']) {
javaexec {
main = 'org.scalatest.tools.Runner'
args = ['-R', 'build/classes/test', '-o']
args += '-m'
args += 'com.passarg.test'
if (project.hasProperty("foo")) {
args += "-Dfoo=" + project.foo
}
classpath = sourceSets.test.runtimeClasspath
}
}
Running the test from command-line:
./gradlew -q spec -Pfoo=bar
Result:
$ ./gradlew -q spec -Pfoo=bar
...
Run starting. Expected test count is: 1
PassArgsToScala:
Arg passed
- must be bar
+ passing arg seem to work ==> foo=bar

I recently had to pass a feature flag to scalatest tests so I could test two implementations of the same API. I used an environment variable for this. It works for sbt & maven. The content of the variable will end up in the configMap. How to deal with the configMap you can check at other posts (or at the post I base this answer on).
You can pass the argument "ABC" as follows:
FEATURE_FLAG="ABC" sbt test
# or
FEATURE_FLAG="ABC" mvn test
sbt setup:
// build.sbt
val featureFlagArg = sys.env.get("FEATURE_FLAG")
.map(ff => s"-DfeatureFlag=$ff")
.map(Tests.Argument(TestFrameworks.ScalaTest, _))
Test / testOptions ++= featureFlagArg
maven setup (using the scalatest maven plugin):
<!-- pom.xml -->
<!-- ... -->
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<config>featureFlag=${env.FEATURE_FLAG}</config>
</configuration>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- ... -->

Related

Scala Mockito Guice and Partial Mocking .... methods get called twice

I have the following code written in Scala, Guice, Mockito and ScalaTest
import javax.inject.Singleton
import com.google.inject.Inject
#Singleton
class TestPartialMock #Inject()(t1: Test1, t2: Test2) {
def test3() = "I do test3"
def workHorse() : List[String] = {
println("+++++ came inside ++++++++")
List(t1.test1(), t2.test2(), test3())
}
}
class MainModule extends ScalaModule {
override def configure() = {
bind[Test1]
bind[Test2]
bind[TestPartialMock]
}
}
and I have written unit test cases with partial mocking
class PartialMockTest extends FunSpec with Matchers {
describe("we are testing workhorse but mock test3") {
it("should return mock for test3") {
val module = new TestModule
val injector = Guice.createInjector(module)
val tpm = injector.getInstance(classOf[TestPartialMock])
val result = tpm.workHorse()
result should contain ("i do test2")
result should contain ("i do test1")
result should contain ("I am mocked")
result should not contain ("I do test3")
}
}
}
class TestModule extends AbstractModule with ScalaModule with MockitoSugar {
override def configure() = {
val module = new MainModule()
val injector = Guice.createInjector(module)
val realobject = injector.getInstance(classOf[TestPartialMock])
val x = spy(realobject)
when(x.test3()).thenReturn("I am mocked")
when(x.workHorse()).thenCallRealMethod()
bind(classOf[TestPartialMock]).toInstance(x)
}
}
My tests are successful and I can see that it mocks the right set of methods and calls the actual implementation of the right set of methods. BUT when I look at the output I see
info] Compiling 5 Scala sources to /Users/IdeaProjects/GuicePartialMock/target/scala-2.12/classes...
[info] Compiling 1 Scala source to /Users/IdeaProjects/GuicePartialMock/target/scala-2.12/test-classes...
+++++ came inside ++++++++
+++++ came inside ++++++++
[info] PartialMockTest:
[info] we are testing workhorse but mock test3
[info] - should return mock for test3
[info] Run completed in 2 seconds, 92 milliseconds.
Why am I seeing the print statement came inside twice?
Edit::
Based on advice of Tavian ... this is the final code which worked
class TestModule extends AbstractModule with ScalaModule with MockitoSugar {
override def configure() = {
val module = new MainModule()
val injector = Guice.createInjector(module)
val realobject = injector.getInstance(classOf[TestPartialMock])
val x = spy(realobject)
when(x.test3()).thenReturn("I am mocked")
bind(classOf[TestPartialMock]).toInstance(x)
}
}
For spies, you need to be careful of expressions like
when(x.workHorse()).thenCallRealMethod()
because x.workHorse() really is invoked in this expression! x doesn't "know" that it's inside a when() call, as the expression is lowered into something like this:
tmp1 = x.workHorse();
tmp2 = when(tmp1);
tmp3 = tmp2.thenCallRealMethod();
Instead, you can write
doCallRealMethod().when(x).workHorse()
which will suppress the invocation of the real workHorse() implementation.
But, you don't need to do any of this for this example—calling real methods is the default behaviour of spies.
As you mentioned in the question title, you have a combination of 3 technologies. Actually 4 technologies including build tool that is used to run the test. You can isolate the problem by
1) Remove Guice and instantiate everything directly
2) Run code as a simple App instead of running as a test by sbt/gradle/maven.
Also it makes sense to print stack trace together with a came inside message to find a caller.

Scalatest with eclipse shows errors while using Matchers

I have an eclipse scala project which uses maven. Eclipse plugins for ScalaIDE and Scalatest are installed. I have tests like:
import org.scalatest._
class ExampleSpec extends FlatSpec with Matchers {
feature("Feature A Test") {
scenario("Foo scenario 1") {
val a = FooClass().getResult()
a.count shouldBe 1 // IDE shows error: value shouldBe is not a member of Long
a(0).getString(0) shouldBe "FOO" // IDE shows error: value shouldBe is not a member of String
}
}
}
The maven compilation and the tests run fine, but in eclipse when I open this file, I see an error in eclipse wherever I am using a Matcher as mentioned in the comments above. Eg.
value shouldBe is not a member of Long
What am I missing? A scala test file shows hundreds of problems.
After adding the following dummy code:
case class Bar() {
def count = Array(Bar())
def getString(x: Int) = Array("aqq")
def apply[T](x: Int) = this
}
case class FooClass() {
def getResult() = Bar()
}
and changing FlatSpec to FeatureSpec as this is the syntax you are using in your ExampleSpec, the code compiles without any issues.
If it's still not the case for you I can suggest creating simple build.sbt and generating project with Eclipse sbt plugin.
I know this is old, but I had the same issue with eclipse (late 2018), and I was able to fix this by making sure the test was NOT in the default package. That is, add "package org.scalatest.examples.flatspec" to the beginning of your test, as an example, and move the test into that package.

Setting to get SBT to fail fast (stop) on an error

I am running a multi-project SBT (v0.13) build and would like it to fail fast (stop) on the first error (compile) encountered in a sub-project.
The current behavior is that when something fails to compile in a sub-project, the build continues (to compile all the other sub-projects).
Is there a way/setting to make SBT stop and exit as soon as the first sub-project with a compile error is encountered?
In short, to my knowledge, no, SBT can not "fail fast" on a compiler or test error.
To my knowledge SBT doesn't control this. SBT is just going to invoke the appropriate test framework when it inspects your unit tests. The test framework can then decide what order to run the tests in, to run them concurrently, how to report issues, etc. Of course each test framework has its own features, configuration, and conventions. The two most widely used test frameworks for Scala are ScalaTest and Specs2.
Fortunately you can get the behavior you've requested in either Specs2 or ScalaTest. I've provided simple unit test examples below that fail early.
ScalaTest
You can get fail-fast behavior in ScalaTest for single test Suites by mixing in the CancelAfterFailure trait. For example, this test would execute the first test, fail the second, and show the third as cancelled.
class SomeSuite extends FunSuite with CancelAfterFailure with Assertions {
test("first") {
println("first")
}
test("second") {
assert(false)
}
test("third") {
println("third")
}
}
Specs2
Similar to ScalaTest you can control behavior in Specs2 on a per-Specification basis. To get fail-fast-like behavior you need to add two Arguments to your Specification: sequential and stopOnFail. See the docs for a full list of arguments you can set. (You do need both if you want an obvious linear ordering since Specs2 by default will execute your tests concurrently!)
class SomeSpec extends Specification {
sequential
stopOnFail
"specs2" should {
"first" in {
println("first")
ok
}
"second" in {
"foo" must equalTo ("bar")
}
"third" in {
println("third")
}
}
}
I've discovered that throwing a new java.lang.VirtualMachineError() {} halts the task immediately, while all other kinds of exceptions I have tried so far were swallowed.
I've tried ThreadDeath, InterruptedException, LinkageError, and ControlThrowable, and of course, the usual RuntimeException and such (the list came from scala.util.control.NonFatal)
<project root>/project/BuildUtils.scala
import sbt._
import sbt.Def._
object BuildUtils {
private def details(inc: Incomplete): Seq[Throwable] =
inc.directCause.toSeq ++ inc.causes.flatMap(details)
implicit class TaskSyntax[A](private val task: Initialize[Task[A]]) extends AnyVal {
def haltWhenFailed: Initialize[Task[A]] = task.result.map {
case Inc(cause) =>
throw new VirtualMachineError({
s"""Task has failed during execution.
|TaskNode: ${cause.node}
|DirectCause: ${details(cause).map(_.getMessage).distinct}
|""".stripMargin
}) {}
case Value(value) => value
}
}
}
<project root>/build.sbt
import BuildUtils._
lazy val ciBuild = taskKey[Unit]("Compile and run test")
ciBuild := {
val t1 # _ = (Test / compile).haltWhenFailed.value
val t2 # _ = (Test / test).haltWhenFailed.value
}
ciBuild will halt at first compilation error with rather verbose stacktrace and the specified message.

Setting tagsToExclude in SBT

ScalaTest allows the setting of tags to be excluded via a filter called tagsToExclude.
How can I configure my SBT build to set this value?
Attempt 1
The CLI of ScalaTest specifies the -l flag for tags to exclude.
SBT allows the setting of CLI params like so:
testOptions in Test += Tests.Argument(
TestFrameworks.ScalaTest, "-l", "DataFileTest")`
But this seems to have no effect (i.e. the test still executes).
For reference the test looks like:
object DataFileTest extends org.scalatest.Tag("com.mydomain.DataFileTest")
class MyDataFileDependantSpec extends FunSpec
with Matchers
with BeforeAndAfter
with BeforeAndAfterAll {
describe("Something") {
it("reads a file and does things", DataFileTest) {
...
}
}
testOptions in Test ++= Seq(Tests.Argument(TestFrameworks.ScalaTest,
"-l", "org.scalatest.tags.Slow")
This works.
See if the problem is to do with the full path name of DataFileTest.

Excluding a ScalaTest test when calling my tests from within sbt

I want to write a test that calls a remote server and validates the response because the server may change (it's not under my control). To do this I figure I'd give it a tag (RemoteTest) and then exclude it when calling the runner:
sbt> test-only * -- -l RemoteTest
However, when doing this all my tests are run, including RemoteTest. How do I call the runner from within sbt so that it is excluded?
If you have the following:-
package com.test
import org.scalatest.FlatSpec
import org.scalatest.Tag
object SlowTest extends Tag("com.mycompany.tags.SlowTest")
object DbTest extends Tag("com.mycompany.tags.DbTest")
class TestSuite extends FlatSpec {
"The Scala language" must "add correctly" taggedAs(SlowTest) in {
val sum = 1 + 1
assert(sum === 2)
}
it must "subtract correctly" taggedAs(SlowTest, DbTest) in {
val diff = 4 - 1
assert(diff === 3)
}
}
To exclude DbTest tag, you would do:-
test-only * -- -l com.mycompany.tags.DbTest
Note that you'll need to include the full tag name. If it's still not working for you, would you mind sharing part of source code that's not working?