I am using Scala 2.10 with ScalaMock 3.6.
I have a quite simple test case with 4 test scenarios. I have created a mock object for those tests to use (imitating file system):
class ProcessingOperatorTest extends FlatSpec with Matchers with BeforeAndAfterEach with MockFactory {
...
val fakeFS = mock[FileIO]
(fakeFS.createFile _).expects(*).returns(true).anyNumberOfTimes()
(fakeFS.exist _).expects(where { (p: String) => p.contains(existing) }).returns(true).anyNumberOfTimes()
(fakeFS.exist _).expects(where { (p: String) => p.contains(notExisting) }).returns(false).anyNumberOfTimes()
behavior of "Something"
it should "test 1" in {
...
}
it should "test 2" in {
...
}
it should "test 3" in {
...
}
it should "test 4" in {
...
}
Now:
1st test does not use any of mocked methods (but needs the mock object)
2nd test uses only existing mock method
3rd test uses both existing and not existing mock methods
4th test uses all methods, (also createFile)
Now, for some reason, when running all those tests together, 4th test fails giving me the following error. If running separately, it will pass.
Unexpected call: <mock-1> FileIO.exist(notExisting)
Expected:
inAnyOrder {
}
Actual:
<mock-1> FileIO.exist(notExisting)
ScalaTestFailureLocation: scala.Option at (Option.scala:120)
org.scalatest.exceptions.TestFailedException: Unexpected call: <mock-1> FileIO.exist(notExisting)
...
The other walkaround is to copy-paste the mock declaration and its behaviour inside 4th it should { ... } test scenario. Tests work then (separately, and all together).
Why is global mock instance failing?
I can try to prepare a similar test scenario as separate sbt project if needed.
Mix in org.scalatest.OneInstancePerTest as described here:
class ProcessingOperatorTest extends FlatSpec
with Matchers
with BeforeAndAfterEach
with MockFactory
with OneInstancePerTest {
...
}
Related
I have a suite of scalatest tests that test different endpoints of a RESTful API.
I really want them separated into different files for best organization.
My problem is how to start something (an HTTP server in my case, but it doesn't matter what it is) before all the tests and shut it down after all the tests are done.
I know about BeforeAndAfterAll, but that only accomplishes before/after inside one test file. I need something like that, but for all tests, for example:
-- start http server before tests
-- run all test suites
-- shut down http server
The intended way to do this is to use nested suites. Suite has a nestedSuites method that returns an IndexedSeq[Suite] (in 2.0, in 1.9.1 it was a List[Suite]). Suite also has a runNestedSuites method that is responsible for executing any nested suites. By default runNestedSuites calls nestedSuites, and on each returned Suite either invokes run directly, or if a Distributor is passed, puts the nested suites in the distributor so that they can be run in parallel.
So what you really probably want to do is make Foo and Bar into classes, and return instances of them from the nestedSuites method of EndpointTests. There's a class that makes that easy called Suites. Here's an example of its use:
import org.scalatest._
import matchers.MustMatchers
class Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
class Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
class EndpointTests extends Suites(new Foo, new Bar) with BeforeAndAfterAll {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
One potential problem, though, is that if you are using discovery to find Suites to run, all three of EndpointTests, Foo, and Bar will be discovered. In ScalaTest 2.0 you can annotate Foo and Bar with #DoNotDiscover, and ScalaTest's Runner will not discover them. But sbt still will. We are currently enhancing sbt so that it passes over otherwise discoverable Suites that are annotated with DoNotDiscover, but this will be in sbt 0.13, which isn't out yet. In the meantime you can get sbt to ignore them by adding an unused constructor parameter to Foo and Bar.
Ok, found a way. It seems (unless someone here can correct me) that Scalatest does not have the facility of a "master" suite. But... you can kinda build one.
You can compose a suite from traits. So using my endpoint example:
class EndpointTests extends FunSpec with MustMatchers with BeforeAndAfterAll
with Foo with Bar {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
Ok, but what about the tests? Notice the with Foo with Bar. I'm bringing the dependent tests in as traits.
See here:
trait Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
trait Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
Alternatively you can just use an object.
object TestServer {
startServer()
}
When you access the object it will be initialised, starting the server.
Just create a common trait in the body of which you access the object.
Then mixin that trait into all your tests. Done.
If your server runs in daemon mode (e.g. a Play! application in test mode) it will be automatically shut down after all tests are run.
I want to set up a test in Scala that creates a mock config to provide certain values. I'm using ScalaTest 3.0.1, ScalaMock 3.4.2,
and typesafe 1.3.1. The goal is to mock values for the config before running tests. The docs at
http://www.scalatest.org/user_guide/testing_with_mock_objects and http://scalamock.org/user-guide/features/ seem to offer
a couple options. First, here's some target code:
import com.typesafe.config.Config
class Sample(config: Config) {
private val aValue = config.getDouble("aValue")
}
It seems like it should be possible to set everything up one time, or to set up everything before each test. This attempt fails:
class SampleSpec extends FlatSpec with MockFactory with BeforeAndAfterAll {
private val mockConfig = mock[Config]
override def beforeAll {
(mockConfig.getDouble _).expects("aValue").returning(1.0).anyNumberOfTimes()
}
"testOne" should "return 1" in {
new Sample(mockConfig)
}
"testTwo" should "return 1" in {
new Sample(mockConfig)
}
}
The first test succeeds but second test in the fixture fails, and this error is produced:
Unexpected call: <mock-1> Config.getDouble(aValue)
Expected:
inAnyOrder {
}
Actual:
<mock-1> Config.getDouble(aValue)
ScalaTestFailureLocation: scala.Option at (Option.scala:120)
org.scalatest.exceptions.TestFailedException: Unexpected call: <mock-1> Config.getDouble(aValue)
Expected:
inAnyOrder {
}
Here's an alternative approach:
class SampleSpec extends FlatSpec with MockFactory with BeforeAndAfter {
private val mockConfig = mock[Config]
before {
(mockConfig.getDouble _).expects("aValue").returning(1.0)
}
"testOne" should "return 1" in {
new Sample(mockConfig)
}
"testTwo" should "return 1" in {
new Sample(mockConfig)
}
}
It produces this exception:
An exception or error caused a run to abort: assertion failed: Null expectation context - missing withExpectations?
java.lang.AssertionError: assertion failed: Null expectation context - missing withExpectations?
Why does the first attempt fail? The test specifies that getDouble can be invoked any number of times, yet the second
test fails as if anyNumberOfTimes() wasn't used. How should this be coded so that the method can be mocked once and
invoked repeatedly? Why does the second attempt fail? Is there a method to reset the mock so it can be reused?
I'd also like to point out the documentation page for this, with a slightly different style (using a trait):
http://scalamock.org/user-guide/sharing-scalatest/#fixture-contexts
For example:
class SampleSpec extends FlatSpec with OneInstancePerTest with MockFactory {
private val mockConfig = mock[Config]
(mockConfig.getDouble _).expects("aValue").returning(1.0).anyNumberOfTimes
"testOne" should "return 1" in {
new Sample(mockConfig)
}
"testTwo" should "return 1" in {
new Sample(mockConfig)
}
}
Recreate the mock every single time, manually is the only way I can get it to work:
class SampleSpec extends FlatSpec with MockFactory {
private def mockConfig = {
val mocked = mock[Config]
(mocked.getDouble _).expects("aValue").returning(1.0).anyNumberOfTimes()
mocked
}
"testOne" should "return 1" in {
new Sample(mockConfig)
}
"testTwo" should "return 1" in {
new Sample(mockConfig)
}
}
which is easy because your tests don't change at all. You're simply moving the logic from the "globally local" variable and into the local scope of an individual test.
I'm trying to run tests by sharing behaviour in traits. When I run such a test, the test always aborts early when there's a failure, not running the rest of the tests.
Here's my test:
trait TestBehaviour extends ShouldMatchers {
def failEarly = fail("this fails before the test is run")
def failEarlyAgain = fail("this also fails before the test is run")
}
class Test extends FlatSpec with TestBehaviour {
it must behave like failEarly
it must behave like failEarlyAgain
}
If I run this in the ScalaTest IDE for Eclipse, no tests are ever shown - the RUN ABORTED text is just shown in the console:
*** RUN ABORTED ***
An exception or error caused a run to abort: this fails before the test is run
So it looks like the first test is being run early.
What I would expect to see is both the tests are run, and fail (with different reason messages).
change
def failEarly = fail("this fails before the test is run")
def failEarlyAgain = fail("this also fails before the test is run")
to
def failEarly() = fail("this fails before the test is run")
def failEarlyAgain() = fail("this also fails before the test is run")
In your version you are invoking the methods defined (because they don't have a parameter list). In my version you are passing a function created from the method (without invoking it)
It seems that the templated test, should contain an actual test-case; like so
import org.scalatest.FlatSpec
trait TestTemplate { this: FlatSpec =>
def failEarly() = {
it should "fail at the right moment" in {
fail("this fails once for each test that should behave like me")
}
}
}
class MyTest extends FlatSpec with TestTemplate {
"This Test" should behave like failEarly()
"Some other test" should behave like failEarly()
}
The example above seems to work for me.
I have a suite of scalatest tests that test different endpoints of a RESTful API.
I really want them separated into different files for best organization.
My problem is how to start something (an HTTP server in my case, but it doesn't matter what it is) before all the tests and shut it down after all the tests are done.
I know about BeforeAndAfterAll, but that only accomplishes before/after inside one test file. I need something like that, but for all tests, for example:
-- start http server before tests
-- run all test suites
-- shut down http server
The intended way to do this is to use nested suites. Suite has a nestedSuites method that returns an IndexedSeq[Suite] (in 2.0, in 1.9.1 it was a List[Suite]). Suite also has a runNestedSuites method that is responsible for executing any nested suites. By default runNestedSuites calls nestedSuites, and on each returned Suite either invokes run directly, or if a Distributor is passed, puts the nested suites in the distributor so that they can be run in parallel.
So what you really probably want to do is make Foo and Bar into classes, and return instances of them from the nestedSuites method of EndpointTests. There's a class that makes that easy called Suites. Here's an example of its use:
import org.scalatest._
import matchers.MustMatchers
class Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
class Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
class EndpointTests extends Suites(new Foo, new Bar) with BeforeAndAfterAll {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
One potential problem, though, is that if you are using discovery to find Suites to run, all three of EndpointTests, Foo, and Bar will be discovered. In ScalaTest 2.0 you can annotate Foo and Bar with #DoNotDiscover, and ScalaTest's Runner will not discover them. But sbt still will. We are currently enhancing sbt so that it passes over otherwise discoverable Suites that are annotated with DoNotDiscover, but this will be in sbt 0.13, which isn't out yet. In the meantime you can get sbt to ignore them by adding an unused constructor parameter to Foo and Bar.
Ok, found a way. It seems (unless someone here can correct me) that Scalatest does not have the facility of a "master" suite. But... you can kinda build one.
You can compose a suite from traits. So using my endpoint example:
class EndpointTests extends FunSpec with MustMatchers with BeforeAndAfterAll
with Foo with Bar {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
Ok, but what about the tests? Notice the with Foo with Bar. I'm bringing the dependent tests in as traits.
See here:
trait Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
trait Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
Alternatively you can just use an object.
object TestServer {
startServer()
}
When you access the object it will be initialised, starting the server.
Just create a common trait in the body of which you access the object.
Then mixin that trait into all your tests. Done.
If your server runs in daemon mode (e.g. a Play! application in test mode) it will be automatically shut down after all tests are run.
I've inherited some JUnit tests written in scala that need to be fixed to use #BeforeClass semantics. I understand that the #BeforeClass annotation must be applied to static methods only. I understand that methods defined in "companion" objects (as opposed to scala classes) are static. How can I get a test method to be called once prior to the individual instance methods in a test class?
object TestClass {
#BeforeClass
def stuff() {
// beforeclass stuff
}
}
class TestClass {
#Test
...
}
seems to work...
I had to move to specs2 to implement this feature with Scala. Just adding an example to help people with the same problem as the orginal poster who don't yet know specs2.
The specs2 way uses the concept of a "step" to accomplish test suite set-up and tear-down. If you run with a JUnitRunner all your Ant scripts and IDEs that use JUnit will still know how to run it. Here's an example using a mutable specification from specs2:
import org.specs2.mutable.Specification
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner
#RunWith(classOf[JUnitRunner])
class MutableSpecs2ExampleTest extends Specification {
var firstStep: String = null
var secondStep: String = null
var thirdStep: String = null
//Steps are guaranteed to run serially
step{println("Loading Spring application context...");firstStep="Hello World"}
step{println("Setting up mocks...");secondStep = "Hello Scala"}
//The fragments should be run in parallel by specs2
"Some component Foo in my project" should{
" pass these tests" in {
println("Excersizing some code in Foo")
firstStep must startWith("Hello") and endWith("World")
}
" pass theses other tests" in {
println("Excersizing some other code in Foo")
firstStep must have size(11)
}
}
"Some component Bar in my project" should{
" give the correct answer" in {
println("Bar is thinking...")
secondStep must startWith("Hello") and endWith("Scala")
thirdStep must be equalTo null
}
}
step{println("Tearing down resources after tests...");thirdStep = "Hello Specs2"}
}
And here's an example with a non-mutable specification:
import org.specs2.Specification
import org.specs2.specification.Step
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner
#RunWith(classOf[JUnitRunner])
class Specs2ExampleTest extends Specification{
var firstStep: String = null
var secondStep: String = null
var thirdStep: String = null
def is =
"This is a test with some set-up and tear-down examples" ^
p^
"Initialize" ^
Step(initializeDependencies())^
Step(createTestData())^
"Component Foo should" ^
"perform some calculation X " !testX^
"perform some calculation Y" !testY^
p^
"Tidy up" ^
Step(removeTestData())^
end
def testX = {
println("testing Foo.X")
firstStep must be equalTo("Hello World")
}
def testY = {
println("testing Foo.Y")
secondStep must be equalTo("Hello Scala")
thirdStep must be equalTo null
}
def initializeDependencies(){
println("Initializing Spring applicaiton context...")
firstStep = "Hello World"
}
def createTestData(){
println("Inserting test data into the db...")
secondStep = "Hello Scala"
}
def removeTestData(){
println("Removing test data from the db...")
println("Tearing down resources...")
thirdStep = "Hello Specs2"
}
}
You don't specify if you mean inherited in the OO-programming sense or the "taken over from someone else" sense of the word.
In the latter case, I'd advise you to re-write the thing using either ScalaTest or Specs2 and expose it as a JUnit test (both frameworks support this) so that it'll integrate with whatever other tools and processes you already have in place.