Hi everybody I have these suites:
#DoNotDiscover
class Suite1 extends FunSuite with Suite{
test("Test1"){
println("Running test1")
println("Using data inserted by MasterDataSuite")
}
}
#DoNotDiscover
class Suite2 extends FunSuite with Suite{
test("Test2"){
println("Running test2")
println("Using data inserted by MasterDataSuite")
}
}
class MasterDataSuite extends Suites(new Suite1,new Suite2) with BeforeAndAfterAllConfigMap {
override def beforeAll(configMap: ConfigMap) {
//inserting in DB
}
override def afterAll(configMap: ConfigMap) {
//deleting in DB
}
}
If I run by command line, the data necessary in MasterData class is inserted, all good so far.
My big problem is if I want to run a single test in Suite1 or Suite2 the test will always fail because it can not find the data necessary to run successfully (this data should be inserted by beforeAll method). I am not sure if I need to set up something in Intellij, or I need to set up something from code or If it is possible.
Quick notes:
Why use that instead of before and after per Suite?: A: Because There are hundreds of tests that already exist in the project. Then is so long check test by test to guarantee correct data per Suite.
That works in the pipeline and by command line.
Thanks in advance
Related
I have a suite of scalatest tests that test different endpoints of a RESTful API.
I really want them separated into different files for best organization.
My problem is how to start something (an HTTP server in my case, but it doesn't matter what it is) before all the tests and shut it down after all the tests are done.
I know about BeforeAndAfterAll, but that only accomplishes before/after inside one test file. I need something like that, but for all tests, for example:
-- start http server before tests
-- run all test suites
-- shut down http server
The intended way to do this is to use nested suites. Suite has a nestedSuites method that returns an IndexedSeq[Suite] (in 2.0, in 1.9.1 it was a List[Suite]). Suite also has a runNestedSuites method that is responsible for executing any nested suites. By default runNestedSuites calls nestedSuites, and on each returned Suite either invokes run directly, or if a Distributor is passed, puts the nested suites in the distributor so that they can be run in parallel.
So what you really probably want to do is make Foo and Bar into classes, and return instances of them from the nestedSuites method of EndpointTests. There's a class that makes that easy called Suites. Here's an example of its use:
import org.scalatest._
import matchers.MustMatchers
class Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
class Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
class EndpointTests extends Suites(new Foo, new Bar) with BeforeAndAfterAll {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
One potential problem, though, is that if you are using discovery to find Suites to run, all three of EndpointTests, Foo, and Bar will be discovered. In ScalaTest 2.0 you can annotate Foo and Bar with #DoNotDiscover, and ScalaTest's Runner will not discover them. But sbt still will. We are currently enhancing sbt so that it passes over otherwise discoverable Suites that are annotated with DoNotDiscover, but this will be in sbt 0.13, which isn't out yet. In the meantime you can get sbt to ignore them by adding an unused constructor parameter to Foo and Bar.
Ok, found a way. It seems (unless someone here can correct me) that Scalatest does not have the facility of a "master" suite. But... you can kinda build one.
You can compose a suite from traits. So using my endpoint example:
class EndpointTests extends FunSpec with MustMatchers with BeforeAndAfterAll
with Foo with Bar {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
Ok, but what about the tests? Notice the with Foo with Bar. I'm bringing the dependent tests in as traits.
See here:
trait Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
trait Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
Alternatively you can just use an object.
object TestServer {
startServer()
}
When you access the object it will be initialised, starting the server.
Just create a common trait in the body of which you access the object.
Then mixin that trait into all your tests. Done.
If your server runs in daemon mode (e.g. a Play! application in test mode) it will be automatically shut down after all tests are run.
I want to know in what order the tests are running, because I want to run
my initial test setup tests only once before all the tests start running.
If I have one initial setup test class, and one essential test class it would be fine:
class EssentialTesting #Inject()(setupTests: SetupTest) extends ....{
setupTests.runInitialSetup()
.....
}
However if I have number of testings classes the setupTests.runInitialSetup will be repeated in each class. How to deal with this duplication?
Make your Setups class like:
class SetUpTest extends SpecificationLike with BeforeAndAfterAll{
override def beforeAll() = {
// runs before each test
}
override val afterAll() = {
// runs after each test
}
}
Now in your test file, you can extend your SetUpTest, like :
class Test extends PlaySpecification with SetupTest
Now you don't need to call the runinitialsetup every time, just put it into beforeAll method and it will get executed before each test. And SpecificationLike is required as BeforeAndAfterAll has self type reference for SpecificationLike.
I have more of a philosophical confusion in-regards to the usefulness of methods like 'beforeAll' in scalaTest.
I have been looking for an answer why the need to even have constructs like beforeAll? I do understand that there is a reason why this design decision was taken but not able to think it through. Can anyone help?
e.g.
Suggested way as per tutorials online,
class TestExample extends FunSuite with BeforeAndAfterAll {
private var _tempDir: File = _
protected def tempDir: File = _tempDir
override def beforeAll(): Unit = {
super.beforeAll()
_tempDir = Utils.createTempDir(namePrefix = this.getClass.getName)
}
test("...") {
// using the variable in the function
}
}
vs
class TestExample extends FunSuite with BeforeAndAfterAll {
private val tempDir: File = Utils.createTempDir(namePrefix =
this.getClass.getName)
}
test("...") {
// Use the initialized variable here.
}
If you have cleanup to do in afterAll, I think it is symmetric to do setup in beforeAll. Also if you need to do some side effect that doesn't involve initializing instance variables, that can go in beforeAll. In the example you gave, though, where you don't have any cleanup to do in afterAll and all you're doing before all tests is initializing instance variables, I'd do with plain old initialization.
One other difference between val initializers and beforeAll is val initializers happen when the class is instantiated, whereas beforeAll happens later, when the instance is executed. If you want to delay the initialization until the class is run, you can use lazy vals.
One point worth noting is that some runners (such as the ScalaTest ant task, and Intellij IDEA), will instantiate all tests instances before running any tests. If your setup code happens to interact with any global variables or external state, then you probably want to defer those interactions until the test is run.
As a simple (contrived) example, suppose your code under test includes
Object Singleton {
var foo = ""
}
and you have two test classes:
class Test1 extends FunSuite {
Singleton.foo = "test1"
test("...") {
Singleton.foo should be("test1")
}
}
class Test1 extends FunSuite {
Singleton.foo = "test2"
test("...") {
Singleton.foo should be("test2")
}
}
If both classes are instantiated before any tests are run, then at least one of your two tests will fail. Conversely, if you defer your initialize work until beforeAll, you'll not see the same interference between tests.
for reference: How make tests always run in same order in Scalatest?
I plan to test my application by calling controllers/routes and comparing the responses to my expected ones.
I do not want to mock my persistence layer, so I can test it too. My approach would now be to execute tests in order to reflect user actions. Example:
Test 1: User registers
--> Test 2: (depends on a existing user) User creates profile
--> Test 3: (depends on a user with existing profile) User changes profile
So to save time, I do not want to mock anything for Test 2 and Test 3 but instead just work on the same database all the time and use the data generated by preceding tests.
Is this approach ok and how would one specify the execution order in Specs2 or ScalaTest?
Having no dependencies between individual test suites is preferred for at least two reasons:
Being concerned with the order in which the suites are executed makes the test execution harder to understand
If suite A depends on suite B, changing something in suite B may break suite A, meaning it is harder to find the cause of a failing test.
Because of these drawbacks I would recommend you to properly setup your persistence layer at the beginning of each acceptance test; at the expense of execution time. Note that you can tag your tests and only execute your slow acceptance tests occasionally to not slow down your development cycles.
If you want to implement dependent tests in ScalaTest nonetheless, you can create a nested test suite as is suggested in the question you linked:
Assuming your persistence layer:
object Users {
var users: List[User] = Nil
def apply(i: Int): User = users(i)
def register(user: User): Unit = users = user :: users
def isEmpty: Boolean = users.isEmpty
}
class User(var profile: Option[Profile] = None) {
def createProfile(): Unit = profile = Some(new Profile)
}
class Profile(var content: String = "") {
def update(newContent: String): Unit = content = newContent
}
and your individual tests:
#DoNotDiscover
class Test1 extends FlatSpec with ShouldMatchers {
"register" should "store a new user" in {
Users.register(new User)
Users should not be 'empty
}
}
#DoNotDiscover
class Test2 extends FlatSpec with ShouldMatchers {
"createProfile" should "create a new user profile" in {
val user = Users(0)
user.createProfile()
user.profile shouldBe 'defined
}
}
#DoNotDiscover
class Test3 extends FlatSpec with ShouldMatchers {
"update" should "update the content of the profile" in {
val newContent = "Test"
val profile = Users(0).profile.get
profile.update(newContent)
profile.content shouldBe newContent
}
}
you can nest them in an acceptance test suite:
class AcceptanceTests extends Suites(
new Test1,
new Test2,
new Test3
) with SequentialNestedSuiteExecution
The #DoNotDiscover annotation is necessary to prevent the test runner from executing the nested tests separatly (as they are itself test suites). Mixing in the trait SequentialNestedSuiteExecution guarantees that the nested tests are executed in the given order.
I have a suite of scalatest tests that test different endpoints of a RESTful API.
I really want them separated into different files for best organization.
My problem is how to start something (an HTTP server in my case, but it doesn't matter what it is) before all the tests and shut it down after all the tests are done.
I know about BeforeAndAfterAll, but that only accomplishes before/after inside one test file. I need something like that, but for all tests, for example:
-- start http server before tests
-- run all test suites
-- shut down http server
The intended way to do this is to use nested suites. Suite has a nestedSuites method that returns an IndexedSeq[Suite] (in 2.0, in 1.9.1 it was a List[Suite]). Suite also has a runNestedSuites method that is responsible for executing any nested suites. By default runNestedSuites calls nestedSuites, and on each returned Suite either invokes run directly, or if a Distributor is passed, puts the nested suites in the distributor so that they can be run in parallel.
So what you really probably want to do is make Foo and Bar into classes, and return instances of them from the nestedSuites method of EndpointTests. There's a class that makes that easy called Suites. Here's an example of its use:
import org.scalatest._
import matchers.MustMatchers
class Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
class Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
class EndpointTests extends Suites(new Foo, new Bar) with BeforeAndAfterAll {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
One potential problem, though, is that if you are using discovery to find Suites to run, all three of EndpointTests, Foo, and Bar will be discovered. In ScalaTest 2.0 you can annotate Foo and Bar with #DoNotDiscover, and ScalaTest's Runner will not discover them. But sbt still will. We are currently enhancing sbt so that it passes over otherwise discoverable Suites that are annotated with DoNotDiscover, but this will be in sbt 0.13, which isn't out yet. In the meantime you can get sbt to ignore them by adding an unused constructor parameter to Foo and Bar.
Ok, found a way. It seems (unless someone here can correct me) that Scalatest does not have the facility of a "master" suite. But... you can kinda build one.
You can compose a suite from traits. So using my endpoint example:
class EndpointTests extends FunSpec with MustMatchers with BeforeAndAfterAll
with Foo with Bar {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
Ok, but what about the tests? Notice the with Foo with Bar. I'm bringing the dependent tests in as traits.
See here:
trait Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
trait Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
Alternatively you can just use an object.
object TestServer {
startServer()
}
When you access the object it will be initialised, starting the server.
Just create a common trait in the body of which you access the object.
Then mixin that trait into all your tests. Done.
If your server runs in daemon mode (e.g. a Play! application in test mode) it will be automatically shut down after all tests are run.