I have a suite of scalatest tests that test different endpoints of a RESTful API.
I really want them separated into different files for best organization.
My problem is how to start something (an HTTP server in my case, but it doesn't matter what it is) before all the tests and shut it down after all the tests are done.
I know about BeforeAndAfterAll, but that only accomplishes before/after inside one test file. I need something like that, but for all tests, for example:
-- start http server before tests
-- run all test suites
-- shut down http server
The intended way to do this is to use nested suites. Suite has a nestedSuites method that returns an IndexedSeq[Suite] (in 2.0, in 1.9.1 it was a List[Suite]). Suite also has a runNestedSuites method that is responsible for executing any nested suites. By default runNestedSuites calls nestedSuites, and on each returned Suite either invokes run directly, or if a Distributor is passed, puts the nested suites in the distributor so that they can be run in parallel.
So what you really probably want to do is make Foo and Bar into classes, and return instances of them from the nestedSuites method of EndpointTests. There's a class that makes that easy called Suites. Here's an example of its use:
import org.scalatest._
import matchers.MustMatchers
class Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
class Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
class EndpointTests extends Suites(new Foo, new Bar) with BeforeAndAfterAll {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
One potential problem, though, is that if you are using discovery to find Suites to run, all three of EndpointTests, Foo, and Bar will be discovered. In ScalaTest 2.0 you can annotate Foo and Bar with #DoNotDiscover, and ScalaTest's Runner will not discover them. But sbt still will. We are currently enhancing sbt so that it passes over otherwise discoverable Suites that are annotated with DoNotDiscover, but this will be in sbt 0.13, which isn't out yet. In the meantime you can get sbt to ignore them by adding an unused constructor parameter to Foo and Bar.
Ok, found a way. It seems (unless someone here can correct me) that Scalatest does not have the facility of a "master" suite. But... you can kinda build one.
You can compose a suite from traits. So using my endpoint example:
class EndpointTests extends FunSpec with MustMatchers with BeforeAndAfterAll
with Foo with Bar {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
Ok, but what about the tests? Notice the with Foo with Bar. I'm bringing the dependent tests in as traits.
See here:
trait Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
trait Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
Alternatively you can just use an object.
object TestServer {
startServer()
}
When you access the object it will be initialised, starting the server.
Just create a common trait in the body of which you access the object.
Then mixin that trait into all your tests. Done.
If your server runs in daemon mode (e.g. a Play! application in test mode) it will be automatically shut down after all tests are run.
Related
I have a suite of scalatest tests that test different endpoints of a RESTful API.
I really want them separated into different files for best organization.
My problem is how to start something (an HTTP server in my case, but it doesn't matter what it is) before all the tests and shut it down after all the tests are done.
I know about BeforeAndAfterAll, but that only accomplishes before/after inside one test file. I need something like that, but for all tests, for example:
-- start http server before tests
-- run all test suites
-- shut down http server
The intended way to do this is to use nested suites. Suite has a nestedSuites method that returns an IndexedSeq[Suite] (in 2.0, in 1.9.1 it was a List[Suite]). Suite also has a runNestedSuites method that is responsible for executing any nested suites. By default runNestedSuites calls nestedSuites, and on each returned Suite either invokes run directly, or if a Distributor is passed, puts the nested suites in the distributor so that they can be run in parallel.
So what you really probably want to do is make Foo and Bar into classes, and return instances of them from the nestedSuites method of EndpointTests. There's a class that makes that easy called Suites. Here's an example of its use:
import org.scalatest._
import matchers.MustMatchers
class Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
class Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
class EndpointTests extends Suites(new Foo, new Bar) with BeforeAndAfterAll {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
One potential problem, though, is that if you are using discovery to find Suites to run, all three of EndpointTests, Foo, and Bar will be discovered. In ScalaTest 2.0 you can annotate Foo and Bar with #DoNotDiscover, and ScalaTest's Runner will not discover them. But sbt still will. We are currently enhancing sbt so that it passes over otherwise discoverable Suites that are annotated with DoNotDiscover, but this will be in sbt 0.13, which isn't out yet. In the meantime you can get sbt to ignore them by adding an unused constructor parameter to Foo and Bar.
Ok, found a way. It seems (unless someone here can correct me) that Scalatest does not have the facility of a "master" suite. But... you can kinda build one.
You can compose a suite from traits. So using my endpoint example:
class EndpointTests extends FunSpec with MustMatchers with BeforeAndAfterAll
with Foo with Bar {
override def beforeAll(configMap: Map[String, Any]) {
println("Before!") // start up your web server or whatever
}
override def afterAll(configMap: Map[String, Any]) {
println("After!") // shut down the web server
}
}
Ok, but what about the tests? Notice the with Foo with Bar. I'm bringing the dependent tests in as traits.
See here:
trait Foo extends FunSpec with MustMatchers {
describe("Message here...") {
it("Must do something") { }
it("Must be ok") { }
}
}
trait Bar extends FunSpec with MustMatchers {
describe("Hello you...") {
it("One more!") { }
}
}
Alternatively you can just use an object.
object TestServer {
startServer()
}
When you access the object it will be initialised, starting the server.
Just create a common trait in the body of which you access the object.
Then mixin that trait into all your tests. Done.
If your server runs in daemon mode (e.g. a Play! application in test mode) it will be automatically shut down after all tests are run.
I'm writing tests for my Play application and I want to run them with a real server so that I can fake all the answers from the external services.
In order to do that I extend PlaySpec and GuiceOneServerPerSuite and I override the method fakeApplication to create my routes and give them to the Guice Application
class MySpec extends PlaySpec with GuiceOneServerPerSuite {
override def fakeApplication(): Application =
GuiceApplicationBuilder().appRoutes(app => {
case ("POST", "/url/") => app.injector.instanceOf(classOf[DefaultActionBuilder]) { Ok }
}).globalApp(true).build()
"Something" should {
"work well" in {
val wsClient = app.injector.instanceOf[WSClient]
val service = new MyService(wsClient)
service.method() mustBe ""
app.injector.instanceOf[DBApi].databases().foreach(_.getConnection().close())
}
}
}
I have multiple test suites like this one and if I run them alone they work fine, but if I run them all together they fill up the connection pool and then everything fails with: org.postgresql.util.PSQLException: FATAL: sorry, too many clients already.
My considerations: I think it happens because at each test suite a new Play Guice Application is created. I also tried to close the connections of all databases manually but didn't solve the problem.
We had the same problems, so we are separating these 2 use cases (running all or just one Test-Suite).
This makes running all tests much faster - as Play Environment is only started once.
The Suite looks like:
class AcceptanceSpecSuite
extends PlaySpec
with GuiceOneAppPerSuite
with BeforeAndAfter {
// all specs
override def nestedSuites: immutable.IndexedSeq[AcceptanceSpec] = Vector(
// api
new DatabaseTaskSpec,
new HistoryPurgeTaskSpec,
...
)
override def fakeApplication(): Application =
// your initialization
}
Now each Spec looks like:
#DoNotDiscover // important that it is run only if called explicitly
class DatabaseTaskSpec extends AcceptanceSpec {
...
The Parent class now we can switch between GuiceOneServerPerSuite and ConfiguredApp:
trait AcceptanceSpec
extends PlaySpec
you need:
// with GuiceOneServerPerSuite // if you want to test only one Test
with ConfiguredApp // if you want to test all
with Logging
with ScalaFutures
with BeforeAndAfter {
...
I know it's a bit of a hack - so I am also interested in a more elegant solution;).
You can put your DB instance as a singleton, if you do that, he won´t create multiple instance, therefore won´t fill the connection pool.
Something like that:
#Singleton
object TestDBProperties extends DBProperties {
override val db: Database = Database.forURL(
url = "jdbc:h2:mem:testdb;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE;",
driver = "org.h2.Driver")
}
Hope this helps.
I want to know in what order the tests are running, because I want to run
my initial test setup tests only once before all the tests start running.
If I have one initial setup test class, and one essential test class it would be fine:
class EssentialTesting #Inject()(setupTests: SetupTest) extends ....{
setupTests.runInitialSetup()
.....
}
However if I have number of testings classes the setupTests.runInitialSetup will be repeated in each class. How to deal with this duplication?
Make your Setups class like:
class SetUpTest extends SpecificationLike with BeforeAndAfterAll{
override def beforeAll() = {
// runs before each test
}
override val afterAll() = {
// runs after each test
}
}
Now in your test file, you can extend your SetUpTest, like :
class Test extends PlaySpecification with SetupTest
Now you don't need to call the runinitialsetup every time, just put it into beforeAll method and it will get executed before each test. And SpecificationLike is required as BeforeAndAfterAll has self type reference for SpecificationLike.
I'm having trouble writing tests for a mixin to my Play application that runs in it's own thread separate from play. I've tried over-writing WithApplication.provideApplication method with no luck. I get an inheriting conflicting methods error. (one from the real app "MyRunnableSystemWrapper", one from my mocked fake mixin called "MyMockedSystemWrapper").
execute(system) runs my system that is tested elsewhere and has sideaffects (connects to networked services, thus failing this test when such things are not available. Good news is I have a mocked service of my system wrapper that uses a system which does NOT have side affects and DB/Network calls are mocked out. However I do not know how to give THIS MOCKED version of my app to "WithApplication" test.
Reduced Code for clarity:
class Application extends Controller with MyRunnableSystemWrapper {
val pool: ExecutorService = Executors.newFixedThreadPool(1)
val system = new MyRunnableSystem() //system is abstract in MRSW ^^^ above
pool.execute(system)
def index = Action {
OK("HI")
}
}
My Test:
class MyAppTest(implicit ee: ExecutionEnv) extends Specification {
abstract class WithMyMockApp extends WithApplication {
def provideApplication = new controllers.Application with MyMockedSystemWrapper // This imports MyRunnableSystemWrapper
}
"Sending a GET request" should {
"Respond with OK" in new WithMyMockApp {
val response = route(app, FakeRequest(GET, "/")).get
status(response) mustEqual OK
}
}
}
If I'm not running my Runnable in the correct place and should be calling it somewhere else to make this testing easier, let me know!
You could inject your system wrapper instead of extending it
trait SystemWrapper {
def execute(system: RunnableSystem)
}
class MyRunnableSystemWrapper extends SystemWrapper {...}
class MyMockedSystemWrapper extends SystemWrapper {...}
class Application #Inject() (systemWrapper SystemWrapper) extends Controller {
Then you need to tell Guice which implementation of SystemWrapper you want for runtime and which one for test. One way of doing this is by using different Guice modules for runtime/test which you set in your .conf files.
I have one superclass with WordSpecLike and several subclasses for each backend.
abstract class A extends TestKit(_system) with WordSpecLike {
"backend" must {
"do something useful" in {
//test useful stuff
}
"print something pretty" in {
//test pretty print
}
}
}
class B extends A {
}
class C extends A {
}
C and B tests different backends but the thing is I need to turn on/off each backend-test separately for integration tests (using exclude groups).
Obviously, I can't use taggedAs.
Using a separate traits like in a bellow example didn't work:
trait backendB { this: Tag =>
override val name = "backendB"
}
class B extends A with backendB {
//...
}
}
Pure java annotation didn't work ether:
#TagAnnotation
#Retention(RetentionPolicy.RUNTIME)
#Target({ElementType.METHOD, ElementType.TYPE})
public #interface backendB {
}
So, the question is: Have I any other options to run each backend test under a special group or I have to copy-paste code?
Scala version 2.10
ScalaTest version 2.2.2
Java version 1.8.0_20
Can't use sbt instead of maven, can't bump scala version.
If I understand correctly, you want sometimes to exclude class B and run just class C. Other times exclude class C and run just class B. If that's correct, then normally you would annotate B and C with a respective tag annotation (make using Java). For example, doing this would cause all tests of B to be tagged as backendB:
#backendB
class B extends A {
}
What you didn't show is where the actor system is coming from. One thing about tags is that it only excludes tests, an instance of B will still be created. That means any state it holds onto will be initialized, and that might in your case mean an actor system. This might be what you meant by "didn't work." If so, my usual advice is to make that state lazily initialized. The instance will still be created, but because no tests run, he state will never be accessed, and since it is lazy, will never be initialized.
Since there are a lot of unknowns in this reply, you might do better on scalatest-users, where it is easier to go back and forth.
Ok, my final solution is not so elegant as it could be but it's work at least (still kinda more java-way solution).
Several small changes:
trait A-suite extends WordSpecLike {
val tagToExclude = Tag("here we can write anything")
def createTestBackend: Backend
"backend" must {
"do something useful" taggedAs(tagToExclude) in {
val backend = createTestBackend
//test useful stuff
}
"print something pretty" taggedAs(tagToExclude) in {
val backend = createTestBackend
//test pretty print
}
}
}
And test classes for different backends:
`
class backendA(override val tagToExclude = "backendA") extends A-suite {
override def createTestBackend: Backend = new backendA
}
class backendB(override val tagToExclude = "backendB") extends A-suite {
override def createTestBackend: Backend = new backendB
}
Now exclude groups work properly with maven.