I am trying to test some db dependent stuff with specs2 in scala. The goal is to test for "db running" and then execute the test. I figured out that i can use orSkip from the Matcher class if the db is down.
The problem is, that i am getting output for the one matching condition (as PASSED) and the example is marked as SKIPPED. What i want instead: Only execute one test that is marked as "SKIPPED" in case the test db is offline. And here is the code for my "TestKit"
package net.mycode.testkit
import org.specs2.mutable._
import net.mycode.{DB}
trait MyTestKit {
this: SpecificationWithJUnit =>
def debug = false
// Before example
step {
// Do something before
}
// Skip the example if DB is offline
def checkDbIsRunning = DB.isRunning() must be_==(true).orSkip
// After example
step {
// Do something after spec
}
}
And here the code for my spec:
package net.mycode
import org.specs2.mutable._
import net.mycode.testkit.{TestKit}
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner
#RunWith(classOf[JUnitRunner])
class MyClassSpec extends SpecificationWithJUnit with TestKit with Logging {
"MyClass" should {
"do something" in {
val sut = new MyClass()
sut.doIt must_== "OK"
}
"do something with db" in {
checkDbIsRunning
// Check only if db is running, SKIP id not
}
}
Out now:
Test MyClass should::do something(net.mycode.MyClassSpec) PASSED
Test MyClass should::do something with db(net.mycode.MyClassSpec) SKIPPED
Test MyClass should::do something with db(net.mycode.MyClassSpec) PASSED
And output i want it to be:
Test MyClass should::do something(net.mycode.MyClassSpec) PASSED
Test MyClass should::do something with db(net.mycode.MyClassSpec) SKIPPED
I think you can use a simple conditional to do what you want:
class MyClassSpec extends SpecificationWithJUnit with TestKit with Logging {
"MyClass" should {
"do something" in {
val sut = new MyClass()
sut.doIt must_== "OK"
}
if (DB.isRunning) {
// add examples here
"do something with db" in { ok }
} else skipped("db is not running")
}
}
Have you tried using the args(skipAll=true) argument? See a few examples here.
Unfortunately (as far as I know), you cannot skip a single example in a unit specification. You can, however, skip the specification structure with this argument like this, so you might have to create separate specifications:
class MyClassSpec extends SpecificationWithJUnit {
args(skipAll = false)
"MyClass" should {
"do something" in {
success
}
"do something with db" in {
success
}
}
}
A new feature addressing this has been added to specs 2.3.10.
Related
I have a set of input cases stored in a file.
I would like each case to be a specific scalatest "test", i.e., reported in the console as an individual test and failed individually.
Unfortunately, experimentation and Google suggest that this capability might not be present?
E.g., this seems to be the common case (eliding for simplicity)
class MyTestingGoop extends FunSuite {
val input : Seq[SpecificTestCase] = ...
test("input data test") {
forAll(input) { case => ... }
}
//...
}
Ideally, each case presents as a separate test. How can this be done with ScalaTest?
You can do this:
class MyTestingGoop extends FunSuite {
val input : Seq[SpecificTestCase] = ...
forAll(input) {
test("testing input" + input) {
// do something with the test
}
}
}
The only limit is that input has a unique toString.
Basically calling test in Funsuite registers the test and later runs it so as long as your test creation is done as part of the class construction and each test has a unique string, you should be fine.
I'm working with Scala project, using Specs2 for testing.
How could I know the test name i'm currently within it (form the test context)?
This is the code structure of my tests:
class TestsE2E{
trait Context extends E2EScope {
...
}
"Test" should {
"do stuff" in new Context {
...
}
}
}
If you just use the test name to log it, you can create a custom ExamplesFactory to intercept the name:
import org.specs2.mutable.{FragmentsBuilder, ExamplesFactory}
trait TestNameLogger extends ExamplesFactory { this: FragmentsBuilder =>
override implicit def exampleFactory: ExampleFactory = new TestNameLoggerMutableExampleFactory
class TestNameLoggerMutableExampleFactory extends MutableExampleFactory {
override def newExample(e: Example): Example = {
println(e.desc.raw)
super.newExample(e)
}
}
}
This solution is a bit involved and the API for doing this will change with the upcoming specs2 3.0. Another alternative is to create your own Notifier.
In our project we use Scala Specs2 together with Selenium.
I'm trying to implement screenshot-on-failure mechanism "in a classic way (link)" for my tests, using JUnit annotations, but, the rule doesn't called on test failure at all.
The structure of the test is as follows:
class Tests extends SpecificationWithJUnit{
trait Context extends LotsOfStuff {
#Rule
val screenshotOnFailRule = new ScreenshotOnFailRule(driver)
}
"test to verify stuff that will fail" should {
"this test FAILS" in new Context {
...
}
}
The ScreenshotOnFailRule looks like this:
class ScreenshotOnFailRule (webDriver: WebDriver) extends TestWatcher {
override def failed(er:Throwable, des:Description) {
val scrFile = webDriver.asInstanceOf[TakesScreenshot].getScreenshotAs(OutputType.FILE)
FileUtils.copyFile(scrFile, new File(s"/tmp/automation_screenshot${Platform.currentTime}.png"))
}
}
I understand that probably it doesn't work now because the tests aren't annotated with #Test annotation.
Is it possible to annotate the Specs2 tests with JUnit #Rule annotation?
According to this question it seems as if JUnit Rules aren't supported. But you could try to make use of the AroundExample trait:
import org.specs2.execute.{AsResult, Result}
import org.specs2.mutable._
import org.specs2.specification.AroundExample
class ExampleSpec extends Specification with AroundExample {
// execute tests in sequential order
sequential
"The 'Hello world' string" should {
"contain 11 characters" in {
"Hello world" must have size (10)
}
// more tests..
}
override protected def around[T](t: => T)(implicit ev: AsResult[T]): Result = {
try {
AsResult.effectively(t)
} catch {
case e: Throwable => {
// take screenshot here
throw e
}
}
}
}
Running tests as described here
"Spec" should {
"example" in new WithApplication {
...
}
}
is unacceptably slow for me. This is because new WithApplication is starting and stoping framework at every example. Don't get me wrong, a framework itself loads very fast, but if database is configured (surprise!), situation becomes terrible.
Here are some measurments:
"The database layer" should {
"test1" in {
1 must be equalTo(1)
}
...
"test20" in {
1 must be equalTo(1)
}
}
Execution time: 2 seconds. Same test with WithApplication at every example consumes 9 seconds
I was able to achive much better results thanks to this answer
import play.api.Play
import play.api.test.FakeApplication
import org.specs2.mutable.Specification
import scalikejdbc._
class MySpec extends Specification {
var fake: FakeApplication = _
step {fake = FakeApplication(...)}
step {Play.start(fake)}
"The database layer" should {
"some db test" in {
DB localTx { implicit session =>
...
}
}
"another db test" in {
DB localTx { implicit session =>
...
}
}
step {Play.stop()}
}
}
Pros: performance boost
Cons:
need to copy-paste setup and tear-down code because don't know how to
reuse it (by reuse I mean something like "class MySpec extends
Specification with NoWasteOfTime"
new WithApplication() calls Helpers.running which looks like this
synchronized {
try {
Play.start(fakeApp)
block
} finally {
Play.stop()
play.api.libs.ws.WS.resetClient()
}
}
so I can't completely emulate Helpers.running behaviour (resetClient is not visible for my code) without reflection.
Please suggest how to break cons or different approach how accomplish my issue.
I don't know if it is the best possible solution but in this thread:
Execute code before and after specification
You can read a solution for reusable code. I implemented it with little modifications. For me the beforeAll step did not run and added the sequential modifier.
import org.specs2.mutable._
import org.specs2.specification._
class PlayAppSpec extends Specification with BeforeAllAfterAll{
sequential
lazy val app : FakeApplication = {
FakeApplication()
}
def beforeAll(){
Play.start(app)
}
def afterAll(){
Play.stop()
}
}
import org.specs2.specification.Step
trait BeforeAllAfterAll extends Specification {
// see specs2 User Guide, linked below
override def map(fragments: =>Fragments) = {
beforeAll()
fragments ^ Step(afterAll)
}
def beforeAll()
def afterAll()
}
I think the map would be better with Step(...) ^ fragments ^ Step(...) but it did not run the beforeAll for me. The user guide at "Global setup/teardown" says to use a lazy val.
Overall it was a pain to set up this. My problem was
Exception in thread "Thread-145" java.net.SocketException: Connection reset
Or
Configuration error[Cannot connect to database [default]] (Configuration.scala:559)
When reusing the same FakeApplication:
SQLException: Attempting to obtain a connection from a pool that has already been shutdown.
I think it is much more logical this way than always creating a new application for every "in" block or adding all tests into one block.
I've inherited some JUnit tests written in scala that need to be fixed to use #BeforeClass semantics. I understand that the #BeforeClass annotation must be applied to static methods only. I understand that methods defined in "companion" objects (as opposed to scala classes) are static. How can I get a test method to be called once prior to the individual instance methods in a test class?
object TestClass {
#BeforeClass
def stuff() {
// beforeclass stuff
}
}
class TestClass {
#Test
...
}
seems to work...
I had to move to specs2 to implement this feature with Scala. Just adding an example to help people with the same problem as the orginal poster who don't yet know specs2.
The specs2 way uses the concept of a "step" to accomplish test suite set-up and tear-down. If you run with a JUnitRunner all your Ant scripts and IDEs that use JUnit will still know how to run it. Here's an example using a mutable specification from specs2:
import org.specs2.mutable.Specification
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner
#RunWith(classOf[JUnitRunner])
class MutableSpecs2ExampleTest extends Specification {
var firstStep: String = null
var secondStep: String = null
var thirdStep: String = null
//Steps are guaranteed to run serially
step{println("Loading Spring application context...");firstStep="Hello World"}
step{println("Setting up mocks...");secondStep = "Hello Scala"}
//The fragments should be run in parallel by specs2
"Some component Foo in my project" should{
" pass these tests" in {
println("Excersizing some code in Foo")
firstStep must startWith("Hello") and endWith("World")
}
" pass theses other tests" in {
println("Excersizing some other code in Foo")
firstStep must have size(11)
}
}
"Some component Bar in my project" should{
" give the correct answer" in {
println("Bar is thinking...")
secondStep must startWith("Hello") and endWith("Scala")
thirdStep must be equalTo null
}
}
step{println("Tearing down resources after tests...");thirdStep = "Hello Specs2"}
}
And here's an example with a non-mutable specification:
import org.specs2.Specification
import org.specs2.specification.Step
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner
#RunWith(classOf[JUnitRunner])
class Specs2ExampleTest extends Specification{
var firstStep: String = null
var secondStep: String = null
var thirdStep: String = null
def is =
"This is a test with some set-up and tear-down examples" ^
p^
"Initialize" ^
Step(initializeDependencies())^
Step(createTestData())^
"Component Foo should" ^
"perform some calculation X " !testX^
"perform some calculation Y" !testY^
p^
"Tidy up" ^
Step(removeTestData())^
end
def testX = {
println("testing Foo.X")
firstStep must be equalTo("Hello World")
}
def testY = {
println("testing Foo.Y")
secondStep must be equalTo("Hello Scala")
thirdStep must be equalTo null
}
def initializeDependencies(){
println("Initializing Spring applicaiton context...")
firstStep = "Hello World"
}
def createTestData(){
println("Inserting test data into the db...")
secondStep = "Hello Scala"
}
def removeTestData(){
println("Removing test data from the db...")
println("Tearing down resources...")
thirdStep = "Hello Specs2"
}
}
You don't specify if you mean inherited in the OO-programming sense or the "taken over from someone else" sense of the word.
In the latter case, I'd advise you to re-write the thing using either ScalaTest or Specs2 and expose it as a JUnit test (both frameworks support this) so that it'll integrate with whatever other tools and processes you already have in place.