In our project we use Scala Specs2 together with Selenium.
I'm trying to implement screenshot-on-failure mechanism "in a classic way (link)" for my tests, using JUnit annotations, but, the rule doesn't called on test failure at all.
The structure of the test is as follows:
class Tests extends SpecificationWithJUnit{
trait Context extends LotsOfStuff {
#Rule
val screenshotOnFailRule = new ScreenshotOnFailRule(driver)
}
"test to verify stuff that will fail" should {
"this test FAILS" in new Context {
...
}
}
The ScreenshotOnFailRule looks like this:
class ScreenshotOnFailRule (webDriver: WebDriver) extends TestWatcher {
override def failed(er:Throwable, des:Description) {
val scrFile = webDriver.asInstanceOf[TakesScreenshot].getScreenshotAs(OutputType.FILE)
FileUtils.copyFile(scrFile, new File(s"/tmp/automation_screenshot${Platform.currentTime}.png"))
}
}
I understand that probably it doesn't work now because the tests aren't annotated with #Test annotation.
Is it possible to annotate the Specs2 tests with JUnit #Rule annotation?
According to this question it seems as if JUnit Rules aren't supported. But you could try to make use of the AroundExample trait:
import org.specs2.execute.{AsResult, Result}
import org.specs2.mutable._
import org.specs2.specification.AroundExample
class ExampleSpec extends Specification with AroundExample {
// execute tests in sequential order
sequential
"The 'Hello world' string" should {
"contain 11 characters" in {
"Hello world" must have size (10)
}
// more tests..
}
override protected def around[T](t: => T)(implicit ev: AsResult[T]): Result = {
try {
AsResult.effectively(t)
} catch {
case e: Throwable => {
// take screenshot here
throw e
}
}
}
}
Related
So I have the following bit of code
class MetricsLogger {
def measure[T](name:String)(operation: => T): T = {
val startTime = System.currentTimeMillis
val result = try {
operation
} finally {
logMetric(Metric(name, System.currentTimeMillis - startTime, StandardUnit.Milliseconds))
}
result
}
}
Where log Metric is some kind of side effect (e.g. upload metric to cloudwatch).
Now I am doing this like
def measuredOp = measure("metricName") { someOperation }
Here some operation is making some network calls.
Now I have to stub measured op.
So my stub is as the follows:-
val loggingMetrics = mock[MetricsLogger] // mock is from MockitoSugar trait
and my stubbing is like
Mockito.when(loggingMetrics.measure(Matchers.anyString())(Matchers.anyObject())).thenReturn(???)
So obviously my stubbing is wrong, but I cannot figure how to stub this properly.
Mockito doesn't support that as by-name parameters it's a concept that doesn't exists in Java, however, mockito-scala does support this from version 0.4.0 (try 0.4.0 or 0.4.2, ignore 0.4.1)
I just run a quick test like this
import org.mockito.{ MockitoSugar, ArgumentMatchersSugar }
class StackOverflowTest extends WordSpec with MockitoSugar with scalatest.Matchers with ArgumentMatchersSugar {
"mock[T]" should {
"it should work with by-name params" in {
val loggingMetrics = mock[MetricsLogger]
when(loggingMetrics.measure(any)(any)).thenReturn("it worked!")
loggingMetrics.measure("test")("") shouldBe "it worked!"
}
}
}
Disclaimer: I'm a maintainer of that library although it's part of the official Mockito suite
I have been developing a command-line tool which calls System.exit() (don't want to use exceptions instead of) on certain inputs.
I am familiar with Java: How to test methods that call System.exit()? and its the most elegant approach.
Unfortunately, it is not enough pure, due to I had to add the dependencies to system-rules, junit-interface
Is there any common pattern for dealing with System.exit in specs2 which is more pure than my current approach which don't use specs2?
import org.junit.Rule;
import org.junit.Test;
import org.junit.contrib.java.lang.system.ExpectedSystemExit;
public class ConverterTest {
#Rule
public final ExpectedSystemExit exit = ExpectedSystemExit.none();
#Test
public void emptyArgs() {
exit.expectSystemExit();
Converter.main(new String[]{});
}
#Test
public void missingOutArgument() {
exit.expectSystemExitWithStatus(1);
Converter.main(new String[]{"--in", "src/test/resources/078.xml.gz"});
}
}
If you really wish to go with a method using System.exit(), the simplest way to test it was actually called is to replace your SecurityManager with one that'll throw an ExitException (subclassing SecurityException) when System.exit() is called:
class SystemExitSpec
import java.security.Permission
import org.specs2.mutable.Specification
import org.specs2.specification.BeforeAfterAll
sealed case class ExitException(status: Int) extends SecurityException("System.exit() is not allowed") {
}
sealed class NoExitSecurityManager extends SecurityManager {
override def checkPermission(perm: Permission): Unit = {}
override def checkPermission(perm: Permission, context: Object): Unit = {}
override def checkExit(status: Int): Unit = {
super.checkExit(status)
throw ExitException(status)
}
}
abstract class SystemExitSpec extends Specification with BeforeAfterAll {
sequential
override def beforeAll(): Unit = System.setSecurityManager(new NoExitSecurityManager())
override def afterAll(): Unit = System.setSecurityManager(null)
}
test ConverterSpec
import org.specs2.execute.Failure
import scala.io.Source
class ConverterSpec extends SystemExitSpec {
"ConverterSpec" should {
"empty args" >> {
try {
Converter.main(Array[String]())
Failure("shouldn't read this code")
} catch {
case e: ExitException =>
e.status must_== 1
}
1 must_== 1
}
}
First option: use some exception instead of System.exit.
Second option: call application in separate thread and check return codes.
Third option: mock System.exit. There are many possibilities to do that, mentioned one is quite good.
However, there is no specs2-specific pattern to work with System.exit. Personally I'd suggest first or second options.
I'm working with Scala project, using Specs2 for testing.
How could I know the test name i'm currently within it (form the test context)?
This is the code structure of my tests:
class TestsE2E{
trait Context extends E2EScope {
...
}
"Test" should {
"do stuff" in new Context {
...
}
}
}
If you just use the test name to log it, you can create a custom ExamplesFactory to intercept the name:
import org.specs2.mutable.{FragmentsBuilder, ExamplesFactory}
trait TestNameLogger extends ExamplesFactory { this: FragmentsBuilder =>
override implicit def exampleFactory: ExampleFactory = new TestNameLoggerMutableExampleFactory
class TestNameLoggerMutableExampleFactory extends MutableExampleFactory {
override def newExample(e: Example): Example = {
println(e.desc.raw)
super.newExample(e)
}
}
}
This solution is a bit involved and the API for doing this will change with the upcoming specs2 3.0. Another alternative is to create your own Notifier.
I am trying to test some db dependent stuff with specs2 in scala. The goal is to test for "db running" and then execute the test. I figured out that i can use orSkip from the Matcher class if the db is down.
The problem is, that i am getting output for the one matching condition (as PASSED) and the example is marked as SKIPPED. What i want instead: Only execute one test that is marked as "SKIPPED" in case the test db is offline. And here is the code for my "TestKit"
package net.mycode.testkit
import org.specs2.mutable._
import net.mycode.{DB}
trait MyTestKit {
this: SpecificationWithJUnit =>
def debug = false
// Before example
step {
// Do something before
}
// Skip the example if DB is offline
def checkDbIsRunning = DB.isRunning() must be_==(true).orSkip
// After example
step {
// Do something after spec
}
}
And here the code for my spec:
package net.mycode
import org.specs2.mutable._
import net.mycode.testkit.{TestKit}
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner
#RunWith(classOf[JUnitRunner])
class MyClassSpec extends SpecificationWithJUnit with TestKit with Logging {
"MyClass" should {
"do something" in {
val sut = new MyClass()
sut.doIt must_== "OK"
}
"do something with db" in {
checkDbIsRunning
// Check only if db is running, SKIP id not
}
}
Out now:
Test MyClass should::do something(net.mycode.MyClassSpec) PASSED
Test MyClass should::do something with db(net.mycode.MyClassSpec) SKIPPED
Test MyClass should::do something with db(net.mycode.MyClassSpec) PASSED
And output i want it to be:
Test MyClass should::do something(net.mycode.MyClassSpec) PASSED
Test MyClass should::do something with db(net.mycode.MyClassSpec) SKIPPED
I think you can use a simple conditional to do what you want:
class MyClassSpec extends SpecificationWithJUnit with TestKit with Logging {
"MyClass" should {
"do something" in {
val sut = new MyClass()
sut.doIt must_== "OK"
}
if (DB.isRunning) {
// add examples here
"do something with db" in { ok }
} else skipped("db is not running")
}
}
Have you tried using the args(skipAll=true) argument? See a few examples here.
Unfortunately (as far as I know), you cannot skip a single example in a unit specification. You can, however, skip the specification structure with this argument like this, so you might have to create separate specifications:
class MyClassSpec extends SpecificationWithJUnit {
args(skipAll = false)
"MyClass" should {
"do something" in {
success
}
"do something with db" in {
success
}
}
}
A new feature addressing this has been added to specs 2.3.10.
I've inherited some JUnit tests written in scala that need to be fixed to use #BeforeClass semantics. I understand that the #BeforeClass annotation must be applied to static methods only. I understand that methods defined in "companion" objects (as opposed to scala classes) are static. How can I get a test method to be called once prior to the individual instance methods in a test class?
object TestClass {
#BeforeClass
def stuff() {
// beforeclass stuff
}
}
class TestClass {
#Test
...
}
seems to work...
I had to move to specs2 to implement this feature with Scala. Just adding an example to help people with the same problem as the orginal poster who don't yet know specs2.
The specs2 way uses the concept of a "step" to accomplish test suite set-up and tear-down. If you run with a JUnitRunner all your Ant scripts and IDEs that use JUnit will still know how to run it. Here's an example using a mutable specification from specs2:
import org.specs2.mutable.Specification
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner
#RunWith(classOf[JUnitRunner])
class MutableSpecs2ExampleTest extends Specification {
var firstStep: String = null
var secondStep: String = null
var thirdStep: String = null
//Steps are guaranteed to run serially
step{println("Loading Spring application context...");firstStep="Hello World"}
step{println("Setting up mocks...");secondStep = "Hello Scala"}
//The fragments should be run in parallel by specs2
"Some component Foo in my project" should{
" pass these tests" in {
println("Excersizing some code in Foo")
firstStep must startWith("Hello") and endWith("World")
}
" pass theses other tests" in {
println("Excersizing some other code in Foo")
firstStep must have size(11)
}
}
"Some component Bar in my project" should{
" give the correct answer" in {
println("Bar is thinking...")
secondStep must startWith("Hello") and endWith("Scala")
thirdStep must be equalTo null
}
}
step{println("Tearing down resources after tests...");thirdStep = "Hello Specs2"}
}
And here's an example with a non-mutable specification:
import org.specs2.Specification
import org.specs2.specification.Step
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner
#RunWith(classOf[JUnitRunner])
class Specs2ExampleTest extends Specification{
var firstStep: String = null
var secondStep: String = null
var thirdStep: String = null
def is =
"This is a test with some set-up and tear-down examples" ^
p^
"Initialize" ^
Step(initializeDependencies())^
Step(createTestData())^
"Component Foo should" ^
"perform some calculation X " !testX^
"perform some calculation Y" !testY^
p^
"Tidy up" ^
Step(removeTestData())^
end
def testX = {
println("testing Foo.X")
firstStep must be equalTo("Hello World")
}
def testY = {
println("testing Foo.Y")
secondStep must be equalTo("Hello Scala")
thirdStep must be equalTo null
}
def initializeDependencies(){
println("Initializing Spring applicaiton context...")
firstStep = "Hello World"
}
def createTestData(){
println("Inserting test data into the db...")
secondStep = "Hello Scala"
}
def removeTestData(){
println("Removing test data from the db...")
println("Tearing down resources...")
thirdStep = "Hello Specs2"
}
}
You don't specify if you mean inherited in the OO-programming sense or the "taken over from someone else" sense of the word.
In the latter case, I'd advise you to re-write the thing using either ScalaTest or Specs2 and expose it as a JUnit test (both frameworks support this) so that it'll integrate with whatever other tools and processes you already have in place.