How to prioritize tests in scala? - scala

I am using scalatest's FunSuite to run my tests. I want to define 2 separate test classes that have a few tests in common, so I created a BaseTest class which I extended in MainTest1 and MainTest2, in which I define additional specific tests (see code snippets below). I want the tests in MainTest1 and MainTest2 to execute in a specific order, e.g. test 4, 2, 1 and then 3. How do I achieve this?
class BaseTest extends FunSuite with BeforeAndAfter{
test("Check Content of file){
//code to check content of a file
}
test("File is not empty"){
//code to check file is not empty
}
test("Check a particular word in file"){
//code to check particular word in file
}
}
class MainTest1 extends BaseTest{
test("Check create file"){
//code to check file creation
}
}
class MainTest2 extends BaseTest{
test("Check download file"){
//code to check file downloaded properly
}
}

You can define your tests as methods and call them in the order you see fit from the children classes.
trait BaseTest extends FunSuite {
protected def test1(): Unit = test("This is test 1") { ... }
protected def test2(): Unit = test("This is test 1") { ... }
}
class MainTest1 extends BaseTest {
// Arrange these is any order you see fit
test3()
test2()
test1()
protected def test3(): Unit = test("This is a test specific to MainTest1") { ... }
}
class MainTest2 extends BaseTest {
// Arrange these is any order you see fit
test2()
test3()
test1()
protected def test3(): Unit = test("This is a test specific to MainTest2") { ... }
}

Related

How do I run a funsuite test within a for loop?

I am very new to scala so trying to understand how funsuite testing works. I have a bunch of tests that take a method data() to set it as a particular method from a variety of methods namely method1(): Long, method2(): Long etc
I figured defining the method1/2/... etc within the class would be fine and then sequentially go through them for my tests rather than having to write duplicate tests for each method, but seems like the data cannot be found by the tests. I also tried putting both the tests inside the for loop but then I am having issues with the IDE not being able to find tests. What am I doing wrong?
Example
class controlTest extends Anyfunsuite {
protected def method1(): Long = { a() }
protected def method2(): Long = { b() }
protected def method3(): Long = { c() }
for(i <- Seq(method1(), method2(), method3()) {
def data(): Long = i
}
test("testing1"){
data()
....
}
test("testing2"){
data()
...
}

Scala: How to test methods that call System.exit()?

I have been developing a command-line tool which calls System.exit() (don't want to use exceptions instead of) on certain inputs.
I am familiar with Java: How to test methods that call System.exit()? and its the most elegant approach.
Unfortunately, it is not enough pure, due to I had to add the dependencies to system-rules, junit-interface
Is there any common pattern for dealing with System.exit in specs2 which is more pure than my current approach which don't use specs2?
import org.junit.Rule;
import org.junit.Test;
import org.junit.contrib.java.lang.system.ExpectedSystemExit;
public class ConverterTest {
#Rule
public final ExpectedSystemExit exit = ExpectedSystemExit.none();
#Test
public void emptyArgs() {
exit.expectSystemExit();
Converter.main(new String[]{});
}
#Test
public void missingOutArgument() {
exit.expectSystemExitWithStatus(1);
Converter.main(new String[]{"--in", "src/test/resources/078.xml.gz"});
}
}
If you really wish to go with a method using System.exit(), the simplest way to test it was actually called is to replace your SecurityManager with one that'll throw an ExitException (subclassing SecurityException) when System.exit() is called:
class SystemExitSpec
import java.security.Permission
import org.specs2.mutable.Specification
import org.specs2.specification.BeforeAfterAll
sealed case class ExitException(status: Int) extends SecurityException("System.exit() is not allowed") {
}
sealed class NoExitSecurityManager extends SecurityManager {
override def checkPermission(perm: Permission): Unit = {}
override def checkPermission(perm: Permission, context: Object): Unit = {}
override def checkExit(status: Int): Unit = {
super.checkExit(status)
throw ExitException(status)
}
}
abstract class SystemExitSpec extends Specification with BeforeAfterAll {
sequential
override def beforeAll(): Unit = System.setSecurityManager(new NoExitSecurityManager())
override def afterAll(): Unit = System.setSecurityManager(null)
}
test ConverterSpec
import org.specs2.execute.Failure
import scala.io.Source
class ConverterSpec extends SystemExitSpec {
"ConverterSpec" should {
"empty args" >> {
try {
Converter.main(Array[String]())
Failure("shouldn't read this code")
} catch {
case e: ExitException =>
e.status must_== 1
}
1 must_== 1
}
}
First option: use some exception instead of System.exit.
Second option: call application in separate thread and check return codes.
Third option: mock System.exit. There are many possibilities to do that, mentioned one is quite good.
However, there is no specs2-specific pattern to work with System.exit. Personally I'd suggest first or second options.

Scala Specs2: how to get current test name?

I'm working with Scala project, using Specs2 for testing.
How could I know the test name i'm currently within it (form the test context)?
This is the code structure of my tests:
class TestsE2E{
trait Context extends E2EScope {
...
}
"Test" should {
"do stuff" in new Context {
...
}
}
}
If you just use the test name to log it, you can create a custom ExamplesFactory to intercept the name:
import org.specs2.mutable.{FragmentsBuilder, ExamplesFactory}
trait TestNameLogger extends ExamplesFactory { this: FragmentsBuilder =>
override implicit def exampleFactory: ExampleFactory = new TestNameLoggerMutableExampleFactory
class TestNameLoggerMutableExampleFactory extends MutableExampleFactory {
override def newExample(e: Example): Example = {
println(e.desc.raw)
super.newExample(e)
}
}
}
This solution is a bit involved and the API for doing this will change with the upcoming specs2 3.0. Another alternative is to create your own Notifier.

How to use JUnit's #Rule annotation with Scala Specs2 tests?

In our project we use Scala Specs2 together with Selenium.
I'm trying to implement screenshot-on-failure mechanism "in a classic way (link)" for my tests, using JUnit annotations, but, the rule doesn't called on test failure at all.
The structure of the test is as follows:
class Tests extends SpecificationWithJUnit{
trait Context extends LotsOfStuff {
#Rule
val screenshotOnFailRule = new ScreenshotOnFailRule(driver)
}
"test to verify stuff that will fail" should {
"this test FAILS" in new Context {
...
}
}
The ScreenshotOnFailRule looks like this:
class ScreenshotOnFailRule (webDriver: WebDriver) extends TestWatcher {
override def failed(er:Throwable, des:Description) {
val scrFile = webDriver.asInstanceOf[TakesScreenshot].getScreenshotAs(OutputType.FILE)
FileUtils.copyFile(scrFile, new File(s"/tmp/automation_screenshot${Platform.currentTime}.png"))
}
}
I understand that probably it doesn't work now because the tests aren't annotated with #Test annotation.
Is it possible to annotate the Specs2 tests with JUnit #Rule annotation?
According to this question it seems as if JUnit Rules aren't supported. But you could try to make use of the AroundExample trait:
import org.specs2.execute.{AsResult, Result}
import org.specs2.mutable._
import org.specs2.specification.AroundExample
class ExampleSpec extends Specification with AroundExample {
// execute tests in sequential order
sequential
"The 'Hello world' string" should {
"contain 11 characters" in {
"Hello world" must have size (10)
}
// more tests..
}
override protected def around[T](t: => T)(implicit ev: AsResult[T]): Result = {
try {
AsResult.effectively(t)
} catch {
case e: Throwable => {
// take screenshot here
throw e
}
}
}
}

How do I stub out a protected method in a object in scala

Example:
object Test {
def test = {
doTest
}
protected def doTest = {
// do something
}
}
class MockTest extends WordSpec with Mockito{
"" in {
val t = spy(Test)
// how do i stub out doTest?
}
}
I have a Test class with a protected doTest method. How do I stub out this protected method?
I would advise to make doTest package private, so that clients of your object won't be able to call it, but you'll be able to test it from within the same package.
package com.company.my.foo
object Test {
def test = {
doTest
}
private[foo] def doTest = {
// do something
}
}
and
package com.company.my.foo
class MockTest extends WordSpec with Mockito{
"" in {
val t = spy(Test)
when(t.doTest()).thenReturn("foo")
}
}
Test is a singleton (all Scala objects are), you can subclass a class, but not an object. Hence protected is a bit meaningless here - you're saying this method should only be accessible to subclasses, but it's not possible to have any (since Test is an object).
You have a couple options depending on what is best suited to your needs. 1) you can make Test a class and then extend it, or 2) change the access level of doTest to public [which is the default in Scala if you don't specify an access modifier]