I am running some tests with ScalaTest which rely on connections to test servers to be present. I currently created my own Spec similar to this:
abstract class ServerDependingSpec extends FlatSpec with Matchers {
def serverIsAvailable: Boolean = {
// Check if the server is available
}
}
Is it possible to ignore (but not fail) tests when this method returns false?
Currently I do it in a "hackish" way:
"Something" should "do something" in {
if(serverIsAvailable) {
// my test code
}
}
but I want something like
whenServerAvailable "Something" should "do something" in {
// test code
}
or
"Something" should "do something" whenServerAvailable {
// test code
}
I think I should define my custom tag, but I can only refer to the source code of in or ignore and I do not understand how I should plug in my custom implementations.
How should I accomplish this?
You can use assume to conditionally cancel a test:
"Something" should "do something" in {
assume(serverIsAvailable)
// my test code
}
or you can test for some condition yourself, and then use cancel:
"Something" should "do something" in {
if(!serverIsAvailable) {
cancel
}
// my test code
}
You can use Tags to achieve this:
Documentation on how to use Tags : http://www.scalatest.org/user_guide/tagging_your_tests
Adding and removing tagged test with command line parameters: http://www.scalatest.org/user_guide/using_the_runner#specifyingTagsToIncludeAndExclude
Example Code:
import org.scalatest.Tag
object ServerIsAvailable extends Tag("biz.neumann.ServerIsAvailable")
"Something" should "do something" taggedAs(ServerIsAvailable) in {
// your test here
}
Running the tests
Running the tests is a bitt tricky. It only works for testOnly and testQuick not test.
In the example testOnly is short for testOnly *
sbt "testOnly -- -l biz.neumann.ServerAvailable"
Here is some trick to skip a test based on a condition:
object WhenServerAvailable extends Tag(if (serverIsAvailable) "" else classOf[Ignore].getName)
"Something" should "do something" taggedAs WhenServerAvailable in { ... }
Related
I'm trying to dynamically run some basic tests in the following way (this is a pseudo code of my test, my actual tests are a bit more complicated):
class ExampleTest extends AnyWordSpec {
def runTests() = {
for( n <- 1 until 10){
testNumber(n)
}
}
def testNumber(n: Int) = {
"testNumber" when {
s"gets the number ${n}" should {
"fail if the number is different than 0" {
assert(n == 0)
}
}
}
}
runTests()
}
When I try to run my tests in IntelliJ all the tests run as expected. But when using sbt tests It says that all my tests passed even though non of my tests actually got executed (I get the message "1 tests suite executed. 0 tests where executed").
How can I fix this? Is there any other simple way to dynamically create tests in scala?)
As mentioned in a comment, I'm not sure what the problem with sbt is.
Another way you can try to create test dynamically is using property-based testing.
One possible way is using table-driven property checks, as in the following example (which you can see in action here on Scastie):
import org.scalatest.wordspec.AnyWordSpec
import org.scalatest.prop.TableDrivenPropertyChecks._
class ExampleTest extends AnyWordSpec {
private val values = Table("n", (1 to 10): _*)
"testNumber" when {
forAll(values) { n =>
s"gets the number ${n}" should {
"fail if the number is different than 0" in {
assert(n == 0)
}
}
}
}
}
Of course all these tests are going to fail. ;-)
Links to the ScalaTest documentation about the topic:
property-based testing
table-driven property checks
generator-driven property checks (especially useful to test a lot of cases within the same case)
I have scalatest codes like following:
class myTest extends FlatSpec with ParallelTestExecution {
val testSuiteId: String = GenerateSomeRandomId()
it should "print test id" in {
println(testSuiteId)
}
it should "print test id again" in {
println(testSuiteId)
}
}
The two tests cannot print the testSuiteId I generate before them. Instead they regenerate the ID and print it. I understand that because of ParallelTestExecution which extends OneInstancePerTest, each test here runs on its own instance and have a copy of the variable "testSuiteId".
But I do want a fixed Id for this test suite and each test case in this suite have access to this fixed it without modifying it. I tried to create the fixed id in BeforeAll{ } but still it didn't work.
How should I achieve what I want?
One way to work around it would be to put the shared state in some sort of external object:
object SuiteId {
lazy val id: String = GenerateSomeRandomId()
}
Admittedly this is very much a hack, and I wouldn't be surprised if scalatest has a way to handle this built-in which I am unaware of.
I came across an issue earlier where I couldn't run an indivdual scala test, it would always try to run all of them even if I set the configuration to just be running one test. Does anyone know of any settings/configuration I can change to get it to run?
class MyTest extends PlaySpec {
val setTo = new AfterWord("set to")
"Setting" when setTo {
"value a" in {
//test stuff
}
"value b" in {
//test stuff
}
}
Turns out it was the use of the AfterWord that was messing up my test, once I removed it the tests ran fine. I'm not sure why they're incompatible but if you want to run individual tests, don't use an AfterWord.
I have been searching for an answer for something that seems so trivial...
Anyway, here's my code:
class Test {
def done() {
println("Well done, you've completed the test!")
}
}
def test = new Test()
test.done()
Yet nothing is being printed.
Oh silly boo boo!
I was parsing the script and not evaluating it which led to it not being executed properly...
In specs2, what is the proper way to express a pattern of subtests that only execute if its "parent" test returned a result without throwing an exception?
I have a function maybeGiveMeAThing, and it can either return a Thing, or throw exceptions.
A call looks like this:
val thing: Thing = maybeGiveMeAThing("foo", "bar" "baz")
I want to test that with a certain set of inputs, that maybeGiveMeAThing successfully returns a Thing without throwing an exception, and using the Thing returned, do further tests to ensure that it is the correct Thing returned for the parameters given to maybeGiveMeAThing.
The way I have the tests currently set up, if the call to maybeGiveMeAThing throws an exception, the entire test suite gets aborted. This would be the logic I prefer:
If a Thing was returned successfully, proceed with a set of subtests that analyze the contents of thing
If maybeGiveMeAThing threw an exception (any exception), skip the subtests that analyze thing, but continue with the rest of the tests
My existing test code looks roughly like:
// ...
"with good parameters" in {
var thing: Thing = null
"return a Thing without throwing an exception" in {
thing = maybeGiveMeAThing("some", "good", "parameters", "etc.")
} should not(throwA[Exception])
"the Thing returned should contain a proper Foo" in {
thing.foo mustBe "bar"
}
//... etc ...
}
// ...
}
...although this feels like it's way off from the right way to do it. What would be the proper way?
(I would like to avoid using vars if I can help it.)
One possibility is to use a condition as in #alexwriteshere answer:
"parent test" in {
val thing: Option[Thing] = maybeGiveMeAThing("foo", "bar" "baz")
thing match {
case Some(thing) =>
"child test 1" in ok
"child test 2" in ok
case None =>
"skipped tests" in skipped
}
}
However you need to add an Example in the None case so that the block of the in method has an acceptable type.
The big drawback with this approach though is that the specification is being executed while being defined. This means that:
if there's an exception with the maybeGiveMeAThing then the whole specification will blow up
if you choose to exclude the examples using the thing, you will still have it being built
Another option is to use a Step saying that any previous failure will skip all the next examples:
class MySpec extends mutable.Specification {
"This is a test with a thing" >> {
lazy val thing: Option[Thing] = maybeGiveMeAThing("foo", "bar" "baz")
"the thing should be ok" >> {
thing must beOk
}
step(stopOnFail = true)
"other examples with thing" >> {
"ex1" >> ok
"ex2" >> ok
}
}
}
A simple if/else with the child cases will do what you want. Or, you could make the "maybeGiveMeAThing" method return an option, and then match it.
"parent test" in {
val thing: Option[Thing] = maybeGiveMeAThing("foo", "bar" "baz")
thing match {
case Some(thing) =>
"child test 1" in {
...
}
"child test 2" in {
...
}
case None =>
// don't run additional tests
}
}
If maybeGiveMeAThing has to return an exception, you could catch it and make the method return an Option.