I am facing the following perplexing behaviour when unit testing classes with variables.
For the sake of a simple example, let's assume I have the following class:
// Case classes are not an alternative in my use case.
final class C(var i: Int = 0) {
def add(that: C): Unit = {
i += that.i
}
override def toString: String = {
s"C($i)"
}
}
For which I concocted the below trivial and seemingly harmless unit test:
import org.junit.runner.RunWith
import org.scalacheck.Gen
import org.scalatest.junit.JUnitRunner
import org.scalatest.prop.GeneratorDrivenPropertyChecks
import org.scalatest.{MustMatchers, WordSpec}
#RunWith(classOf[JUnitRunner])
class CUnitTest extends WordSpec with MustMatchers with GeneratorDrivenPropertyChecks {
private val c: C = new C()
forAll (Gen.choose(1, 100).map(new C(_))) { x =>
s"Adding $x to $c" must {
val expectedI = c.i + x.i
c.add(x)
s"result in its .i property becoming $expectedI" in {
c.i mustBe expectedI
}
}
}
}
Where all test cases except the last fail:
For example, the first three test cases fail with the below results:
org.scalatest.exceptions.TestFailedException: 414 was not equal to 68
org.scalatest.exceptions.TestFailedException: 414 was not equal to 89
org.scalatest.exceptions.TestFailedException: 414 was not equal to 151
Now, playing around the unit test and moving the c.add(x) part inside the in clause:
import org.junit.runner.RunWith
import org.scalacheck.Gen
import org.scalatest.junit.JUnitRunner
import org.scalatest.prop.GeneratorDrivenPropertyChecks
import org.scalatest.{MustMatchers, WordSpec}
#RunWith(classOf[JUnitRunner])
class CUnitTest extends WordSpec with MustMatchers with GeneratorDrivenPropertyChecks {
private val c: C = new C()
forAll (Gen.choose(1, 100).map(new C(_))) { x =>
s"Adding $x to $c" must {
val expectedI = c.i + x.i
s"result in its .i property becoming $expectedI" in {
c.add(x)
c.i mustBe expectedI
}
}
}
}
And all test cases except the first fail:
For example, the second and the third test cases fail with the following messages:
org.scalatest.exceptions.TestFailedException: 46 was not equal to 44
org.scalatest.exceptions.TestFailedException: 114 was not equal to 68
In addition, c.i doesn't seem to increase at all in the test case description as I intended and expected it to be.
Clearly, the order of execution inside ScalaTest clauses are not top-down. Something happens earlier or later than in the order it's written, or perhaps doesn't happen at all depending on which clause it is inside, yet I can't wrap my head around it.
What's going on and why?
Furthermore, how could I achieve the desired behaviour (c.i increasing, all test cases passing)?
Consider rewriting the test like so
import org.scalacheck.Gen
import org.scalatest._
import org.scalatestplus.scalacheck.ScalaCheckDrivenPropertyChecks
class HelloSpec extends WordSpec with MustMatchers with ScalaCheckDrivenPropertyChecks {
private val c: C = new C()
"class C" must {
"add another class C" in {
forAll (Gen.choose(1, 100).map(new C(_))) { x =>
val expectedI = c.i + x.i
c.add(x)
c.i mustBe expectedI
}
}
}
}
Note how here forAll is on the "inside" of the test body which means we have a single test which is using multiple inputs provided by forAll to test the system C. When it is on the "outside" like so
forAll (Gen.choose(1, 100).map(new C(_))) { x =>
s"Adding $x to $c" must {
...
s"result in its .i property becoming $expectedI" in {
...
}
}
}
then forAll is misused to generate multiple tests where each has a single test input, however the purpose of forAll is to generate multiple inputs for system under test, not multiple tests. Furthermore, design of CUnitTest results in subsequent test depending on the state of the previous test which is buggy and harder to maintain. Ideally tests would run in isolation of each other where all the state needed is provided afresh as part of a test fixture.
Few side notes: #RunWith(classOf[JUnitRunner]) should not be necessary, and GeneratorDrivenPropertyChecks is deprecated.
Related
I am trying to learn how schedules work, and I just want to display a simple message to the console using a schedule.
import zio.console._
import zio._
import scala.util.Try
import zio.duration._
import zio.clock._
object MyApp extends zio.App {
def run(args: List[String]) =
myAppLogic.exitCode
val myAppLogic =
for {
sequential = Schedule.recurs(10) andThen Schedule.spaced(1.second)
_ <- ZIO.effectTotal{ "hello" }.retry(sequential)
} yield ()
}
I am getting this error:
This error handling operation assumes your effect can fail. However,
your effect has Nothing for the error type, which means it cannot
fail, so there is no need to handle the failure. To find out which
method you can use instead of this operation, please see the reference
chart at: https://zio.dev/docs/can_fail [error] _ <-
ZIO.effectTotal{ "hello" }.retry(sequential)
I am returning a Task[String] which has a throwable error type. If I replace effectTotal with effect then I don't get a compile error, but the program just exists without repeating the word hello even once.
What concept am I missing here?
First of all you can't have = in the first line of a for-comprehension. You need to wrap your pure value in ZIO.succeed or leave it outside the for-comprehension.
"Hello" is a pure string. You could use ZIO.effectTotal but technically it doesn't produce any side effect so you should also use succeed here.
retry retries on failure, but effectTotal / succeed cannot fail. You should be using repeat instead.
import zio._
import zio.duration._
object MyApp extends zio.App {
def run(args: List[String]) =
myAppLogic.exitCode
val myAppLogic =
for {
sequential <- ZIO.succeed(Schedule.recurs(10) andThen Schedule.spaced(1.second))
_ <- ZIO.succeed("hello").repeat(sequential)
} yield ()
}
Now with errors fixed it still doesn't display your texts because you are just producing values (not a side effect) without printing them (side effect).
The correct code should be something like:
import zio._
import zio.duration._
import zio.console._
object MyApp extends zio.App {
def run(args: List[String]) =
myAppLogic.exitCode
//leave this out of the for-comprehension
val sequential = Schedule.recurs(10) andThen Schedule.spaced(1.second)
val myAppLogic = putStrLn("hello").repeat(sequential)
// Or you could suspend the `println` side effect with `effectTotal` (doesn't fail but produces side effects)
// val myAppLogicAlt = ZIO.effectTotal(println("hello")).repeat(sequential)
}
Gen.sequence seems to be ignoring size of given Traversable. Is that by design? I'm using version 1.14.0 with Scala 2.13. Following generator
Gen.sequence[List[Int], Int](List.fill(3)(Gen.const(5)))
sometimes generates List of size 1. What am I missing ?
Sample test
import org.scalacheck.Gen
import org.scalatest.concurrent.ScalaFutures
import org.scalatest.{Assertions, FlatSpec, Inside, Inspectors, Matchers, OptionValues}
import org.scalatestplus.scalacheck.ScalaCheckPropertyChecks
class RuleSpec extends FlatSpec with Matchers with Assertions with OptionValues with Inside with Inspectors with ScalaFutures with ScalaCheckPropertyChecks {
val filterGen = Gen.listOfN(3, Gen.const(5))
val inputGen = Gen.pick(10, 5 to 15).map(_.toList).filter(_.nonEmpty)
"A HasAny rule with partially matching filter" should "validate input" in {
forAll(filterGen, inputGen) { case (filter, input) =>
val result = HasAny(filter: _*).valid(input)
println(s"$result: $filter ${input}\n")
result should be(true)
}
}
}
This might be due to Test Case Minimisation
One interesting feature of ScalaCheck is that if it finds an argument
that falsifies a property, it tries to minimise that argument before
it is reported. This is done automatically when you use the
Prop.property and Prop.forAll methods to create properties, but not if
you use Prop.forAllNoShrink.
Hence try using Prop.forAllNoShrink like so
Prop.forAllNoShrink(filterGen, inputGen) { case (filter, input) => ...
If using ScalaTest's forAll then try the suggestion in Add support for not shrinking values in GeneratorDrivenPropertyChecks #584 by creating the following trait
trait NoShrink {
implicit def noShrink[A]: Shrink[A] = Shrink(_ => Stream.empty)
}
and mix it in the spec like so
class RuleSpec extends FlatSpec ... with ScalaCheckPropertyChecks with NoShrink {
...
forAll(filterGen, inputGen) { case (filter, input) => ...
I would expect this test to pass:
import org.scalamock.scalatest.MockFactory
import org.scalatest.{FlatSpec, Matchers}
class FruitImpl {
case class FruitName(name: String)
def getFruitName: Option[FruitName] = {
Some(FruitName("apple"))
}
}
class FruitSpec extends FlatSpec with Matchers with MockFactory {
val f = mock[FruitImpl]
(f.getFruitName _).expects().returning(None)
behavior of "getFruitName method"
it should "return None" in {
f.getFruitName should === (None)
}
}
But it fails with:
[error] my/path/QuestionTest.scala:13: overriding method getFruitName in class FruitImpl of type => Option[this.FruitName];
[error] method getFruitName has incompatible type
[error] val f = mock[FruitImpl]
[error] ^
This works, though:
import org.scalamock.scalatest.MockFactory
import org.scalatest.{FlatSpec, Matchers}
case class FruitName(name: String)
class FruitImpl {
def getFruitName: Option[FruitName] = {
Some(FruitName("apple"))
}
}
class FruitSpec extends FlatSpec with Matchers with MockFactory {
val f = mock[FruitImpl]
(f.getFruitName _).expects().returning(None)
behavior of "getFruitName method"
it should "return None" in {
f.getFruitName should === (None)
}
}
The only difference is that the case class FruitName is defined outside of the class FruitImpl. Why does one version of the code fails and the other doesn't? What should one do to fix the error in the first example?
Without looking at the ScalaMock code, I'd say that the mock is not a true derivation of FruitImpl in the OO sense. Its purpose is to allow method interception, so it only deals with the facade. It follows then that the mock actually has no definition of the path dependent type FruitName, and so cannot work with a method signature that depends on it.
This is precisely why it does work when the FruitName definition is moved out of FruitImpl. It now exists independently of the mock, who's method signatures depending on it then work as expected.
I want to test my code with Property based testing style.
I have some generator, that always generates Some(data), the data is never filtered out.
I want to run let's say 100 tests, with the same set of data across the runs. Now, I'm doing it like this, but it is not very idiomatic code:
import org.scalacheck._
import java.util.Random
import org.scalatest._
class ExplainItOnStackOverflow extends PropSpec {
property("Just a test property to explain my problem!") {
val g = Gen.choose(1,100)
for (i <- 1 to 100) {
val Some(res) = g.apply(Gen.Params(rng = new Random(i)))
assert (res > 0)
}
}
}
The class PropSpec has it's own method runAll, it seems to me like I should want to use it to run these property checks.
I'm using specs2 (v1.8.2) on with Scala (v2.9.1) to write acceptance tests. Following the example at http://etorreborre.github.com/specs2/guide/org.specs2.guide.SpecStructure.html#Contexts, I have the following Specification and context case class:
import org.specs2._
class testspec extends SpecificationWithJUnit { def is =
"test should" ^
"run a test in a context" ! context().e1
}
case class context() {
def e1 = 1 must beEqualTo(1)
}
I get a compiler error:
error: value must is not a member of Int def e1 = 1 must beEqualTo(1)
when compiling the context case class.
Obviously I'm new to specs2 (and to Scala). References to the appropriate documentation would be greatly appreciated.
Obviously the doc is wrong (was wrong, I fixed it now).
The correct way to write declare the case class for a context is usually to include it in the Specification scope:
import org.specs2._
class ContextSpec extends Specification { def is =
"this is the first example" ! context().e1
case class context() {
def e1 = List(1,2,3) must have size(3)
}
}
Otherwise if you want to reuse the context in another specification, you can, as Dario wrote, access the MustMatchers functionalities either by importing the MustMatchers object methods or by inheriting the MustMatchers trait.
must is not a member of Int, because to "must" is not known in the context of your "context" class. Put the method "e1" inside your specification class and it should work. E.g.
import org.specs2._
class TestSpec extends Specification { def is =
"test should" ^
"run a test in a context" ! e1 ^
end
def e1 = 1 must beEqualTo(1)
}
edit
Ah, I see what you want ;-). This should work like this:
To have the matchers in the scope of the context class you have to import the MustMatchers.
import org.specs2._
import matcher.MustMatchers._
class ContextSpec extends Specification { def is =
"this is the first example" ! context().e1
}
case class context() {
def e1 = List(1,2,3) must have size(3)
}