ScalaCheck shrinking command data in stateful testing - scala

Question

When doing stateful testing with ScalaCheck the library can shrink the commands needed to find a certain bug. Like in the counter example from the userguide: https://github.com/typelevel/scalacheck/blob/master/doc/UserGuide.md. But what if the commands took arguments and i wanted ScalaCheck to shrink the data inside the commands as well? See the scenario where i am testing the Counter below:
package Counter
case class Counter() {
private var n = 1
def increment(incrementAmount: Int) = {
if (n%100!=0) {
n += incrementAmount
}
}
def get(): Int = n
}
The counter is programmed with a bug. It should not increment with the given amount if n%100 == 0. So if the value of n is x*100, where x is any positive integer, the counter is not incremented. I am testing the counter with the ScalaCheck stateful test below:
import Counter.Counter
import org.scalacheck.commands.Commands
import org.scalacheck.{Gen, Prop}
import scala.util.{Success, Try}
object CounterCommands extends Commands {
type State = Int
type Sut = Counter
def canCreateNewSut(newState: State, initSuts: Traversable[State],
runningSuts: Traversable[Sut]): Boolean = true
def newSut(state: State): Sut = new Counter
def destroySut(sut: Sut): Unit = ()
def initialPreCondition(state: State): Boolean = true
def genInitialState: Gen[State] = Gen.const(1)
def genCommand(state: State): Gen[Command] = Gen.oneOf(Increment(Gen.chooseNum(1, 200000).sample.get), Get)
case class Increment(incrementAmount: Int) extends UnitCommand {
def run(counter: Sut) = counter.increment(incrementAmount)
def nextState(state: State): State = {state+incrementAmount}
def preCondition(state: State): Boolean = true
def postCondition(state: State, success: Boolean) = success
}
case object Get extends Command {
type Result = Int
def run(counter: Sut): Result = counter.get()
def nextState(state: State): State = state
def preCondition(state: State): Boolean = true
def postCondition(state: State, result: Try[Int]): Prop = result == Success(state)
}
}
Everytime the increment command is chosen it is given some arbitrary integer between 1 and 200000 as argument. Running the test gave the following output:
! Falsified after 28 passed tests.
> Labels of failing property:
initialstate = 1
seqcmds = (Increment(1); Increment(109366); Increment(1); Increment(1); Inc
rement(104970); Increment(27214); Increment(197045); Increment(1); Increm
ent(54892); Get => 438600)
> ARG_0: Actions(1,List(Increment(1), Increment(109366), Increment(1), Incr
ement(1), Increment(104970), Increment(27214), Increment(197045), Increme
nt(1), Increment(54892), Get),List())
> ARG_0_ORIGINAL: Actions(1,List(Get, Get, Increment(1), Increment(109366),
Get, Get, Get, Get, Increment(1), Get, Increment(1), Increment(104970),
Increment(27214), Get, Increment(197045), Increment(1), Increment(54892),
Get, Get, Get, Get, Get, Increment(172491), Get, Increment(6513), Get, I
ncrement(57501), Increment(200000)),List())
ScalaCheck did indeed shrink the commands needed to find the bug (as can be seen in ARG_0) but it did not shrink the data inside the commands. It ended up with a much larger counter value (438600) than what is actually needed to find the bug. If the first increment command was given 99 as argument the bug would be found.
Is there any way in ScalaCheck to shrink the data inside the commands when running stateful tests? The ScalaCheck version used is 1.14.1.
EDIT:
I tried simplifying the bug (and only incrementing if n!=10) and added the shrinker that was suggested by Levi and could still not get it to work. The whole runnable code can be seen below:
package LocalCounter
import org.scalacheck.commands.Commands
import org.scalacheck.{Gen, Prop, Properties, Shrink}
import scala.util.{Success, Try}
case class Counter() {
private var n = 1
def increment(incrementAmount: Int) = {
if (n!=10) {
n += incrementAmount
}
}
def get(): Int = n
}
object CounterCommands extends Commands {
type State = Int
type Sut = Counter
def canCreateNewSut(newState: State, initSuts: Traversable[State],
runningSuts: Traversable[Sut]): Boolean = true
def newSut(state: State): Sut = new Counter
def destroySut(sut: Sut): Unit = ()
def initialPreCondition(state: State): Boolean = true
def genInitialState: Gen[State] = Gen.const(1)
def genCommand(state: State): Gen[Command] = Gen.oneOf(Increment(Gen.chooseNum(1, 40).sample.get), Get)
case class Increment(incrementAmount: Int) extends UnitCommand {
def run(counter: Sut) = counter.increment(incrementAmount)
def nextState(state: State): State = {state+incrementAmount}
def preCondition(state: State): Boolean = true
def postCondition(state: State, success: Boolean) = success
}
case object Get extends Command {
type Result = Int
def run(counter: Sut): Result = counter.get()
def nextState(state: State): State = state
def preCondition(state: State): Boolean = true
def postCondition(state: State, result: Try[Int]): Prop = result == Success(state)
}
implicit val shrinkCommand: Shrink[Command] = Shrink({
case Increment(amt) => Shrink.shrink(amt).map(Increment(_))
case Get => Stream.empty
})
}
object CounterCommandsTest extends Properties("CounterCommands") {
CounterCommands.property().check()
}
Running the code gave the following output:
! Falsified after 4 passed tests.
> Labels of failing property:
initialstate = 1
seqcmds = (Increment(9); Increment(40); Get => 10)
> ARG_0: Actions(1,List(Increment(9), Increment(40), Get),List())
> ARG_0_ORIGINAL: Actions(1,List(Increment(9), Increment(34), Increment(40)
, Get),List())
Which is not the minimal example.

Answer 1

You should be able to define a custom Shrink for Command along these lines:
implicit val shrinkCommand: Shrink[Command] = Shrink({
case Increment(amt) => shrink(amt).map(Increment(_))
case Get => Stream.empty
}
Note that, because Stream is deprecated in Scala 2.13, you may need to disable warnings in Scala 2.13 (Scalacheck 1.15 will allow LazyList to be used to define shrinks).