In the Scalacheck documentation for stateful testing an ATM maschine is mentioned as a use case. For it to work, the commands need parameters, for example the PIN or the withdrawal amount. In the given example the methods in the class Counter don't have parameters.
Now my question is how I could test a method like this in scalachecks stateful testing:
class Counter {
private var n = 0
def inc(i: Int) = n += i
...
}
The run and nextState methods of a command don't offer a parameter. Adding a Random.nextInt wouldn't work, because in run and nextState the value would differ and the test fails:
case object Inc extends UnitCommand {
def run(sut: Sut): Unit = sut.inc(Random.nextInt)
def nextState(state: State): State = state + Random.nextInt
...
}
Is there any way to pass a parameter to the Sut?
As you may notice from how genCommand, ScalaCheck Commands actually does something like a Cartesian product between initial states generated by genInitialState and series of commands generated by genCommand. So if some of your commands actually need a parameter, you need to convert them into classes from objects and provide a Gen for them. So modifying the example from the docs you will need something like this:
/** A generator that, given the current abstract state, should produce
* a suitable Command instance. */
def genCommand(state: State): Gen[Command] = {
val incGen = for (v <- arbitrary[Int]) yield Inc(v)
val decGen = for (v <- arbitrary[Int]) yield Dec(v)
Gen.oneOf(incGen, decGen, Gen.oneOf(Get, Reset))
}
// A UnitCommand is a command that doesn't produce a result
case class Inc(dif: Int) extends UnitCommand {
def run(sut: Sut): Unit = sut.inc(dif)
def nextState(state: State): State = state + dif
// This command has no preconditions
def preCondition(state: State): Boolean = true
// This command should always succeed (never throw an exception)
def postCondition(state: State, success: Boolean): Prop = success
}
case class Dec(dif: Int) extends UnitCommand {
def run(sut: Sut): Unit = sut.dec(dif)
def nextState(state: State): State = state - dif
def preCondition(state: State): Boolean = true
def postCondition(state: State, success: Boolean): Prop = success
}
Note that if your parameters are just constants rather than variables (as is in the case of PIN-code), you should either hard-code them in the commands or make the whole specification class rather than object and pass those parameters from the outside.
Related
When doing stateful testing with ScalaCheck the library can shrink the commands needed to find a certain bug. Like in the counter example from the userguide: https://github.com/typelevel/scalacheck/blob/master/doc/UserGuide.md. But what if the commands took arguments and i wanted ScalaCheck to shrink the data inside the commands as well? See the scenario where i am testing the Counter below:
package Counter
case class Counter() {
private var n = 1
def increment(incrementAmount: Int) = {
if (n%100!=0) {
n += incrementAmount
}
}
def get(): Int = n
}
The counter is programmed with a bug. It should not increment with the given amount if n%100 == 0. So if the value of n is x*100, where x is any positive integer, the counter is not incremented. I am testing the counter with the ScalaCheck stateful test below:
import Counter.Counter
import org.scalacheck.commands.Commands
import org.scalacheck.{Gen, Prop}
import scala.util.{Success, Try}
object CounterCommands extends Commands {
type State = Int
type Sut = Counter
def canCreateNewSut(newState: State, initSuts: Traversable[State],
runningSuts: Traversable[Sut]): Boolean = true
def newSut(state: State): Sut = new Counter
def destroySut(sut: Sut): Unit = ()
def initialPreCondition(state: State): Boolean = true
def genInitialState: Gen[State] = Gen.const(1)
def genCommand(state: State): Gen[Command] = Gen.oneOf(Increment(Gen.chooseNum(1, 200000).sample.get), Get)
case class Increment(incrementAmount: Int) extends UnitCommand {
def run(counter: Sut) = counter.increment(incrementAmount)
def nextState(state: State): State = {state+incrementAmount}
def preCondition(state: State): Boolean = true
def postCondition(state: State, success: Boolean) = success
}
case object Get extends Command {
type Result = Int
def run(counter: Sut): Result = counter.get()
def nextState(state: State): State = state
def preCondition(state: State): Boolean = true
def postCondition(state: State, result: Try[Int]): Prop = result == Success(state)
}
}
Everytime the increment command is chosen it is given some arbitrary integer between 1 and 200000 as argument. Running the test gave the following output:
! Falsified after 28 passed tests.
> Labels of failing property:
initialstate = 1
seqcmds = (Increment(1); Increment(109366); Increment(1); Increment(1); Inc
rement(104970); Increment(27214); Increment(197045); Increment(1); Increm
ent(54892); Get => 438600)
> ARG_0: Actions(1,List(Increment(1), Increment(109366), Increment(1), Incr
ement(1), Increment(104970), Increment(27214), Increment(197045), Increme
nt(1), Increment(54892), Get),List())
> ARG_0_ORIGINAL: Actions(1,List(Get, Get, Increment(1), Increment(109366),
Get, Get, Get, Get, Increment(1), Get, Increment(1), Increment(104970),
Increment(27214), Get, Increment(197045), Increment(1), Increment(54892),
Get, Get, Get, Get, Get, Increment(172491), Get, Increment(6513), Get, I
ncrement(57501), Increment(200000)),List())
ScalaCheck did indeed shrink the commands needed to find the bug (as can be seen in ARG_0) but it did not shrink the data inside the commands. It ended up with a much larger counter value (438600) than what is actually needed to find the bug. If the first increment command was given 99 as argument the bug would be found.
Is there any way in ScalaCheck to shrink the data inside the commands when running stateful tests? The ScalaCheck version used is 1.14.1.
EDIT:
I tried simplifying the bug (and only incrementing if n!=10) and added the shrinker that was suggested by Levi and could still not get it to work. The whole runnable code can be seen below:
package LocalCounter
import org.scalacheck.commands.Commands
import org.scalacheck.{Gen, Prop, Properties, Shrink}
import scala.util.{Success, Try}
case class Counter() {
private var n = 1
def increment(incrementAmount: Int) = {
if (n!=10) {
n += incrementAmount
}
}
def get(): Int = n
}
object CounterCommands extends Commands {
type State = Int
type Sut = Counter
def canCreateNewSut(newState: State, initSuts: Traversable[State],
runningSuts: Traversable[Sut]): Boolean = true
def newSut(state: State): Sut = new Counter
def destroySut(sut: Sut): Unit = ()
def initialPreCondition(state: State): Boolean = true
def genInitialState: Gen[State] = Gen.const(1)
def genCommand(state: State): Gen[Command] = Gen.oneOf(Increment(Gen.chooseNum(1, 40).sample.get), Get)
case class Increment(incrementAmount: Int) extends UnitCommand {
def run(counter: Sut) = counter.increment(incrementAmount)
def nextState(state: State): State = {state+incrementAmount}
def preCondition(state: State): Boolean = true
def postCondition(state: State, success: Boolean) = success
}
case object Get extends Command {
type Result = Int
def run(counter: Sut): Result = counter.get()
def nextState(state: State): State = state
def preCondition(state: State): Boolean = true
def postCondition(state: State, result: Try[Int]): Prop = result == Success(state)
}
implicit val shrinkCommand: Shrink[Command] = Shrink({
case Increment(amt) => Shrink.shrink(amt).map(Increment(_))
case Get => Stream.empty
})
}
object CounterCommandsTest extends Properties("CounterCommands") {
CounterCommands.property().check()
}
Running the code gave the following output:
! Falsified after 4 passed tests.
> Labels of failing property:
initialstate = 1
seqcmds = (Increment(9); Increment(40); Get => 10)
> ARG_0: Actions(1,List(Increment(9), Increment(40), Get),List())
> ARG_0_ORIGINAL: Actions(1,List(Increment(9), Increment(34), Increment(40)
, Get),List())
Which is not the minimal example.
You should be able to define a custom Shrink for Command along these lines:
implicit val shrinkCommand: Shrink[Command] = Shrink({
case Increment(amt) => shrink(amt).map(Increment(_))
case Get => Stream.empty
}
Note that, because Stream is deprecated in Scala 2.13, you may need to disable warnings in Scala 2.13 (Scalacheck 1.15 will allow LazyList to be used to define shrinks).
When incrementing concurrent counters by key in ConcurrentHashMap is it safe to use regular Int for value or do we have to use AtomicInteger? For example consider the following two implementations
ConcurrentHashMap[String, Int]
final class ExpensiveMetrics(implicit system: ActorSystem, ec: ExecutionContext) {
import scala.collection.JavaConverters._
private val chm = new ConcurrentHashMap[String, Int]().asScala
system.scheduler.schedule(5.seconds, 60.seconds)(publishAllMetrics())
def countRequest(key: String): Unit =
chm.get(key) match {
case Some(value) => chm.update(key, value + 1)
case None => chm.update(key, 1)
}
private def resetCount(key: String) = chm.replace(key, 0)
private def publishAllMetrics(): Unit =
chm foreach { case (key, value) =>
// publishMetric(key, value.doubleValue())
resetCount(key)
}
}
ConcurrentHashMap[String, AtomicInteger]
final class ExpensiveMetrics(implicit system: ActorSystem, ec: ExecutionContext) {
import scala.collection.JavaConverters._
private val chm = new ConcurrentHashMap[String, AtomicInteger]().asScala
system.scheduler.schedule(5.seconds, 60.seconds)(publishAllMetrics())
def countRequest(key: String): Unit =
chm.getOrElseUpdate(key, new AtomicInteger(1)).incrementAndGet()
private def resetCount(key: String): Unit =
chm.getOrElseUpdate(key, new AtomicInteger(0)).set(0)
private def publishAllMetrics(): Unit =
chm foreach { case (key, value) =>
// publishMetric(key, value.doubleValue())
resetCount(key)
}
}
Is the former implementation safe? If not, at what point in the snippet can race-condition be introduced and why?
The context of the question are AWS CloudWatch metrics which can get very expensive on high-frequency APIs if posted on each request. So I am trying to "batch" them up and publish them periodically.
The first implementation is not correct, because the countRequest method is not atomic. Consider this sequence of events:
Threads A and B both call countRequest with key "foo"
Thread A obtains the counter value, let's call it x
Thread B obtains the counter value. It's the same value x, because Thread A hasn't updated the counter yet.
Thread B updates the map with the new counter value, x+1
Thread A updates the map, and because it obtained the counter value before B wrote the new counter value, it also writes x+1.
The counter should be x+2, but it is x+1. It's a classic lost update problem.
The second implementation has a similar problem due to the use of the `getOrElseUpdate` method. `ConcurrentHashMap` does not have that method, therefore the Scala wrapper needs to emulate it. I think the implementation is the one inherited from `scala.collection.mutable.MapOps`, and it is defined like so:
```
def getOrElseUpdate(key: K, op: => V): V =
get(key) match {
case Some(v) => v
case None => val d = op; this(key) = d; d
}
```
This is obviously not atomic.
To implement this correctly, use the compute method on ConcurrentHashMap.
This method will execute atomically, so you won't need an AtomicInteger.
I am creating in Scala and Cats a function that does some I/O and that will be called by other parts of the code. I'm also learning Cats and I want my function to:
Be generic in its effect and use a F[_]
Run on a dedicated thread pool
I want to introduce async boundaries
I assume that all my functions are generic in F[_] up to the main method because I'm trying to follow these Cat's guidelines
But I struggle to make these constraint to work by using ContextShift or ExecutionContext. I have written a full example here and this is an exctract from the example:
object ComplexOperation {
// Thread pool for ComplexOperation internal use only
val cs = IO.contextShift(
ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor())
)
// Complex operation that takes resources and time
def run[F[_]: Sync](input: String): F[String] =
for {
r1 <- Sync[F].delay(cs.shift) *> op1(input)
r2 <- Sync[F].delay(cs.shift) *> op2(r1)
r3 <- Sync[F].delay(cs.shift) *> op3(r2)
} yield r3
def op1[F[_]: Sync](input: String): F[Int] = Sync[F].delay(input.length)
def op2[F[_]: Sync](input: Int): F[Boolean] = Sync[F].delay(input % 2 == 0)
def op3[F[_]: Sync](input: Boolean): F[String] = Sync[F].delay(s"Complex result: $input")
}
This clearly doesn't abstract over effects as ComplexOperation.run needs a ContextShift[IO] to be able to introduce async boundaries. What is the right (or best) way of doing this?
Creating ContextShift[IO] inside ComplexOperation.run makes the function depend on IO which I don't want.
Moving the creation of a ContextShift[IO] on the caller will simply shift the problem: the caller is also generic in F[_] so how does it obtain a ContextShift[IO] to pass to ComplexOperation.run without explicitly depending on IO?
Remember that I don't want to use one global ContextShift[IO] defined at the topmost level but I want each component to decide for itself.
Should my ComplexOperation.run create the ContextShift[IO] or is it the responsibility of the caller?
Am I doing this right at least? Or am I going against standard practices?
So I took the liberty to rewrite your code, hope it helps:
import cats.effect._
object Functions {
def sampleFunction[F[_]: Sync : ContextShift](file: String, blocker: Blocker): F[String] = {
val handler: Resource[F, Int] =
Resource.make(
blocker.blockOn(openFile(file))
) { handler =>
blocker.blockOn(closeFile(handler))
}
handler.use(handler => doWork(handler))
}
private def openFile[F[_]: Sync](file: String): F[Int] = Sync[F].delay {
println(s"Opening file $file with handler 2")
2
}
private def closeFile[F[_]: Sync](handler: Int): F[Unit] = Sync[F].delay {
println(s"Closing file handler $handler")
}
private def doWork[F[_]: Sync](handler: Int): F[String] = Sync[F].delay {
println(s"Calculating the value on file handler $handler")
"The final value"
}
}
object Main extends IOApp {
override def run(args: List[String]): IO[ExitCode] = {
val result = Blocker[IO].use { blocker =>
Functions.sampleFunction[IO](file = "filePath", blocker)
}
for {
data <- result
_ <- IO(println(data))
} yield ExitCode.Success
}
}
You can see it running here.
So, what does this code does.
First, it creates a Resource for the file, since close has to be done, even on guarantee or on failure.
It is using Blocker to run the open and close operations on a blocking thread poo (that is done using ContextShift).
Finally, on the main, it creates a default Blocker for instance, for **IO*, and uses it to call your function; and prints the result.
Fell free to ask any question.
There is some data that I have pulled from a remote API, for which I use a Future-style interface. The data is structured as a linked-list. A relevant example data container is shown below.
case class Data(information: Int) {
def hasNext: Boolean = ??? // Implemented
def next: Future[Data] = ??? // Implemented
}
Now I'm interested in adding some functionality to the data class, such as map, foreach, reduce, etc. To do so I want to implement some form of IterableLike such that it inherets these methods.
Given below is the trait Data may extend, such that it gets this property.
trait AsyncIterable[+T]
extends IterableLike[Future[T], AsyncIterable[T]]
{
def hasNext : Boolean
def next : Future[T]
// How to implement?
override def iterator: Iterator[Future[T]] = ???
override protected[this] def newBuilder: mutable.Builder[Future[T], AsyncIterable[T]] = ???
override def seq: TraversableOnce[Future[T]] = ???
}
It should be a non-blocking implementation, which when acted on, starts requesting the next data from the remote data source.
It is then possible to do cool stuff such as
case class Data(information: Int) extends AsyncIterable[Data]
val data = Data(1) // And more, of course
// Asynchronously print all the information.
data.foreach(data => println(data.information))
It is also acceptable for the interface to be different. But the result should in some way represent asynchronous iteration over the collection. Preferably in a way that is familiar to developers, as it will be part of an (open source) library.
In production I would use one of following:
Akka Streams
Reactive Extensions
For private tests I would implement something similar to following.
(Explanations are below)
I have modified a little bit your Data:
abstract class AsyncIterator[T] extends Iterator[Future[T]] {
def hasNext: Boolean
def next(): Future[T]
}
For it we can implement this Iterable:
class AsyncIterable[T](sourceIterator: AsyncIterator[T])
extends IterableLike[Future[T], AsyncIterable[T]]
{
private def stream(): Stream[Future[T]] =
if(sourceIterator.hasNext) {sourceIterator.next #:: stream()} else {Stream.empty}
val asStream = stream()
override def iterator = asStream.iterator
override def seq = asStream.seq
override protected[this] def newBuilder = throw new UnsupportedOperationException()
}
And if see it in action using following code:
object Example extends App {
val source = "Hello World!";
val iterator1 = new DelayedIterator[Char](100L, source.toCharArray)
new AsyncIterable(iterator1).foreach(_.foreach(print)) //prints 1 char per 100 ms
pause(2000L)
val iterator2 = new DelayedIterator[String](100L, source.toCharArray.map(_.toString))
new AsyncIterable(iterator2).reduceLeft((fl: Future[String], fr) =>
for(l <- fl; r <- fr) yield {println(s"$l+$r"); l + r}) //prints 1 line per 100 ms
pause(2000L)
def pause(duration: Long) = {println("->"); Thread.sleep(duration); println("\n<-")}
}
class DelayedIterator[T](delay: Long, data: Seq[T]) extends AsyncIterator[T] {
private val dataIterator = data.iterator
private var nextTime = System.currentTimeMillis() + delay
override def hasNext = dataIterator.hasNext
override def next = {
val thisTime = math.max(System.currentTimeMillis(), nextTime)
val thisValue = dataIterator.next()
nextTime = thisTime + delay
Future {
val now = System.currentTimeMillis()
if(thisTime > now) Thread.sleep(thisTime - now) //Your implementation will be better
thisValue
}
}
}
Explanation
AsyncIterable uses Stream because it's calculated lazily and it's simple.
Pros:
simplicity
multiple calls to iterator and seq methods return same iterable with all items.
Cons:
could lead to memory overflow because stream keeps all prevously obtained values.
first value is eagerly gotten during creation of AsyncIterable
DelayedIterator is very simplistic implementation of AsyncIterator, don't blame me for quick and dirty code here.
It's still strange for me to see synchronous hasNext and asynchronous next()
Using Twitter Spool I've implemented a working example.
To implement spool I modified the example in the documentation.
import com.twitter.concurrent.Spool
import com.twitter.util.{Await, Return, Promise}
import scala.concurrent.{ExecutionContext, Future}
trait AsyncIterable[+T <: AsyncIterable[T]] { self : T =>
def hasNext : Boolean
def next : Future[T]
def spool(implicit ec: ExecutionContext) : Spool[T] = {
def fill(currentPage: Future[T], rest: Promise[Spool[T]]) {
currentPage foreach { cPage =>
if(hasNext) {
val nextSpool = new Promise[Spool[T]]
rest() = Return(cPage *:: nextSpool)
fill(next, nextSpool)
} else {
val emptySpool = new Promise[Spool[T]]
emptySpool() = Return(Spool.empty[T])
rest() = Return(cPage *:: emptySpool)
}
}
}
val rest = new Promise[Spool[T]]
if(hasNext) {
fill(next, rest)
} else {
rest() = Return(Spool.empty[T])
}
self *:: rest
}
}
Data is the same as before, and now we can use it.
// Cool stuff
implicit val ec = scala.concurrent.ExecutionContext.global
val data = Data(1) // And others
// Print all the information asynchronously
val fut = data.spool.foreach(data => println(data.information))
Await.ready(fut)
It will trow an exception on the second element, because the implementation of next was not provided.
I can't seem to figure out how to chain together these functions, any help or advice would be appreciated.
// Generic approach to adding flags to a command string
trait UpdateCommandString {
def update[T](option: Option[T], flagName: String)(implicit command: String): String = {
if (option.isEmpty)
command
else if (option.get.isInstanceOf[Boolean]) {
if (option.get.asInstanceOf[Boolean])
s"$command $flagName"
command
} else
s"$command $flagName ${option.get.asInstanceOf[String]}"
}
}
// One example of flags (the program I'm using has literally 50+ flags
// so there will be a number of case classes that group them into related
// sets)
case class Flags(cache: Option[String] = None,
errorlog: Option[String] = None,
accesslog: Option[String] = None,
verbose: Option[Boolean] = Some(false),
auth: Option[Boolean] = Some(false)) extends UpdateCommandString {
def applyToCommand(implicit command: String): String = {
// These seem to apply separately, but I want to chain
// them together!
update(cache, "-cache")
update(errorlog, "-error")
update(accesslog, "-access")
update(auth, "-do-auth")
}
}
// An example of what I'm trying to do
// Given a base command string and a bunch of case classes to apply
// to that string, I'd like to be able to call applyToCommand and
// get back the modified command string
var command = "run_system"
val f = Flags(Some("asdfasdf"), None, None, Some(true), Some(false))
command = f.applyToCommand(command)
I would recommend a complete redesign of your current approach.
Every member of your Flags class should be it's own case class, extending a common Flag class.
So you can define functions to combine different flags to one configuration. This configuration can than, in a final step, be used to build your result string.
abstract class Flag(name: String, parameter : Option[String])
case class Cache(parameter : Option[String]) extends Flag("-cache", parameter)
case class ErrorLog(parameter : Option[String]) extends Flag("-errorlog", parameter)
//...
type Config = List[Flag]
def applyToCommand(commandName : String, config : Config) = {
def buildString(f:Flag) =
s" $f.name${f.parameter.map(" " ++ _).getOrElse("")}"
val flagsString = config.map(buildString).mkString("")
s"$commandName" ++ flagString
}
//Now you can it simply use it as I described above
val config = List(Cache(Some("asdf")), ErrorLog(None))
applyToCommand("run_system", config)
This makes your code more flexible and easier to refactor.
At last here are some advises how you could modify this design to better fit your needs:
If you need to group your flags, you can put them in objects or separate files. Or if you want to change their behavior based on the group you can enhance the class hierarchy and add an intermediate layer.
You can move the parameter from Flag down to the case classes, so every Flag can define if it needs parameters, if yes how many and if those are optional or not.
You could also implement buildString at the case classes so every flag can decide how to format it self.
If you want do add new Flags you simply add a new class and that's it, no need to add anything to an unrelated class.
As explained #bmaderbacher, I think you should separate the different flags in the different case class.
But to answer your question, you should modify applyToCommand:
def applyToCommand(implicit command: String): String = {
var s = update(cache, "-cache")(command)
s = update(errorlog, "-error")(s)
s = update(accesslog, "-access")(s)
s = update(auth, "-do-auth")(s)
s
}
At this point it should be clear that you didn't make the right choice for your Flag class.
I'll do something like that:
trait Flag {
def toString: String
}
case class Command(value: String) {
def add(flag: Flag) = Command(value + ' ' + flag.toString)
def +(flag: Flag) = add(flag)
}
case class Cache(size: Int) extends Flag {
def toString = s"--cache $size"
}
case object Auth extends Flag {
def toString = "--auth"
}
Now you can do something like:
val command = Command("run") + Cache(500) + Auth