I have a org.scalatest.FunSpec with org.scalatest.Matchers test that does the following e.g.
val tol = 1e-10
val res = 1.000000000000001
val ref = 1.000000000000000
res should be (ref +- tol)
but it does so in a loop for multiple cases keyed by name, of course I can't change the granularity of the tested code so I get a collection of values with those names associated to them. Therefore for a test above I need to place an extra context or extra description name to reflect to which name it applies. I need something like:
val name : String = ...
res should be (ref +- tol) for name
I can't use it and describe at this point because they are already outside.
It really depends on what you're trying to do, and you should probably add a more complete example of what you're trying to achieve, but you could use describe in the loop. For example:
class TempTest extends FunSpec with Matchers {
describe("Some example test") {
(1 to 10).foreach { i => // your loop here
describe(s"Scenario $i") {
it("should be equal to itself") {
i shouldBe i
}
}
}
}
}
UPDATE: You could use withClue to add more context to the matcher e.g.:
withClue("Some clarifying message") {
i shouldBe 5
}
This will add the clue string to the error if the conditions fails.
Probably GivenWhenThen can be used in order to add context to the tests reporting. I am not sure how exactly you wrap your multiple tests in the loop, but here is the idea:
import org.scalatest.{GivenWhenThen, WordSpec}
/**
* Created by alex on 10/3/16.
*/
class Temp extends WordSpec with GivenWhenThen{
val names = List("Alex", "Dana")
for(name <- names)yield{
"Reversing a name " + name + " two times" should {
"result in the same name" in{
Given("name " + name)
When("reversed two times")
val reversed = name.reverse.reverse
Then("it should be the same")
assert(name === reversed)
}
}
}
}
Related
I am writing some Specs2 specifications; that looks like:
class ComponentSpecification extends Specification with Mockito {
private val dependency = mock[Dependency]
private val subject = new Component(..)
"methodOne" should {
"handle happy path" in {
val result = subject.methodOne("Param1", 42)
result must ...
there was one(dependency).something()
}
"deal with border case" in {
val result = subject.methodOne("", -1)
result must ...
there was one(dependency).something()
}
}
}
However, those tests fails because the mock[Dependency] is shared.
One solution would be to make them sequential and reset the mock before each test but this look odd and as written in the doc about "Parallel by default":
it encourages to write independent examples when the result of a given example should not be influenced by others
Another would be to move the val to the test itself. But while I should be able to reduce the duplication with this still looks like a strange structure. And looks like the subject is stateful while it should not.
I can also try to use a less strict approach by verifying with there was atLestOne(dependency).something() but:
this does not validate that the method was called in this specific test case and
argument capture and validation is painful.
So my question is:
How can I create readable tests with detailed verifications on mock.
Thanks a lot.
Scopes can provide fresh state to each test like so
class ComponentSpecification extends mutable.Specification with Mockito {
trait FooScope extends Scope {
val dependency = mock[Dependency]
val subject = new Component(dependency)
}
"methodOne" should {
"handle happy path" in new FooScope {
val result = subject.methodOne("Param1", 42)
there was one(dependency).something()
}
"deal with border case" in new FooScope {
val result = subject.methodOne("", -1)
there was one(dependency).something()
}
}
}
where there is no need to reset the mock before each test.
I was going to accept the answer from #Mario Galic. However, regarding the comment of #Eric (author of Specs2) I ended with a method that creates the context as expected but removes duplication. By using pattern matching I extract the interesting part :
class ComponentSpecification extends mutable.Specification with Mockito {
def givenOneCallToMethodOneWithDependency(s:String, i:Int):(Result, Dependency) = {
val dependency = mock[Dependency]
val subject = new Component(dependency)
val result = subject.methodOne(s, i)
(result, dependency)
}
"methodOne" should {
"handle happy path" in new FooScope {
val (result, dependency) = givenOneCallToMethodOneWithDependency("Param1", 42)
there was one(dependency).something()
}
"deal with border case" in new FooScope {
val (result, dependency) = givenOneCallToMethodOneWithDependency("", -1)
there was one(dependency).something()
}
}
}
Consider the following code:
import chisel3._
import chisel3.util._
import chisel3.iotesters._
class Inverter extends Module {
val io = IO(new Bundle {
val a = Input(UInt(4.W))
val s = Output(UInt(4.W))
})
io.s := ~io.a
}
class InverterTester(c: Inverter) extends PeekPokeTester(c) {
poke(c.io.a, 8)
step(1)
expect(c.io.s, 8) // Should be 7 here
}
object TestMain extends App {
chisel3.iotesters.Driver.execute(args, () => new Inverter()) {
c => new InverterTester(c)
}
}
Now I run sbt 'test:runMain TestMain' and got this line (info is in purple):
[info] [0.002] EXPECT AT 1 io_s got 7 expected 8 FAIL
And the exit value of sbt is zero.
I need that line to be an error (with red color):
[error] [0.002] EXPECT AT 1 io_s got 7 expected 8 FAIL
As well as making the above sbt command exit with a non-zero value.
How can I achieve it with minimal change to existing code?
First the easy part. You can get a non-zero result code by using the result of chisel.execute like this.
val result = chisel3.iotesters.Driver.execute(args, () => new Inverter()) {
c => new InverterTester(c)
}
System.exit(if(result) 0 else 1)
Changing the logging level, unfortunately requires changing each of the separate backends in the chisel-testers repo. The following is an example of changing the TreadleBackend.scala, one of the three.
def expect(signal: InstanceId, expected: BigInt, msg: => String)
(implicit logger: TestErrorLog, verbose: Boolean, base: Int) : Boolean = {
signal match {
case port: Element =>
val name = portNames(port)
val got = treadleTester.peek(name)
val good = got == expected
if (!good) {
logger error
s"""EXPECT AT $stepNumber $msg $name got ${bigIntToStr(got, base)} expected ${bigIntToStr(expected, base)}""" +
s""" ${if (good) "PASS" else "FAIL"}"""
}
else if (verbose) {
logger info
s"""EXPECT AT $stepNumber $msg $name got ${bigIntToStr(got, base)} expected ${bigIntToStr(expected, base)}""" +
s""" ${if (good) "PASS" else "FAIL"}"""
}
if(good) treadleTester.expectationsMet += 1
good
case _ => false
}
}
This would not be an unreasonable issue to file, I think a logger.error would make more sense on an expect failure. There is some concern that changing this could have unexpected consequences for existing users, who are looking for that string.
But I'd like to encourage you to take a look at freechipsproject/chisel-testers2 repo. It's where the team is putting most of their testing development time. It would be easier to change, it has a lot of other nice features for building unit tests, and we are looking at ways we can make it better than chisel-testers.
I am having a problem while mocking a method that is being called in another method.
For example: Below in my main class.
class Trial extends TrialTrait {
def run(): String ={
val a = createA()
val b = a.split(" ")
val c = b.size
val d = c + " words are there"
d
}
def createA(): String = {
var a = "above all the things that have been done, one thing remained in silent above all the things that have been done one thing remained in silent above all the that "
a
}
}
Below is my mock code.
class TryMock4 extends FunSuite with BeforeAndAfterEach with MockFactory {
val trial = new Trial
val st = stub[TrialTrait]
test("Mocking the DataFrame") {
val input = "above all the things that have been done, one thing remained in silent above "
(st.createA _).when().returns(input)
val expected = "14 words are there"
val actual = st.run()
Assert.assertEquals(expected,actual)
}
}
What I am trying to do is, passing mocking data to createA and use that in the run method.
But, it is giving null value after running the run method.
Could you please suggest how it can be achieved?
I don't think you need a mock in this case, just a regular override should suffice.
class TrialTest extends FlatSpec with Matchers {
behavior of "Trial"
it should "count words" in {
val input = "above all the things that have been done, one thing remained in silent above "
val trial = new Trial {
override def createA(): String = input
}
val expected = "14 words are there"
val actual = trial.run()
actual should be (expected)
}
}
However, should you really want to use a mock here, it's possible with scalamock.
You can define our own class that makes part of the class final (the bit you don't want to mock), see below:
class TrialTestWithMock extends FlatSpec with Matchers with MockFactory {
behavior of "Trial"
it should "count words" in {
val input = "above all the things that have been done, one thing remained in silent above "
class FinalTrial extends Trial {
final override def run(): String = super.run()
}
val trial = mock[FinalTrial]
(trial.createA _).expects().returning(input).anyNumberOfTimes()
val expected = "14 words are there"
val actual = trial.run()
actual should be (expected)
}
}
EDIT2:
So another heads up on this:
I still have no idea why this happens, but I have now a similar problem with jOOQ and the Dialect I have to it. My code here looks like this:
object MyDB {
private lazy val dialect = SQLDialect.POSTGRES
def withSession[T](f: DSLContext => T) = f(DSL.using(getConnectionPool, dialect))
}
if I remove the "lazy" it blows up when I try to execute jOOQ queries in line 552 of https://github.com/jOOQ/jOOQ/blob/version-3.2.0/jOOQ/src/main/java/org/jooq/impl/DefaultRenderContext.java
That happens to be a line where the dialect is evaluated. After I added the lazy everything works as expected.
Maybe this is an issue with the threading of LiftWeb and the executing thread does not see the correct value of the val? I have no idea...
EDIT:
I have found a way to do what I want simply by adding a lazy to the values in the first, broken version. So with lazy vals it all works well.
However I'll let this stay open, as I have no idea how to explain this behavior.
Original Post:
So I am trying to use Parameterized Queries in Slick.
My code is below, my problem is that I get an NPE (see comments) when I try to run this from within the webapplication (liftweb, container started with sbt) (the application creates an object of the class PlayerListCollector that is given the string "cola")
When I execute the object as App from within Eclipse the println at the bottom works just fine.
class PlayerListCollector(term: String) {
import PlayerListCollector._
val searchResult = executeSearch(term)
}
object PlayerListCollector extends Loggable with App{
private val searchNameCurrent = Parameters[String].flatMap {
case (term) => {
for {
p <- Players if p.uberName.isNotNull
n <- p.displayName if (n.displayName.toLowerCase.like(term))
} yield (p.id, n.displayName)
}
}
private def executeSearch(term: String) = {
val lowerTerm = "%"+term.toLowerCase()+"%"
logger info "HELLO " +lowerTerm // prints HELLO %cola%
val foo = searchNameCurrent(lowerTerm) // NPE right in this line
logger info foo // never executed from here on ...
val byCurrent = foo.list
logger info byCurrent
[...]
}
// this works if run directly from within eclipse!
println(DB withSession {
searchNameCurrent("%cola%").list
})
}
The problem vanishes when I change the code to look like this:
[...]
object PlayerListCollector extends Loggable with App{
private def executeSearch(term: String) = {
val searchNameCurrent = Parameters[String].flatMap {
case (term) => {
for {
p <- Players if p.uberName.isNotNull
n <- p.displayName if (n.displayName.toLowerCase.like(term))
} yield (p.id, n.displayName)
}
}
val lowerTerm = "%"+term.toLowerCase()+"%"
logger info "HELLO " +lowerTerm // prints HELLO %cola%
val foo = searchNameCurrent(lowerTerm) // executes just fine when the query is in a local val
logger info foo
val byCurrent = foo.list
logger info byCurrent // prints expected output
[...]
}
[...]
}
I have no idea whatsoever why this happens.
Isn't the whole point of a paramterized query to put it in a val that is only once filled with a value so it does not need to be compiled multiple times?
So it turns out I used the App-Trait (http://www.scala-lang.org/api/current/index.html#scala.App) on these objects.
Reading the big fat caveat tells us what is happening I guess.
Is there some preferred way to design a Specs2 test, with lots of tests that depend on the results of previous tests?
Below, you'll find my current test suite. I don't like the vars inbetween the test fragments. They're "needed" though, since some tests generate ID numbers that subsequent tests reuses.
Should I perhaps store the ID numbers in a Specs2 Context instead, or create a separate Object that holds all mutable state? And place only test fragments in the specification object? Or is there some even better approach?
If a test fails, I'd like to cancel the remaining test at the same depth. Can I make the test fragments depend upon each other? (I know I can cancel remaining matchers in a single test fragment (by using mutable tests, or via orSkip), but what about cancelling whole fragments?)
.
object DatabaseSpec extends Specification {
sequential
"The Data Access Object" should {
var someId = "" // These var:s feels error prone, is there a better way?
"save an object" >> {
someId = database.save(something)
someId must_!= ""
// I'd like to cancel the remaining tests, below, at this "depth",
// if this test fragmen fails. Can I do that?
// (That is, cancel "load one object", "list all objects", etc, below.)
}
"load one object" >> {
anObject = database.load(someId)
anObject.id must_== someId
}
"list all objects" >> {
objs = database.listAll()
objs.find(_.id == someId) must beSome
}
var anotherId = ""
...more tests that create another object, and
...use both `someId` and `anotherId`...
var aThirdId = ""
...tests that use `someId`, `anotherId` and `aThirdId...
}
"The Data Access Object can also" >> {
...more tests...
}
}
There are 2 parts to your question: using vars for storing intermediary state, and stopping examples when one is failing.
1 - Using vars
There are some alternatives to using vars when using a mutable specification.
You can use lazy vals representing the steps of your process:
object DatabaseSpec extends mutable.Specification {
sequential
"The Data Access Object" should {
lazy val id1 = database.save(Entity(1))
lazy val loaded = database.load(id1)
lazy val list = database.list
"save an object" >> { id1 === 1 }
"load one object" >> { loaded.id === id1 }
"list all objects" >> { list === Seq(Entity(id1)) }
}
object database {
def save(e: Entity) = e.id
def load(id: Int) = Entity(id)
def list = Seq(Entity(1))
}
case class Entity(id: Int)
}
Since those values are lazy, they will only be called when the examples are executed.
If you're ready to change the structure of your current specification you can also use the latest 1.12.3-SNAPSHOT and group all those small expectations into one example:
"The Data Access Object provides a save/load/list api to the database" >> {
lazy val id1 = database.save(Entity(1))
lazy val loaded = database.load(id1)
lazy val list = database.list
"an object can be saved" ==> { id1 === 1 }
"an object can be loaded" ==> { loaded.id === id1 }
"the list of all objects can be retrieved" ==> {
list === Seq(Entity(id1))
}
}
If any of those expectations fail then the rest will not be executed and you will get a failure message like:
x The Data Access Object provides a save/load/list api to the database
an object can not be saved because '1' is not equal to '2' (DatabaseSpec.scala:16)
Another possibility, which would require 2 small improvements, would be to use the Given/When/Then way of writing specifications but using "thrown" expectations inside Given and When steps. As you can see in the User Guide, the Given/When/Then steps extract data from strings and pass typed information to the next Given/When/Then:
import org.specs2._
import specification._
import matcher.ThrownExpectations
class DatabaseSpec extends Specification with ThrownExpectations { def is =
"The Data Access Object should"^
"save an object" ^ save^
"load one object" ^ load^
"list all objects" ^ list^
end
val save: Given[Int] = groupAs(".*") and { (s: String) =>
database.save(Entity(1)) === 1
1
}
val load: When[Int, Int] = groupAs(".*") and { (id: Int) => (s: String) =>
val e = database.load(id)
e.id === 1
e.id
}
val list: Then[Int] = groupAs(".*") then { (id: Int) => (s: String) =>
val es = database.list
es must have size(1)
es.head.id === id
}
}
The improvements, which I'm going to do, are:
catch failure exceptions to report them as failures and not errors
remove the necessity to use groupAs(".*") and when there's nothing to extract from the string description.
In that case it should be enough to write:
val save: Given[Int] = groupAs(".*") and { (s: String) =>
database.save(Entity(1)) === 1
1
}
Another possibility would be to allow to directly write:
val save: Given[Int] = groupAs(".*") and { (s: String) =>
database.save(Entity(1)) === 1
}
where a Given[T] object can be created from a String => MatchResult[T] because the MatchResult[T] object already contains a value of type T, that would become a "Given".
2 - Stop the execution after a failing example
Using the implicit WhenFail Around context is certainly the best way to do what you want (unless you go with the expectations descriptions as shown above the G/W/T example).
Note on step(stepOnFail = true)
The step(stepOnFail = true) works by interrupting the following examples if one example in the previous block of concurrent examples failed. However, when you're using sequential, that previous block is limited to just one example. Hence what you're seeing. Actually I think that this is a bug and that all the remaining examples should not be executed, whether you're using sequential or not. So stay tuned for a fix coming up this week-end.
(Concerning question 1: I don't know if there's some better alternative to the vars inside the examples. Perhaps my examples are simply too long, and perhaps I should split my Spec:s into many smaller specs.)
Concerning question 2, I found in this email by etorreborre that stopping subsequent tests can be done like so:
"ex1" >> ok
"ex2" >> ok
"ex3" >> ko
step(stopOnFail=true)
"ex4" >> ok
(Ex4 will be skipped if ex1, ex2 or ex3 fails. (This doesn't work as expected in Specs2 < 1.12.3 if you're using a sequential spec, however.))
Here's another way: According to this Specs2 Googl groups email by etorreborre one can have subsequent tests stop on failure, like so:
("example2" would be skipped, but "example3" and "4" would run)
class TestSpec extends SuperSpecification {
sequential
"system1" >> {
implicit val stop = WhenFail()
"example1" >> ko
"example2" >> ok
}
"system2" >> {
implicit val stop = WhenFail()
"example3" >> ok
"example4" >> ok
}
}
case class WhenFail() extends Around {
private var mustStop = false
def around[R <% Result](r: =>R) = {
if (mustStop) Skipped("one example failed")
else if (!r.isSuccess) { mustStop = true; r }
else r
}
}
In this email by etorreborre there's a method to cancel subsequent specifications if an example fails, if you've include a list of specifications:
sequential ^ stopOnFail ^
"These are the selenium specifications" ^
include(childSpec1, childSpec2, childSpec3)
And you'd need to edit test options in build.sbt so the child specs aren't executed again indepentendly after they've been included. From the email:
testOptions := Seq(Tests.Filter(s =>
Seq("Spec", "Selenium").exists(s.endsWith(_)) &&
! s.endsWith("ChildSpec")))
Specs doc states that you may use .orSkip to skip the rest of the example in case of a failure
"The second example will be skipped" >> {
1 === 2
(1 === 3).orSkip
}
But I have not tried it personally