Dynamicaly Created Tests sbt - scala

I'm trying to dynamically run some basic tests in the following way (this is a pseudo code of my test, my actual tests are a bit more complicated):
class ExampleTest extends AnyWordSpec {
def runTests() = {
for( n <- 1 until 10){
testNumber(n)
}
}
def testNumber(n: Int) = {
"testNumber" when {
s"gets the number ${n}" should {
"fail if the number is different than 0" {
assert(n == 0)
}
}
}
}
runTests()
}
When I try to run my tests in IntelliJ all the tests run as expected. But when using sbt tests It says that all my tests passed even though non of my tests actually got executed (I get the message "1 tests suite executed. 0 tests where executed").
How can I fix this? Is there any other simple way to dynamically create tests in scala?)

As mentioned in a comment, I'm not sure what the problem with sbt is.
Another way you can try to create test dynamically is using property-based testing.
One possible way is using table-driven property checks, as in the following example (which you can see in action here on Scastie):
import org.scalatest.wordspec.AnyWordSpec
import org.scalatest.prop.TableDrivenPropertyChecks._
class ExampleTest extends AnyWordSpec {
private val values = Table("n", (1 to 10): _*)
"testNumber" when {
forAll(values) { n =>
s"gets the number ${n}" should {
"fail if the number is different than 0" in {
assert(n == 0)
}
}
}
}
}
Of course all these tests are going to fail. ;-)
Links to the ScalaTest documentation about the topic:
property-based testing
table-driven property checks
generator-driven property checks (especially useful to test a lot of cases within the same case)

Related

Mock a function call in scala for unittesting

I would like to write a unittest to the function
import com.github.nscala_time.time.Imports.{DateTime, richReadableInstant}
def myFunction(ts: Long):Long = {
(new DateTime(ts) to DateTime.now()).toDurationMillis
}
Is it possible to somehow mock Datetime.now(), so that the test result does not depend on when the test is running?
From what I can see this library is a wrapper around joda-time (which, as the official documentation recommends, should be dropped in favor of java.time, but I assume you have some constraint that forces you to work on a pre-Java 8 release).
joda-time comes with a collection of static helpers that, among other things, allow you to manage what the response is when a method asks for the "current time" (see their JavaDoc here).
The easiest possible way (but possibly prone to mistakes due to the shared mutable state it relies on) would look like the following:
import com.github.nscala_time.time.Imports.{DateTime, richReadableInstant}
import org.joda.time.DateTimeUtils
DateTimeUtils.setCurrentMillisFixed(42)
def myFunction(ts: Long):Long = {
(new DateTime(ts) to DateTime.now()).toDurationMillis
}
assert(myFunction(42) == 0)
You can play around with this code here on Scastie.
As mentioned, this approach is a bit clunky and relies on shared mutable state, which makes it prone to confusing errors. You can build a nice small helper to make sure you can use a custom clock on a specific test and reset to the system clock once done. The required synchronization means a performance hit, but it's probably acceptable for your tests.
import com.github.nscala_time.time.Imports.{DateTime, richReadableInstant}
import org.joda.time.DateTimeUtils
import org.joda.time.DateTimeUtils.MillisProvider
def myFunction(ts: Long):Long = {
(new DateTime(ts) to DateTime.now()).toDurationMillis
}
final class FixedClock(at: Long) extends MillisProvider {
override def getMillis(): Long = at
}
def withCustomClock[A](clock: MillisProvider)(f: => A): A = {
synchronized {
try {
DateTimeUtils.setCurrentMillisProvider(clock)
f
} finally {
DateTimeUtils.setCurrentMillisSystem() // _always_ reset to the system clock once done
}
}
}
assert(myFunction(42) > 1000000)
withCustomClock(new FixedClock(at = 42)) {
assert(myFunction(42) == 0)
Thread.sleep(1000)
assert(myFunction(42) == 0)
}
assert(myFunction(42) > 1000000)
You can play around with this other example here on this other worksheet on Scastie.

How to initialize shared variables before parallel test in scalatest

I have scalatest codes like following:
class myTest extends FlatSpec with ParallelTestExecution {
val testSuiteId: String = GenerateSomeRandomId()
it should "print test id" in {
println(testSuiteId)
}
it should "print test id again" in {
println(testSuiteId)
}
}
The two tests cannot print the testSuiteId I generate before them. Instead they regenerate the ID and print it. I understand that because of ParallelTestExecution which extends OneInstancePerTest, each test here runs on its own instance and have a copy of the variable "testSuiteId".
But I do want a fixed Id for this test suite and each test case in this suite have access to this fixed it without modifying it. I tried to create the fixed id in BeforeAll{ } but still it didn't work.
How should I achieve what I want?
One way to work around it would be to put the shared state in some sort of external object:
object SuiteId {
lazy val id: String = GenerateSomeRandomId()
}
Admittedly this is very much a hack, and I wouldn't be surprised if scalatest has a way to handle this built-in which I am unaware of.

Couldn't run individual scala test in Intellij

I came across an issue earlier where I couldn't run an indivdual scala test, it would always try to run all of them even if I set the configuration to just be running one test. Does anyone know of any settings/configuration I can change to get it to run?
class MyTest extends PlaySpec {
val setTo = new AfterWord("set to")
"Setting" when setTo {
"value a" in {
//test stuff
}
"value b" in {
//test stuff
}
}
Turns out it was the use of the AfterWord that was messing up my test, once I removed it the tests ran fine. I'm not sure why they're incompatible but if you want to run individual tests, don't use an AfterWord.

Screenshot on every failure using Scalatest

I would like to take a screenshot on every fail test in a Spec or Suite using ScalaTest.
The Scala Test website shows how to take screenshots surrounding every code that might fail with this:
withScreenshot {
drive.findElement(By.id("login")).getAttribute("value") should be ("Login")
}
There is this post that tries to explain, but I could not understand what exactly should be done.
I also found the class ScreenshotOnFailure.scala, but could not use it, once it's private and has a package restriction.
Can anyone tell me if there's a way to intercept any failure and then take a screenshot?
Just to have a final answer I'm writing the way I could solve the problem based on the approach from this post mentioned in the question.
In short, the solution ended up like this (pseudo-code).
trait Screenshots extends FunSpec {
...
override def withFixture(test: NoArgTest): Outcome = {
val outcome = test()
// If the test fails, it will hold an exception.
// You can get the message with outcome.asInstanceOf[Failure].exception
if (outcome.isExceptional) {
// Implement Selenium code to save the image using a random name
// Check: https://stackoverflow.com/questions/3422262/take-a-screenshot-with-selenium-webdriver
}
outcome
}
}
class MySpec extends Screenshots {
...
describe("Scenario A") {
describe("when this") {
it("the field must have value 'A'") {
// It will save a screenshot either if the selector is wrong or the assertion fails
driver.findElement(By.id("elementA")).getAttribute("value") should be ("A")
}
}
}
}
From this point on, all Specs that extend the Screenshot trait will intercept errors and save a screenshot.
Just to complement, surrounding areas with withScreenshot(), as mentioned in the question, saves only failure on assertions, but it does not save a screenshot when the test fails due an element not found (e.g. wrong selector).
With the code above, all failures will save a screenshot.

How to test methods based on Salat with ScalaTest

I'm writing a web-app using Play 2, Salat (for mongoDB bindin). I would like to test some methods, in the Lesson Model (for instance test the fact that I retrieve the right lesson by id). The problem is that I don't want to pollute my current DB with dummy lessons. How can I use a fake DB using Salat and Scala Test ? Here is one of my test file. It creates two lessons, and insert it into the DB, and it runs some tests on it.
LessonSpec extends FlatSpec with ShouldMatchers {
object FakeApp extends FakeApplication()
val newLesson1 = Lesson(
title = "lesson1",
level = 5,
explanations = "expl1",
questions = Seq.empty)
LessonDAO.insert(newLesson1)
val newLesson2 = Lesson(
title = "lesson2",
level = 5,
explanations = "expl2",
questions = Seq.empty)
LessonDAO.insert(newLesson2)
"Lesson Model" should "be retrieved by level" in {
running(FakeApp) {
assert(Lesson.findByLevel(5).size === 2)
}
}
it should "be of size 0 if no lesson of the level is found" in {
running(FakeApp) {
Lesson.findByLevel(4) should be(Nil)
}
}
it should "be retrieved by title" in {
running(FakeApp) {
Lesson.findOneByTitle("lesson1") should be(Some(Lesson("lesson1", 5, "expl1", List())))
}
}
}
I searched on the web but i can't find a good link or project that use Salat and ScalaTest.
Salat developer here. My recommendation would be to have a separate test only database. You can populate it with test data to put your test database in a known state - see the casbah tests for how to do this - and then test against it however you like, clearing out collections as necessary.
I use specs2, not scalatest, but the principle is the same - see the source code for the Salat tests.
Here's a good test to get you started:
https://github.com/novus/salat/blob/master/salat-core/src/test/scala/com/novus/salat/test/dao/SalatDAOSpec.scala
Note that in my base spec I clear out my test database - this gets run before each spec:
trait SalatSpec extends Specification with Logging {
override def is =
Step {
// log.info("beforeSpec: registering BSON conversion helpers")
com.mongodb.casbah.commons.conversions.scala.RegisterConversionHelpers()
com.mongodb.casbah.commons.conversions.scala.RegisterJodaTimeConversionHelpers()
} ^
super.is ^
Step {
// log.info("afterSpec: dropping test MongoDB '%s'".format(SalatSpecDb))
MongoConnection().dropDatabase(SalatSpecDb)
}
And then in SalatDAOSpec, I run my tests inside scopes which create, populate and/or clear out individual collections so that the tests can run in an expected state. One hitch: if you run your tests concurrently on the same collection, they may fail due to unexpected state. The solution is either to run your tests in isolated special purpose collections, or to force your tests to run in series so that operations on a single collection don't step on each other as different test cases modify the collection.
If you post to the Scalatest mailing list (http://groups.google.com/group/scalatest-users), I'm sure someone can recommend the correct way to set this up.
In my applications, I use a parameter in application.conf to specify the Mongo database name. When initializing my FakeApplication, I override that parameter so that my unit tests can use a real Mongo instance but do not see any of my production data.
Glossing over a few details specific to my application, my tests look something like this:
// wipe any existing data
db.collectionNames.foreach { colName =>
if (colName != "system.indexes") db.getCollection(colName).drop
}
app = FakeApplication(additionalConfiguration = Map("mongo.db.name" -> "unit-test"))