I came across an issue earlier where I couldn't run an indivdual scala test, it would always try to run all of them even if I set the configuration to just be running one test. Does anyone know of any settings/configuration I can change to get it to run?
class MyTest extends PlaySpec {
val setTo = new AfterWord("set to")
"Setting" when setTo {
"value a" in {
//test stuff
}
"value b" in {
//test stuff
}
}
Turns out it was the use of the AfterWord that was messing up my test, once I removed it the tests ran fine. I'm not sure why they're incompatible but if you want to run individual tests, don't use an AfterWord.
Related
I'm trying to dynamically run some basic tests in the following way (this is a pseudo code of my test, my actual tests are a bit more complicated):
class ExampleTest extends AnyWordSpec {
def runTests() = {
for( n <- 1 until 10){
testNumber(n)
}
}
def testNumber(n: Int) = {
"testNumber" when {
s"gets the number ${n}" should {
"fail if the number is different than 0" {
assert(n == 0)
}
}
}
}
runTests()
}
When I try to run my tests in IntelliJ all the tests run as expected. But when using sbt tests It says that all my tests passed even though non of my tests actually got executed (I get the message "1 tests suite executed. 0 tests where executed").
How can I fix this? Is there any other simple way to dynamically create tests in scala?)
As mentioned in a comment, I'm not sure what the problem with sbt is.
Another way you can try to create test dynamically is using property-based testing.
One possible way is using table-driven property checks, as in the following example (which you can see in action here on Scastie):
import org.scalatest.wordspec.AnyWordSpec
import org.scalatest.prop.TableDrivenPropertyChecks._
class ExampleTest extends AnyWordSpec {
private val values = Table("n", (1 to 10): _*)
"testNumber" when {
forAll(values) { n =>
s"gets the number ${n}" should {
"fail if the number is different than 0" in {
assert(n == 0)
}
}
}
}
}
Of course all these tests are going to fail. ;-)
Links to the ScalaTest documentation about the topic:
property-based testing
table-driven property checks
generator-driven property checks (especially useful to test a lot of cases within the same case)
I have scalatest codes like following:
class myTest extends FlatSpec with ParallelTestExecution {
val testSuiteId: String = GenerateSomeRandomId()
it should "print test id" in {
println(testSuiteId)
}
it should "print test id again" in {
println(testSuiteId)
}
}
The two tests cannot print the testSuiteId I generate before them. Instead they regenerate the ID and print it. I understand that because of ParallelTestExecution which extends OneInstancePerTest, each test here runs on its own instance and have a copy of the variable "testSuiteId".
But I do want a fixed Id for this test suite and each test case in this suite have access to this fixed it without modifying it. I tried to create the fixed id in BeforeAll{ } but still it didn't work.
How should I achieve what I want?
One way to work around it would be to put the shared state in some sort of external object:
object SuiteId {
lazy val id: String = GenerateSomeRandomId()
}
Admittedly this is very much a hack, and I wouldn't be surprised if scalatest has a way to handle this built-in which I am unaware of.
I would like to take a screenshot on every fail test in a Spec or Suite using ScalaTest.
The Scala Test website shows how to take screenshots surrounding every code that might fail with this:
withScreenshot {
drive.findElement(By.id("login")).getAttribute("value") should be ("Login")
}
There is this post that tries to explain, but I could not understand what exactly should be done.
I also found the class ScreenshotOnFailure.scala, but could not use it, once it's private and has a package restriction.
Can anyone tell me if there's a way to intercept any failure and then take a screenshot?
Just to have a final answer I'm writing the way I could solve the problem based on the approach from this post mentioned in the question.
In short, the solution ended up like this (pseudo-code).
trait Screenshots extends FunSpec {
...
override def withFixture(test: NoArgTest): Outcome = {
val outcome = test()
// If the test fails, it will hold an exception.
// You can get the message with outcome.asInstanceOf[Failure].exception
if (outcome.isExceptional) {
// Implement Selenium code to save the image using a random name
// Check: https://stackoverflow.com/questions/3422262/take-a-screenshot-with-selenium-webdriver
}
outcome
}
}
class MySpec extends Screenshots {
...
describe("Scenario A") {
describe("when this") {
it("the field must have value 'A'") {
// It will save a screenshot either if the selector is wrong or the assertion fails
driver.findElement(By.id("elementA")).getAttribute("value") should be ("A")
}
}
}
}
From this point on, all Specs that extend the Screenshot trait will intercept errors and save a screenshot.
Just to complement, surrounding areas with withScreenshot(), as mentioned in the question, saves only failure on assertions, but it does not save a screenshot when the test fails due an element not found (e.g. wrong selector).
With the code above, all failures will save a screenshot.
I am running some tests with ScalaTest which rely on connections to test servers to be present. I currently created my own Spec similar to this:
abstract class ServerDependingSpec extends FlatSpec with Matchers {
def serverIsAvailable: Boolean = {
// Check if the server is available
}
}
Is it possible to ignore (but not fail) tests when this method returns false?
Currently I do it in a "hackish" way:
"Something" should "do something" in {
if(serverIsAvailable) {
// my test code
}
}
but I want something like
whenServerAvailable "Something" should "do something" in {
// test code
}
or
"Something" should "do something" whenServerAvailable {
// test code
}
I think I should define my custom tag, but I can only refer to the source code of in or ignore and I do not understand how I should plug in my custom implementations.
How should I accomplish this?
You can use assume to conditionally cancel a test:
"Something" should "do something" in {
assume(serverIsAvailable)
// my test code
}
or you can test for some condition yourself, and then use cancel:
"Something" should "do something" in {
if(!serverIsAvailable) {
cancel
}
// my test code
}
You can use Tags to achieve this:
Documentation on how to use Tags : http://www.scalatest.org/user_guide/tagging_your_tests
Adding and removing tagged test with command line parameters: http://www.scalatest.org/user_guide/using_the_runner#specifyingTagsToIncludeAndExclude
Example Code:
import org.scalatest.Tag
object ServerIsAvailable extends Tag("biz.neumann.ServerIsAvailable")
"Something" should "do something" taggedAs(ServerIsAvailable) in {
// your test here
}
Running the tests
Running the tests is a bitt tricky. It only works for testOnly and testQuick not test.
In the example testOnly is short for testOnly *
sbt "testOnly -- -l biz.neumann.ServerAvailable"
Here is some trick to skip a test based on a condition:
object WhenServerAvailable extends Tag(if (serverIsAvailable) "" else classOf[Ignore].getName)
"Something" should "do something" taggedAs WhenServerAvailable in { ... }
I'm writing a web-app using Play 2, Salat (for mongoDB bindin). I would like to test some methods, in the Lesson Model (for instance test the fact that I retrieve the right lesson by id). The problem is that I don't want to pollute my current DB with dummy lessons. How can I use a fake DB using Salat and Scala Test ? Here is one of my test file. It creates two lessons, and insert it into the DB, and it runs some tests on it.
LessonSpec extends FlatSpec with ShouldMatchers {
object FakeApp extends FakeApplication()
val newLesson1 = Lesson(
title = "lesson1",
level = 5,
explanations = "expl1",
questions = Seq.empty)
LessonDAO.insert(newLesson1)
val newLesson2 = Lesson(
title = "lesson2",
level = 5,
explanations = "expl2",
questions = Seq.empty)
LessonDAO.insert(newLesson2)
"Lesson Model" should "be retrieved by level" in {
running(FakeApp) {
assert(Lesson.findByLevel(5).size === 2)
}
}
it should "be of size 0 if no lesson of the level is found" in {
running(FakeApp) {
Lesson.findByLevel(4) should be(Nil)
}
}
it should "be retrieved by title" in {
running(FakeApp) {
Lesson.findOneByTitle("lesson1") should be(Some(Lesson("lesson1", 5, "expl1", List())))
}
}
}
I searched on the web but i can't find a good link or project that use Salat and ScalaTest.
Salat developer here. My recommendation would be to have a separate test only database. You can populate it with test data to put your test database in a known state - see the casbah tests for how to do this - and then test against it however you like, clearing out collections as necessary.
I use specs2, not scalatest, but the principle is the same - see the source code for the Salat tests.
Here's a good test to get you started:
https://github.com/novus/salat/blob/master/salat-core/src/test/scala/com/novus/salat/test/dao/SalatDAOSpec.scala
Note that in my base spec I clear out my test database - this gets run before each spec:
trait SalatSpec extends Specification with Logging {
override def is =
Step {
// log.info("beforeSpec: registering BSON conversion helpers")
com.mongodb.casbah.commons.conversions.scala.RegisterConversionHelpers()
com.mongodb.casbah.commons.conversions.scala.RegisterJodaTimeConversionHelpers()
} ^
super.is ^
Step {
// log.info("afterSpec: dropping test MongoDB '%s'".format(SalatSpecDb))
MongoConnection().dropDatabase(SalatSpecDb)
}
And then in SalatDAOSpec, I run my tests inside scopes which create, populate and/or clear out individual collections so that the tests can run in an expected state. One hitch: if you run your tests concurrently on the same collection, they may fail due to unexpected state. The solution is either to run your tests in isolated special purpose collections, or to force your tests to run in series so that operations on a single collection don't step on each other as different test cases modify the collection.
If you post to the Scalatest mailing list (http://groups.google.com/group/scalatest-users), I'm sure someone can recommend the correct way to set this up.
In my applications, I use a parameter in application.conf to specify the Mongo database name. When initializing my FakeApplication, I override that parameter so that my unit tests can use a real Mongo instance but do not see any of my production data.
Glossing over a few details specific to my application, my tests look something like this:
// wipe any existing data
db.collectionNames.foreach { colName =>
if (colName != "system.indexes") db.getCollection(colName).drop
}
app = FakeApplication(additionalConfiguration = Map("mongo.db.name" -> "unit-test"))