Is there a way to truncate the test results to only show result text for unit tests only when the unit test has failed? I'm working on a Scala project that has 850 unit tests, and the green text from the successful unit tests makes it difficult to focus on just the failures.
Example of what I'm talking about:
[info] - should have colors
[info] - should not be dead
//.... x 100
[info] - animals should not be rainbows *** FAILED ***
[info] -"[rainbow]s" was not equal to "[ponie]s" (HappinessSpec.scala:31)
What I would like is something that just shows the failure(s):
[info] - animals should not be rainbows *** FAILED ***
[info] -"[rainbow]s" was not equal to "[ponie]s" (HappinessSpec.scala:31)
I realize there is the test-quick sbt command, but it's still running 300 successful
unit tests in my case when there are only 30 failures.
Something along the lines of this in terms of usage is what I'm looking for:
sbt> ~ test -showOnlyFailures
I would also be happy with something that shows all of the failures at the end of running
the unit tests. IIRC, this is how RSpec works in Ruby...
After adding the following to build.sbt, scalaTest will show a fail summary after the standart report:
testOptions in Test += Tests.Argument("-oI")
I - show reminder of failed and canceled tests without stack traces
T - show reminder of failed and canceled tests with short stack traces
G - show reminder of failed and canceled tests with full stack traces
There is also a "drop TestSucceeded events" flag, but I have failed to use it:
http://www.scalatest.org/user_guide/using_the_runner
Updated answer for Scalatest 3.
Here's the new syntax for your build.sbt file to reprint all the failures at the bottom of the test suite run:
testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-oI")
You might also want to suppress the info notifications, so it's easier to see the failures.
testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-oNCXELOPQRM")
Here's what all the options represent:
N - drop TestStarting events
C - drop TestSucceeded events
X - drop TestIgnored events
E - drop TestPending events
H - drop SuiteStarting events
L - drop SuiteCompleted events
O - drop InfoProvided events
P - drop ScopeOpened events
Q - drop ScopeClosed events
R - drop ScopePending events
M - drop MarkupProvided events
"-oI" prints the error notifications again at the end of the test suite run whereas "-oNCXELOPQRM" only shows the tests that fail.
See the configuring reporters section on the ScalaTest website for more detail.
For those using maven with the scalatest-maven-plugin, you can add this configuration:
<configuration>
<stdout>I</stdout>
</configuration>
You could try the sbt command
last-grep error test
This should show you the errors/failures only.
Related
I have a pytest suite running in this env:
Test session starts (platform: linux, Python 3.6.1, pytest 3.3.1, pytest-sugar 0.9.1)
plugins: flaky-3.5.3, dependency-0.3.2, forked-0.2, logger-0.4.0, sugar-0.9.1, xdist-1.24.1
I have a parametrized test, decorated with flaky, and it is supposed to be re-run max three times if it fails.
#pytest.mark.flaky(max_runs=3) # re-run this test in case it fails
def test_cucubau(getBauBau_fixture):
assert cucubau(getBauBau_fixture) == True
However, it fails only once, it is not re-run, and my flaky test report is empty.
===Flaky Test Report===
===End Flaky Test Report===
Based on what I read about flaky plugin, the usage should be trivial.. but I'm not able to see what is wrong with my code.
any idea?
I believe you need the pytest-rerunfailures plugin for that to work. Then you should be able to annotate your test with #pytest.mark.flaky(reruns=3).
When I run tests in Play 2.6.x that are using the WsTestClient like:
WsTestClient.withClient { client => ...
I see in the logs:
[info] p.a.t.WsTestClient$SingletonWSClient - createNewClient: name = ws-test-client-1
Which I believe is coming from this line
Why is it logging as Info if it's set at log-level warn? Can I silence the noise somehow?
I am using the latest version of Play framework, along with the following test dependencies from by build.sbt file:
"org.scalatest" %% "scalatest" % "3.0.0",
"org.scalatestplus.play" % "scalatestplus-play_2.11" % "2.0.0-M1"
I have a base specification all of my test cases extend from. I return a Future[Assertion] in each one of my clauses, it looks like this:
trait BaseSpec extends AsyncWordSpec with TestSuite with OneServerPerSuite with MustMatchers with ParallelTestExecution
An example spec looks like this:
"PUT /v1/user/create" should {
"create a new user" in {
wsClient
.url(s"http://localhost:${port}/v1/user")
.put(Json.obj(
"name" -> "username",
"email" -> "email",
"password" -> "hunter12"
)).map { response => response.status must equal(201) }
}
}
I decided to rewrite my current tests using the AsyncWordSpec provided by the newer version of ScalaTest, but when I run the test suite, this is the output that I get:
[info] UserControllerSpec:
[info] PUT /v1/user/create
[info] application - ApplicationTimer demo: Starting application at 2016-11-13T01:29:12.161Z.
[info] application - ApplicationTimer demo: Stopping application at 2016-11-13T01:29:12.416Z after 1s.
[info] application - ApplicationTimer demo: Stopping application at 2016-11-13T01:29:12.438Z after 0s.
[info] application - ApplicationTimer demo: Stopping application at 2016-11-13T01:29:12.716Z after 0s.
[info] application - ApplicationTimer demo: Stopping application at 2016-11-13T01:29:13.022Z after 1s.
[info] ScalaTest
[info] Run completed in 13 seconds, 540 milliseconds.
[info] Total number of tests run: 0
[info] Suites: completed 4, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[success] Total time: 20 s, completed Nov 12, 2016 8:29:13 PM
All of my test classes are found, built, and seemingly run by the test runner when invoking sbt test. I have also tried using the IDEA test runner, and it reports Empty Test Suite under each one of my test classes. I have exhaustively attempted to RTFM but I cannot see what I am doing wrong. The synchronous versions of my tests are running totally fine.
EDIT 1: A friend suggested to attempt doing whenReady() { /* clause */ } on my Future[WSResponse], but this too has failed.
I was having the same problem while using a test suite with multiple traits. I got it to work by removing all the other traits except for AsyncFlatSpec. I will add them back one at a time as I need them.
I'm playing with Mongo database through the Reactive Mongo driver
import org.slf4j.LoggerFactory
import reactivemongo.api.MongoDriver
import reactivemongo.api.collections.default.BSONCollection
import reactivemongo.bson.BSONDocument
import scala.concurrent.Future
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
object Main {
val log = LoggerFactory.getLogger("Main")
def main(args: Array[String]): Unit = {
log.info("Start")
val conn = new MongoDriver().connection(List("localhost"))
val db = conn("test")
log.info("Done")
}
}
My build.sbt file:
lazy val root = (project in file(".")).
settings(
name := "simpleapp",
version := "1.0.0",
scalaVersion := "2.11.4",
libraryDependencies ++= Seq(
"org.reactivemongo" %% "reactivemongo" % "0.10.5.0.akka23",
"ch.qos.logback" % "logback-classic" % "1.1.2"
)
)
When I run: sbt compile run
I get this output:
$ sbt compile run
[success] Total time: 0 s, completed Apr 25, 2015 5:36:51 PM
[info] Running Main
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
17:36:52.328 [run-main-0] INFO Main - Start
17:36:52.333 [run-main-0] INFO Main - Done
And application doesn't stop.... :/
I have to press Ctrl + C to kill it
I've read that MongoDriver() creates ActorSystem so I tried to close connection manually with conn.close() but I get this:
[info] Running Main
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
17:42:23.252 [run-main-0] INFO Main - Start
17:42:23.258 [run-main-0] INFO Main - Done
17:42:23.403 [reactivemongo-akka.actor.default-dispatcher-2] ERROR reactivemongo.core.actors.MongoDBSystem - (State: Closing) UNHANDLED MESSAGE: ChannelConnected(-973180998)
[INFO] [04/25/2015 17:42:23.413] [reactivemongo-akka.actor.default-dispatcher-3] [akka://reactivemongo/deadLetters] Message [reactivemongo.core.actors.Closed$] from Actor[akka://reactivemongo/user/$b#-1700211063] to Actor[akka://reactivemongo/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [04/25/2015 17:42:23.414] [reactivemongo-akka.actor.default-dispatcher-3] [akka://reactivemongo/user/$a] Message [reactivemongo.core.actors.Close$] from Actor[akka://reactivemongo/user/$b#-1700211063] to Actor[akka://reactivemongo/user/$a#-1418324178] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
And app doesn't exit also
So, what am i doing wrong? I can'f find answer...
And it seems to me that official docs doesn't explain whether i should care about graceful shutdown at all.
I don't have much experience with console apps, i use play framework in my projects but i want to create sub-project that works with mongodb
I see many templates (in activator) such as: Play + Reactive Mongo, Play + Akka + Mongo but there's no Scala + Reactive Mongo that would explain how to work properly :/
I was having the same problem. The solution I found was invoking close on both object, the driver and the connection:
val driver = new MongoDriver
val connection = driver.connection(List("localhost"))
...
connection.close()
driver.close()
If you close only the connection, then the akka system remains alive.
Tested with ReactiveMongo 0.12
This looks like a known issue with Reactive Mongo, see the relevant thread on GitHub
A fix for this was introduced in this pull request #241 by reid-spencer, merged on the 3rd of February 2015
You should be able to fix it by using a newer version. If no release has been made since February, you could try checking out a version that includes this fix and building the code yourself.
As far as I can see, there's no mention of this bugfix in the release notes for version 0.10.5
Bugfixes:
BSON library: fix BSONDateTimeNumberLike typeclass
Cursor: fix exception propagation
Commands: fix ok deserialization for some cases
Commands: fix CollStatsResult
Commands: fix AddToSet in aggregation
Core: fix connection leak in some cases
GenericCollection: do not ignore WriteConcern in save()
GenericCollection: do not ignore WriteConcern in bulk inserts
GridFS: fix uploadDate deserialization field
Indexes: fix parsing for Ascending and Descending
Macros: fix type aliases
Macros: allow custom annotations
The name of the committer does not appear as well:
Here is the list of the commits included in this release (since 0.9, the top commit is the most recent one):
$ git shortlog -s -n refs/tags/v0.10.0..0.10.5.x.akka23
39 Stephane Godbillon
5 Andrey Neverov
4 lucasrpb
3 Faissal Boutaounte
2 杨博 (Yang Bo)
2 Nikolay Sokolov
1 David Liman
1 Maksim Gurtovenko
1 Age Mooij
1 Paulo "JCranky" Siqueira
1 Daniel Armak
1 Viktor Taranenko
1 Vincent Debergue
1 Andrea Lattuada
1 pavel.glushchenko
1 Jacek Laskowski
Looking at the commit history for 0.10.5.0.akka23 (the one you reference in build.sbt), it seems the fix was not merged into it.
I have a Play! project where I would like to add some code coverage information. So far I have tried JaCoCo and scct. The former has the problem that it is based on bytecode, hence it seems to give warning about missing tests for methods that are autogenerated by the Scala compiler, such as copy or canEqual. scct seems a better option, but in any case I get many errors during tests with both.
Let me stick with scct. I essentially get errors for every test that tries to connect to the database. Many of my tests load some fixtures into an H2 database in memory and then make some assertions. My Global.scala contains
override def onStart(app: Application) {
SessionFactory.concreteFactory = Some(() => connection)
def connection() = {
Session.create(DB.getConnection()(app), new MySQLInnoDBAdapter)
}
}
while the tests usually are enclosed in a block like
class MySpec extends Specification {
def app = FakeApplication(additionalConfiguration = inMemoryDatabase())
"The models" should {
"be five" in running(app) {
Fixtures.load()
MyModels.all.size should be_==(5)
}
}
}
The line running(app) allows me to run a test in the context of a working application connected to an in-memory database, at least usually. But when I run code coverage tasks, such as scct coverage:doc, I get a lot of errors related to connecting to the database.
What is even more weird is that there are at least 4 different errors, like:
ObjectExistsException: Cache play already exists
SQLException: Attempting to obtain a connection from a pool that has already been shutdown
Configuration error [Cannot connect to database [default]]
No suitable driver found for jdbc:h2:mem:play-test--410454547
Why is that launching tests in the default configuration is able to connect to the database, while running in the context of scct (or JaCoCo) fails to initialize the cache and the db?
specs2 tests run in parallel by default. Play disables parallel execution for the standard unit test configuration, but scct uses a different configuration so it doesn't know not to run in parallel.
Try adding this to your Build.scala:
.settings(parallelExecution in ScctPlugin.ScctTest := false)
Alternatively, you can add sequential to the beginning of your test classes to force all possible run configurations to run sequentially. I've got both in my files still, as I think I had some problems with the Build.scala solution at one point when I was using an early release candidate of Play.
A better option for Scala code coverage is Scoverage which gives statement line coverage.
https://github.com/scoverage/scalac-scoverage-plugin
Add to project/plugins.sbt:
addSbtPlugin("com.sksamuel.scoverage" % "sbt-scoverage" % "1.0.1")
Then run SBT with
sbt clean coverage test
You need to add sequential in the beginning of your Specification.
class MySpec extends Specification {
sequential
"MyApp" should {
//...//
}
}