It would be great if one can take advantage of sbt incremental compilation and avoid having to recompile the whole project every time you run your tests, which is exactly what happens when you do clean. The following sequence though:
sbt> coverage
sbt> test
sbt> coverageReport
doesn't re-generate the coverage report but this one does:
sbt> clean
sbt> coverage
sbt> test
sbt> coverageReport
e.g The following sequence will always generate the same coverage report (i.e. the report that was generated the first time around):
sbt> coverage
sbt> test
sbt> coverageReport
here I change one of my test files and again run:
sbt> coverage
sbt> test
sbt> coverageReport
I am going to copy comments posted on this question:
By Mateusz Kubuszok:
From my experience that is exactly the problem with scoverage - during coverage instrumentation it only notices parts of code affected in some way. If you clean it will measure coverage on whole code. If you repeat measurement after that, it will only measure on... I guess parts touched by incremental compilation.
By Grzegorz Slowikowski:
SBT compilation is incremental, Scoverage uses a sequence to identify statements it instruments. This sequence is not stored anywhere, it's being reset on every compilation. For incremental compilation different statements would reuse the same first sequence numbers (1, 2, 3, ...) and it would lead to inconsistent scoverage data. Maybe it would be possible to store and reuse the sequence in subsequent builds, but nobody tried it.
Related
I want to do test coverage on my Scala project, for which I use Maven as a build tool.
So I found this:
https://github.com/scoverage/scoverage-maven-plugin
And I looked here:
http://scoverage.github.io/scoverage-maven-plugin/1.3.0/check-mojo.html
So now to check test coverage, I run this:
mvn test
And then:
mvn scoverage:check
However, this only makes the tests to be run. I get no information about coverage.
Also, I tried:
mvn scoverage:report
But the result is the same.
So how can I use this tool or another to get test coverage info in a Scala/Maven project?
I have only used Scoverage with SBT, but chances are the usage is the same.
mvn scoverage:check will only generate some metadata - XML - and compare the generated coverage values against any coverage minimum you might have set up.
With mvn scoverage:report you will get some formatted reports. More docs here.
I am trying to merge two modules into single module. Both are successfully running modules. I merge two modules. And trying to run the test cases.
i am compiling source and testcases by using sbt commands:
sbt
clean
compile
project module-read
test:compile
it:test
Till test:compile everything working fine but after it:test, it showing lot of compilation issues.
Could I know best way of compiling?
The test:compile task will only compile tests within the src/test/scala folder as per the default sbt test configuration.
In order to compile your integration tests (in src/it/scala) you will have to run it:compile .
See http://www.scala-sbt.org/0.13.5/docs/Detailed-Topics/Testing.html#integration-tests for more info.
When I run unit tests for my sbt project (with sbt clean coverage test), I get code coverage of ~77%.
When I run integration tests (sbt clean coverage it:test), I get code coverage of ~10%.
When I run both (sbt clean coverage test it:test), I get code coverage of ~84%.
I'd like to set an aggressive code coverage minimum and fail the build if it's not met, but if I add these build settings:
coverageMinimum := 83
coverageFailOnMinimum := true
...and then run sbt clean coverage test it:test, the coverage minimum is checked after the unit tests, before the integration tests can run, and the build fails:
[error] Coverage is below minimum [77.0% < 83.0%]
If I put it:test before test, it's even worse ([10.0% < 83.0%]).
Is there any way to stipulate that the 83% minimum should apply only after both unit and integration tests have run? Or am I doomed to setting the coverage minimum meetable by my unit tests alone, and always remembering to put test before it:test on the command line?
Automatic post-test coverage minimum check was removed in version 1.3.4 (see issue https://github.com/scoverage/sbt-scoverage/issues/132).
Upgrade plugin version to latest 1.3.5 and call coverageReport after all tests, e.g.:
sbt clean coverage test it:test coverageReport
I was trying to use jacoco to integrate test report to my sbt project. https://github.com/sbt/jacoco4sbt
I added jacoco.settings into build.sbt
I also added addSbtPlugin("de.johoop" % "jacoco4sbt" % "2.1.6") into plugins.sbt
When I run sbt jacoco:check, it is working fine. However, when I try to look at how many tasks for jacoco, sbt tasks doesn't show anything related to jacoco.
I have to go to source code to look at it.
https://github.com/sbt/jacoco4sbt/blob/master/src/main/scala/de/johoop/jacoco4sbt/Keys.scala
May I know why jacoco is not shown for sbt tasks command and what is preferable way to look at all the available tasks for the plugins
Edit:
I suspect that the statement lazy val Config = config("jacoco") extend(Test) hide means jacoco extends Test task, so it wont show it in the sbt tasks, but I am not sure.
By running
> tasks -v
Edit: If that doesn't work consider adding more "v"s, such as tasks -vvv, or even tasks -V to see all the tasks.
I see, for instance, cover:
This is a list of tasks defined for the current project.
It does not list the scopes the tasks are defined in; use the 'inspect' command for that.
Tasks produce values. Use the 'show' command to run the task and print the resulting value.
check Executes the tests and saves the execution data in 'jacoco.exec'.
classesToCover compiled classes (filtered by includes and excludes) that will be covered
clean Cleaning JaCoCo's output-directory.
compile Compiles sources.
console Starts the Scala interpreter with the project classes on the classpath.
consoleProject Starts the Scala interpreter with the sbt and the build definition on the classpath and useful imports.
consoleQuick Starts the Scala interpreter with the project dependencies on the classpath.
copyResources Copies resources to the output directory.
cover Executes the tests and creates a JaCoCo coverage report.
coveredSources Covered Sources.
Note also what it says at the beginning (wrapped for clarity):
It does not list the scopes the tasks are defined in;
use the 'inspect' command for that.
which leads to
> inspect cover
[info] No entry for key.
[info] Description:
[info] Executes the tests and creates a JaCoCo coverage report.
[info] Delegates:
[info] *:cover
[info] {.}/*:cover
[info] */*:cover
[info] Related:
[info] jacoco:cover
So you know to run jacoco:cover
I have not found any documentation on how to do this. For JUnit the equivalent would be:
mvn -Dtest=org.apache.spark.streaming.InputStreamSuite test
tl;dr mvn test -Dsuites="some.package.SpecsClass"
I found an answer from here and it works:(https://groups.google.com/forum/#!topic/scalatest-users/Rr0gy61dg-0)
run test 'a pending test' in HelloSuite, and all tests in HelloWordSpec:
mvn test -Dsuites='org.example.
HelloSuite #a pending test, org.example.HelloWordSpec'
run all tests in HelloSuite containing 'hello':
mvn test -Dsuites='org.example.HelloSuite hello'
for more details: http://scalatest.org/user_guide/using_the_scalatest_maven_plugin
Found the answer: it is
-DwildcardSuites
So here is the example command line:
mvn -pl streaming -DwildcardSuites=org.apache.spark.streaming.InputStreamSuite test
Update Newer versions of scalatest use
-Dsuites
So the syntax would be:
mvn -pl streaming -Dsuites=org.apache.spark.streaming.InputStreamSuite test
Note that if you have some Java tests in the same module, as much of spark does, you need to turn them off -which you can do by telling surefire to run a test that isn't there
Here is the test that I've just been running
mvn test -Dtest=moo -DwildcardSuites=org.apache.spark.deploy.yarn.ClientSuite
That skips the java test and only runs the scala one.
One thing which scalatest doesn't seem to do is let you run a single test within a suite, the way maven surefire does. That's not ideal if you have one failing test in a big suite.
[Correction 2016-08-22: looks like you can ask for a specific suite by name; look at the other answers below. Happy to be wrong].