When I run unit tests for my sbt project (with sbt clean coverage test), I get code coverage of ~77%.
When I run integration tests (sbt clean coverage it:test), I get code coverage of ~10%.
When I run both (sbt clean coverage test it:test), I get code coverage of ~84%.
I'd like to set an aggressive code coverage minimum and fail the build if it's not met, but if I add these build settings:
coverageMinimum := 83
coverageFailOnMinimum := true
...and then run sbt clean coverage test it:test, the coverage minimum is checked after the unit tests, before the integration tests can run, and the build fails:
[error] Coverage is below minimum [77.0% < 83.0%]
If I put it:test before test, it's even worse ([10.0% < 83.0%]).
Is there any way to stipulate that the 83% minimum should apply only after both unit and integration tests have run? Or am I doomed to setting the coverage minimum meetable by my unit tests alone, and always remembering to put test before it:test on the command line?
Automatic post-test coverage minimum check was removed in version 1.3.4 (see issue https://github.com/scoverage/sbt-scoverage/issues/132).
Upgrade plugin version to latest 1.3.5 and call coverageReport after all tests, e.g.:
sbt clean coverage test it:test coverageReport
Related
Note: We are executing this as part of CI build in Teamcity
Step 1: Getting coverage details
addSbtPlugin("org.scoverage" % "sbt-scoverage" % "1.6.1")
Step 2: SBT functions to generate coverage report
clean compile coverage test coverageReport
Step 3: Run Sonar scan to upload to sonar server
%teamcity.tool.sonar-qube-scanner%\bin\sonar-scanner
-Dsonar.projectKey="%Sonar.ProjectKey%"
-Dsonar.projectName="%Sonar.ProjectName%"
-Dsonar.projectVersion="Latest"
-Dsonar.login="%Sonar.UserToken%"
-Dsonar.host.url="%Sonar.ServerUrl%"
-Dsonar.links.ci=%system.TeamCity_Url%/viewType.html?buildTypeId=%system.teamcity.buildType.id%
-Dsonar.branch.name=%teamcity.build.branch%
-Dsonar.sources="%Sonar.Sources%"
-Dsonar.scala.coverage.reportPaths="%Sonar.ScoverageReportpath%"
So with these steps we are able to get test coverage % however , we do not get the Unit Test execution details, e.g. Test count, Test Pass% ( the "test" tab under measures> coverage > (coverage details) is not there)
Any ideas what might be a workaround or alternate for this?
you must have to be include
sonar.tests=src/test/scala
sonar.junit.reportPaths= test-to/test-reports
Hope this will work for you
It would be great if one can take advantage of sbt incremental compilation and avoid having to recompile the whole project every time you run your tests, which is exactly what happens when you do clean. The following sequence though:
sbt> coverage
sbt> test
sbt> coverageReport
doesn't re-generate the coverage report but this one does:
sbt> clean
sbt> coverage
sbt> test
sbt> coverageReport
e.g The following sequence will always generate the same coverage report (i.e. the report that was generated the first time around):
sbt> coverage
sbt> test
sbt> coverageReport
here I change one of my test files and again run:
sbt> coverage
sbt> test
sbt> coverageReport
I am going to copy comments posted on this question:
By Mateusz Kubuszok:
From my experience that is exactly the problem with scoverage - during coverage instrumentation it only notices parts of code affected in some way. If you clean it will measure coverage on whole code. If you repeat measurement after that, it will only measure on... I guess parts touched by incremental compilation.
By Grzegorz Slowikowski:
SBT compilation is incremental, Scoverage uses a sequence to identify statements it instruments. This sequence is not stored anywhere, it's being reset on every compilation. For incremental compilation different statements would reuse the same first sequence numbers (1, 2, 3, ...) and it would lead to inconsistent scoverage data. Maybe it would be possible to store and reuse the sequence in subsequent builds, but nobody tried it.
I want to do test coverage on my Scala project, for which I use Maven as a build tool.
So I found this:
https://github.com/scoverage/scoverage-maven-plugin
And I looked here:
http://scoverage.github.io/scoverage-maven-plugin/1.3.0/check-mojo.html
So now to check test coverage, I run this:
mvn test
And then:
mvn scoverage:check
However, this only makes the tests to be run. I get no information about coverage.
Also, I tried:
mvn scoverage:report
But the result is the same.
So how can I use this tool or another to get test coverage info in a Scala/Maven project?
I have only used Scoverage with SBT, but chances are the usage is the same.
mvn scoverage:check will only generate some metadata - XML - and compare the generated coverage values against any coverage minimum you might have set up.
With mvn scoverage:report you will get some formatted reports. More docs here.
What sbt task does intellij IDEA 14 use to run scala tests with coverage? I'm test my spark code and need to prevent them running in parallel.
I've added the following to my build.sbt file and it prevents tests running in parallel when they aren't generating coverage reports:
parallelExecution in Test := false
However this has no effect when running with coverage. I tried using something similar, but with ScctTest instead of Test, but sbt couldn't resolve it.
So, what coverage plugin does intellij use, and how can I disable parallel test execution when running tests with coverage? Running sbt tasks doesn't show anything containing the word coverage. I haven't enabled the Emma plugin in intellij - only the default Coverage one is enabled and it has no information.
Try to add to settings:
fork in ThisBuild in Test:= false
Hi I am using sonar and I love it...
But I have a somewhat strange behavior with my latest proj.
If I put all my tests in a JUnit 4 #TestSuite JaCoCo gives me a coverage of 86,2% ( Groove baby !!! ) when I run mvn sonar:sonar
If I let the tests by themselves I get 0% coverage even with the Unit test success
100,0%
I don't need a TestSuite for this proj any clue about why JaCoCo is doing this?
BTW: I am using Maven 2.2.1 + Sonar 3.4.1 + sonar-maven-plugin 1.0.
You do not need to define a test suite if your tests respect the "Test*.java" & "*Test.java" naming convention.
You can have a look at the following sample application for which tests are analysed by JaCoCo: https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage/ut/maven/ut-maven-jacoco-runTests