sbt-scoverage + specs2 not working; no tests discovered during analysis - specs2

I am unable to get sbt-scoverage to discover and analyze our tests. We should have pretty good test coverage (at least 80%); all tests run and pass fine. But the scoverage report shows nearly zero coverage (~ 3%).
The only thing I can think of, is that we are using specs2 (and also scalacheck). Most of the examples I've seen use ScalaTest.
For example: We have four classes in the models package. They are showing zero coverage, even though the test results show that they have been tested thoroughly:
[info] Compiling 5 Scala sources to /Users/zbeckman/Projects/Accenture/Trident_IDC_Training/target/scala-2.11/test-classes...
[info] TestCreditModel
[info] + Given a credit
[info] When validation is performed
[info] Then credits between 1 and 60 should be acceptable
[info] + And credits less than 1 or greater than 60 should not
[info]
[info] + Given a category and attribute credits
[info] When validation is performed
[info] Then we should get validated category credits
[info] + And we should get validated attribute credits
[info] + And invalid category credits should not pass validation
[info] + And invalid attribute credits should not pass validation
[info]
[info] Total for specification TestCreditModel
[info] Finished in 86 ms
[info] 6 examples, 600 expectations, 0 failure, 0 error
[info]
[info] TestCategoryModel
[info] + Given a well formed category
[info] When validation is performed
[info] Categories with good abbreviations and descriptions should be valid
[info] + And those with invalid abbreviations should not
[info] + And those with invalid descriptions should not
[info]
[info]
[info] Total for specification TestCategoryModel
[info] Finished in 167 ms
[info] 3 examples, 300 expectations, 0 failure, 0 error
But I see scoverage is definitely running:
[info] Compiling 11 Scala sources and 1 Java source to /Users/zbeckman/Projects/Accenture/Trident_IDC_Training/target/scala-2.11/classes...
[info] [info] Cleaning datadir [/Users/zbeckman/Projects/Accenture/Trident_IDC_Training/target/scala-2.11/scoverage-data]
[info] [info] Beginning coverage instrumentation
[info] [info] Instrumentation completed [31 statements]
[info] [info] Wrote instrumentation file [/Users/zbeckman/Projects/Accenture/Trident_IDC_Training/target/scala-2.11/scoverage-data/scoverage.coverage.xml]
[info] [info] Will write measurement data to [/Users/zbeckman/Projects/Accenture/Trident_IDC_Training/target/scala-2.11/scoverage-data]
But the coverage report is clearly not picking up the classes that we are testing:
[IDC] $ coverageReport
[info] Waiting for measurement data to sync...
[info] Reading scoverage instrumentation [/Users/zbeckman/Projects/Accenture/Trident_IDC_Training/target/scala-2.11/scoverage-data/scoverage.coverage.xml]
[info] Reading scoverage measurements...
[info] Generating scoverage reports...
[info] Written Cobertura report [/Users/zbeckman/Projects/Accenture/Trident_IDC_Training/target/scala-2.11/coverage-report/cobertura.xml]
[info] Written XML coverage report [/Users/zbeckman/Projects/Accenture/Trident_IDC_Training/target/scala-2.11/scoverage-report/scoverage.xml]
[info] Written HTML coverage report [/Users/zbeckman/Projects/Accenture/Trident_IDC_Training/target/scala-2.11/scoverage-report/index.html]
[info] Statement coverage.: 3.23%
[info] Branch coverage....: 100.00%
[info] Coverage reports completed
[error] Coverage is below minimum [3.23% < 80.0%]
[trace] Stack trace suppressed: run last *:coverageReport for the full output.
[error] (*:coverageReport) Coverage minimum was not reached
[error] Total time: 1 s, completed Aug 25, 2016 1:11:27 PM
When viewing the report, I see the four classes in the models package show completely untested.
We have the following in our built.sbt:
coverageEnabled := true
coverageMinimum := 80
coverageFailOnMinimum := true
Obviously, since we have the above settings, and since scoverage is failing to find the tests, we get a build failure:
[error] Coverage is below minimum [3.23% < 80.0%]
[error] (*:coverageReport) Coverage minimum was not reached
[error] Total time: 1 s, completed Aug 25, 2016 1:39:12 PM

Related

maven does not print name of tested class; replaces class name with TestSuite

I'm running this project with testNG, i get the test result(successful one) as
[RemoteTestNG] detected TestNG version 6.13.1
Success!!
Dec 07, 2017 4:44:06 PM org.openqa.selenium.remote.ProtocolHandshake
createSession
INFO: Detected dialect: OSS
PASSED: begin
PASSED: google
===============================================
Default test
Tests run: 2, Failures: 0, Skips: 0
===============================================
===============================================
Default suite
Total tests run: 2, Failures: 0, Skips: 0
===============================================
But when running as Maven test, it doesn't print the name of passed test cases(or any test). It just gives number of tests run. I want the passed cases to be printed here as well.
P.S. - Class is in src/test/java. Tried changing it to src/main/java as suggested in one of issue. but maven skipped the tests altogether in that way as Surefire-plugin runs all tests that are in test source directory src/test/java.
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building MavenFrstProj 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # MavenFrstProj ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
C:\Users\Shrikanth\workspace\MavenFrstProj\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) #
MavenFrstProj ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources)
# MavenFrstProj ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
C:\Users\Shrikanth\workspace\MavenFrstProj\src\test\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) #
MavenFrstProj ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.20:test (default-test) # MavenFrstProj --
-
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
Success!!
Dec 07, 2017 4:45:58 PM org.openqa.selenium.remote.ProtocolHandshake
createSession
INFO: Detected dialect: OSS
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.017
s - in TestSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ---------------------------------------------------------------------
---
[INFO] BUILD SUCCESS
[INFO] ---------------------------------------------------------------------
---
[INFO] Total time: 11.972 s
[INFO] Finished at: 2017-12-07T16:46:02+05:30
[INFO] Final Memory: 12M/178M
[INFO] ------------------------------------------------------------------------
Note : Also checking the xml report(TestSuite) generated in target/surefire-reports, it replaces class name as TestSuite and in console output it prints Running TestSuite instead of Running <classname>. May be thats the root, but not sure. How do I fix it. Please help!
By default, the Surefire Plugin will automatically include all test classes with the following wildcard patterns:
"**/Test*.java" - includes all of its subdirectories and all Java filenames that start with "Test".
"**/*Test.java" - includes all of its subdirectories and all Java filenames that end with "Test".
"**/*Tests.java" - includes all of its subdirectories and all Java filenames that end with "Tests".
"**/*TestCase.java" - includes all of its subdirectories and all Java filenames that end with "TestCase".
Make sure your test class names follow this pattern otherwise , add each class inclusively in pom.xml

How to force SBT to use Javadoc instead of Scaladoc?

I have Java based a Play & Akka project that is built with SBT, is there any way to tell SBT to run Javadoc instead of Scaladoc when building the project? The SBT documentation says that
“sbt will run javadoc if there are only Java sources in the project. If there are any Scala sources, sbt will run scaladoc.”
and there are no Scala files in the project but Scaladoc is still run. I assume that this is because the Play plugin is converting the view templates and conf/routes file into Scala code before compilation happens.
tl;dr Change sources in (Compile, doc) to exclude *.scala files so scaladoc doesn't kick in.
As you can read below sources in (Compile, doc) holds all sources, both managed and unmanaged in a project:
> inspect compile:doc::sources
[info] Task: scala.collection.Seq[java.io.File]
[info] Description:
[info] All sources, both managed and unmanaged.
With some filtering you can achieve your goal quite easily - use the following in build.sbt:
sources in (Compile, doc) <<= sources in (Compile, doc) map { _.filterNot(_.getName endsWith ".scala") }
A sample project's session (comments with // are mine to enhance reading):
// display current setting's value
> show compile:doc::sources
[info] ArrayBuffer(/Users/jacek/sandbox/sbt-learning-space/src/main/scala/x.scala, /Users/jacek/sandbox/sbt-learning-space/src/main/java/Hello.java)
[success] Total time: 0 s, completed Aug 24, 2014 9:31:34 PM
// generate docs - scaladoc kicks in since there are scala files
> doc
[info] Updating {file:/Users/jacek/sandbox/sbt-learning-space/}sbt-learning-space...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Main Scala API documentation to /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api...
model contains 3 documentable templates
[info] Main Scala API documentation successful.
Hello
[success] Total time: 1 s, completed Aug 24, 2014 9:32:15 PM
// change the value of sources in the current session only - use session save to retain it
> set sources in (Compile, doc) <<= sources in (Compile, doc) map { _.filterNot(_.getName endsWith ".scala") }
[info] Defining compile:doc::sources
[info] The new value will be used by compile:doc
[info] Reapplying settings...
[info] Set current project to hello (in build file:/Users/jacek/sandbox/sbt-learning-space/)
// display current setting's value - it only holds the single java file
> show compile:doc::sources
[info] ArrayBuffer(/Users/jacek/sandbox/sbt-learning-space/src/main/java/Hello.java)
[success] Total time: 0 s, completed Aug 24, 2014 9:33:23 PM
// generate docs - javadoc is properly generated
> doc
[info] Main Java API documentation to /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api...
[info] Loading source file /Users/jacek/sandbox/sbt-learning-space/src/main/java/Hello.java...
[info] Constructing Javadoc information...
[info] Standard Doclet version 1.7.0_65
[info] Building tree for all the packages and classes...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/Hello.html...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/package-frame.html...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/package-summary.html...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/package-tree.html...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/constant-values.html...
[info] Building index for all the packages and classes...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/overview-tree.html...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/index-all.html...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/deprecated-list.html...
[info] Building index for all classes...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/allclasses-frame.html...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/allclasses-noframe.html...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/index.html...
[info] Generating /Users/jacek/sandbox/sbt-learning-space/target/scala-2.10/api/help-doc.html...
[info] Main Java API documentation successful.
Hello
[success] Total time: 1 s, completed Aug 24, 2014 9:34:01 PM
You could try and integrate genjavadoc into sbt, in order to generate a javadoc.
The ScalaDoc can still be generated using the normal doc task,
whereas the JavaDoc can be generated using genjavadoc:doc.
Another approach, allowed with scala 2.12.0-RC1 (Sept. 2016) and SI 4826 is to let scaladoc run! It will process javadoc comments!
Scaladoc already processes Java sources and produces Scaladoc for them.
It just ignores the doc comments.
Many of our Scala codebases have at least some Java in them for various reasons.
It'd be nice if Scaladoc could generate full documentation for these mixed codebases, including all doc comments.

Can't run GwtTestCase when updating GWT to 2.6.0

After updating my GWT version to 2.6.0, i got this error when running my old GwtTestCase :
[ERROR] The -out option is deprecated. This option will be removed in future GWT release and will throw an error if it is still used. Please use -war option instead.
[INFO] <b>[ERROR] RunStyleHtmlUnit: Unknown browser name FF3, expected browser name: one of [IE8, IE9, Chrome, FF17]</b>
[INFO] [ERROR] shell failed in doStartup method
[INFO] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.682 sec <<< FAILURE!
[INFO] testSerializeDeserialize(m6.sherpa.portal.widget.base.client.components.frame.GwtTestSerializer) Time elapsed: 1.641 sec <<< ERROR!
[INFO] com.google.gwt.junit.JUnitFatalLaunchException: Shell failed to start
[INFO] at com.google.gwt.junit.JUnitShell.getUnitTestShell(JUnitShell.java:732)
[INFO] at com.google.gwt.junit.JUnitShell.runTest(JUnitShell.java:705)
[INFO] at com.google.gwt.junit.client.GWTTestCase.runTest(GWTTestCase.java:421)
[INFO] at junit.framework.TestCase.runBare(TestCase.java:141)
[INFO] at junit.framework.TestResult$1.protect(TestResult.java:122)
[INFO] at junit.framework.TestResult.runProtected(TestResult.java:142)
[INFO] at junit.framework.TestResult.run(TestResult.java:125)
[INFO] at junit.framework.TestCase.run(TestCase.java:129)
[INFO] at com.google.gwt.junit.client.GWTTestCase.run(GWTTestCase.java:247)
[INFO] at junit.framework.TestSuite.runTest(TestSuite.java:255)
[INFO] at junit.framework.TestSuite.run(TestSuite.java:250)
[INFO] at junit.framework.TestSuite.runTest(TestSuite.java:255)
[INFO] at junit.framework.TestSuite.run(TestSuite.java:250)
[INFO] at junit.framework.TestSuite.runTest(TestSuite.java:255)
[INFO] at junit.framework.TestSuite.run(TestSuite.java:250)
[INFO] at org.codehaus.mojo.gwt.test.MavenTestRunner.doRun(MavenTestRunner.java:105)
[INFO] at junit.textui.TestRunner.start(TestRunner.java:183)
[INFO] at org.codehaus.mojo.gwt.test.MavenTestRunner.main(MavenTestRunner.java:63)
Any Idea ?
Best way to fix it is to just update your gwt-maven-plugin to 2.6.0 too.
In previous versions (up to and including 2.5.1), the <htmlunit> configuration property defaulted to FF3, but that value is no longer valid in GWT 2.6.0. gwt-maven-plugin 2.6.0 now defaults the property to FF17.
It looks like GWT 2.6.0 no more supports FireFox v3. FF3 has been replaced with FF17.

Per-configuration classpath dependencies doesn't work with test->test in SBT?

I've got a multi-project build with scalania main project as well as exercises and answers (sub)projects.
The scalania project is hosted on GitHub.
I'm trying to set up a SBT project configuration where the test classes are part of the exercises project while the answers project provides the solutions.
I read Per-configuration classpath dependencies in the official documentation of SBT and ended up with the following configuration in the scalania main project:
lazy val exercises = project
lazy val answers = project.dependsOn(exercises % "test->test")
It doesn't seem to work and upon test execution I used to get:
> project answers
[info] Set current project to scalania-answers (in build file:/Users/jacek/oss/scalania/)
> test
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for answers/test:test
[success] Total time: 1 s, completed Oct 27, 2013 1:06:51 AM
It was until I changed answers/build.sbt to the following:
scalaSource in Test := (scalaSource in LocalProject("exercises") in Test).value
It works fine now.
> reload
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
[info] Loading project definition from /Users/jacek/oss/scalania/project
[info] Set current project to scalania-answers (in build file:/Users/jacek/oss/scalania/)
> project answers
[info] Set current project to scalania-answers (in build file:/Users/jacek/oss/scalania/)
> testOnly *s99.P01*
[info] Formatting 19 Scala sources {file:/Users/jacek/oss/scalania/}answers(test) ...
[info] Compiling 19 Scala sources to /Users/jacek/oss/scalania/answers/target/scala-2.10/test-classes...
[info] P01Spec
[info]
[info] P01 solution should
[info] + Find the last element of a list
[info]
[info]
[info] Total for specification P01Spec
[info] Finished in 151 ms
[info] 1 example, 0 failure, 0 error
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 74 s, completed Oct 27, 2013 1:09:07 AM
What's wrong with using project.dependsOn(exercises % "test->test") only? Am I missing something in the build configuration?
Declaring a dependency on tests in another project just makes the classpath available. Running its tests doesn't happen by default because otherwise tests would run multiple times in the common situation of just reusing code.
To run tests in another project, add the discovered tests from the other project to those for the current project:
definedTests in Test :=
(definedTests in Test).value ++
(definedTests in exercises in Test).value

Running ScalaZ3 on Mac

Running a program that uses ScalaZ3. This is on a Max with OS XAt run time I get an exception as follows
Could not find /lib-bin/libscalaz3.dylib
java.io.FileNotFoundException: Could not find /lib-bin/libscalaz3.dylib
at z3.Z3Wrapper.loadLib(Z3Wrapper.java:74)
at z3.Z3Wrapper.loadFromJar(Z3Wrapper.java:52)
at z3.Z3Wrapper.<clinit>(Z3Wrapper.java:32)
at z3.scala.Z3Config.<init>(Z3Config.scala:6)
Later I get a second stack trace
java.lang.UnsatisfiedLinkError: z3.Z3Wrapper.mkConfig()J
at z3.Z3Wrapper.mkConfig(Native Method)
at z3.scala.Z3Config.<init>(Z3Config.scala:6)
at thesis.z3option.Z3SessionCore$class.open(Z3SessionCore.scala:89)
I'm using the jar file at
jar-releases/64/scala-2.9.1/scalaz3-3.2.c.jar
in the github respository.
Edit: Here are the errors I got on step (3) of the set up process: sbt package
n060h152:ScalaZ3 theo$ sbt package
[info] Building project ScalaZ3 4.0a against Scala 2.9.2
[info]
[info] == compute-checksum ==
[info] == compute-checksum ==
[info]
[info] == compile ==
[info] Source analysis: 0 new/modified, 0 indirectly invalidated, 0 removed.
[info] Compiling main sources...
[info] Nothing to compile.
[info] Post-analysis: 273 classes.
[info] == compile ==
[info]
[info] == javah ==
[info] Generating JNI C headers
[info] javah -classpath /Users/theo/Documents/ScalaZ3/target/scala_2.9.2/classes -d /Users/theo/Documents/ScalaZ3/src/c z3.Z3Wrapper
[error] Cannot find annotation method 'bytes()' in type 'scala.reflect.ScalaSignature': class file for scala.reflect.ScalaSignature not found
[error] Error: Class scala.ScalaObject could not be found.
[info] == javah ==
[error] Error running javah: Non-zero exit code.
[info]
[info] Total time: 4 s, completed 24-May-2013 2:52:36 PM
[info]
[info] Total session time: 6 s, completed 24-May-2013 2:52:36 PM