Scala: Code coverage for projects with tests in separate modules - scala

For a variety of admin reasons, our projects have the following structure:
someproject-core
someproject-api
someproject-test
The idea is all the tests are in a separate module.
Question: What plugin can I use to get accurate test coverage?

Use Scoverage: https://github.com/scoverage/sbt-scoverage
It supports multi-module SBT projects and measures statement coverage instead of line or branch coverages.
BTW there is a nice plugin for Sonar to browse coverage reports:
https://github.com/RadoBuransky/sonar-scoverage-plugin

Related

Code RED tools for Scala

I have been a Java developer and have been using code red tools like FindBugs, CheckStyle, PMD-CPD etc., I have integrated these plugin in Jenkins for automated reviews as well.
When I moved to scala, I want to continue with Maven and Jenkins. But I am not able to find code red tools supported in eclipse, maven-plugin & Jenkins plugins for Scala. Can anyone give some pointer to this to address the static code analysis, code coverage etc., ?
There are many cool tools for integrated with scala, i provide list of tools preferred by me below:
Scalastyle https://github.com/scalastyle/scalastyle
Scapegoat https://github.com/sksamuel/scalac-scapegoat-plugin
Wart remover https://github.com/typelevel/wartremover
Linter https://github.com/HairyFotr/linter
CPD https://github.com/sbt/cpd4sbt
Abide https://github.com/scala/scala-abide
UPD
Firstly, for integration between scala and jenkins you must use sbt plugin in jenkins. Secondly, there are several useful plugins for code quality in jenkins (scapegoat, see link above, it's flexible for your case wiht scala and jenkins), scoverageplugin https://wiki.jenkins-ci.org/display/JENKINS/Scoverage+Plugin for code coverage metrics and of course own commands of sbt, they are very helpful, like sbt doc, sbt test. I found nice article about jenkins and scala http://yeghishe.github.io/2015/02/28/continuous-integration-for-scala-projects.html

Using Maven for JUnit testing

I started learning JUnit testing within Eclipse. The plugin that is used to show the test results presents a nice clean view of the test and you can click on items that take you to the areas in code that are under test.
When I started working with Maven, I noticed that you can have Maven carry out your JUnit tests as well. However, because Maven is a command line process and the results get written to the console, the JUnit results get sent there as well. And it looks like crap. You have to parse your way through all the console text to find the results of the test. Everything is just plain old text.
This raises the question as to what purpose Maven has in testing with JUnit (or any other test frameworks for that matter)? I must be missing the point. Why would anyone want to read a large text dump when the Eclipse plugin provides an elegant way of viewing, executing and evaluating tests?
Generally speaking, you build on the command line because you've got everything working correctly in your IDE. The command line build is the final sanity check and possibly the way in which you are releasing your software (e.g. mvn release:perform etc.).
While there are a few plugins that make Maven test output slightly nicer, the expectation is that the tests will pass. If they don't, fire up Eclipse and run the tests again.
Maven is a build tool. Its goal and purpose is to produce repeatable and protable result. This is especially important when we talk about build/continuous integration servers.
So the normal workflow is/should be: Developers usually develop using their IDE (eclipse), they run their tests in the IDE, because it is developer centric and more comfortable.
The build server, lacking a graphical environment runs the build tool, i.e. maven.
Sometimes, the results between maven and the eclipse might differ, in that case might become necessary for developers to also run maven on their machine.
Another reason to use maven directly might be integration test which specifically us maven lifecycle integrations for say starting and stopping a server.
Some specific points: Maven quite comfortably shows you which tests failed in a summary:
00:46:21.988 Results :
00:46:21.988
00:46:21.988 Failed tests:
00:46:21.988 MyTest.testPersistErrorStateNewTransaction:48 Test for 'testPersistErrorStateNewTransaction()' not yet implemented!
00:46:21.988
00:46:21.988 Tests in error:
00:46:21.988 MyOtherTest.testMethod
00:46:21.989
00:46:21.989 Tests run: 1162, Failures: 1, Errors: 1, Skipped: 491
00:46:21.989
00:46:22.003 [ERROR] There are test failures.
00:46:22.004
Also, when run, from maven, you can still open the results of the surefire tests in your eclipse junit view by double-clicking on a test result (.xml) in the surefire-reports directory.
what purpose Maven has in testing with JUnit?
Unit testing is one of the part in Application Development(usually do while coding/developing components), Apache Maven is the project management tool, (as Duncan told) it helps us to releasing software.
includes - Dependency Management, Module Management, Plugin Management, and reporting configuration for tests.
Maven Objectives:
Making the build process easy
Providing a uniform build system
Providing quality project information
Providing guidelines for best practices development
Allowing transparent migration to new features
please look at below threads for more details about Maven:
Why do so few people use Maven?
Why maven ? What are the benefits?
Why would anyone want to read a large text dump when the Eclipse plugin provides an elegant way of viewing, executing and evaluating tests?
For software release purpose we need maintain statistics of the application(how many scenarios covered, test cases passed..etc). Maven supports different plugins to format the results, Maven Surefire Plugin, maven-site-plugin..etc plugins help us to generate reports different formats.
please refer below threads for more details:
JUnit3 and Junit4 XML Reports with Maven
Is there a decent HTML Junit report plugin for Maven?

Test suite generator for guava project

In the google-collections gtug videos there was a mention of test suite generator and plans to open source it.
Currently in guava git project there are two maven modules
guava-testlib
guava-tests
Are these two modules the next version of the test suite generator?
At cursory glance the code within these modules seemed very much specific to guava project.
Is there an easy way to utilize the library/code/strategies used in test suite generator?
Yes, the automatic test suite generator for testing collection implementations is the bulk of what's in guava-testlib. This post explains how to use it.

Display SCCT coverage in Jenkins

We have got SCCT to produce some pretty web-pages showing the coverage of our unit tests. And now we want to integrate this into our Jenkins continuous integration build.
I have created a Jenkins job to run the SCCT tests but how to a) fail the build if the coverage is below n% and b) how to display these pretty web-pages?
The build is a maven-multi-module project.
Cheers
We are using the jenkins cobertura plugin. SCCT emits a cobertura-format coverage xml file for each module (I didn't spot at first) and the cobertura plugin collates these. One problem is that the cobertura plugin needed to be downgraded to 1.3 as otherwise it thinks that the mojo hasn't been run and doesn't do anything.
Since this is possible for other coverage tools like cobertura via a jenkins plugin, you could either
convert the output of SCCT to something looking like the results of an existing tool and use the matching jenkins plugin.
write your own jenkins plugin, possibly borrowing code from existing plugins for other coverage tools.

Run All the Tests

Is there a way to run all the tests in multiple eclipse projects?
I have a maven multi module project and want to use emma to show me code, that is not covered by any tests, not matter in which submodule it lies. So my idea is to have a single emma-coverage run, that includes all the tests of all my modules.
is there a way to do this?
You could have a look at jacoco which runs your tests in an ant/emma context and generates a report - the report component is able to merge the results from each submodule into a single report.