We have got SCCT to produce some pretty web-pages showing the coverage of our unit tests. And now we want to integrate this into our Jenkins continuous integration build.
I have created a Jenkins job to run the SCCT tests but how to a) fail the build if the coverage is below n% and b) how to display these pretty web-pages?
The build is a maven-multi-module project.
Cheers
We are using the jenkins cobertura plugin. SCCT emits a cobertura-format coverage xml file for each module (I didn't spot at first) and the cobertura plugin collates these. One problem is that the cobertura plugin needed to be downgraded to 1.3 as otherwise it thinks that the mojo hasn't been run and doesn't do anything.
Since this is possible for other coverage tools like cobertura via a jenkins plugin, you could either
convert the output of SCCT to something looking like the results of an existing tool and use the matching jenkins plugin.
write your own jenkins plugin, possibly borrowing code from existing plugins for other coverage tools.
Related
I'm a Cucumber and Eclipse beginner and have a few questions and hope you can help me to get through this: I created a sample cucumber test scenario, a sample test steps and a cucumber runner. The scenarios runs fine within eclipse IDE (Neon). I used Maven as the dependency manager. I also installed the Maven command line module. The step code is Java.
Here is the (basic) question: How do I create a jar file from my cucumber test scenario so that execute it via command line so that I can bring the test scenario to Jenkins CI? Is there anything I need to do with Maven BEFORE I can build the jar file?
Thanks a lot folks!
If you run Cucumber using the JUnit runner, then all you have to do to run from a command line is to execute Maven and make sure you invoke a life cycle phase that includes running the unit tests. One way would be
mvn test
An example that might get you up and running can be found at
https://github.com/cucumber/cucumber-java-skeleton
For a variety of admin reasons, our projects have the following structure:
someproject-core
someproject-api
someproject-test
The idea is all the tests are in a separate module.
Question: What plugin can I use to get accurate test coverage?
Use Scoverage: https://github.com/scoverage/sbt-scoverage
It supports multi-module SBT projects and measures statement coverage instead of line or branch coverages.
BTW there is a nice plugin for Sonar to browse coverage reports:
https://github.com/RadoBuransky/sonar-scoverage-plugin
I am developing a plugin for Eclipse, and my plugin need the JaCoCo/Eclemma to analyse the coverage of a Java Project.
What is the best way to integrate the JaCoCo/Eclemma in my plugin?
In my plugin I have a button and when I click in this button, I want to call JaCoCo/Eclemma, something like this:
Coverage = JaCoCo.run( /* Projecto */ );
Jacoco architecture is quite different than other code coverage tools because it does not instrument the code, but uses a JRE agent to capture execution information.
I'm quite new to Jacoco and I'm not an Eclipse expert, but I guess your plugin must somehow:
create an execution
attach the jacoco agent to that execution
execute the tests
retrieve the jacoco output (I suggest the xml format)
parse the output and use it to augment the java source code view
If you are willing to accept a suggestion I like the way Jacoco reports coverage in its HTML report: green for covered lines, red for uncovered and yellow for partial branch coverage.
I would definitely install such plugin!
I'm considering using SBT for a new scala project, but I'm not sure which CI server - hudson / cruise / whatever has support for it. Any ideas?
I know SBT is a little Maven like, but I don't think it can work as a drop in replacement for Maven in the eyes of a CI server.
By now you would have worked through this. But I found simple integration possible by getting Hudson to call a bash script that invoked SBT. I didn't do any further integration. The solution was able to trigger a build and show the output. That is all I needed at the time.
In my experience, Travis CI https://travis-ci.org works great with SBT. See http://about.travis-ci.org/docs/user/languages/scala/ for details on using them together.
Some of my own SBT-based projects on Travis:
https://travis-ci.org/SethTisue/lens-examples
https://travis-ci.org/NetLogo/NetLogo-Headless
https://travis-ci.org/NetLogo/Tortoise
I'm trying to put together a Continuous Integration server that will do the following:
Work with subversion
Use NUnit tests (fail build on failed tests)
Use partcover (fail build on < X% coverage)
Run code against FxCop (fail build on FxCop warnings, given settings)
Run code against StyleCop (fail build on StyleCop warnings, given settings)
Not as important:
Be able to run from a sln file
Be able to publish the application (ClickOnce is setup for the project already)
I'm using TeamCity right now and it doesn't seem to do 3 or 5, and it doesn't have a runner for the newest NUnit.
From the list of plugins that hudson has, it looks like it can do all of these except 3 (and the not as important requests). I've considered writing a plugin for hudons to use partcover, but that's adding more time to setting up a build server.
NAnt can be used as a build script which will build your projects and then execute NUnit and FXCop.
Another option, which is what I use at work, is create a build script for MSBuild and use the MSBuild Community Tasks which support running FXCop & NUnit among other things.
So for my setup CCNet pulls down the source from SVN then calls MSBuild with the main build file. Inside there it builds the projects, runs NUnit, NCover, FXCop, StyleCop etc. and merges the results which are then displayed on the CCNet webpage. Each task can also be set so if there's a failure the build fails.
I haven't used TeamCity but there should be a way to pull down the source and then run an MSBuild or NAnt build script which will then handle the build steps.
It's not a continuous integration server if it's run from a sln file. Perhaps you're mixing build tool and continuous integration. Many CI servers today does nothing but run build scripts made for other tools like NAnt or Maven. Look at NAnt first if it's what you're looking for. NAnt is able to do the build and execute other tools like FXCop (using NAntContrib library). You use CI server to run a build script on a regular basis.