I'm trying to set up simple code coverage reports for a team coding in mixed Scala/Java at approx. a 90/10 ratio and running into some serious roadblocks. I've previously set up & administrated Sonar to great success with a Java-only team, but it doesn't appear to be an option.
Sonar w/Scala plugin is buggy and appears to support Scala-only projects, not mixed ones.
SCCT integrates with our maven build, but fails out with false-negative test failures repeatedly.
Undercover has been my best luck so far; It's integrated with our maven build & generates reports, but they aren't archived or hosted anywhere as they would be with Sonar. There also appears to be no central index to make it simple to navigate the generated reports.
I've read the answers here on StackOverflow, but they largely date back to 2010 and suggest that no decent solution is available. Has this changed?
Is there something obvious I'm missing?
About Sonar side:
yes, the Scala Sonar Plugin development is currently stalled. It was initiated by the community, but nobody has offered to take it over yet. If there are some volunteers, we'll be glad to guide and help them.
concerning the support of several languages inside a single project, support will be coming in Sonar. I can't give you a roadmap for it, but we're currently thinking about how to add this support in Sonar in the next releases, so this is a short term issue.
You can either use SCCT or JaCoCo.
SCCT: It supports Scala up to version 2.10, but development seems to be a stalled for a about 9 months. It supports Scala natively and works with both, Maven and SBT.
JaCoCo is under sctive development. It supports any version of Scala, but not natively, but on bytecode level. So you might get some artifacts, e.g. some code gets only partial coveragege, because the generated bytecode has some theoretical code path JaCoCo sees (but which can never be executed from Scala code).
JaCoCo can be a little tricky to set up with Maven and Scala. Here a few tricks:
Use the variant with the agent launcher. Do not use the variant with preprocessing bytecode.
When using JaCoCo with Maven: There is a Maven task (jacoco:prepare-agent) which will produce the correct expression for the agent launcher and stores it into a property. You can then use this property as a command line parameter when running the Java virtual machine.
Parametrize the agent launcher, so that multiple launches (e.g. for running different tests) write to the same log file. Some IDE plugins will have problems with parsing such a file, but the JaCoCo Hudson plugin for example works fine.
Related
I'm currently using JUnit 5.1, but I want to switch to JUnit 5.5 (at least 5.4) to use new features of that version (mainly the #Order annotation), but I'm really struggling to properly set it up.
Unfortunately I cannot simply switch to a newer eclipse version, because I'm tied a specific version we use at my workplace.
As there is no JAR-download, i've tried to build it from source. The only guidelines concerning this are saying to build it with Gradle, but I don't understand how to do this.
What I tried so far:
- downloaded the repository as an archive (zip) and imported it to eclipse, which resulted in multiple projects and <1.000 errors
- imported the repository into eclipse (via import wizard), but failed to find a way to make a build from it
I was able to run some test with JUnit 5.4 features, but i achieved this by downloading different modules as JARs from some maven-repository I found in a JUnit issue on github and including them as external JARs into the classpath. This was very tedious because of the number of different JARs & the way they were organized in the repository folders.
To me this feels like a workaround, as I still have no idea how to solve the initial problem of building JUnit from source.
I'm quite new to eclipse and have a basic understanding of what Gradle is, but I've never used it so far.
I don't expect a detailed step-by-step guide on how to achieve this (although I wouldn't complain ;) ), but I'd really appreciate it, if you could give me an outline of how building from source is done in general and maybe additional references for me to read about certain steps (e.g. setting up gradle).
I started learning JUnit testing within Eclipse. The plugin that is used to show the test results presents a nice clean view of the test and you can click on items that take you to the areas in code that are under test.
When I started working with Maven, I noticed that you can have Maven carry out your JUnit tests as well. However, because Maven is a command line process and the results get written to the console, the JUnit results get sent there as well. And it looks like crap. You have to parse your way through all the console text to find the results of the test. Everything is just plain old text.
This raises the question as to what purpose Maven has in testing with JUnit (or any other test frameworks for that matter)? I must be missing the point. Why would anyone want to read a large text dump when the Eclipse plugin provides an elegant way of viewing, executing and evaluating tests?
Generally speaking, you build on the command line because you've got everything working correctly in your IDE. The command line build is the final sanity check and possibly the way in which you are releasing your software (e.g. mvn release:perform etc.).
While there are a few plugins that make Maven test output slightly nicer, the expectation is that the tests will pass. If they don't, fire up Eclipse and run the tests again.
Maven is a build tool. Its goal and purpose is to produce repeatable and protable result. This is especially important when we talk about build/continuous integration servers.
So the normal workflow is/should be: Developers usually develop using their IDE (eclipse), they run their tests in the IDE, because it is developer centric and more comfortable.
The build server, lacking a graphical environment runs the build tool, i.e. maven.
Sometimes, the results between maven and the eclipse might differ, in that case might become necessary for developers to also run maven on their machine.
Another reason to use maven directly might be integration test which specifically us maven lifecycle integrations for say starting and stopping a server.
Some specific points: Maven quite comfortably shows you which tests failed in a summary:
00:46:21.988 Results :
00:46:21.988
00:46:21.988 Failed tests:
00:46:21.988 MyTest.testPersistErrorStateNewTransaction:48 Test for 'testPersistErrorStateNewTransaction()' not yet implemented!
00:46:21.988
00:46:21.988 Tests in error:
00:46:21.988 MyOtherTest.testMethod
00:46:21.989
00:46:21.989 Tests run: 1162, Failures: 1, Errors: 1, Skipped: 491
00:46:21.989
00:46:22.003 [ERROR] There are test failures.
00:46:22.004
Also, when run, from maven, you can still open the results of the surefire tests in your eclipse junit view by double-clicking on a test result (.xml) in the surefire-reports directory.
what purpose Maven has in testing with JUnit?
Unit testing is one of the part in Application Development(usually do while coding/developing components), Apache Maven is the project management tool, (as Duncan told) it helps us to releasing software.
includes - Dependency Management, Module Management, Plugin Management, and reporting configuration for tests.
Maven Objectives:
Making the build process easy
Providing a uniform build system
Providing quality project information
Providing guidelines for best practices development
Allowing transparent migration to new features
please look at below threads for more details about Maven:
Why do so few people use Maven?
Why maven ? What are the benefits?
Why would anyone want to read a large text dump when the Eclipse plugin provides an elegant way of viewing, executing and evaluating tests?
For software release purpose we need maintain statistics of the application(how many scenarios covered, test cases passed..etc). Maven supports different plugins to format the results, Maven Surefire Plugin, maven-site-plugin..etc plugins help us to generate reports different formats.
please refer below threads for more details:
JUnit3 and Junit4 XML Reports with Maven
Is there a decent HTML Junit report plugin for Maven?
I have recently completed the Scala course on Coursera, and since then I have been looking forward to getting my hands dirty with Scala again. I have written code for some years but I neither educated to be nor work as a programmer, so it took me a while to get a good opportunity but now that I have some time to invest and a good project to work on it's time...
Except I can't seem to get things set up properly, which I find really frustrating. I have OpenJDK 1.7.0_25 running on my Linux machine. I have downloaded and installed the Bundle Scala IDE build for Eclipse (just like we used in the course). And I got ScalaTest both as a jar file and the Eclipse plug-in.
I have a simple project (so far) and no matter what I do I can't seem to get my builds and tests in order. First off how exactly am I supposed to set up my project so that my classes and tests are actually run properly? All the assignments we got were projects that had the same structure, so do I have to have:
project
|--src
|--main
|--scala
|--test
|--scala
structure? If so why is it not the default way the project is setup when I create a new project? Do I create these folders manually, as packages or as source folders? The whole thing gets pretty murky..
I should mention that I tried to "Mavenize" the project using the contextual menu in Eclipse, added my ScalaTest dependency. The first thing that happens is that I get compile errors, at every point of dependency in my code. So clearly the library is not visible, in other words Maven does not seem to be doing much of management. I thought the whole point of Maven was to get and maintain dependencies as the project evolves. I concluded that I do not fully understand the way Maven works and thus I eventually gave up on Maven, once again, and went back to doing things manually.
Secondly, I can't seem to run my tests; the Run As... menu item does not include ScalaTest as it's mentioned in the documentation of ScalaTest Eclipse Plug-in. I have double checked that the plugin is installed. If I instead try to run using JUnitRunner then my tests are not recognized as valid tests. I have JUnit and ScalaTest on my build path, so it's got to be something else.
I suppose my overarching question is as follows:
given the Scala IDE build of Eclipse and ScalaTest, just exactly how am I supposed to set up my project (in Eclipse) so that I can just focus on writing my code and testing it, and hopefully not have any other headaches?
I work alone, and this project is not a product I need to deliver to some client. In other words I do not need to adhere to strict professionalism here. Honestly I just want to be able to code, get better acquainted with Scala and hopefully build a small data analysis tool that I will be using from time to time.
Thanks in advance!
Try using the sbt eclipse plugin:
https://github.com/typesafehub/sbteclipse
This is of course assumes that you use sbt as you build tool. If you don't at the moment you can find instructions on installation and usage here: http://www.scala-sbt.org/
Personally I've been using typesafe giter8 template (https://github.com/typesafehub/scala-sbt.g8) to setup my Scala projects, and then I use the sbt plugin mentioned above to generate eclipse project files.
Scala is somewhat Maven-based (sometimes implicitly), that's why you use that structure.
The easiest way I think is to create a simple Sbt/Maven POM and create the Eclipse project configurations (like with sbt eclipse). There you can set the dependencies (like the actual version of JUnit, Scalatest to use), so you can use the ScalaTest plugin easily.
In case of other issues, feel free to ask at the ScalaTest mailing list, Chee Seng and Bill Venners can help you a lot there.
The Scala IDE website has a full documentation on how to run unit testing frameworks with the IDE, have a look ! If you find missing elements, the bug tracker of the scala-IDE project is here.
Not long before I had digged into Ant and use it doing simple practice(compile,unit test,package).Now my usual work is under Eclipse for java development,the usual work can all be simply done through Eclipse' function:in-time compiling、click button to deploy、various wizard..
So I am wondering under Eclipse do we have a need for Ant.
And if it is, at which aspect should we use Ant along with Eclipse.
Building applications is not about just building them in your IDE. You need to setup an automated build and deploy method that can be run for, say, continuous integration and deployment. The built package has to be deloyed to various environments and finally production. You cannot build them out of your own box. You cannot build it out of your IDE. You need separate build system. That is where build tools like ANT come in.
Also note that the capabilities of IDEs for doing build, deploy is limited and that is not their strength. It is, however, possible to make use of ANT and its build scripts and run them out of eclipse. This way, you have a single set of scripts for both your local ( developer ) build and the main build process. With ANT being JAVA based, you can also suit your build tool to your needs by writing custom tasks etc.
Also, when you make use of something as standard as ANT for your build system, you can move to a different IDE like Intellij IDEA and still make use of the build system that you know.
Do you need Ant if you can build through the IDE? Of course not! You can see that yourself.
Should you still have a IDE independent build with standard tools? Of course!
Note I didn't say Ant. There are a number of build tools such as Maven or Gradle that are made to build Java programs. However, there are reasons why you want an IDE independent build system:
You have a Continuous Integration Server: You do have one. Right? If you don't get one. A Continuous Integration Server automatically builds your project (and runs every test) each time a change is committed to the version control system. It catches mistakes. It also provides an OFFICIAL BUILD. No more It worked on my machine! excuses. If the Continuous Integration Server can't build it and successfully run the tests, the software isn't good.
You have others who need to work with your project on a different computer in a different environment: The problem with IDEs is that its specific to your system. Someone else checks out your project, and they may be using a different IDE. They maybe have a slightly different setup. An independent build scripting language like Ant guarantees that they can rebuild your project as intended.
If you share a project with someone else, you need an independent standard method way of being able to build that project and to know exactly what is required for that build to work. IDEs can hide that information.
I am a benchmark tester using Ant everyday. Our environment has 7 test machines and 1 console machine. For each test run, the ant script updates the working copy, restarts the database, build the product and the Grinder with proper properties and deploy them to the machines, and starts them and the monitoring softwares. With the 1000-line-buildfile, I can just type in an "ant A B C D E"-like line in the terminal on the console machine. I don't think it's a good idea to do this in a IDE like eclipse or NetBeans or anything else.
Also, in my company's product, ant is widely used. We use ant to build sample data for the benchmark test; we use ant to update Tomcat which is going to be bundled with our product; we also use ant to build and run selenium tests.
What's more, when you want to release the source code, including the IDE project files is not a good idea because there are a lot of IDEs. And what if the user just want to get a fresh build from your latest code (if you don't provide nightly build)? Providing a set of ant (or other build tool) buildfiles is a good idea. With this, you can manage your classpath for the build.
The shortest answer here wold be flexibility. The IDEs may came with pre-defined functionalities and may even allow you to choose some options using wizards. It is like generating html with Dreamweaver or such tools. It will never be as flexible as the pure thing. For example NetBeans uses Ant underneath.
Quoted from: Netbeans: Build Tools
The default NetBeans IDE project system is built directly on top of
the Ant build system. All of the project commands, such as "Build
Project" and "Run File in Debugger", call targets in the project's Ant
script. You can therefore build and run your project outside the IDE
exactly as it is built and run inside the IDE.
Even after you use tools for that purpose you are most likely still to brush up. So, what I am saying is, what the IDEs do for you is only a subset of what you can do with the pure ant or maven( if you use it). For simple tasks such as compile and run, IDEs do pretty much a good job! But if you have a use case as complex as deploying on CI IDEs won't help you. You need the whole flexibility of your build tools.
Item 2 of the Joel Test is "Can you make a build in one step", but what is the best (or commonly accepted) way to achieve this? What are the pros and cons of using the IDE's command-line interface to do a headless build, as opposed to maintaining a build script that is completely independent of the IDE (e.g. using Ant or Maven in the case of a Java project).
I ask this question because I am experimenting with Maven, and was a bit surprised to find that even when using the m2eclipse plugin it's not really feasible to turn off Eclipse's Java Builder and delegate the whole build process to Maven while working on the project. It seems that if I want to migrate the project to Maven, I'll end up having to maintain two equivalent but different build processes, for example the command-line build will use the DataNucleus Maven plugin for enhancing JDO classes, while the IDE build will use the DataNuclues plugin for Eclipse; the command-line build will use the Tomcat Maven plugin, while the IDE will use Eclipse's web tools platform. This redundancy seems unfortunate.
I gather that NetBeans uses Ant for its build, which sounds like it would solve this problem. But unfortunately I'm using neither NetBeans nor Ant!
EDITED TO ADD: I found that I can set up a Maven builder in Eclipse with specific goals that call some of the Maven plugins that I'm using (e.g. in my case the goals "datanucleus:enhance process-resources" take care of JDO enhancement, resource copying, and native2ascii conversion). This leaves java compilation up to Eclipse, but still achieves some degree of integration with Maven.
The eclipse builder gives you intermediate and repeated building, but the "Joel Test" is really about being able to go from source to ready-to-deploy something in a single step. If you're using Maven, there's several ways of going about it - including just invoking Maven as a command line script from within Eclipse.
mvn package
Is what I most frequently used for a quick run through the whole build setup. You can extend Maven through the POM and inject some additional mechanisms if you want to. If you get a little more complex, it's often most effective to start using multi-module POM and maven setups to get functional tests integrated and running with the rest of the code.
Basically, let eclipse "do it's thing" with it's builders and take advantage of what it provides, but when you're ready to use the end product have it invoked from the command line through a continuous integration server (Hudson is a nice easy one to get and set up - free too: http://hudson-ci.org/). Presumably if you're looking at using Maven, you'll also have an instance of Archiva or Nexus set up as your DSL for the resulting libraries. You can have hudson invoke "mvn deploy" for regular checkin builds, or "mvn release:prepare && mvn release:perform" for when you're ready to cut the release (separate builds in Hudson work best for this)
You can do a headless build in an IDE. In eclipse you can execute an arbitrary shell command as an "External Tool". This is also true in IDEA and netbeans with a little effort. At the very least, this is a convenient test of the headless build that you should run whenever you make changes to the build configuration.
Also, I'd like to add that the build should be one-step as a minimum requirement. It should also be easy to set up and easy to debug. If it takes longer than an couple of hours to set up a new developers environment, then the one-step process is likely to be less than optimal. This is with the caveat that if you add or replace team members quarterly or annually this is less of an issue.