While researching on the subject of automating my integration tests, I found out a nice plugin in the maven world called FailSafe. it gives me phases like pre-integration-test, post-integration-test and integration-test.
By tying into these phases, I can have other plugins which can start/stop and run docker images.
The plugin also has a nice way in which I can differentiate between UnitTests and IntegrationTests (even though both are written in JUNIT).
Well now the question is how can I do the same thing with Scala / SBT combination?
my requirement is
Write Integration tests in SpecFlow.
Integration tests are treated differently than unit tests.
First Unit Tests are run.
Then docker containers are created and run
then integration tests are run.
docker contains are shut-down.
test results are captured in files. (just like surefire/failsafe plugins).
Is this possible in Scala/sbt combo?
A simple solution is to run $ sbt "~ it:test" (make sure integration test are in a package named 'it') for integration test which will automatically run every time source code is updated. Furthermore, $sbt "~ test" for automated unit testing. If you are using a IDE such as IntelliJ IDEA, you can make it easier to run this in a custom configuration from the IDE. Hope this helps a little bit. I run these all the time when working.
I found the answer to the question. SBT provides means to do integration test and also setup and cleanup methods to do things like creation / destruction of docker containers
http://www.scala-sbt.org/0.13/docs/Testing.html
Related
I am working on a Java application project in NetBeans. I have added JUnit tests, with the #Test annotation. The tests work fine when I run them from the ‘Run Tests’ menu item.
I want to run the same tests from a command-line script. I do not know how NetBeans does this. I want a single command for testing the whole project. How can I achieve this?
Netbeans uses Ant as a build tool, and its default Ant configuration contains a number of useful targets. You can list them with the command ant -projecthelp (executed in your project directory), which should output, amongst other things,
test Run unit tests.
test-single Run single unit test.
test-single-method Run single unit test.
So, simply executing ant test will run all your unit tests on the command line. This will run the tests, write the test results to the standard output, and finish up with a brief summary (including a BUILD SUCCESSFUL message if all the tests passed, or BUILD FAILED if not).
The ant test command will return an appropriate exit code (0 if all tests passed, 1 if any failed), which can be useful -- for example, if you want to add a pre-commit hook to your version control system to forbid commits with failing unit tests.
You should build your project on the command-line with a build tool like Maven, Ant or Gradle and not rely on Netbeans for building it. Please search for this tools.
You should also consider to read a book about software development like Head First Software Development. They explain why and how to us a build tool (e.g. chapter 6 1/2 of Head First Software Development).
How could I debug or run the BasicSimulation.scala?
Now I use sbt command to run:
testOnly simulations:BasicSimulation
Is there any other easy way to debug? Thanks.
You can add a simple class and run it by right click->run . Something like here: https://github.com/puppetlabs/gatling-puppet-load-test/blob/master/simulation-runner/src/main/scala/com/puppetlabs/gatling/runner/PuppetGatlingRunner.scala
If you want to run gatling tests inside the intellij, you can go to the edit configurations in the run toolbar on the top left and add a new sbt task.
On the task field enter testOnly simulations:BasicSimulation and that's it.
Normally you should be able to debug it by clicking the debug button, but it doesn't seems to be possible for the gatling tests at least in my configuration.
According to actual documentation this is done like:
gatling:testOnly *BasicSimulation*
Here the Documentation
As with any SBT testing framework, you’ll be able to run Gatling
simulations using SBT standard test, testOnly, testQuick, etc… tasks.
However, since the SBT Plugin introduces many customizations that we
don’t want interfering with unit tests, those commands are integrated
into custom configurations, meaning you’ll need to prefix them with
gatling or gatling-it, eg. gatling:test or gatling-it:test.
(I tested this in sbt console)
Everything is nicely explained in the official documentation page of the gatling project:
http://gatling.io/docs/2.2/extensions/maven_archetype/
Just use their archetype to generate a project in maven and then import in any decent IDE and voilà
The contents of my jar file does not contain my junit tests. I tried creating a main class that calls my tests with "Junit Core" but that too doesn't get added to my jar file. What am I missing?
Your production package (your jar) is not supposed to contain the test code (your unit tests).
Test code is supposed to be invoked only when the tests are been executed.
You shouldn't do it, but if you want the test code to be included in your package just put it in the same root folder of your production code. Ex: if you are using Maven, your folders structure might be something like this:
project_name/src/main/java/
-> production code
project_name/src/test/java/
-> test code
In that case, change it to:
project_name/src/main/java/
-> production code
-> test code
Be aware that doing that you will have some problems, like the tests not executing during a normal build. We might help you better if you put a question asking how to accomplish your goal (what is your goal?).
Update based on the goal explained in the comments: Would be better if this tool (Silk Central) could run your tests using Maven or Ant, and I'm pretty sure it can.
I've been used different tools to run my Junit (or other XUnit frameworks) tests. What I do is to configure the tool to:
Get the source code from the version control.
Run the build/tests by calling Maven or Ant installed along the tool.
Read the Junit outputs.
Sometimes the tool has its own integration with Junit and you don't have even to use Maven or Ant.
I mean, your central build or tests tool should be able to run your tests just like you do in your own machine.
Take a look on this or search by "Silk Central junit" in the web:
http://community.microfocus.com/borland/test/silk_central/w/wiki/465.zero-maintenance-junit-testing-using-classpathsuite-in-sctm.aspx
I started learning JUnit testing within Eclipse. The plugin that is used to show the test results presents a nice clean view of the test and you can click on items that take you to the areas in code that are under test.
When I started working with Maven, I noticed that you can have Maven carry out your JUnit tests as well. However, because Maven is a command line process and the results get written to the console, the JUnit results get sent there as well. And it looks like crap. You have to parse your way through all the console text to find the results of the test. Everything is just plain old text.
This raises the question as to what purpose Maven has in testing with JUnit (or any other test frameworks for that matter)? I must be missing the point. Why would anyone want to read a large text dump when the Eclipse plugin provides an elegant way of viewing, executing and evaluating tests?
Generally speaking, you build on the command line because you've got everything working correctly in your IDE. The command line build is the final sanity check and possibly the way in which you are releasing your software (e.g. mvn release:perform etc.).
While there are a few plugins that make Maven test output slightly nicer, the expectation is that the tests will pass. If they don't, fire up Eclipse and run the tests again.
Maven is a build tool. Its goal and purpose is to produce repeatable and protable result. This is especially important when we talk about build/continuous integration servers.
So the normal workflow is/should be: Developers usually develop using their IDE (eclipse), they run their tests in the IDE, because it is developer centric and more comfortable.
The build server, lacking a graphical environment runs the build tool, i.e. maven.
Sometimes, the results between maven and the eclipse might differ, in that case might become necessary for developers to also run maven on their machine.
Another reason to use maven directly might be integration test which specifically us maven lifecycle integrations for say starting and stopping a server.
Some specific points: Maven quite comfortably shows you which tests failed in a summary:
00:46:21.988 Results :
00:46:21.988
00:46:21.988 Failed tests:
00:46:21.988 MyTest.testPersistErrorStateNewTransaction:48 Test for 'testPersistErrorStateNewTransaction()' not yet implemented!
00:46:21.988
00:46:21.988 Tests in error:
00:46:21.988 MyOtherTest.testMethod
00:46:21.989
00:46:21.989 Tests run: 1162, Failures: 1, Errors: 1, Skipped: 491
00:46:21.989
00:46:22.003 [ERROR] There are test failures.
00:46:22.004
Also, when run, from maven, you can still open the results of the surefire tests in your eclipse junit view by double-clicking on a test result (.xml) in the surefire-reports directory.
what purpose Maven has in testing with JUnit?
Unit testing is one of the part in Application Development(usually do while coding/developing components), Apache Maven is the project management tool, (as Duncan told) it helps us to releasing software.
includes - Dependency Management, Module Management, Plugin Management, and reporting configuration for tests.
Maven Objectives:
Making the build process easy
Providing a uniform build system
Providing quality project information
Providing guidelines for best practices development
Allowing transparent migration to new features
please look at below threads for more details about Maven:
Why do so few people use Maven?
Why maven ? What are the benefits?
Why would anyone want to read a large text dump when the Eclipse plugin provides an elegant way of viewing, executing and evaluating tests?
For software release purpose we need maintain statistics of the application(how many scenarios covered, test cases passed..etc). Maven supports different plugins to format the results, Maven Surefire Plugin, maven-site-plugin..etc plugins help us to generate reports different formats.
please refer below threads for more details:
JUnit3 and Junit4 XML Reports with Maven
Is there a decent HTML Junit report plugin for Maven?
I'm finding it difficult to phrase this question well, as there are quite a few generic terms (run, configuration, launch, etc.). Here goes:
You can save run configurations in a .launch file. (in the Run Configuration Dialog, under the Common tab, Save as a shared file.
We check these in to SVN. The developers can pass them around, and it helps getting new devs running a working application quicker.
I'd like to check these out as part of our build and use them to programatically run the application, the tests, etc, without spinning up the whole IDE.
What would be the best way to run a .launch file outside of the UI?
Edit: I am trying to unify the tests run on the build server and the IDE. I do not
particularly want to give up integrated debugging, which would be the case with an ant script to run the tests .
This is probably more a problem for integration testing with multiple bundles, or unit testing a whole bundle, where you'd like to mock up extensions.
there is an eclipse plugin built over JUnit, called TPTP. It provides an automation client which can be used to launch the test from eclipse with no gui. maybe it helps
Ant4Eclipse may provide a good starting point on how to do this.
Unfortunately, this is limited to Java Applications and JUnit configurations; I am more interested in PDE applications and Plugin JUnit tests.
I have recently had alot of success building an Eclipse RCP app inside a Hudson CI server using Eclipse Buckminster. It took a bit of doing, but once I setup both features, made my RCP product be based on features, and added the Buckminster query files and the like, it worked. There is a Hudson/Jenkins Buckminster plugin that allowed me to have hudson build the application.
After saving the launch configurations for each test fragment, I created hudson commands to invoke them (yes one line per test fragment unfortunately), but after that I got the automated CI build that I wanted.
You could also use the shell command Eclipse uses. To get it:
Run your program in Eclipse
Go to the "Debug" view
Right-click on the process (probably the second item in the tree) and select "Properties"
Copy shell command and delete the agentlib flag to run in bash
I think you don't need to use the .launch configurations to run the tests. If you build an application using the Eclipse Build System, then you can use the AntRunner application from Eclipse to run your units tests. This doesn't start the whole IDE.
This article describes how to run the tests during your build process. With this process, you use a special "Test" Eclipse and load the plugins you want to test.
Perhaps running the configurations the way you would run your own custom run configurations would help here. It is described in this article.