I am running Scala (and Spark) tests using FunSuite in Eclipse (Kepler). I have been able to successfully run the tests. I then created a new package (in the same source folder and project) and am running into a problem.
When I select run as:
I am able to successfully run my tests using both the 1st (ScalaTest – File) and 3rd (ScalaTest – Test) options. However, when I try and run method 2 (ScalaTest – Suite), I get:
***RUN ABORTED ***
java.lang.ClassNotFoundException: com.domain.gotime.rdd.Test1
Why is that I am getting this error for the Suite only? I am planning on this being run automatically with Jenkins – and want to know if I need to fix this, and if so how?
I have looked over this page but am still confused as to why it isn't working.
Please check that com.domain.gotime.rdd.Test1 is in the classpath of the auto-created Run Configuration.
Related
To speed up our development workflow we split the tests and run each part on multiple agents in parallel. However, compiling test sources seem to take most of the time for the testing steps.
To avoid this, we pre-compile the tests using sbt test:compile and build a docker image with compiled targets.
Later, this image is used in each agent to run the tests. However, it seems to recompile the tests and application sources even though the compiled classes exists.
Is there a way to make sbt use existing compiled targets?
Update: To give more context
The question strictly relates to scala and sbt (hence the sbt tag).
Our CI process is broken down in to multiple phases. Its roughly something like this.
stage 1: Use SBT to compile Scala project into java bitecode using sbt compile We compile the test sources in the same test using sbt test:compile The targes are bundled in a docker image and pushed to the remote repository,
stage 2: We use multiple agents to split and run tests in parallel.
The tests run from the built docker image, so the environment is the
same. However, running sbt test causes the project to recompile even
through the compiled bitecode exists.
To make this clear, I basically want to compile on one machine and run the compiled test sources in another without re-compiling
Update
I don't think https://stackoverflow.com/a/37440714/8261 is the same problem because unlike it, I don't mount volumes or build on the host machine. Everything is compiled and run within docker but in two build stages. The file modified times and paths are retained the same because of this.
The debug output has something like this
Initial source changes:
removed:Set()
added: Set()
modified: Set()
Invalidated products: Set(/app/target/scala-2.12/classes/Class1.class, /app/target/scala-2.12/classes/graph/Class2.class, ...)
External API changes: API Changes: Set()
Modified binary dependencies: Set()
Initial directly invalidated classes: Set()
Sources indirectly invalidated by:
product: Set(/app/Class4.scala, /app/Class5.scala, ...)
binary dep: Set()
external source: Set()
All initially invalidated classes: Set()
All initially invalidated sources:Set(/app/Class4.scala, /app/Class5.scala, ...)
Recompiling all 304 sources: invalidated sources (266) exceeded 50.0% of all sources
Compiling 302 Scala sources and 2 Java sources to /app/target/scala-2.12/classes ...
It has no Initial source changes, but products are invalidated.
Update: Minimal project to reproduce
I created a minimal sbt project to reproduce the issue.
https://github.com/pulasthibandara/sbt-docker-recomplile
As you can see, nothing changes between the build stages, other than running in the second stage in a new step (new container).
While https://stackoverflow.com/a/37440714/8261 pointed at the right direction, the underlying issue and the solution for this was different.
Issue
SBT seems to recompile everything when it's run on different stages of a docker build. This is because docker compresses images created in each stage, which strips out the millisecond portion of the lastModifiedDate from sources.
SBT depends on lastModifiedDate when determining if sources have changed, and since its different (the milliseconds part) the build triggers a full recompilation.
Solution
Java 8:
Setting -Dsbt.io.jdktimestamps=true when running SBT as recommended in https://github.com/sbt/sbt/issues/4168#issuecomment-417655678 to workaround this issue.
Newer:
Follow recomendation in https://github.com/sbt/sbt/issues/4168#issuecomment-417658294
I solved the issue by setting SBT_OPTS env variable in the docker file like
ENV SBT_OPTS="${SBT_OPTS} -Dsbt.io.jdktimestamps=true"
The test project has been updated with this workaround.
Using SBT:
I think there is already an answer to this here: https://stackoverflow.com/a/37440714/8261
It looks tricky to get exactly right. Good luck!
Avoiding SBT:
If the above approach is too difficult (i.e. getting sbt test to consider that your test classes do not need re-compiling), you could instead avoid using sbt but instead run your test suite using java directly.
If you can get sbt to log the java command that it is using to run your test suite (e.g. using debug logging), then you could run that command on your test runner agents directly, which would completely preclude sbt re-compiling things.
(You might need to write the java command into a script file, if the classpath is too long to pass as a command-line argument in your shell. I have previously had to do that for a large project.)
This would be a much hackier approach that the one above, but might be quicker to get working.
A possible solution might be defining your own sbt task without dependencies or try to change the test task. For example you could create a task to run a JUnit runner if that was your testing framework. To define a task see this on Implementing Tasks.
You could even go as far as compiling sending the code and running the remotes from the same task as it is any scala code you want. From the sbt reference manual
You could be defining your own task, or you could be planning to redefine an existing task. Either way looks the same; use := to associate some code with the task key
I am trying to run some unit tests in teamcity build configuration. I am using NUnit.ConsoleRunner.3.6.1 . However It says "Has no TestFixture". But when I tried to run it in command prompt using same runner it completed successfuly. What are the possible reasons ?
Really, more info is needed to judge what your problem is. However, most errors of that type indicate that the nunit framework has not been deployed with the test assembly that uses it.
I'm new to using Eclipse for Grails (using STS) and I'm trying to figure out an easy way to run the unit tests. I've seen that I can do it by right clicking Run As > Grails Command (test-app). This works but is slow and the test output goes to the test report html page and has no apparent clickable stack traces.
I can also do Run As > JUnit Test, which appears to be much faster and gives me the traditional JUnit console available in non-Grails tests. When running unit tests, is there a difference in the two? Is the grails command setting up other things or doing anything else?
You are performing a full blown test with all bells and whistles on. :)
According to the docs:
test-app: Runs all Grails unit and integration tests and generates reports.
Setting up the container for the integration tests is what makes it more 'expensive'.
You can limit the test cases that are being run by using 'unit:' as a parameter to indicate that only unit tests need to be run. (When not using JUnit directly from eclipse)
In your case you could do:
test-app unit:
or for a specific FooBarTests.groovy file:
test-app unit: FooBar
optionally you can add -echoOut or -echoErr to get more verbose output.
Check out the docs for more info and different phases of testing.
I developed a simple scala app that uses casbah to query the DB for the command line argument passed to it. For example
$ querydb.scala execution 10
it will run a casbah query to find 10 records matching execution in mongo. Now i have two questions.
1) How do i test this in my local. If i click execute in intellij it is just running the program, i am not able to pass command line arguments to my program.
2) How do i deploy it to run on my server, it is just going to used as console app in my ubuntu server, but im not sure how i should deploy this, which files i should put up on the server and how do i execute it in server, and stuff like that.
Any pointers would be useful for me.
or try to use sbt, IDEA has a plugin with sbt, the wiki of it has an explanation on how to use it.
I usually use sbt directly in Terminal instead of running in IDE.
1) First you need to find "Select Run/Debug Configuration" button at the top of your screen
Click on it and choose edit
Create new one, if you haven't got it yet.
Your program parameters should be written in "Program parameters" field
2) Compile your .scala files with scalac and you'll got .class files.
Then deploy it, as you usually do with java code. Hence you don't need to install scala on target machine - all you need is JDK.
I googled this unexpected error message and there not a single result.
I am using Eclipse Helios (3.6) with Makegood plugin to run PHPUnit test.
PHPUnit is working just fine.
I can also use Makegood to one test class.
But when I run all test, Makegood refuse to do it and display
'Launching <currentfilename>' has encountered a problem.
The main script is not found.
Looks like there are some internal issue with Makegood. I just don t know how to get started debugging this. Is this a eclipse or makegood error message? What does it mean ? Is there any log or debug mode I could use to understand what s happen ?
Recently, I've encountered this problem when executing the Run All Test command. Then the project has no PHP script under the specified test folders. Since the Xdebug implementation of PDT requires a PHP file, test cannot be run in such state.
To prevent this, MakeGood checks whether the project has at least a PHP file under the specified test folders, and skips a test run if the project has no PHP scripts. But even so this error is raised by any reason...
I created a issue http://redmine.piece-framework.com/issues/310 to fix this problem.
Thank you for using MakeGood.