Has anyone tried RTS tools like Ekstazi or STARTS to perform Regression test selection (RTS) for scala SBT (not maven) projects where tests are written using ScalaTests?
I tried by adding following library dependencies in build.sbt but it seems like RTS is not performed when I ran unit test cases (using sbt test command):
https://mvnrepository.com/artifact/org.ekstazi/org.ekstazi.core/5.3.0
https://mvnrepository.com/artifact/org.ekstazi/org.ekstazi.scalatest/5.3.0
Can someone please tell me what is the correct way?
Related
After switching to Java 11, sbt tests started to fail with the exception "No configuration setting found for key 'akka'". We are using sbt assembly plugin on the project but since the tests are run not inside jar but using sbt <module_name>/test, looks like there are some issues with building test resources/paths.
List of things I did:
Added concat MergeStrategy for 'reference.conf' file. Jar that is assembled has all files inside and there are no issues with that, so it is only a test step issue.
Checked class path for tests using 'export <module_name>/test:fullClasspath' in sbt before and after migrating to Java 11. They are the same.
Everything else except of a test where I create actor system works good.
Code that creates actor system:
object MyObjectTest extends AsyncFunSuite {
private implicit val system: ActorSystem = ActorSystem("MyActorSystem")
private implicit val ex: ExecutionContext = system.dispatcher
}
At the moment I do not have any ideas what should I check next, so any suggestions would be appreciated.
P.S. running tests inside IDEA works good, but we have CI/CD job that runs tests using sbt test command.
I guess there are some issues when you are using sbt + Java 11. I tried to reproduce this error using a small project with the same configuration but was not been able to do it. However, the issue was fixed using the following option in sbt:
Test / fork := true
According to sbt documentation, this property creates a separate JVM for running all tests. I think all configurations have been set up correctly for another JVM.
Hope this answer will help someone in future.
In my Scala project, my Specs2 tests are structured as follows:
src/test/scala
-> my.package
---> my.package.sub1
------> SomeTest1
------> SomeTest2
---> my.package.sub2
------> SomeTest3
I'm using SBT to build all of this, and I can use sbt test to run all tests in my package.
I'd like to use IntelliJ IDEA's built-in Specs2 run configuration support. I point it to use all tests in my.package.
Running this yields the error message Error running <run config name>: Not found suite class. It cannot find Specs2 test suites. IDEA runs my tests if I point it to a subpackage.
How do I configure IDEA to look in all packages and run all the test suites it finds?
I've managed to run all my Specs2 tests in IDEA 13.1.4 and the more recent 14.0.1 using All in package for Test kind and In whole project or In single module for Search for tests. I left Test Package field empty.
I had to create this configuration manually.
You may want to use Ctrl+Shift+F10 to create a Specs2 configuration and then modify it accordingly.
I have a typical sbt (0.13) build and have added the jacoco4sbt plugin to my build.
addSbtPlugin("de.johoop" % "jacoco4sbt" % "2.1.1")
I use specs2 to run my tests (2.2.2).
If I run
~>sbt
>test
all my tests get run (120 of them). However, if I do
>jacoco:test
it runs 0 tests, as if the jacoco configuration cannot find them.
A quick search reveals that there is an issue with jacoco4sbt and Play because Play sets parallelExecution to false. However, I am not using Play, and parallelExecution is set to True for both configurations. I have tried to set them both to false to no avail.
Any idea what might be going wrong?
n.b. The project I am working on is open source, so I created a branch where I put my attempt at adding jacoco4sbt. Feel free to clone it and see what is happening for yourself.
https://github.com/jedesah/scala-codesheet-api/tree/jacoco
I had this issue, but upgraded to Specs2 2.2.3 and jacoco4sbt started producing output from that point.
For what it's worth, I had the same problem when using specs2. When I switched to ScalaTest, jacoco4sbt started detecting my tests.
I have a very basic configuration too, so I don't know we're missing something or if there's something wrong in the current jacoco4sbt version. I did try version 2.1.0 of jacoco4sbt but had the same results.
I know I can filter which tests to run using Tests.Filter(s => s.endsWith("Test")).
However, Scalatest (and I'm sure other testing frameworks) has an #DoNotDiscover annotation for when running the framework directly.
I was wondering, is there a way to make SBT recognize this annotation?
Thanks
EDIT: I just found out that Tests.Filter(s => s.endsWith("Test")) isn't what I want. I have some tests that shouldn't be ran in Jenkins. I would like these tests to not run when I type sbt test. However, I would like these tests to run if called directly sbt test-only some.test.
I think you can't do it with ScalaTest 1.x, the latest 2.0.M6-SNAP36 should have it supported though:
https://oss.sonatype.org/content/groups/public/org/scalatest/scalatest_2.10/2.0.M6-SNAP36/
Is there a way to build tests with SBT without running them?
My own use case is to run static analysis on the test code by using a scalac plugin. Another possible use case is to run some or all of the test code using a separate runner than the one built into SBT.
Ideally there would be a solution to this problem that applies to any SBT project. For example, Maven has a test-compile command that can be used just to compile the tests without running them. It would be great if SBT had the same thing.
Less ideal, but still very helpful, would be solutions that involve modifying the project's build files.
Just use the Test / compile command.
Test/compile works for compiling your unit tests.
To compile integration tests you can use IntegrationTest/compile.
Another hint to continuously compile on every file change: ~Test/compile
We have a build.sbt file that is used for multiple projects. Doing sbt test:compile compiled the tests for every single project and took over 30 minutes.
I found out I can compile only the tests for a specific project named xyz by doing:
sbt xyz/test:compile
Using sbt version 1.5.0 and higher test:compile returns deprecation warning.
Use Test / compile.
(docs)