When using the cobertura-maven-plugin for Java, I get a nice <instrumentation> block in the config where I can put the incredibly useful <ignoreMethodAnnotation> block.
Best thing to happen to coverage since the gin martini.
Now I'm using scoverage-sbt, but I can't seem to find where I can configure it! The available keys in build.scala are limited. I can do package exclusion and file exclusion but there's nowhere to tell cobertura anything else.
Is there a -D I can supply on the SBT command line, maybe?
There is no similar configuration in Scoverage ATM.
update:
You can use special comments to exclude a block of code from instrumentation:
// $COVERAGE-OFF
...
// $COVERAGE-ON$
One way to pass parameters and commands into SBT from the command line is:
$ sbt 'set coverageEnabled := true' clean coverage test coverageReport coverageAggregate codacyCoverage
Where you call SBT once and then separate each parameter or command by a space.
In this example, I first set the property coverageEnabled := true and then run several commands in sequence: clean, coverage, test, coveraReport, coverageAggregate and finally codacyCoverage
Please note that setting properties like this requires that you enclose your statements in single quotes, for example:
$ sbt 'set coverageEnabled := true'...
Related
I have an sbt subproject that includes end to end tests. These are run as e2e:test. I have defined my config as
I have defined a tag in the same subproject.
object HealthCheckTest extends Tag("HealthCheckTest")
I am tagging some of my end to end tests with HealthCheckTest as follows:
it("should be able to verify the data", HealthCheckTest)
I want to run only the health check tests from command line. I am trying to do this via:
sbt 'project e2e' e2e:testOnly -- -n HealthCheckTest
but this leads to all of the e2e tests being run. I have tried giving the full path to the tag (com.s.p.e2etests.HealthCheckTest), but that does not work either.
Occasionally I get warnings about -- and - being deprecated; however, all documentation online says to use this syntax including scalatest docs.
How can I run just my tagged e2e tests?
I have also tried to create a separate task for health check tests but could only figure out how to filter based on test class name, not by tag.
The commandline you posted was missing quotes. Try it like this:
sbt 'project e2e' 'testOnly -- -n HealthCheckTest'
The sbt executable treats each argument as a full line to run in the sbt console, so you must place quotes around each full line.
Note that while this won't run the test cases, it will still instantiate test class and print out test case names.
I usually do below command in my CI:
clean update compile test publish
However, I'd like to exclude 1 (or a few) test class from the sbt command line.
How can I do this? (I don't want to change my code to use ignore, etc)
Two possible options
test-only See http://www.scalatest.org/user_guide/using_scalatest_with_sbt
Tags http://www.scalatest.org/user_guide/tagging_your_tests
Just to elaborate on the 2 correct options #Gonfva suggested above:
To use testOnly you should run:
sbt "testOnly org.fully.qualified.domainn.name.ASpec"
When the argument is the FQDN of the class. You can use multiple classes separate them by space. This can be used with glob as well. For example:
sbt "testOnly *ASpec"
Using tags. First, define a tag:
import org.scalatest.Tag
object CustomTag extends Tag("tagName")
Then, define a test with this tag:
it should "test1" taggedAs CustomTag in { println("test1") }
Now, in order to include tests using this tag, run:
sbt "testOnly * -- -n tagName"
Note: * is a wild card. It can be any glob as described in section 1.
In order to exclude this tag, you need to run:
sbt "testOnly * -- -l aaa"
Including or excluding tests is dependent on the test framework you are using. The command you will be employing in SBT is not completely parsed by SBT, but partially parsed by SBT, partially parsed by the test framework you are using.
So, if you are suing scalameta/munit, you may enter a command like this:
sbt> myproject/Test/testOnly MyProjectTest -- --exclude-tags=tag1,tag2,tag3
If you are using another test framework, the specific syntax after -- will be probably different.
When you are writing your test cases, you have to add tags somewhere, obviously. More details can be found below:
scalameta/munit: https://scalameta.org/munit/docs/filtering.html
scalatest: https://www.scalatest.org/user_guide/tagging_your_tests (stolen from another answer)
I find it incredibly awkward to have to restart sbt with special flags if I want to run the tests (or a main) with debug enabled. It's also a pain if the main or test is usually in a forked JVM:
How to set fork in Test when -jvm-debug given on command line?
Is there any simple way to conditionally do a run, test, test-quick or test-only and ask for debugging to be enabled in the forked process? e.g. with syntax like test-only -jdb
I don't really want to have to write my own Tasks to do this... maintaining them is going to be a nightmare. But I guess that would allow syntax like module/jdb:test-only
While Eugene accurately mentions that we could provide debug:testOnly out of the box, the following should help you out:
val DebugTest = config("dtest") extend Test
lazy val myproject =
project.configs(DebugTest).
settings(inConfig(DebugTest)(Defaults.testSettings):_*).
settings(
fork in DebugTest := true,
javaOptions in DebugTest += "debugging options",
definedTests in DebugTest := (definedTests in Test).value
)
This should allow you to do dtest:testOnly *, dtest:run and dtest:test on myproject. The forked JVM (note fork in DebugTest := true) will use whatever debugging options you've provided.
I don't think there's any simple feature that enables debugging out of the box.
As you mentioned on sbt-dev list, making a custom configuration like debug:testOnly sounds like a good strategy.
For my specific reason, I'd like to override test task to run not all tests, but to behave like test-only TopTestSuiteName param1 param2. Is it possible?
I tried to go through documentation on tasks and examples, but still far from understanding what is possible and what's not.
Or maybe I could create custom task and somehow use existing test-only functionality? I feel it is reasonable but completely cannot understand - where do I start.
I would suggest you leave test alone and add a custom thing instead.
How about an alias? You could just add this to your .sbt definition:
addCommandAlias("myTest", "testOnly TopTestSuiteName -- param1 param2")
But if you actually need myTest to be a task (for example if you want another task to depend on it), then here's code (in .sbt format for sbt 0.13) for reusing testOnly functionality in a custom task:
val myTest = taskKey[Unit]("call testOnly with some special args")
myTest in Test :=
(testOnly in Test).toTask(" TopTestSuiteName -- param1 param2").value
In my specs2 tests, I frequently use helper functions to test groups of conditions at once. Unfortunately, that makes the line number output of failed tests useless, since all failures are on the same line.
Google turned up that there's a "failtrace" option that will output the stack trace of failure. However, I can't find an example of how to actually use that. Is it in build.sbt? Is it used on the SBT command line? Is it set somehow in the constructor of the Specification class?
You can set the failtrace option at the specification level:
class MySpec extends org.specs2.mutable.Specification {
args.report(failtrace = true)
...
}
You can also pass failtrace on the sbt command line:
sbt> test-only *MySpec* -- failtrace
Eric's solution can also be applied to all tests run by SBT by including the following in your build.sbt file:
testOptions += Tests.Argument(TestFrameworks.Specs2, "failtrace")