Cannot get uTest to see my tests - scala

I'm trying to get uTest to work with ScalaJS and SBT. SBT is compiling the files, and uTest is running, but it simply ignores my tests. Try as I might I cannot find any difference between my code and the tutorial examples.
build.sbt:
enablePlugins(ScalaJSPlugin)
name := "Scala.js Stuff"
scalaVersion := "2.11.5" // or any other Scala version >= 2.10.2
scalaJSStage in Global := FastOptStage
libraryDependencies += "com.lihaoyi" %% "utest" % "0.3.0"
testFrameworks += new TestFramework("utest.runner.Framework")
src/test/scala/com/mysite/jovian/GeometryTest.scala:
package com.mysite.jovian
import utest._
object GeometryTest extends TestSuite {
def tests = TestSuite {
'addPoints {
val p: Point = new Point(3,4)
val q: Point = new Point(4,3)
val expected: Point = new Point(8,8)
assert(p.plus(q).equals(expected))
throw new Exception("foo")
}
'fail {
assert(1==2)
}
}
}
Output:
> reload
[info] Loading project definition from /Users/me/Dropbox (Personal)/mysite/flocks/project
[info] Set current project to Scala.js Stuff (in build file:/Users/me/Dropbox%20(Personal)/mysite/flocks/)
> test
[success] Total time: 1 s, completed Mar 6, 2015 7:01:41 AM
> test-only -- com.mysite.jovian.GeometryTest
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for test:testOnly
[success] Total time: 1 s, completed Mar 6, 2015 7:01:49 AM
If I introduce a syntax error, sbt test does see it:
> test
[info] Compiling 1 Scala source to /Users/me/Dropbox (Personal)/mysite/flocks/target/scala-2.11/test-classes...
[error] /Users/me/Dropbox (Personal)/mysite/flocks/src/test/scala/com/mysite/jovian/GeometryTest.scala:21: not found: value blablablablabla
[error] blablablablabla
[error] ^
[error] one error found
[error] (test:compile) Compilation failed
[error] Total time: 1 s, completed Mar 6, 2015 7:03:54 AM
So it's definitely seeing the code, it just doesn't seem to think that "tests" contains any tests.
Otherwise, in the non-test code, SBT+ScalaJS seems to be working fine...
Thanks for any help, I am mystified.

Your mistake lies in the dependency on uTest:
libraryDependencies += "com.lihaoyi" %% "utest" % "0.3.0"
This is a JVM dependency. To use the Scala.js-enabled dependency, use %%% instead of %%, like this:
libraryDependencies += "com.lihaoyi" %%% "utest" % "0.3.0"
Additionally, you probably want this dependency only in the Test configuration, so add % "test" a the end:
libraryDependencies += "com.lihaoyi" %%% "utest" % "0.3.0" % "test"

Related

Why won't sbt run ScalaTest's example test?

Please assume the normal "newbie" appologies.
I'm using scala 2.12.10; In build.sbt I added ScalaTest:
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.8" % "test"
I'm adding some scala.js to an existing java project so my scala source paths are normal but I made some empty directories for java and resources and use a different target so there would be no collision with existing code:
javaSource in Compile := baseDirectory.value / "src/main/scalajs_java"
javaSource in Test := baseDirectory.value / "src/test/scalajs_java"
resourceDirectory in Compile := baseDirectory.value / "src/main/scalajs_resources"
resourceDirectory in Test := baseDirectory.value / "src/test/scalajs_resources"
target := baseDirectory.value / "scalajs_target"
I put the ScalaTest example file ExampleSpec.scala in src/test/scala/ExampleSpec.scala. The example ran fine using org.scalatest.run as described in http://www.scalatest.org/.
The test classpath looks reasonable, I thought:
sbt:RedsPro-ScalaJS> show test:fullClasspath
[info] * Attributed(/Users/tballard/git/redspro/reds/scalajs_target/scala-2.12/test-classes)
[info] * Attributed(/Users/tballard/git/redspro/reds/scalajs_target/scala-2.12/classes)
[info] * Attributed(/Users/tballard/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.10.jar)
[info] * Attributed(/Users/tballard/.ivy2/cache/org.scala-js/scalajs-library_2.12/jars/scalajs-library_2.12-0.6.31.jar)
[info] * Attributed(/Users/tballard/.ivy2/cache/org.scala-js/scalajs-dom_sjs0.6_2.12/jars/scalajs-dom_sjs0.6_2.12-0.9.7.jar)
[info] * Attributed(/Users/tballard/.ivy2/cache/org.scala-js/scalajs-test-bridge_2.12/jars/scalajs-test-bridge_2.12-0.6.31.jar)
[info] * Attributed(/Users/tballard/.ivy2/cache/org.scala-js/scalajs-test-interface_2.12/jars/scalajs-test-interface_2.12-0.6.31.jar)
[info] * Attributed(/Users/tballard/.ivy2/cache/org.scalatest/scalatest_2.12/bundles/scalatest_2.12-3.0.8.jar)
[info] * Attributed(/Users/tballard/.ivy2/cache/org.scalactic/scalactic_2.12/bundles/scalactic_2.12-3.0.8.jar)
[info] * Attributed(/Users/tballard/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.12.10.jar)
[info] * Attributed(/Users/tballard/.ivy2/cache/org.scala-lang.modules/scala-xml_2.12/bundles/scala-xml_2.12-1.2.0.jar)
After compiling, the test is where I would expect it:
> ls scalajs_target/scala-2.12/test-classes/
ExampleSpec.class ExampleSpec.sjsir JS_DEPENDENCIES org/
However, sbt apparently isn't convinced there are any tests:
sbt:RedsPro-ScalaJS> show definedTestNames
[info] *
[success] Total time: 1 s, completed Dec 1, 2019, 11:37:51 AM
sbt:RedsPro-ScalaJS> testOnly ExampleSpec
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for Test / testOnly
[success] Total time: 1 s, completed Dec 1, 2019, 12:44:35 PM
A point in the right direction would be greatly appreciated.
You need to use %%% instead of %% when depending on ScalaTest. In general, you always need to use %%% in Scala.js.

Scalatest GeneratorDrivenPropertyChecks init seed

I'm using Scalatest 3.1.0-SNAP13 and cannot find how to specify the init seed option from this PR.
I'm using SBT to run the test so if there is a way to specify this option in build.sbt would be ideal.
The -S flag seems to be disabled in 3.1.x:
parseLongArgument(seedArgs, "-S") match {
case Some(seed) => // Randomizer.defaultSeed.getAndSet(Some(seed))
println("Note: -S for setting the Randomizer seed is not yet supported.")
case None => // do nothing
}
However it is seems to be enabled in 3.2.x, so try
libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.0-M1" % Test
and
testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-S", "1568769615146")
I managed to get it working with:
// specify an initial seed 0
Test / testOptions += Tests.Argument(TestFrameworks.ScalaTest, "-S", "0")
Result:
[info] SummaryTest:
[info] - hello *** FAILED *** (49 milliseconds)
[info] GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation. (SummaryTest.scala:8)
[info] Falsified after 0 successful property evaluations.
[info] Message: 0 was not greater than 1
[info] Location: (SummaryTest.scala:10)
[info] Occurred when passed generated values (
[info] ""
[info] )
[info] Init Seed: 0

How do I specify specific test from the build.sbt

How do I can specify the test from the build.sbt file , I wanted to run one test only and I used the filter as in the sbt docs, but it doesn't work with me, this is my code I have two test classes and in my sbt I specify test1 to be rub but it seems that the two test are running at the same time any one know what I should do ?
Test1Demo.scala
import org.scalatest.{FlatSpec, Matchers}
class Test1Demo extends FlatSpec with Matchers{
"value of x " should " be 9 " in { assert(my.App.x == 9) }
}
Test2Demo.scala
import org.scalatest.{FlatSpec, Matchers}
class Test2Demo extends FlatSpec with Matchers{
"value of y " should " be 8 " in { assert(my.App2.y == 8) }
}
build.sbt
version := "0.1"
scalaVersion := "2.12.8"
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.5" % Test
testOptions in Test := Seq(Tests.Filter(s => s.startsWith("Test1")))
the output :
[info] Done updating.
[info] Compiling 2 Scala sources to /home/****/target/scala-2.12/classes ...
[info] Done compiling.
[info] Compiling 2 Scala sources to /home/****/target/scala-2.12/test-classes ...
[info] Done compiling.
[info] Test2Demo:
[info] value of y
[info] - should be 8
[info] Test1Demo:
[info] value of x
[info] - should be 9
[info] Run completed in 6 seconds, 365 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 2, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 264 s, completed Apr 15, 2019 2:47:10 PM
If you want to run value of x test from Test1Demo:
testOnly *Test1Demo -- -z value
This sbt command will run only the tests whose name includes the substring "value".
For exact match rather than substring, use -t instead of -z.
Pay attention to -- (two -, not one)

sbt play cross build project setup: uTest runner doesn't seperate client/server projects correctly

I'm using an build.sbt which has cross compile settings and basically is an adapted version of "Play with scala-js example" and having some trouble getting a clean setup for my tests. Specifically, when running my server tests, my client tests also get executed (which is something that I want to avoid).
I followed the instructions from Cannot get uTest to see my tests
and added
libraryDependencies += "com.lihaoyi" %%% "utest" % "0.3.0"
My tests for some reason didn't get executed until I added
testFrameworks += new TestFramework("utest.runner.Framework")
to every project definition. Also not adding
"com.lihaoyi" %% "utest" % "0.3.1" % "test"
to the server side triggers a series of
not found: object utest [error] import utest._
-style errors.
To my impression I shouldn't have to add these additional settings at all if having a clean setup. Here's my sbt file:
import sbt.Project.projectToRef
lazy val clients = Seq(client)
lazy val scalaV = "2.11.7"
lazy val server = (project in file("server")).settings(
scalaVersion := scalaV,
scalaJSProjects := clients,
pipelineStages := Seq(scalaJSProd/*, gzip*/),
resolvers += "scalaz-bintray" at "https://dl.bintray.com/scalaz/releases",
libraryDependencies ++= Seq(
"com.vmunier" %% "play-scalajs-scripts" % "0.3.0",
"be.doeraene" %% "scalajs-pickling-play-json" % "0.4.0"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).enablePlugins(PlayScala).
aggregate(clients.map(projectToRef): _*).
dependsOn(sharedJvm)
lazy val client = (project in file("client")).settings(
scalaVersion := scalaV,
persistLauncher := true,
persistLauncher in Test := false,
libraryDependencies ++= Seq(
"org.scala-js" %%% "scalajs-dom" % "0.8.0"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).enablePlugins(ScalaJSPlugin, ScalaJSPlay).
dependsOn(sharedJs)
lazy val shared = (crossProject.crossType(CrossType.Pure) in file("shared")).
settings(scalaVersion := scalaV,
libraryDependencies ++= Seq(
"com.lihaoyi" %%% "utest" % "0.3.1",
"be.doeraene" %%% "scalajs-pickling-core" % "0.4.0",
"com.lihaoyi" %%% "pprint" % "0.3.6"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).
jsConfigure(_ enablePlugins ScalaJSPlay)
lazy val sharedJvm = shared.jvm
lazy val sharedJs = shared.js
// loads the Play project at sbt startup
onLoad in Global := (Command.process("project server", _: State)) compose (onLoad in Global).value
And here a summary of my problems:
when I run client/test, only client tests are executed
when running play-with-scalajs-example/test, client + shared tests are executed
and strangely when running server/test my server AND client tests are executed
How could I modify my project setup to
find my server tests when running server/test
running all tests when running play-with-scalajs-example/test
and additionally including the shared tests when running server/test or client test?
And on another node, is there a way to disable scalatest? It leads to a rather unreadable testoutput:
[info] 1/2 TestSimpleServerSuite.absolutely simple test on the server side Success
[info] 2/2 TestSimpleServerSuite Success
[info] utest
[info] -----------------------------------Results-----------------------------------
[info]
[info]
[info] Tests: 0
[info] Passed: 0
[info] Failed: 0
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[info] 1/2 TestSimpleClientSuite.absolutely simple test on the client side Success
[info] 2/2 TestSimpleClientSuite Success
[info] 1/2 SimpleClient.TestSimpleClientSuite.absolutely simple test on the client side Success
[info] 2/2 SimpleClient.TestSimpleClientSuite Success
[info] ScalaCheck
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] ScalaTest
[info] Run completed in 1 second, 751 milliseconds.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[info] utest
[info] -----------------------------------Results-----------------------------------
[info] SimpleClient.TestSimpleClientSuite Success
[info] absolutely simple test on the client side Success
[info] TestSimpleClientSuite Success
[info] absolutely simple test on the client side Success
[info]
[info] Tests: 4
[info] Passed: 4
[info] Failed: 0
[info] Passed: Total 4, Failed 0, Errors 0, Passed 4
[success] Total time: 11 s, completed 16.10.2015 03:25:59
Thanks a bunch and kind regards
sjrd's comment about removing aggregate pointed me to right path.
Removing it stopped
server/test
from executing both server and client tests. As sjrd pointed out, the aggregate function used also runs every task that is run on server on the client.
Aggregation means that running a task on the aggregate project will
also run it on the aggregated projects.
(See: sbt doc)
As I also wanted to run shared tests on both client and server project when running test, I modified the aggregate function for the server project definition and added an additional aggregate to the client project definition. :
Server def.:
lazy val server = (project in file("server")).settings(
scalaVersion := scalaV,
scalaJSProjects := clients,
pipelineStages := Seq(scalaJSProd/*, gzip*/),
resolvers += "scalaz-bintray" at "https://dl.bintray.com/scalaz/releases",
libraryDependencies ++= Seq(
"com.vmunier" %% "play-scalajs-scripts" % "0.3.0",
"be.doeraene" %% "scalajs-pickling-play-json" % "0.4.0",
"com.lihaoyi" %% "utest" % "0.3.1" % "test"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).enablePlugins(PlayScala).
/*
* Executes shared tests compiled to JVM with server/test
*/
aggregate(projectToRef(sharedJvm)). // Former: aggregate(clients.map(projectToRef): _*). before
dependsOn(sharedJvm)
Client def.:
lazy val client = (project in file("client")).settings(
scalaVersion := scalaV,
persistLauncher := true,
persistLauncher in Test := false,
libraryDependencies ++= Seq(
"org.scala-js" %%% "scalajs-dom" % "0.8.0"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).enablePlugins(ScalaJSPlugin, ScalaJSPlay).
/*
* Executes shared tests compiled to JS with client/test
*/
aggregate(projectToRef(sharedJs)).
dependsOn(sharedJs)
When running tests with play-with-scalajs-example/test now runs all tests, including the shared tests both compiled seperately for JS and JVM.
As for having to explicitly include the utest dependencies, it seems that client and server projects don't derive from crossProject and are thus disconnected - will look into finding a better way of handling that.

How can I use JMH for Scala benchmarks together with sbt?

I have tried to use jmh together with sbt, but so far I have not managed to set it up properly so that .scala based benchmarks work.
As the combination sbt + .java based benchmarks works, I tried to start from that base.
I am using sbt 0.13.1.
.java based benchmarks using sbt
build.sbt
import AssemblyKeys._
name := "scala-benchmark"
version := "1.0"
scalaVersion := "2.10.3"
scalacOptions += "-deprecation"
libraryDependencies += "org.openjdk.jmh" % "jmh-core" % "0.5.5"
libraryDependencies += "org.openjdk.jmh" % "jmh-java-benchmark-archetype" % "0.5.5"
libraryDependencies += "org.openjdk.jmh" % "jmh-generator-annprocess" % "0.5.5"
libraryDependencies += "org.openjdk.jmh" % "jmh-generator-bytecode" % "0.5.5"
assemblySettings
jarName in assembly := "microbenchmarks.jar"
test in assembly := {}
mainClass in assembly := Some("org.openjdk.jmh.Main")
To get a single "fat" jar at the end, the sbt-assembly plugin is required:
project/assembly.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")
A simple benchmark:
src/main/java/app/benchmark/java/benchmark2/Benchmark2.java
package app.benchmark.java.benchmark2;
import org.openjdk.jmh.annotations.GenerateMicroBenchmark;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
public class Benchmark2 {
#GenerateMicroBenchmark
public int run() {
int result = 0;
for (int i = 0; i < 10; i++) {
result += i * i;
}
return result;
}
}
Running sbt assembly gives this output:
$ sbt assembly
[...]
[info] Compiling 2 Scala sources and 2 Java sources to ...
[warn] warning: Supported source version 'RELEASE_6' from annotation processor 'org.openjdk.jmh.generators.GenerateMicroBenchmarkProcessor' less than -source '1.8'
[warn] 1 warning
[info] Including: jmh-java-benchmark-archetype-0.5.5.jar
[info] Including: jmh-generator-bytecode-0.5.5.jar
[info] Including: jopt-simple-4.6.jar
[info] Including: jmh-generator-reflection-0.5.5.jar
[info] Including: jmh-generator-annprocess-0.5.5.jar
[info] Including: asm-4.2.jar
[info] Including: commons-math3-3.2.jar
[info] Including: jmh-core-0.5.5.jar
[info] Including: scala-library.jar
[...]
[info] Packaging /home/scala-2.10/vc/rhaag/scala/scala-benchmark/target/scala-2.10/microbenchmarks.jar ...
and the resulting microbenchmarks.jar contains everything required to run the benchmarks:
$ java -jar target/scala-2.10/microbenchmarks.jar -wi 3 -i 3 -f 1 app.benchmark.java.benchmark2.Benchmark2.run
[...]
Benchmark Mode Samples Mean Mean error Units
a.b.j.b.Benchmark2.run thrpt 3 607555,968 70243,275 ops/ms
So far so good.
Scala benchmarks using sbt
From that base I tried to switch to .scala based benchmarks:
build.sbt
Replacing the Java archetype with the Scala one
libraryDependencies += "org.openjdk.jmh" % "jmh-scala-benchmark-archetype" % "0.5.5"
does not work, as the download fails.
This works:
libraryDependencies += "org.openjdk.jmh" % "jmh-scala-benchmark-archetype" % "0.5.5" from "http://repo1.maven.org/maven2/org/openjdk/jmh/jmh-scala-benchmark-archetype/0.5.5/jmh-scala-benchmark-archetype-0.5.5.jar"
Another simple benchmark:
src/main/scala/app/benchmark/scala/benchmark2/Benchmark2.scala
package app.benchmark.scala.benchmark2
import org.openjdk.jmh.annotations.GenerateMicroBenchmark
import org.openjdk.jmh.runner.Runner
import org.openjdk.jmh.runner.RunnerException
import org.openjdk.jmh.runner.options.Options
import org.openjdk.jmh.runner.options.OptionsBuilder
class Benchmark2 {
#GenerateMicroBenchmark
def run() = {
Seq.range(0, 10).map(i => i * i).sum
}
}
Now sbt assembly creates the jar file, but target/scala-2.10/microbenchmarks.jar#META-INF/MicroBenchmarks does not list the Scala benchmarks, and these are not shown by java -jar target/scala-2.10/microbenchmarks.jar -l either.
Resources:
JMH with maven (there is a link to an ant based approach as well)
A Japanese page where I got the inital sbt setup from
JMH with gradle
How can I integrate the (bytecode based) JMH processor for Scala?
Or from another perspective: Why is the (annotation based) JMH processor picked up automatically and produces Java based benchmarks?
I have implemented an sbt-jmh plugin that actually works: https://github.com/ktoso/sbt-jmh
Currently building benchmarks.jar is not supported, but you can simply type run -i 10 .*MyBenchmark.* and it will work as expected (doing all the multi-step compilation for you).
I hope this helps, cheers!