How to apply common test configuration to all projects? - scala

I'm migrating an old project to Scala 3. The build.sbt is as follows:
import Dependencies._
inThisBuild(
Seq(
scalaVersion := "2.12.7",
scalacOptions ++= Seq(
"-unchecked",
// more
)
)
++ inConfig(Test)(Seq(
testOptions += Tests.Argument(TestFrameworks.ScalaTest, "-o", "-e"),
// more
))
)
lazy val root = (project in file("."))
.aggregate(
`test-util`
)
lazy val `test-util` = project
Now, I want to separate stuff inside inThisBuild for legibility.
import Dependencies._
ThisBuild / scalaVersion := "3.0.1"
ThisBuild / scalacOptions ++= Seq(
"-unchecked",
// more
)
lazy val testSettings = inConfig(Test)(
Seq(
testOptions += Tests.Argument(TestFrameworks.ScalaTest, "-o", "-e"),
// more
))
lazy val root = (project in file("."))
.aggregate(
`test-util`
)
.settings(testSettings)
lazy val `test-util` = project
As you can see, I'm having to apply the testSettings for each project. Ideally, I'd like to do something like ThisBuild / Test := testSettings but that is not valid syntax.
Is there a way to apply the testSettings to all projects without having to explicitly set .settings(testSettings)?
Edit:
I understand I can write each line of testSettings with ThisBuild / Test prefix, but I’d rather not repeat the same prefix. I’m looking for something like what I’ve done with scalacOptions.

Is there a way to apply the testSettings to all projects without having to explicitly set .settings(testSettings)
Consider creating an auto plugin which can inject settings automatically in all the sub-projects, for example in project/CommonTestSettings.scala
import sbt._
import Keys._
object CommonTestSettings extends sbt.AutoPlugin {
override def requires = plugins.JvmPlugin
override def trigger = allRequirements
override lazy val projectSettings =
inConfig(Test)(
Seq(
testOptions += Tests.Argument(TestFrameworks.ScalaTest, "-o", "-e")
// more
)
)
}
You can test with show testOptions which should reveal the common settings in all the sub-projects, for example in my project where root aggregates foo and bar I get something like
sbt:sbt-multi-project> show testOptions
[info] foo / Test / testOptions
[info] List(Argument(Some(TestFramework(org.scalatest.tools.Framework, org.scalatest.tools.ScalaTestFramework)),List(-o, -e)))
[info] bar / Test / testOptions
[info] List(Argument(Some(TestFramework(org.scalatest.tools.Framework, org.scalatest.tools.ScalaTestFramework)),List(-o, -e)))
[info] Test / testOptions
[info] List(Argument(Some(TestFramework(org.scalatest.tools.Framework, org.scalatest.tools.ScalaTestFramework)),List(-o, -e)))

As you can done with scalaVersion and scalacOptions, you can do with Test. For example:
lazy val testSettings = inConfig(Test)(
Seq(
testOptions += Tests.Argument(TestFrameworks.ScalaTest, "-o", "-e"),
// more
))
Can be rewritten as:
ThisBuild / Test / testOptions += Test.Argument(TestFrameworks.ScalaTest, "-o", "-e")
Is it this that you want? Or do you want to pass a sequence directly?

Related

How to run main project with all sub-project in scala SBT

I'm using multi-project build in SBT to create a simple microservice architecture. I defined build.sbt as:
addCommandAlias("rebuild", ";clean; compile; package")
lazy val oc = Project(id = "oc",
base = file("."),
settings = commonSettings).aggregate(
persistence,
core,
configuration,
integration).dependsOn(
persistence,
core,
configuration,
integration)
.configs (MultiJvm)
lazy val persistence = Project(...)
lazy val core = Project(...)
lazy val log = Project(...)
lazy val configuration = Project(...)
lazy val integration = Project(...)
lazy val commonSettings = Defaults.coreDefaultSettings ++
basicSettings ++
net.virtualvoid.sbt.graph.Plugin.graphSettings
lazy val basicSettings = Seq(
version := PROJECT_VERSION,
organization := ORGANIZATION,
scalaVersion := SCALA_VERSION,
scalacOptions in Compile ++= Seq("-deprecation", "-feature", "-unchecked", "-Xlog-reflective-calls", "-Xlint" , "-encoding", "utf8"),
javacOptions in Compile ++= Seq("-Xlint:unchecked", "-Xlint:deprecation"),
javaOptions in run ++= Seq("-Xms128m", "-Xmx1024m"),
libraryDependencies ++= Seq( ...
),
fork in run := true,
// disable parallel tests
parallelExecution in Test := false,
licenses := Seq(("CC0", url("http://creativecommons.org/publicdomain/zero/1.0"))),
fork in Test := true
)
For single sub-project I can easily build the project. But I cannot build all projects at once to check the working as a whole system. Any help from the experts on building the system would be appreciable

How to reference a custom SBT Setting in sub-projects

Somewhat similar to this question, how can reference a custom setting in a sub project.
In build.sbt:
import sbt.Keys._
val finagleVersion = settingKey[String]("Defines the Finagle version")
val defaultSettings = Defaults.coreDefaultSettings ++ Seq(
finagleVersion in ThisBuild := "6.20.0",
organization in ThisBuild := "my.package",
scalaVersion in ThisBuild := "2.10.4",
version in ThisBuild := "0.1-SNAPSHOT"
)
lazy val root = project.in(file(".")).aggregate(thrift).settings(
publishArtifact in (Compile, packageBin) := false,
publishArtifact in (Compile, packageDoc) := false,
publishArtifact in (Compile, packageSrc) := false
)
lazy val thrift = project.in(file("thrift"))
In thrift/build.sbt:
name := "thrift"
// doesn't work
libraryDependencies ++= Seq(
"com.twitter" %% "finagle-thriftmux" % (finagleVersion in LocalRootProject).value
)
.sbt files cannot see the definitions (e.g., vals) in other .sbt files, even if they are part of the same build.
However, all .sbt files in a build can see/import the content of project/*.scala files. So you'll have to declare your val finagleVersion in a .scala file:
project/CustomKeys.scala:
import sbt._
import Keys._
object CustomKeys {
val finagleVersion = settingKey[String]("Defines the Finagle version")
}
Now, in your .sbt files, just
import CustomKeys._
and you're good to go.

How to use custom plugin in multi-module project?

In a multi-module project, with one module implementing a custom SBT plugin with custom TaskKey, how this plugin can be imported for the project settings of another submodule.
If the submodule using the plugin is define in submodule1/build.sbt, then submodule1/project/plugins.sbt is not loaded.
If plugin is registered in project/plugins.sbt it will fails when loading top/aggregate project as plugin is not necessarily already built.
Is there any other way to define a custom task requiring custom dependency so that it can be used by a submodule?
Here is how I finally make it works:
import sbt._
import Keys._
object MyBuild extends Build {
private lazy val myGenerator =
// Private project with generator code and its specific dependencies
// (e.g. Javassist)
Project(id = "my-generator",
base = file("project") / "my-generator").settings(
name := "my-generator",
javacOptions in Test ++= Seq("-Xlint:unchecked", "-Xlint:deprecation"),
autoScalaLibrary := false,
scalacOptions += "-feature",
resolvers +=
"Typesafe Snapshots" at "http://repo.typesafe.com/typesafe/releases/",
libraryDependencies ++= Seq( // Dependencies required to generate classes
"org.javassist" % "javassist" % "3.18.2-GA")
)
// Some custom setting & task
lazy val generatedClassDirectory = settingKey[File](
"Directory where classes get generated")
lazy val generatedClasses = taskKey[Seq[(File, String)]]("Generated classes")
lazy val myProject =
Project(id = "my-project", base = file("my-project")).settings(
name := "my-project",
javacOptions in Test ++= Seq("-Xlint:unchecked", "-Xlint:deprecation"),
autoScalaLibrary := false,
scalacOptions += "-feature",
libraryDependencies ++= Seq(/* ... */),
generatedClassDirectory := {
// Defines setting for path to generated classes
val dir = target.value / "generated_classes"
if (!dir.exists) dir.mkdirs()
dir
},
generatedClasses <<= Def.task { // Define task generating .class files
// first get classloader including generator and its dependencies
val cp = (fullClasspath in (myGenerator, Compile)).value
val cl = classpath.ClasspathUtilities.toLoader(cp.files)
// then loaded generator class, and instantiate with structural type
val genClass = cl loadClass "my.custom.GeneratorClass"
val generator = genClass.newInstance.
asInstanceOf[{def writeTo(out: File): File}]
// finally we can call the
val outdir = generatedClassDirectory.value
val generated = generator writeTo outdir
val path = generated.getAbsolutePath
// Mappings describing generated classes
Seq[(File, String)](generated -> path.
drop(outdir.getAbsolutePath.length+1))
} dependsOn(compile in (myGenerator, Compile))/* awkward? */,
managedClasspath in Compile := {
// Add generated classes to compilation classpath,
// so it can be used in my-project sources
val cp = (managedClasspath in Compile).value
cp :+ Attributed.blank(generatedClassDirectory.value)
},
// Make sure custom class generation is done before compile
compile in Compile <<= (compile in Compile) dependsOn generatedClasses,
mappings in (Compile, packageBin) := {
val ms = mappings.in(Compile, packageBin).value
ms ++ generatedClasses.value // add generated classes to package
}
).dependsOn(myGenerator/* required even if there dependsOn(compile in (myGenerator, Compile)) */)
}
Not sure there is better solution, especially about the redondant dependsOn(compile in (myGenerator, Compile)) and .dependsOn(myGenerator).

Play sub-projects: how to convert to build.sbt

I've a working multi-module Play 2.2 application which is organized like this...
myApp
+ app
+ conf
+ project
+ build.properties
+ Build.scala
+ plugin.sbt
... where Build.scala contains the following statements:
import sbt._
import Keys._
import play.Project._
object ApplicationBuild extends Build {
val appName = "myApp"
val appVersion = "1.0-SNAPSHOT"
val authDependencies = Seq(
"se.radley" %% "play-plugins-salat" % "1.3.0"
)
val mainDependencies = Seq(
"se.radley" %% "play-plugins-salat" % "1.3.0"
)
lazy val auth = play.Project(
appName + "-auth",
appVersion,
authDependencies,
path = file("modules/auth")).settings(
lessEntryPoints <<= baseDirectory(customLessEntryPoints),
routesImport += "se.radley.plugin.salat.Binders._",
templatesImport += "org.bson.types.ObjectId",
testOptions in Test := Nil,
resolvers ++= Seq(Resolvers.sonatype, Resolvers.scalaSbt)
)
lazy val main = play.Project(
appName,
appVersion,
mainDependencies).settings(
scalacOptions += "-language:reflectiveCalls",
routesImport += "se.radley.plugin.salat.Binders._",
templatesImport += "org.bson.types.ObjectId",
testOptions in Test := Nil,
lessEntryPoints <<= baseDirectory(customLessEntryPoints),
resolvers ++= Seq(Resolvers.sonatype, Resolvers.scalaSbt)
).dependsOn(auth).aggregate(auth)
def customLessEntryPoints(base: File): PathFinder = {
(base / "app" / "assets" / "stylesheets" / "bootstrap" * "bootstrap.less") +++
(base / "app" / "assets" / "stylesheets" * "*.less")
}
}
object Resolvers {
val scalaSbt = Resolver.url("Scala Sbt", url("http://repo.scala-sbt.org/scalasbt/sbt-plugin-snapshots"))(Resolver.ivyStylePatterns)
val sonatype = Resolver.sonatypeRepo("snapshots")
}
Now reading the Play 2.2 documentation it looks like I should convert my project to build.sbt:
The following example uses a build.scala file to declare a play.Project. This approach was the way Play applications were defined prior to version 2.2. The approach is retained in order to support backward compatibility. We recommend that you convert to the build.sbt based approach or, if using a build.scala, you use sbt’s Project type and project macro.
Is there any working example that describes how to replace project/build.scala with build.sbt? I read some short articles here and there... but I was unable to get a working Play project.
There is no urgent need to convert your build to build.sbt. build.sbt is simpler, but basically just gets compiled into Build.scala.
The other answer to this question will work, but is perhaps a little verbose. Start with the SBT documentation:
http://www.scala-sbt.org/0.13.0/docs/Getting-Started/Multi-Project.html
Now, create specify your main project and sub projects, and put your main project settings into your main build.sbt file:
lazy val auth = project.in(file("modules/auth"))
lazy val main = project.in(file(".")).dependsOn(auth).aggregate(auth)
playScalaSettings
name := "myApp"
version := "1.0-SNAPSHOT"
libraryDependencies += "se.radley" %% "play-plugins-salat" % "1.3.0"
scalacOptions += "-language:reflectiveCalls"
routesImport += "se.radley.plugin.salat.Binders._"
templatesImport += "org.bson.types.ObjectId"
testOptions in Test := Nil
lessEntryPoints <<= baseDirectory(customLessEntryPoints)
resolvers ++= Seq(Resolvers.sonatype, Resolvers.scalaSbt)
object Resolvers {
val scalaSbt = Resolver.url("Scala Sbt", url("http://repo.scala-sbt.org/scalasbt/sbt-plugin-snapshots"))(Resolver.ivyStylePatterns)
val sonatype = Resolver.sonatypeRepo("snapshots")
}
And now, in modules/auth/build.sbt, put your settings for the auth module:
name := "myApp-auth"
lessEntryPoints <<= baseDirectory(customLessEntryPoints)
routesImport += "se.radley.plugin.salat.Binders._"
templatesImport += "org.bson.types.ObjectId"
testOptions in Test := Nil
resolvers ++= Seq(Resolvers.sonatype, Resolvers.scalaSbt)
Anyway, it might need a bit of tweaking, but hopefully you get the point.
if using a build.scala, you use sbt’s Project type and project macro
Replace play.Project with Project and fix the arguments according to the ScalaDoc, it should be something like
lazy val auth = Project(
appName + "-auth",
file("modules/auth")).settings(
version := appVersion,
libraryDependencies ++= authDependencies,
lessEntryPoints <<= baseDirectory(customLessEntryPoints),
routesImport += "se.radley.plugin.salat.Binders._",
templatesImport += "org.bson.types.ObjectId",
testOptions in Test := Nil,
resolvers ++= Seq(Resolvers.sonatype, Resolvers.scalaSbt)
)
lazy val main = Project(
appName,
file("app")).settings(
version := appVersion,
libraryDependencies ++= mainDependencies,
scalacOptions += "-language:reflectiveCalls",
routesImport += "se.radley.plugin.salat.Binders._",
templatesImport += "org.bson.types.ObjectId",
testOptions in Test := Nil,
lessEntryPoints <<= baseDirectory(customLessEntryPoints),
resolvers ++= Seq(Resolvers.sonatype, Resolvers.scalaSbt)
).dependsOn(auth).aggregate(auth)
Same definitions can be used in build.sbt instead. I would also extract common settings:
val commonSettings = Seq(
version := appVersion,
routesImport += "se.radley.plugin.salat.Binders._",
templatesImport += "org.bson.types.ObjectId",
testOptions in Test := Nil,
lessEntryPoints <<= baseDirectory(customLessEntryPoints),
resolvers ++= Seq(Resolvers.sonatype, Resolvers.scalaSbt)
)
lazy val auth = Project(
appName + "-auth",
file("modules/auth")).settings(commonSettings: _*).settings(
libraryDependencies ++= authDependencies
)
lazy val main = Project(
appName,
file("app")).settings(commonSettings: _*).settings(
libraryDependencies ++= mainDependencies,
scalacOptions += "-language:reflectiveCalls"
).dependsOn(auth).aggregate(auth)

Integrating scala code coverage tool jacoco into a play 2.2.x project

My goal is to integrate jacoco into my play 2.2.0 project.
Different guides on the web I tried to follow mostly added to confusion not to closing in on the goal.
Adding to confusion
Most guides assume the existance of an build.sbt
which as it seems as been superseded by an build.scala with a different
There is a jacoco4sbt and a regular jacoco
which one is most appropiate for use with scala play framework 2
Current state
in plugins.sbt added
addSbtPlugin("de.johoop" % "jacoco4sbt" % "2.1.2")
in build.scala added
import de.johoop.jacoco4sbt.JacocoPlugin._
lazy val jacoco_settings = Defaults.defaultSettings ++ Seq(jacoco.settings: _*)
With these changes i don't get an "jacoco" task in sbt nor in the play console.
What are the appropriate steps to get this working?
Update
As requested the content of the build.scala
import com.typesafe.sbt.SbtNativePackager._
import com.typesafe.sbt.SbtScalariform._
import play.Project._
import sbt.Keys._
import sbt._
import sbtbuildinfo.Plugin._
import de.johoop.jacoco4sbt.JacocoPlugin._
object BuildSettings {
val buildOrganization = "XXXXX"
val buildVersion = "0.1"
val buildScalaVersion = "2.10.2"
val envConfig = "-Dsbt.log.format=false -Dconfig.file=" + Option(System.getProperty("env.config")).getOrElse("local.application")
scalacOptions ++= Seq("-encoding", "UTF-8", "-deprecation", "-unchecked", "-feature")
javaOptions += envConfig
val buildSettings = Defaults.defaultSettings ++ Seq (
organization := buildOrganization,
version := buildVersion,
scalaVersion := buildScalaVersion
)
}
object Resolvers {
val remoteRepoUrl = "XXXXXXXXXXXX" at "http://nexus.cXXXXX/content/repositories/snapshots/"
val publishRepoUrl = "XXXXXXXXXXXX" at "http://nexus.ciXXXXXXXXXXXXXXXX/content/repositories/snapshots/"
}
object Dependencies {
val ods = "XXXXXXXXX" % "XXXXXX-ws" % "2.2.1-SNAPSHOT"
val scalatest = "org.scalatest" %% "scalatest" % "2.0.M8" % "test"
val mockito = "org.mockito" % "mockito-all" % "1.9.5" % "test"
}
object ApplicationBuild extends Build {
import BuildSettings._
import Dependencies._
import Resolvers._
// Sub-project specific dependencies
val commonDeps = Seq(
ods,
scalatest,
mockito
)
//val bN = settingKey[Int]("current build Number")
val gitHeadCommitSha = settingKey[String]("current git commit SHA")
val release = settingKey[Boolean]("Release")
lazy val jacoco_settings = Defaults.defaultSettings ++ Seq(jacoco.settings: _*)
lazy val nemo = play.Project(
"nemo",
path = file("."),
settings = Defaults.defaultSettings ++ buildSettings ++
Seq(libraryDependencies ++= commonDeps) ++
Seq(scalariformSettings: _*) ++
Seq(playScalaSettings: _*) ++
buildInfoSettings ++
jacoco_settings ++
Seq(
sourceGenerators in Compile <+= buildInfo,
buildInfoKeys ++= Seq[BuildInfoKey](
resolvers,
libraryDependencies in Test,
buildInfoBuildNumber,
BuildInfoKey.map(name) { case (k, v) => "project" + k.capitalize -> v.capitalize },
"envConfig" -> envConfig, // computed at project load time
BuildInfoKey.action("buildTime") {
System.currentTimeMillis
} // re-computed each time at compile
),
buildInfoPackage := "com.springer.nemo"
) ++
Seq(resolvers += remoteRepoUrl) ++
Seq(mappings in Universal ++= Seq(
file("ops/rpm/start-server.sh") -> "start-server.sh",
file("ops/rpm/stop-server.sh") -> "stop-server.sh"
))
).settings(version <<= version in ThisBuild)
lazy val nemoPackaging = Project(
"nemoPackaging",
file("nemoPackaging"),
settings = Defaults.defaultSettings ++Seq(Packaging.settings:_*)
)
def publishSettings =
Seq(
publishTo := Option(publishRepoUrl),
credentials += Credentials(
"Repo", "http://mycompany.com/repo", "admin", "admin123"))
}
Note: jacoco is running with this but does not pick up our tests. Output:
jacoco:cover
[info] Compiling 1 Scala source to /home/schl14/work/nemo/target/scala-2.10/classes...
[info] ScalaTest
[info] Run completed in 13 milliseconds.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for nemo/jacoco:test
I solved it by doing this.
Add the following to plugins.sbt
addSbtPlugin("de.johoop" % "jacoco4sbt" % "2.1.2")
In build.scala i added a new import
import de.johoop.jacoco4sbt.JacocoPlugin._
and added jacoco to the config section like this
lazy val xyz = play.Project(
"xyz",
path = file("."),
settings = Defaults.defaultSettings
jacoco.settings ++ //this is the important part.
).settings(parallelExecution in jacoco.Config := false) //not mandatory but needed in `most cases as most test can not be run in parallel`
After these steps jacoco:cover was available in the sbt and play console and also discovers our tests.
A better option for Scala code coverage is Scoverage which gives statement line coverage.
https://github.com/scoverage/scalac-scoverage-plugin
Add to project/plugins.sbt:
addSbtPlugin("com.sksamuel.scoverage" % "sbt-scoverage" % "1.0.1")
Then run SBT with
sbt clean coverage test
To measure the code coverage of Java code jacoco4sbt is the best fit.
Add to project/plugins.sbt:
addSbtPlugin("de.johoop" % "jacoco4sbt" % "2.1.2")
Add at the end of build.sbt:
jacoco.settings
Then run in the terminal:
//or just the sbt command and then use your browser
sbt jacoco:cover && /usr/bin/x-www-browser target/scala-2.10/jacoco/html/index.html
Scala code coverage can also be determined by SCCT.
Add to project/plugins.sbt:
addSbtPlugin("com.github.scct" % "sbt-scct" % "0.2.1")
Add at the end of build.sbt:
ScctPlugin.instrumentSettings
And then to see the coverage:
sbt scct:test && /usr/bin/x-www-browser target/scala_2.10/coverage-report/index.html
Maybe you get the error
Please either restart the browser with --allow-file-access-from-files
or use a different browser.
Maybe you use chrome and the security settings forbid dynamic actions on local files. You can open the page with firefox or use python -m SimpleHTTPServer 8000 to bind it to the http protokoll and open http://localhost:8000/target/scala-2.10/coverage-report/ .
Inspiration which generated classes should be excluded from the report can you find on the mailing list.
My test projects are on GitHub.