build.sbt - iteration over sub projects for common settings in monorepo - scala

I'm implementing a monorepo using SBT. I would like to iterate over my subprojects in order to initialize them (as the have the same configuration) and prevent code duplication.
In my build.sbt:
lazy val root = (project in file("."))
.aggregate(projects: _*)
.settings(
crossScalaVersions := Nil,
publish / skip := true
)
lazy val projects = Seq("projectA", "projectB", "projectC")
.map((projectName: String) => (project in file(projectName))
.settings(
name := projectName,
commonSettings,
libraryDependencies ++= ModulesDependencies.get(projectName))
.project
)
I'm getting the error:
error: project must be directly assigned to a val, such as `val x = project`. Alternatively, you can use `sbt.Project.apply`
Based on the error message, I also tried to use sbt.Project.apply(projectName, file(projectName)).settings(...) instead, but I'm also facing some funny errors.
From what I understand, it seems that SBT expects me to declare as lazy val projectA = (project in file("projectA")).settings(...), which works fine but I would have to duplicate this code for all my sub projects.
Is this iteration that I try to implement even possible?

Utility method might help with some of the duplication, for example
def createProject(projectName: String) = {
Project(projectName, file(projectName))
.settings(
name := projectName,
commonSettings,
libraryDependencies ++= ModulesDependencies.get(projectName)
)
}
lazy val projectA = createProject("projectA")
lazy val projectB = createProject("projectB")
lazy val projectC = createProject("projectC")
lazy val root = (project in file("."))
.aggregate(projectA, projectB, projectB)
.settings(
crossScalaVersions := Nil,
publish / skip := true
)

Related

How can I define my own setting or variable in build.sbt?

I try to define my own setting that calculated by using value of name setting in build.sbt
// ...
val projectName_ = "project_name"
val projectName = projectName_.replace("_", "")
lazy val main_class = settingKey[String]("")
main_class := s"ru.company.${projectName}.${name.value}.Main"
lazy val commonSettings = Seq(
// ...
Compile / mainClass := Some(main_class.value),
assembly / mainClass := Some(main_class.value)
// ...
)
lazy val rollout = taskKey[File](s"rollout_${projectName_}") := {
// Other using of main_class.value
}
lazy val root = (project in file("."))
.aggregate(stg, dm)
.settings(
name := "root"
)
lazy val core = project
.settings(
name := "core",
//...
)
lazy val stg = project.dependsOn(core)
.settings(
name := "stg",
commonSettings,
rollout
)
lazy val dm = project.dependsOn(core)
.settings(
name := "dm",
commonSettings,
rollout
)
But i get error when i try to get value of my setting:
Some(main_class.value)
Reference to undefined settings
How can I define variable with name setting that I will be able to use in settings?
When you do
lazy val main_class = settingKey[String]("")
main_class := s"ru.company.${projectName}.${name.value}.Main"
you're defining settingKey for any place which can access it, but setting its value only for the current project (which is root). For subprojects the value is undefined. So you have to set it for all projects.
Do something like this:
Global / main_class := s"ru.company.${projectName}.${name.value}.Main"
or
ThisBuild / main_class := s"ru.company.${projectName}.${name.value}.Main"
and main_class.value should not longer complain.
See the differences between Global and ThisBuild here.

sbt-assembly does not pick up configuration specific settings

I am updating an old 0.7.x build file from the tool sbt that thankfully removed the reference to "simple" from its name in the meantime.
Something that once worked, does not do so any longer. I had different config entries for platform specific assembly tasks. These include specific filters that for some reason are now called assemblyExcludedJars instead of excludedJars, and specific jar names that for some reason are now called assemblyJarName instead of jarName.
Basically:
val Foo = config("foo") extend Compile
lazy val assemblyFoo = TaskKey[File]("assembly-foo")
lazy val root = Project(id = "root", base = file("."))
// .configs(Foo) // needed? doesn't change anything
.settings(
inConfig(Foo)(inTask(assembly) {
assemblyJarName := "wtf.jar"
}),
scalaVersion := "2.11.7",
assemblyFoo <<= assembly in Foo
)
Now I would expect that if I run sbt assembly-foo or sbt foo:assembly, it would produce a file wtf.jar. But I am getting the default root-assembly-0.1-snapshot.jar. The same problem happens when I try to specify assemblyExcludedJars, they are simply ignored and still included.
If I remove the inConfig it works:
lazy val root = Project(id = "root", base = file("."))
.settings(
inTask(assembly) {
assemblyJarName := "wtf.jar"
},
scalaVersion := "2.11.7",
assemblyFoo <<= assembly in Foo
)
But now I cannot use different jar names for different configurations (which is the whole point).
As described in a blog post by one of sbt's authors and the author of sbt-assembly, this should work. It was also written in this Stackoverflow question. But the example requires an antique version of sbt-assembly (0.9.0 from 2013, before auto plugins etc.) and doesn't seem to apply to the current versions.
If one defines a new configuration, one has to redefine (?) all the tasks one is about to use. Apparently for sbt-assembly, this means running baseAssemblySettings:
val Foo = config("foo") extend Compile
lazy val assemblyFoo = TaskKey[File]("assembly-foo")
lazy val root = Project(id = "root", base = file("."))
.settings(
inConfig(Foo)(baseAssemblySettings /* !!! */ ++ inTask(assembly) {
jarName := "wtf.jar"
}),
scalaVersion := "2.11.7",
assemblyFoo := (assembly in Foo).value
)
Tested with sbt 0.13.9 and sbt-assembly 0.14.1.

Project aggregation in sbt

I am just starting to learn sbt to build scala projects.
Here is my build.sbt file
lazy val commonSettings = Seq(
organization := "com.example",
version := "0.1.0",
scalaVersion := "2.11.7"
)
lazy val task = taskKey[Unit]("An example task")
lazy val root = project.in(file(".")).
aggregate(core).
settings(commonSettings: _*).
settings(
task := { println("Hello!") },
name := "hello",
version := "1.0"
)
lazy val core = project.in( file("SbtScalaProjectFoo") )
My project structure is as follows
SbtScalaProject
|--SbtScalaProjectFoo
|--build.sbt
|--build.sbt
When I try to run "sbt" inside SbtScalaProject I get the following
No project 'core' in 'file:/Users/asattar/Dev/work/SbtScalaProject/'
What am I missing?
Having multiple build.sbt's has proven to make problems for me, too.
I would recommend aggregating all project data into one build.sbt. If you want to modularize the build, consider moving parts into .scala-Files in (the root project's) project/ directory.

SBT/Scala: macro implementation not found

I tried my hand on macros, and I keep running into the error
macro implementation not found: W
[error] (the most common reason for that is that you cannot use macro implementations in the same compilation run that defines them)
I believe I've set up a two pass compilation with the macro implementation being compiled first, and the usage second.
Here is part of the /build.sbt:
lazy val root = (project in file(".")).
settings(rootSettings: _*).
settings(name := "Example").
aggregate(macros, core).
dependsOn(macros, core)
lazy val macros = (project in file("src/main/com/example/macros")).
settings(macrosSettings: _*).
settings(name := "Macros")
lazy val core = (project in file("src/main/com/example/core")).
settings(coreSettings: _*).
settings (name := "Core").
dependsOn(macros)
lazy val commonSettings = Seq(
organization := Organization,
version := Version,
scalaVersion := ScalaVersion
)
lazy val rootSettings = commonSettings ++ Seq(
libraryDependencies ++= commonDeps ++ rootDeps ++ macrosDeps ++ coreDeps
)
lazy val macrosSettings = commonSettings ++ Seq(
libraryDependencies ++= commonDeps ++ macrosDeps
)
lazy val coreSettings = commonSettings ++ Seq(
libraryDependencies ++= commonDeps ++ coreDeps
)
The macro implementation looks like this:
/src/main/com/example/macros/Macros.scala
object Macros {
object Color {
def ColorWhite(c: Context): c.Expr[ObjectColor] = c.Expr[ObjectColor](c.universe.reify(ObjectColor(White())).tree)
}
}
The usage looks like this:
/src/main/com/example/core/Main.scala
object Macros {
import com.example.macros.Macros._
def W: ObjectColor = macro Color.ColorWhite
}
object Main extends App {
import Macros._
println(W)
}
Scala 2.11.6. SBT 0.13.8.
What am I doing wrong?
Thanks for your advice!
Fawlty Project:
The Project on Github
Working Project:
Rearranged the projects to a more correct form:
The cleanedup working project
Your macros and core projects don't contain any files, so they don't cause the problem. The error happens when sbt compiles root, which contains both Main.scala and Macros.scala by the virtue of you saying project in file(".") in the sbt build.

SBT: How to make one task depend on another in multi-project builds, and not run in the root project?

For my multi-project build, I'm trying to create a verify task that just results in scct:test and then scalastyle being executed in order. I would like scct:test to execute for all the subprojects, but not the top-level project. (If it executes for the top-level project, I get "timed out waiting for coverage report" from scct, since there's no source and no tests in that project.) What I had thought to do was to create verify as a task with dependencies on scct:test and scalastyle. This has turned out to be fairly baroque. Here is my Build.scala from my top-level project/ directory:
object MyBuild extends Build {
val verifyTask = TaskKey[Unit]("verify", "Compiles, runs tests via scct:test and then runs scalastyle")
val scctTestTask = (test in ScctPlugin.Scct).scopedKey
val scalastyleTask = PluginKeys.scalastyleTarget.scopedKey
lazy val root = Project("rootProject",
file("."),
settings = Defaults.defaultSettings ++
ScalastylePlugin.Settings ++
ScctPlugin.instrumentSettings ++
ScctPlugin.mergeReportSettings ++
Seq(
verifyTask in Global := {},
verifyTask <<= verifyTask.dependsOn(scctTestTask, scalastyleTask)
)
) aggregate(lift_webapp, selenium_tests)
lazy val subproject_1 = Project(id = "subproject_1", base = file("subproject_1"))
lazy val subproject_2 = Project(id = "subproject_2", base = file("subproject_2"))
}
However, the verify task only seems to exist for the root project; when I run it I don't see the same task being run in the subprojects. This is exactly the opposite of what I want; I'd like to issue sbt verify and have scct:test and scalastyle run in each of the subprojects but not in the top-level project. How might I go about doing that?
solution 1: define verifyTask in subprojects
First thing to note is that if you want some task (verify, test, etc) to run in some projects, you need to define them scoped to the subprojects. So in your case, the most straightforward thing to do this is to define verifyTask in subproject_1 and subproject_2.
lazy val scalaTest = "org.scalatest" %% "scalatest" % "3.0.4"
lazy val verify = taskKey[Unit]("verify")
def verifySettings = Seq(
skip in verify := false,
verify := (Def.taskDyn {
val sk = (skip in verify).value
if (sk) Def.task { println("skipping verify...") }
else (test in Test)
}).value
)
lazy val root = (project in file("."))
.aggregate(sub1, sub2)
.settings(
verifySettings,
scalaVersion in ThisBuild := "2.12.4",
skip in verify := true
)
lazy val sub1 = (project in file("sub1"))
.settings(
verifySettings,
libraryDependencies += scalaTest % Test
)
lazy val sub2 = (project in file("sub2"))
.settings(
verifySettings,
libraryDependencies += scalaTest % Test
)
solution 2: ScopeFilter
There was a recent Reddit thread that mentioned this question, so I'll post what I've done there.
If you want to manually aggregate on some subprojects, there's also a technique called ScopeFilter.
Note that I am using sbt 1.x here, but it should work with sbt 0.13 some minor change.
lazy val packageAll = taskKey[Unit]("package all the projects")
lazy val myTask = inputKey[Unit]("foo")
lazy val root = (project in file("."))
.aggregate(sub1, sub2)
.settings(
scalaVersion in ThisBuild := "2.12.4",
packageAll := {
(packageBin in Compile).all(nonRootsFilter).value
()
},
myTask := {
packageAll.value
}
)
lazy val sub1 = (project in file("sub1"))
lazy val sub2 = (project in file("sub2"))
def nonRootsFilter = {
import sbt.internal.inc.ReflectUtilities
def nonRoots: List[ProjectReference] =
allProjects filter {
case LocalProject(p) => p != "root"
case _ => false
}
def allProjects: List[ProjectReference] =
ReflectUtilities.allVals[Project](this).values.toList map { p =>
p: ProjectReference
}
ScopeFilter(inProjects(nonRoots: _*), inAnyConfiguration)
}
In the above, myTask depends on packageAll, which aggregates (packageBin in Compile) for all non-root subprojects.
sbt:root> myTask
[info] Packaging /Users/xxx/packageall/sub1/target/scala-2.12/sub1_2.12-0.1.0-SNAPSHOT.jar ...
[info] Done packaging.
[info] Packaging /Users/xxx/packageall/sub2/target/scala-2.12/sub2_2.12-0.1.0-SNAPSHOT.jar ...
[info] Done packaging.
[success] Total time: 0 s, completed Feb 2, 2018 7:23:23 PM
I may be wrong, but you are defining the verify task dependency only for the current project.
Maybe you can try:
Seq(
verifyTask in Global := {},
verifyTask <<= (verifyTask in Global).dependsOn(scctTestTask, scalastyleTask)
)
Or you can add the verifyTask settings to all your modules.