I have an SBT Scala multi-project with the following structure:
multiprojectRoot
project/SharedProjectBuildCode.scala
project1
src/sourceFiles
project1-build.sbt
project2
src/sourceFiles
project2-build.sbt
projectN
src/sourceFiles
projectN-build.sbt
multiprojectRoot/project/SharedProjectBuildCode.scala: contains multi-project definitions that use dependsOn to create dependencies on local projects. For example:
lazy val project2 = Project(
...
).dependsOn(project1)
multiprojectRoot/project2/project2-build.sbt: Contains the settings and dependencies for a given project. For example:
name := "project2"
libraryDependencies ++= Seq(
...
"my.company" % "project1" % "1.0"
)
First dependency to project1 is declared with dependsOn on SharedProjectBuildCode.scala file and the second is created on standalone project2-build.sbt build definition file.
So, project2 definition contains either:
an ambiguous dependency to project1 or
a double dependency to project1
We want to keep this project structure, because is the best for our current workflow:
Independent .sbt files serve standalone deployment purposes for each project on our continuous delivery server.
Multi-project .scala file with dependsOn is used to facilitate development, allowing us to avoid things such as continuous publishLocal.
We need to have control for such dependency ambiguities someway. Can you help me?
I think you should have in SharedProjectBuildCode.scala
lazy val root = Project(id = "Main-Project",
base = file(".")) aggregate(project1, project2,..)
lazy val project2 = Project(id = "project2",
base = file("project1")).dependsOn(project1)
...
And don't need to add as dependency in build.sbt anymore.
I was able to control which dependency set loaded on each use case by using the rules of build files loading provided by SBT.
When you load SBT from a given root directory, it looks for *.sbt files on the root directory and also for *.scala on the root/project directory. If you have a multi-project build, then it also reads the definitions of .sbt files that are encountered on child projects, but it will not use project/.scala files on child projects:
.sbt build definition
Multi-project builds
So, I changed my multi-project build the following way:
multiprojectRoot
project/SharedProjectBuildCode.scala
project1
src/sourceFiles
project/DeploymentOnlyCode.scala
project1-build.sbt
project2
src/sourceFiles
project/DeploymentOnlyCode.scala
project2-build.sbt
projectN
src/sourceFiles
project/DeploymentOnlyCode.scala
projectN-build.sbt
This way, depending on the use case I run SBT from the multi-project root or a project internal directory:
Development: SBT is run from multiprojectRoot directory. It takes the advantages of having a multi-project build (such as using dependsOn and avoiding publishLocal).
Production: SBT is run from within a concrete project directory, such as multiprojectRoot/project2. It allows the project to be built as stand-alone, having all dependencies as explicit external (useful for declaring a sequence of dependencies on production, continuous integration server).
Now, a project has 3 instances of code that aggregates their attributes for a final build:
multiprojectRoot/project/SharedProjectBuildCode.scala: Contains local dependencies and other code relevant for multi-project build.
multiprojectRoot/project1/project1-build.sbt: Contains project build attributes, common for multi-project and standalone build of a project, such as name or dependencies that are always external. The same should be done for other multi-project projects of the same level, to be explicitly treated as external dependency artifacts.
multiprojectRoot/project1/project/DeploymentOnlyCode.scala: Contains build attributes that will only be taken into consideration for stand-alone build. The same can be done on other sub-projects, if these require to define deployment specific attributes.
This also gives maximum control on how a project is built, whether is a releasable artifact or not, and handle source code relevant only for a given project, as a complete and independent piece.
Related
I have a main sbt project which depends on two sbt subprojects, 'util', and 'common'. In addition the common project depends on the util project. In the build.sbt of the common project, how should I specify the dependency to the sibling project util, so that I can compile the common project independently?
Many thanks.
Azad
I'm trying to come up with a structure for a large, multi-module sbt project that satisfies the following requirements:
When the root project is built, dependencies are first resolved from the modules available under the root ( i.e. if module A depends on module B version 2, which is the version of B available under the root, satisfy the dependency with whatever the build for B produces )
When modules are built individually, dependencies are resolved from repositories ( local, cache, remote )
I am aware of two vehicles to define dependencies to an sbt project: dependsOn() and the libraryDependencies setting key.
So far, in my naive structure, where all build information for the modules (A, B) was tracked at the root, I simply passed .dependsOn the project references, and the inter-module dependencies were correctly resolved in the build of R
What I would like to do, is to move/track this relationship in the build.sbt file of the modules themselves, which are then hosted in separate repositories (and pulled back occasionally to an "aggregate" tag of the parent project's repo via git submodule)
I've never had any problem doing this with maven (I assume because of being able to refer to a parent explicitly in the module's pom and there being only one way to establish a dependency) but I can't yet wrap my head around how to get it going in sbt
So my question is, will I have to write a custom resolver for this? Is there anything obvious I'm missing here?
Thanks.
I'm also having a similar setup, with an aggregate project with 100+ sub-projects. Sub-project also live in their own repository and can be built/published stand-alone or as part of the aggregate project. I don't need any special resolver for this to work.
I'm just combining both approaches you described:
project A:
groupId := "groupId"
version := "1.0.0-SNAPSHOT"
libraryDependencies += "groupId" %% "B" % version
project B:
groupId := "groupId"
version := "1.0.0-SNAPSHOT"
project R:
lazy val a = (project in file(a)).dependsOn(b)
lazy val b = (project in file(b))
I noticed that sbt is clever enough not to include the dependency on b twice.
Using SBT 0.13.13, our build definition is inherited from an older project. Currently there are a build.sbt and some project/*.scala files. These scala files follow the same pattern. Here is an example:
import sbt._
import sbt.Keys._
object Docs {
lazy val docTask = TaskKey[Unit]("docPackage", "Generate Scaladoc")
lazy val settings = Seq(
docTask := {
val docs = (doc in Compile).value
IO.copyDirectory(docs, new java.io.File("src/main/resources/myapp-scaladoc"), overwrite = true)
},
docTask := (docTask.dependsOn(doc in Compile)).value
)
}
Appendix: .scala build definition says
In the previous versions of sbt, .scala was the only way to create
multi-project build definition
Question: I suppose this mean the use of separate project/*.scala files is discouraged. If so, is it OK to move the code of these *.scala file and put them all in build.sbt?
There is little functional difference between having code in project/ and code in build.sbt.
The advantage to putting code in a .scala file in project/ is that all symbols in it are imported into the namespace of all build.sbt files in all subprojects of your build. This means that helper methods, constants, plugins, or other Scala code can live in a single location, but be called from anywhere.
Ultimately, whether to use this features comes down to whether you want shared items configured in a single location, or in multiple locations.
If you prefer to have your project settings configured entirely in separate build.sbt files in their subproject directories, putting shared settings in the project/ folder is the way to do that.
If you're OK with having shared settings configured in the root build.sbt file (or if you don't use any subproject build.sbt files), you can consolidate all of your shared configuration into it.
I have a main project and a few subprojects. When I want to run tests, currently I have to do sbt test and sbt subProjectName/test. Is there any way of making sbt run all tests or for example all tests in the main project and one of the subprojects.
I am using Build.scala configurations, but can't find a way of setting this.
Thanks!
Therefore sbt supports aggregate.
for in depth details read: http://www.scala-sbt.org/0.13.5/docs/Getting-Started/Multi-Project.html#aggregation
In given example all commands at mainProject will also be run on other-project.
So running mainProject/test will also run otherProject/test.
If mainProject is your base project test will be enough.
in build.sbt
lazy val mainProject =
(project in file("."))
.aggregate(otherProject)
lazy val otherProject = (project in file("other-project"))
I am trying to figure out how idea will recognize thrid party dependencies when using SBT. When I use the sbt plugin gen-idea it seems to download all the necessary dependencies which get put into my ~/.ivy/ directory as expected. How can intellij use these deps?
EDIT:
One thing I noticed is if I make a new idea project instead of just a module then this works? Any idea why this would be? I would like to be able to have multiple sbt modules in the same project.
The sbt-idea plugin works with multi-module sbt project. We have been using it since somewhere around sbt-0.10.0, and currently are at sbt-0.11.2. It seems like you have the dependency part of the build file set up ok, so here's an example of how we do the project setup from a full specification Build.scala file:
object Vcaf extends Build {
import Resolvers._
import Dependencies._
import BuildSettings._
lazy val vcafDb = Project(
id = "vcaf-db",
base = file("./vcaf-db"),
dependencies = Seq(),
settings = buildSettings ++ /* proguard */ SbtOneJar.oneJarSettings ++ Seq(libraryDependencies := dbDeps, resolvers := cseResolvers)
)
lazy val vcaf = Project(
"vcaf",
file("."),
dependencies = Seq(vcafDb),
aggregate = Seq(vcafDb),
settings = buildSettings ++ Seq(libraryDependencies := vcafDeps, resolvers := cseResolvers) ++ webSettings
)
}
In the example, the vcaf-db project is in the a folder within the vcaf project folder. The vcaf-db project does not have it's own build.sbt or Build.scala file. You'll notice that we are specifying libraryDependencies for each project, which may or may not be your missing link.
As ChrisJamesC mentioned, you need to do a "reload" from within SBT (or exit sbt and come back in) to pick up changes to your build definition. After the project is reloaded, you should be able to do a "gen-idea no-classifiers no-sbt-classifiers" and get an intellij project that has the main project, modules, and library access as defined in the build file.
Hope it helps!
If you want multiple SBT modules in one IDEA project, you can use sbt multi-project builds (aka subprojects). Just create a master project that refers to the modules as sub-projects, then run gen-idea on the master. To specify dependencies among the modules you have to use Build.scala (not build.sbt), as in jxstanford's answer or like this:
lazy val foo = Project(id = "foo", base = file("foo"))
lazy val bar = Project(id = "bar", base = file("bar")) dependsOn(foo)
One level of subprojects works fine (with the dependencies correctly reflected in the resulting IDEA project), but nested subprojects don't seem to work. Also, it seems to be an sbt restriction that the subprojects must live in subdirectories of the master project (i.e., file("../foo") is not allowed).
See also How to manage multiple interdependent modules with SBT and IntelliJ IDEA?.