SBT is building external projects in random order - scala

I have project with externl projectes, registered in my build file with RootProject class.
In general, I have 4 folders: app0, app1, app3 and app-all on same level.
There is project folder in the app-all with following Build.scala file:
import sbt._
object AppBuild extends Build {
lazy val app2 = RootProject(file("../app0"))
lazy val app3 = RootProject(file("../app3"))
lazy val app1 = RootProject(file("../app1"))
lazy val all = Seq(japp1, app2, app3)
lazy val root = project.in(file(".")).dependsOn(all.map(ClasspathDependency(_, None)) :_*)
// lazy val root = project.in(file(".")).aggregate(all.map(sbt.Project.projectToRef) :_*)
}
The problem is, that sbt is building those 3 sub-projects in random order.
Once it starts with app1 another time it starts with app3. Same for both aggregate and dependsOn invocation. What are rules of aggregation?
Can I set order somehow?
I expect to set smth like app1.dependsOn(app3) but actually I cannot do it, because it's "project reference" but not real project.
Sbt version 0.13.
--
Note: This is test sample with one file, you can create 4 folders by yourself, try to build everything from apps-all and see what is happeninng.

In latest sbt 0.13 it's done like this.
lazy val `root` = (project in file(".")).
aggregate(`sub-a`, `sub-b`)
lazy val `sub-a` = (project in file("a"))
lazy val `sub-b` = (project in file("b")).
dependsOn(`sub-a`)
According to docs, http://www.scala-sbt.org/0.13/docs/Multi-Project.html#Classpath+dependencies
This also creates an ordering between the projects when compiling them

Related

In SBT, how to reference a subproject of another project?

In SBT it is possible to depend directly on another project. This has the advantage that when running ~compile, a change to the dependency causes a rebuild.
Now I want to depend on a subproject of another project, without depending on any of its siblings. So, for example, I have:
a/
build.sbt
b/
build.sbt
c/
build.sbt
d/
build.sbt
and I want d to depend on b but not on c.
I tried, in d/build.sbt,
lazy val d = ProjectRef(file("../a"), "b")
lazy val root = project.dependsOn(d)
but this gives
Note: Unresolved dependencies path:
com.foo:a_2.10:1.0
+- root:root_2.10:1.0
I could of course do
lazy val d = RootProject(file("../a/b"))
except that it is possible for a/build.sbt to contain additional settings for a/b that won't be pickup up this way. I need a reference that will pick up a/build.sbt but also refer specifically to a/b.
Is there any way to make this sort of reference?
It was just a stupid mistake on my part.
lazy val root = project
is not correct. That line of code creates a subproject in a directory called root/. The correct way to refer to the root project is:
lazy val root = Project(id = "root", base = file("."))
after that,
lazy val d = ProjectRef(file("../a"), "b")
lazy val root = Project(id = "root", base = file(".")).dependsOn(d)
works as it should.

sbt with sub-modules and multiple scala versions

I'm building a scala applications with these module and dependencies :
a shared lib "common"
module "A" depends on "common" and can be built in scala 2.10 only
module "B" depends on "common" and can be built in scala 2.11+ only
I'm trying to have all the 3 modules in a single sbt build :
import sbt._
import Keys._
lazy val root = (project in file("."))
.aggregate(common, A, B)
lazy val common = (project in file("common"))
lazy val A = (project in file("A"))
.dependsOn(common)
lazy val B = (project in file("B"))
.dependsOn(common)
I've read things about crossScalaVersions, but whatever combination I try in root build, or in common, etc, I can't manage to make this work properly.
Any clue ?
I'm using sbt 0.13.8 by the way.
Using bare sbt this might not be possible.
An example build.sbt file representing such situation:
lazy val common = (project in file("common"))
.settings(crossScalaVersions := Seq("2.10.6", "2.11.8"))
lazy val A = (project in file("A"))
.settings(scalaVersion := "2.10.6")
.dependsOn(common)
lazy val B = (project in file("B"))
.settings(scalaVersion := "2.11.8")
.dependsOn(common)
This will work just fine.
Now. A compilation of any project leads to a creation of a package. Even for the root. If you follow the console, at some point it says:
Packaging /project/com.github.atais/target/scala-2.10/root_2.10-0.1.jar
So as you see sbt needs to decide on some Scala version, just to build this jar! That being said your projects A and B must have a common Scala version so they can be aggregated in a common project root.
Therefore, you can't have:
lazy val root = (project in file("."))
.aggregate(common, A, B)
if they do not share any Scala version they could be built with.
But... sbt-cross to the rescue
You can use sbt-cross plugin to help you out.
In project/plugins.sbt, add
addSbtPlugin("com.lucidchart" % "sbt-cross" % "3.2")
And define your build.sbt in a following way:
lazy val common = (project in file("common")).cross
lazy val common_2_11 = common("2.11.8")
lazy val common_2_10 = common("2.10.6")
lazy val A = (project in file("A"))
.settings(scalaVersion := "2.10.6")
.dependsOn(common_2_10)
lazy val B = (project in file("B"))
.settings(scalaVersion := "2.11.8")
.dependsOn(common_2_11)
lazy val root = (project in file("."))
.aggregate(common, A, B)
And then it works :-)!
In my experience sbt multi-module builds are quite finicky to get to work reliably if you require any extra hoops to jump through such as this requirement.
Have you considered the simpler way to achieve this:
publish your common dependency (sbt publish-local if you only need to access it yourself)
make two projects A and B
make both A and B import common as a dependency

SBT: Access managed resources of a subproject?

In an SBT Plugin, I'm trying to access to managed resources of subprojects.
Here is the build file:
import sbt._
import Keys._
import play.Project._
object ApplicationBuild extends Build {
val appName = "demo"
val appVersion = "1.0-SNAPSHOT"
val appDependencies = Seq(
"org.jruby" % "jruby-complete" % "1.7.1"
)
val widgets = play.Project("widgets", appVersion, appDependencies, path = file("widgets"))
val main = play.Project(appName, appVersion, appDependencies, path = file("demo"))
.dependsOn(widgets)
}
I'm working in an SBT plugin defined in plugins.sbt.
Now, I need to use resources files from the subproject (widgets) during compilation of the parent project (demo).
So far the closest I've got to is the buildDependencies settings key - but I'm only getting ProjectRef objects, and the only information is the build base and the project id. I couldn't find a way to get to that project's resources directory.
I'm not familiar with writing plugins, but at least in your build.sbt you can define the resource file.
Or, again in the build.sbt you can create a "common" project that others reference, like:
lazy val common = (project in file("common"))
.settings(
Seq(
includeFilter in unmanagedResources := new SimpleFileFilter(_.getCanonicalPath.startsWith((sourceDirectory.value / "main" / "resources").getCanonicalPath))
)
)
Then other code (e.g. a Task) could reference this like:
lazy val doSomething = taskKey[Seq[File]]("Does something useful")
lazy val doSomethingSetting = doIt := {
val resourceDir = (resourceDirectory in common in Compile).value
println(resourceDir)
}
So your other projects could run this or reference that directory
Hopefully there's a straight forward way to implement one of those solutions for a plugin vs a build?
Unfortunately I do not believe this is possible. I was trying something similar but found the following in the documentation:
Note: At runtime, all plugins for all builds are loaded in a separate, parent class loader of the class loaders for builds. This means that plugins will not see classes or resources from build definitions
See: SBT Plugins

SBT: Run an action on all projects

I have an SBT project that contains multiple subprojects, i.e.:
> projects
[info] In file:/home/me/src/foo/
[info] popen
[info] foobar-core
[info] * foobar-client
[info] foobar-db
Is there a way to run an action in each of these subprojects? I'm looking for something like publish-all, since I currently go through all the subprojects and run publish manually, which gets fairly tedious once there are more than a handful of subprojects.
I'm using sbt-0.11.2 out of inertia, but am willing to upgrade if that helps.
You can define a project that aggregates all the other projects. Each action run on this project will be run on all aggregates. Here is an example from the sbt wiki:
import sbt._
import Keys._
object HelloBuild extends Build {
lazy val root = Project(id = "hello",
base = file(".")) aggregate(foo, bar)
lazy val foo = Project(id = "hello-foo",
base = file("foo"))
lazy val bar = Project(id = "hello-bar",
base = file("bar"))
}
The answer from Kim Stebel works perfectly, but requires an aggregation project.
(The following solution is tested with sbt 1.3.x)
I wanted a more dynamic solution. We have a small sbt plugin, that is used in all our projects anyway. So I added this:
val allProjects = Command.single("allProjects"){ (s,task) =>
val extracted = Project extract s
import extracted._
import currentRef.{ build => curi}
val build = structure.units(curi)
build.defined.keys.map(projectId => s"$projectId/$task").foldRight(s)(_ :: _)
}
override def globalSettings: Seq[Def.Setting[_]] = {
List(
commands += allProjects,
...
)
}
For a single build.sbt just use the first part with the line commands += allProjects added
This creates the command allProjects and can be used like allProjects compile.
The execution happens sequential, so it is blocking.
Better try this
After digging a bit more, I found out, that if no project with path . is created, all projects are by default aggregated in the root project.
Our problem was, that we liked our root project to be named, so we defined it as
lazy val coolProjectName = project.in(file("."))
I changed this in the build.sbt to:
name := "coolProjectName"
This way the default aggregation is still working.
And commands/tasks on aggregated projects are executed in parallel.

How to specify that to build project A another project B has to be built first?

Suppose one guy in my company has an sbt project called commons that's pretty general-purpose. This project is defined in the traditional sbt way: in the main folder with the build definition in project/Build.scala file.
Now some other guy is developing a project called databinding that depends on commons. We want to define this project in the same way, with project/Build.scala.
We have the following directory layout:
dev/
commons/
src/
*.scala files here...
project/
Build.scala
databinding/
src/
*.scala files here...
project/
Build.scala
How can I specify that databinding requires commons to be built first and use the output class files?
I read Multi-project builds, and came up with the following for the build definition of databinding:
object MyBuild extends Build {
lazy val root = Project(id = "databinding", base = file(".")) settings (
// ... omitted
) dependsOn (commons)
lazy val common = Project(id = "commons",
base = file("../commons")
)
}
Except it doesn't work: sbt doesn't like the .. and throws an AssertionError. Apparently, commons should be a folder inside databinding. But these two projects are kept in separate git repositories, which we cannot nest.
How can this dependency be specified properly?
You need to define the multi-project in a root project (or whatever name but this one fits well) that will be define in dev/project/Build.scala.
object RootBuild extends Build {
lazy val root = Project(id = "root", base = file("."))
.settings(...)
.aggregate(commons, databinding)
lazy val commons = Project(id = "commons", base = file("commons"))
.settings(...)
lazy val databinding = Project(id = "databinding", base = file("databinding"))
.settings(...)
.dependsOn(commons)
}
one more thing, SBT doesn't support *.scala configuration files in sub-projects. This means that you will have to migrate the configuration you made on commons/project/Build.scala and databinding/project/Build.scala into respectively commons/build.sbt and databinding/build.sbt.
If some of your configuration are not suitable for a .sbt definition file, you will have to add them in the root project/Build.scala. Obviously, settings defined in root Build.scala are available in *.sbt files.
You should use RootProject (in case of referring to root project of another project) or ProjectRef (in case of referring to subproject of another project).
Here is a sample of using RootProject:
lazy val commons = RootProject(file("../commons"))
lazy val root = Project(id = "databinding", base = file(".")) settings (...) dependsOn (commons)
And here is the sample for using ProjectRef
lazy val commons = ProjectRef(file("../commons"), "sub-project")
lazy val root = Project(id = "databinding", base = file(".")) settings (...) dependsOn (commons)
You could have two separate projects, and just publish one of them locally, adding it as a normal library dependency to the other.