SBT: Run an action on all projects - scala

I have an SBT project that contains multiple subprojects, i.e.:
> projects
[info] In file:/home/me/src/foo/
[info] popen
[info] foobar-core
[info] * foobar-client
[info] foobar-db
Is there a way to run an action in each of these subprojects? I'm looking for something like publish-all, since I currently go through all the subprojects and run publish manually, which gets fairly tedious once there are more than a handful of subprojects.
I'm using sbt-0.11.2 out of inertia, but am willing to upgrade if that helps.

You can define a project that aggregates all the other projects. Each action run on this project will be run on all aggregates. Here is an example from the sbt wiki:
import sbt._
import Keys._
object HelloBuild extends Build {
lazy val root = Project(id = "hello",
base = file(".")) aggregate(foo, bar)
lazy val foo = Project(id = "hello-foo",
base = file("foo"))
lazy val bar = Project(id = "hello-bar",
base = file("bar"))
}

The answer from Kim Stebel works perfectly, but requires an aggregation project.
(The following solution is tested with sbt 1.3.x)
I wanted a more dynamic solution. We have a small sbt plugin, that is used in all our projects anyway. So I added this:
val allProjects = Command.single("allProjects"){ (s,task) =>
val extracted = Project extract s
import extracted._
import currentRef.{ build => curi}
val build = structure.units(curi)
build.defined.keys.map(projectId => s"$projectId/$task").foldRight(s)(_ :: _)
}
override def globalSettings: Seq[Def.Setting[_]] = {
List(
commands += allProjects,
...
)
}
For a single build.sbt just use the first part with the line commands += allProjects added
This creates the command allProjects and can be used like allProjects compile.
The execution happens sequential, so it is blocking.
Better try this
After digging a bit more, I found out, that if no project with path . is created, all projects are by default aggregated in the root project.
Our problem was, that we liked our root project to be named, so we defined it as
lazy val coolProjectName = project.in(file("."))
I changed this in the build.sbt to:
name := "coolProjectName"
This way the default aggregation is still working.
And commands/tasks on aggregated projects are executed in parallel.

Related

How do I do a multi-project build that outputs a jar for every subproject?

I have a project, called Main, and 2 subprojects: One, Two. Here's what my build.sbt looks like:
name := "Main"
version := "0.1"
scalaVersion := "2.12.7"
lazy val root = Project(id = "root", base = file(".")) aggregate(one, two) dependsOn(one, two)
lazy val one = Project(id = "one", base = file("One"))
lazy val two = Project(id = "two", base = file("Two"))
when I run sbt compile package I only get a .jar for Main (Main.jar), but I want to get a .jar for every subproject and not Main: One.jar, Two.jar.
How do I achieve this?
Also I have no idea what aggregate(one, two) dependsOn(one, two) means, do I even need that?
I also want every subproject to build into a fat jar with sbt-assembly.
It actually works now. I was encountering this behavior, where sbt was telling me that my assembly was not packaged, because it was up-to-date, but then I started doing sbt clean assembly, instead of sbt assembly and it works like a charm now.

sbt 0.13.1 multi-project module not found when I change the sbt default scala library to scala 2.11.2

I use the sbt 0.13.1 create the two modules, and I create project/MyBuild.scala to compile this two modules.
MyBuild.scala:
import sbt._
import Keys._
object MyBuild extends Build {
lazy val task = project.in(file("task"))
lazy val root = project.in(file(".")) aggregate(task) dependsOn task
}
When I change the scala library to 2.11.2 by set scalaHome. It will go to maven download the task.jar and failed, that's very strange. Is it a sbt bug?
There is the github test project address: test-sbt-0.13.1
This is because you have defined a custom scalaVersion in your main build.sbt which means it is defined for the root project only. The task project will use the default value:
jjst#ws11:test-sbt-0.13.1$ sbt
[...]
> projects
[info] In file:/home/users/jjost/dev/test-sbt-0.13.1/
[info] * root
[info] task
> show scalaVersion
[info] task/*:scalaVersion
[info] 2.10.4
[info] root/*:scalaVersion
[info] 2.11.2-local
As a consequence, artifacts generated by the task subproject won't be available for the root project to use.
You can solve this by making sure that your projects use the same scalaVersion. The easiest way to do so while keeping the same project structure would be to share common settings like the scala version across projects like so:
object MyBuild extends Build {
val commonSettings = Seq(
scalaVersion := "2.11.2-local",
scalaHome := Some(file("/usr/local/scala-2.11.2/"))
)
lazy val task = (project.in(file("task"))).
settings(commonSettings: _*)
lazy val root = (project.in(file("."))).
settings(commonSettings: _*).
aggregate(task).
dependsOn(task)
}
In practice, you might want to put common settings shared between projects in a dedicated file under project/common.scala, as recommended in Effective sbt.
I'd recommend you to move the sources of root project from root to subfolder (task2), and add them as aggregated. It will remove aggregating and depending on same project:
object MyBuild extends Build {
lazy val task = project.in(file("task"))
lazy val task2 = project.in(file("task2")) dependsOn task
lazy val root = project.in(file(".")) aggregate(task, task2)
}
This works for me, but it's strange that such non-standard project structure works fine with default scalaHome. Seems to be a problem in the way in which sbt resolves such dependency as external.
P.S. This answer doesn't cover the whole story. See #jjst's answer to clarify more.

SBT is building external projects in random order

I have project with externl projectes, registered in my build file with RootProject class.
In general, I have 4 folders: app0, app1, app3 and app-all on same level.
There is project folder in the app-all with following Build.scala file:
import sbt._
object AppBuild extends Build {
lazy val app2 = RootProject(file("../app0"))
lazy val app3 = RootProject(file("../app3"))
lazy val app1 = RootProject(file("../app1"))
lazy val all = Seq(japp1, app2, app3)
lazy val root = project.in(file(".")).dependsOn(all.map(ClasspathDependency(_, None)) :_*)
// lazy val root = project.in(file(".")).aggregate(all.map(sbt.Project.projectToRef) :_*)
}
The problem is, that sbt is building those 3 sub-projects in random order.
Once it starts with app1 another time it starts with app3. Same for both aggregate and dependsOn invocation. What are rules of aggregation?
Can I set order somehow?
I expect to set smth like app1.dependsOn(app3) but actually I cannot do it, because it's "project reference" but not real project.
Sbt version 0.13.
--
Note: This is test sample with one file, you can create 4 folders by yourself, try to build everything from apps-all and see what is happeninng.
In latest sbt 0.13 it's done like this.
lazy val `root` = (project in file(".")).
aggregate(`sub-a`, `sub-b`)
lazy val `sub-a` = (project in file("a"))
lazy val `sub-b` = (project in file("b")).
dependsOn(`sub-a`)
According to docs, http://www.scala-sbt.org/0.13/docs/Multi-Project.html#Classpath+dependencies
This also creates an ordering between the projects when compiling them

How to get Intellij to use dependencies from SBT scala

I am trying to figure out how idea will recognize thrid party dependencies when using SBT. When I use the sbt plugin gen-idea it seems to download all the necessary dependencies which get put into my ~/.ivy/ directory as expected. How can intellij use these deps?
EDIT:
One thing I noticed is if I make a new idea project instead of just a module then this works? Any idea why this would be? I would like to be able to have multiple sbt modules in the same project.
The sbt-idea plugin works with multi-module sbt project. We have been using it since somewhere around sbt-0.10.0, and currently are at sbt-0.11.2. It seems like you have the dependency part of the build file set up ok, so here's an example of how we do the project setup from a full specification Build.scala file:
object Vcaf extends Build {
import Resolvers._
import Dependencies._
import BuildSettings._
lazy val vcafDb = Project(
id = "vcaf-db",
base = file("./vcaf-db"),
dependencies = Seq(),
settings = buildSettings ++ /* proguard */ SbtOneJar.oneJarSettings ++ Seq(libraryDependencies := dbDeps, resolvers := cseResolvers)
)
lazy val vcaf = Project(
"vcaf",
file("."),
dependencies = Seq(vcafDb),
aggregate = Seq(vcafDb),
settings = buildSettings ++ Seq(libraryDependencies := vcafDeps, resolvers := cseResolvers) ++ webSettings
)
}
In the example, the vcaf-db project is in the a folder within the vcaf project folder. The vcaf-db project does not have it's own build.sbt or Build.scala file. You'll notice that we are specifying libraryDependencies for each project, which may or may not be your missing link.
As ChrisJamesC mentioned, you need to do a "reload" from within SBT (or exit sbt and come back in) to pick up changes to your build definition. After the project is reloaded, you should be able to do a "gen-idea no-classifiers no-sbt-classifiers" and get an intellij project that has the main project, modules, and library access as defined in the build file.
Hope it helps!
If you want multiple SBT modules in one IDEA project, you can use sbt multi-project builds (aka subprojects). Just create a master project that refers to the modules as sub-projects, then run gen-idea on the master. To specify dependencies among the modules you have to use Build.scala (not build.sbt), as in jxstanford's answer or like this:
lazy val foo = Project(id = "foo", base = file("foo"))
lazy val bar = Project(id = "bar", base = file("bar")) dependsOn(foo)
One level of subprojects works fine (with the dependencies correctly reflected in the resulting IDEA project), but nested subprojects don't seem to work. Also, it seems to be an sbt restriction that the subprojects must live in subdirectories of the master project (i.e., file("../foo") is not allowed).
See also How to manage multiple interdependent modules with SBT and IntelliJ IDEA?.

How to specify that to build project A another project B has to be built first?

Suppose one guy in my company has an sbt project called commons that's pretty general-purpose. This project is defined in the traditional sbt way: in the main folder with the build definition in project/Build.scala file.
Now some other guy is developing a project called databinding that depends on commons. We want to define this project in the same way, with project/Build.scala.
We have the following directory layout:
dev/
commons/
src/
*.scala files here...
project/
Build.scala
databinding/
src/
*.scala files here...
project/
Build.scala
How can I specify that databinding requires commons to be built first and use the output class files?
I read Multi-project builds, and came up with the following for the build definition of databinding:
object MyBuild extends Build {
lazy val root = Project(id = "databinding", base = file(".")) settings (
// ... omitted
) dependsOn (commons)
lazy val common = Project(id = "commons",
base = file("../commons")
)
}
Except it doesn't work: sbt doesn't like the .. and throws an AssertionError. Apparently, commons should be a folder inside databinding. But these two projects are kept in separate git repositories, which we cannot nest.
How can this dependency be specified properly?
You need to define the multi-project in a root project (or whatever name but this one fits well) that will be define in dev/project/Build.scala.
object RootBuild extends Build {
lazy val root = Project(id = "root", base = file("."))
.settings(...)
.aggregate(commons, databinding)
lazy val commons = Project(id = "commons", base = file("commons"))
.settings(...)
lazy val databinding = Project(id = "databinding", base = file("databinding"))
.settings(...)
.dependsOn(commons)
}
one more thing, SBT doesn't support *.scala configuration files in sub-projects. This means that you will have to migrate the configuration you made on commons/project/Build.scala and databinding/project/Build.scala into respectively commons/build.sbt and databinding/build.sbt.
If some of your configuration are not suitable for a .sbt definition file, you will have to add them in the root project/Build.scala. Obviously, settings defined in root Build.scala are available in *.sbt files.
You should use RootProject (in case of referring to root project of another project) or ProjectRef (in case of referring to subproject of another project).
Here is a sample of using RootProject:
lazy val commons = RootProject(file("../commons"))
lazy val root = Project(id = "databinding", base = file(".")) settings (...) dependsOn (commons)
And here is the sample for using ProjectRef
lazy val commons = ProjectRef(file("../commons"), "sub-project")
lazy val root = Project(id = "databinding", base = file(".")) settings (...) dependsOn (commons)
You could have two separate projects, and just publish one of them locally, adding it as a normal library dependency to the other.