What is the SBT := operator in build.sbt? - scala

I'm new to Scala and SBT. I noticed an unfamiliar operator in the build.sbt of an open source project:
:=
Here are a couple examples of how it's used:
lazy val akkaApp = Project(id = "akka-app", base = file("akka-app"))
.settings(description := "Common Akka application stack: metrics, tracing, logging, and more.")
and it's used a few times in this larger code snippet:
lazy val jobServer = Project(id = "job-server", base = file("job-server"))
.settings(commonSettings)
.settings(revolverSettings)
.settings(assembly := null.asInstanceOf[File])
.settings(
description := "Spark as a Service: a RESTful job server for Apache Spark",
libraryDependencies ++= sparkDeps ++ slickDeps ++ cassandraDeps ++ securityDeps ++ coreTestDeps,
test in Test <<= (test in Test).dependsOn(packageBin in Compile in jobServerTestJar)
.dependsOn(clean in Compile in jobServerTestJar)
.dependsOn(buildPython in jobServerPython)
.dependsOn(clean in Compile in jobServerPython),
testOnly in Test <<= (testOnly in Test).dependsOn(packageBin in Compile in jobServerTestJar)
.dependsOn(clean in Compile in jobServerTestJar)
.dependsOn(buildPython in jobServerPython)
.dependsOn(clean in Compile in jobServerPython),
console in Compile <<= Defaults.consoleTask(fullClasspath in Compile, console in Compile),
fullClasspath in Compile <<= (fullClasspath in Compile).map { classpath =>
extraJarPaths ++ classpath
},
fork in Test := true
)
.settings(publishSettings)
.dependsOn(akkaApp, jobServerApi)
.disablePlugins(SbtScalariform)
My best guess is that it means "declare if not already declared".

The := has essentially nothing to do with the ordinary assignment operator =. It's not a built-in scala operator, but rather a family of methods/macros called :=. These methods (or macros) are members of classes such as SettingKey[T] (similarly for TaskKey[T] and InputKey[T]). They consume the right hand side of the key := value expression, and return instances of type Def.Setting[T] (or similarly, Tasks), where T is the type of the value represented by the key. They are usually written in infix notation. Without syntactic sugar, the invocations of these methods/macros would look as follows:
key.:=(value)
The constructed Settings and Tasks are in turn the basic building blocks of the build definition.
The important thing to understand here is that the keys on the left hand side are not some variables in a code block. Instead of merely representing a memory position in an active stack frame of a function call (as a simple variable would do), the keys on the left hand side are rather complex objects which can be inspected and passed around during the build process.

Related

Scala.js crossProject vs manual setup

To compile the same source code with Scala.js and Scala JVM the documentation says to use crossProject, for instance
lazy val foo = crossProject.in(file(".")).
settings(
name := "foo",
version := "0.1-SNAPSHOT"
)
lazy val fooJVM = foo.jvm
lazy val fooJS = foo.js
However it looks that the same goal can be achieved setting up the modules manually
lazy val fooSettings = Seq(
name := "foo",
version := "0.1-SNAPSHOT",
scalaSource in Compile := baseDirectory.value / ".." / "shared" / "src" / "main" / "scala"
)
lazy val fooJVM = project.in(file("jvm"))
.settings(fooSettings: _*)
lazy val fooJS = project.in(file("js"))
.settings(fooSettings: _*)
.enablePlugins(ScalaJSPlugin)
Does crossProject do something important or it's just more convenient way to setup stuff?
It looks like, in your manual setup, you have separate jvm and js subdirectories under your "shared" area -- it doesn't look like they're the same code at all.
CrossProject is about letting you have a single directory structure, with the same code, that is compiled for both sides. Possibly with some small subdirectories that are specific to one side or another, but in general the focus is that most of the code is shared -- the jvm and js subdirectories, if present (it depends on the mode), are usually just shims to adapt the common code to those sides.
All that said, it is really just to make it more convenient -- I believe you could achieve the same results without CrossProject. But in the common case where you're sharing a lot of code, it's significantly more convenient...

What does "<<=" do in sbt settings?

I am learning to write some more advanced sbt build files, and I came across sbt-proguard's code:
binaryDeps <<= compile in Compile map { _.relations.allBinaryDeps.toSeq },
inputs <<= fullClasspath in Runtime map { _.files },
libraries <<= (binaryDeps, inputs) map { (deps, in) => deps filterNot in.toSet },
outputs <<= artifactPath map { Seq(_) },
I want to know what does <<= mean in this context?
How do I understand the map function at the 3rd line?
The <<= is a method on DefinableSetting (mixed in by TaskKey, InputKey, and SettingKey) which provides a way to initialise a build setting. It's described in the older docs here:
:= assigns a value, overwriting any existing value. <<= uses existing values to initialize a setting.
Essentially, in 0.12 (and current versions, for compatibility) it was a way to define a build setting in terms of some other build setting(s).
As #sjrd points out, in 0.13 a new task setting syntax was introduced allowing you to do this with := instead.
The map in the third line is creating a new settings value by getting only dependency items from binaryDeps which are not already in inputs, i.e. transform these two settings into this new one.

Where are examples of `?`, `??`, `<++=`, `<+=` in build.sbt?

I'm reading the sbt document, found there are some special methods I've never used:
?
??
<++=
<+=
Where can I find any examples of them?
SBT 0.13 did a great job eliminating the need for these operators and simplify build definition to :=, += and ++= with the help of macros and the special "extractor" .value. So no need for these operators anymore. The only thing I'm still using is ~= were you can apply some function to the value of some setting, but it also can be expressed with := and .value
In your question, I think you've mixed two sets of operations - one with <+= and <++= that was or are about to be "deprecated" in favour of :=, += and ++=, and another with ? and ?? that's unfortunately not very often used since all can be expressed with :=, += and ++= (and people often find using 3 enough for their use cases).
Read the official documentation of sbt in More operations about ? and ??.
As for examples:
?
lazy val unintiedKey = settingKey[String]("Unitialized key")
lazy val someKey = settingKey[String]("Key to check the value of another")
someKey := unintiedKey.?.value getOrElse "new value"
What do you think is going to be printed out with show someKey given the above build.sbt?
> show someKey
[info] new value
When you add the following to the build.sbt to have the uninitedKey setting initialized:
unintiedKey := "Another value"
someKey changes, too:
> show unintiedKey
[info] Another value
> show someKey
[info] Another value
??
Let's define a build with the following build.sbt:
lazy val unintiedKey = settingKey[String]("Unitialized key")
lazy val someKey = settingKey[String]("Key to check the value of another")
someKey := (unintiedKey ?? "uninitedKey had no value").value
Guess what the value of someKey is going to be?
> show someKey
[info] uninitedKey had no value
The key to understand the operations (that make up the sbt.SettingKey API) is to understand what a setting is in sbt - it's a pair of a key and an initialization that gets transformed into a useable setting when a scope gets applied to it.

How to define project-specific settings/keys in sbt 0.13

I have a simple multi project, whereby the root aggregates projects a and b. The root project loads this plugin I'm writing that is supposed to allow easy integration with the build system in our company.
lazy val a = project in file("a")
lazy val b = project in file("b")
Now, I'd like to define some Settings in the plugin that don't make sense in the root project, and can have different values for each sub-project. However, if I just define them like
object P extends Plugin {
val kind = settingKey[Kind]("App or Lib?")
val ourResolver = settingKey[Resolver]("...")
override def projectSettings = Seq(
// I want this to only be defined in a and b, where `kind` is defined
ourResolver <<= kind { k => new OurInternalResolver(k) }
)
}
then sbt will complain that ourResolver cannot be defined because kind is not defined in the root project.
Is there a way to specify a scope for this setting (ourResolver), such that the setting would be defined in every aggregated project except the root project?
Or do I have to make it into an SettingKey[Option[_]] and set it to None by default?
Edit: I have quite a large set of settings which progressively depend on kind and then ourResolver and so on, and these settings should only be defined where (read: "in the project scopes where") kind is defined. Added ourResolver to example code to reflect this.
However, if I just define them like....
There's nothing really magical about setting keys. Keys are just String entry tied with the type. So You can define your key the way you're doing now just fine.
Is there a way to specify a scope for this setting, such that the setting would be defined in every aggregated project except the root project?
A setting consists of the following four things:
scope (project, config, task)
a key
dependencies
an ability to evaluate itself (def evaluate(Settings[Scope]): T)
If you did not specify otherwise, your settings are already scoped to a particular project in the build definition.
lazy val a = (project in file("a")).
settings(commonSettings: _*).
settings(
name := "a",
kind := Kind.App,
libraryDependencies ++= aDeps(scalaVersion.value)
)
lazy val b = (project in file("b")).
settings(commonSettings: _*).
settings(
name := "b",
kind := Kind.Lib,
libraryDependencies ++= bDeps(scalaVersion.value)
)
lazy val root = (project in file(".")).
settings(commonSettings: _*).
settings(
name := "foo",
publishArtifact := false
).
aggregate(a, b)
In the above, kind := Kind.App setting is scoped to project a. So that would answer your question literally.
sbt will complain that kind is not defined in the root project.
This is the part I'm not clear what's going on. Does this happen when you load the build? Or does it happen when you type kind into the sbt shell? If you're seeing it at the startup that likely means you have a task/setting that's trying to depend on kind key. Don't load the setting, or reimplement it so something that doesn't use kind.
Another way to avoid this issue may be to give up on using root aggregation. Switch into subproject and run tasks or construct explicit aggregation using ScopeFilter.
I've managed to ultimately do this by leveraging derive, a method that creates DerivedSettings. These settings will then be automatically expanded by sbt in every scope where their dependencies are defined.
Therefore I can now write my plugin as:
object P extends Plugin {
val kind = settingKey[Kind]("App or Lib?")
val ourResolver = settingKey[Resolver]("...")
def derivedSettings = Seq(
// I want this to only be defined in a and b, where `kind` is defined
ourResolver <<= kind { k => new OurInternalResolver(k) }
) map (s => Def.derive(s, trigger = _ != streams.key))
override def projectSettings = derivedSettings ++ Seq( /* other stuff */ )
}
And if I define kind in a and b's build.sbt, then ourResolver will also end up being defined in those scopes.

Run custom task automatically before/after standard task

I often want to do some customization before one of the standard tasks are run. I realize I can make new tasks that executes existing tasks in the order I want, but I find that cumbersome and the chance that a developer misses that he is supposed to run my-compile instead of compile is big and leads to hard to fix errors.
So I want to define a custom task (say prepare-app) and inject it into the dependency tree of the existing tasks (say package-bin) so that every time someone invokes package-bin my custom tasks is run right before it.
I tried doing this
def mySettings = {
inConfig(Compile)(Seq(prepareAppTask <<= packageBin in Compile map { (pkg: File) =>
// fiddle with the /target folder before package-bin makes it into a jar
})) ++
Seq(name := "my project", version := "1.0")
}
lazy val prepareAppTask = TaskKey[Unit]("prepare-app")
but it's not executed automatically by package-bin right before it packages the compile output into a jar. So how do I alter the above code to be run at the right time ?
More generally where do I find info about hooking into other tasks like compile and is there a general way to ensure that your own tasks are run before and after a standard tasks are invoked ?.
Extending an existing task is documented the SBT documentation for Tasks (look at the section Modifying an Existing Task).
Something like this:
compile in Compile <<= (compile in Compile) map { _ =>
// what you want to happen after compile goes here
}
Actually, there is another way - define your task to depend on compile
prepareAppTask := (whatever you want to do) dependsOn compile
and then modify packageBin to depend on that:
packageBin <<= packageBin dependsOn prepareAppTask
(all of the above non-tested, but the general thrust should work, I hope).
As an update for the previous answer by #Paul Butcher, this could be done in a bit different way in SBT 1.x versions since <<== is no longer supported. I took an example of a sample task to run before the compilation that I use in one of my projects:
lazy val wsdlImport = TaskKey[Unit]("wsdlImport", "Generates Java classes from WSDL")
wsdlImport := {
import sys.process._
"./wsdl/bin/wsdl_import.sh" !
// or do whatever stuff you need
}
(compile in Compile) := ((compile in Compile) dependsOn wsdlImport).value
This is very similar to how it was done before 1.x.
Also, there is another way suggested by official SBT docs, which is basically a composition of tasks (instead of dependencies hierarchy). Taking the same example as above:
(compile in Compile) := {
val w = wsdlImport.value
val c = (compile in Compile).value
// you can add more tasks to composition or perform some actions with them
c
}
It feels like giving more flexibility in some cases, though the first example looks a bit neater, as for me.
Tested on SBT 1.2.3 but should work with other 1.x as well.