I'm reading the sbt document, found there are some special methods I've never used:
?
??
<++=
<+=
Where can I find any examples of them?
SBT 0.13 did a great job eliminating the need for these operators and simplify build definition to :=, += and ++= with the help of macros and the special "extractor" .value. So no need for these operators anymore. The only thing I'm still using is ~= were you can apply some function to the value of some setting, but it also can be expressed with := and .value
In your question, I think you've mixed two sets of operations - one with <+= and <++= that was or are about to be "deprecated" in favour of :=, += and ++=, and another with ? and ?? that's unfortunately not very often used since all can be expressed with :=, += and ++= (and people often find using 3 enough for their use cases).
Read the official documentation of sbt in More operations about ? and ??.
As for examples:
?
lazy val unintiedKey = settingKey[String]("Unitialized key")
lazy val someKey = settingKey[String]("Key to check the value of another")
someKey := unintiedKey.?.value getOrElse "new value"
What do you think is going to be printed out with show someKey given the above build.sbt?
> show someKey
[info] new value
When you add the following to the build.sbt to have the uninitedKey setting initialized:
unintiedKey := "Another value"
someKey changes, too:
> show unintiedKey
[info] Another value
> show someKey
[info] Another value
??
Let's define a build with the following build.sbt:
lazy val unintiedKey = settingKey[String]("Unitialized key")
lazy val someKey = settingKey[String]("Key to check the value of another")
someKey := (unintiedKey ?? "uninitedKey had no value").value
Guess what the value of someKey is going to be?
> show someKey
[info] uninitedKey had no value
The key to understand the operations (that make up the sbt.SettingKey API) is to understand what a setting is in sbt - it's a pair of a key and an initialization that gets transformed into a useable setting when a scope gets applied to it.
Related
I'm new to Scala and SBT. I noticed an unfamiliar operator in the build.sbt of an open source project:
:=
Here are a couple examples of how it's used:
lazy val akkaApp = Project(id = "akka-app", base = file("akka-app"))
.settings(description := "Common Akka application stack: metrics, tracing, logging, and more.")
and it's used a few times in this larger code snippet:
lazy val jobServer = Project(id = "job-server", base = file("job-server"))
.settings(commonSettings)
.settings(revolverSettings)
.settings(assembly := null.asInstanceOf[File])
.settings(
description := "Spark as a Service: a RESTful job server for Apache Spark",
libraryDependencies ++= sparkDeps ++ slickDeps ++ cassandraDeps ++ securityDeps ++ coreTestDeps,
test in Test <<= (test in Test).dependsOn(packageBin in Compile in jobServerTestJar)
.dependsOn(clean in Compile in jobServerTestJar)
.dependsOn(buildPython in jobServerPython)
.dependsOn(clean in Compile in jobServerPython),
testOnly in Test <<= (testOnly in Test).dependsOn(packageBin in Compile in jobServerTestJar)
.dependsOn(clean in Compile in jobServerTestJar)
.dependsOn(buildPython in jobServerPython)
.dependsOn(clean in Compile in jobServerPython),
console in Compile <<= Defaults.consoleTask(fullClasspath in Compile, console in Compile),
fullClasspath in Compile <<= (fullClasspath in Compile).map { classpath =>
extraJarPaths ++ classpath
},
fork in Test := true
)
.settings(publishSettings)
.dependsOn(akkaApp, jobServerApi)
.disablePlugins(SbtScalariform)
My best guess is that it means "declare if not already declared".
The := has essentially nothing to do with the ordinary assignment operator =. It's not a built-in scala operator, but rather a family of methods/macros called :=. These methods (or macros) are members of classes such as SettingKey[T] (similarly for TaskKey[T] and InputKey[T]). They consume the right hand side of the key := value expression, and return instances of type Def.Setting[T] (or similarly, Tasks), where T is the type of the value represented by the key. They are usually written in infix notation. Without syntactic sugar, the invocations of these methods/macros would look as follows:
key.:=(value)
The constructed Settings and Tasks are in turn the basic building blocks of the build definition.
The important thing to understand here is that the keys on the left hand side are not some variables in a code block. Instead of merely representing a memory position in an active stack frame of a function call (as a simple variable would do), the keys on the left hand side are rather complex objects which can be inspected and passed around during the build process.
I'm trying to run a code generator, and passing it the filename to write the output:
resourceGenerators in (proj, Compile) += Def.task {
val file = (resourceManaged in (proj, Compile)).value / "swagger.yaml"
(runMain in (proj, Compile)).toTask(s"api.swagger.SwaggerDump $file").value
Seq(file)
}.value
However, this gives me:
build.sbt:172: error: Illegal dynamic reference: file
(runMain in (proj, Compile)).toTask(s"api.swagger.SwaggerDump $file").value
Your code snippet has two problems:
You use { ... }.value instead of { ... }.taskValue. The type of resource generators is Seq[Task[Seq[File]]] and when you do value, you get Seq[File] not Task[Seq[File]]. That causes a legitimate compile error.
The dynamic variable file is used as the argument of toTask, which the current macro implementation prohibits.
Why static?
Sbt forces task implementations to have static dependencies on other tasks. Otherwise, sbt cannot perform task deduplication and cannot provide correct information in the inspect commands. That means that whichever task evaluation you perform inside a task cannot depend on a variable (a value known only at runtime), as your file in toTask does.
To overcome this limitation, there exists dynamic tasks, whose body allows you to return a task. Every "dynamic dependency" has to be defined inside a dynamic task, and then you can depend on the hoisted up dynamic values in the task that you return.
Dynamic solution
The following Scastie is the correct implementation of your task. I copy-paste the code so that folks can have a quick look, but go to that Scastie to check that it successfully compiles and runs.
resourceGenerators in (proj, Compile) += Def.taskDyn {
val file = (resourceManaged in (proj, Compile)).value / "swagger.yaml"
Def.task {
(runMain in (proj, Compile))
.toTask(s"api.swagger.SwaggerDump $file")
.value
Seq(file)
}
}.taskValue
Discussion
If you had fixed the taskValue error, should your task implementation correctly compile?
In my opinion, yes, but I haven't looked at the internal implementation good enough to assert that your task implementation does not hinder task deduplication and dependency extraction. If it does not, the illegal reference check should disappear.
This is a current limitation of sbt that I would like to get rid of, either by improving the whole macro implementation (hoisting up values and making sure that dependency analysis covers more cases) or by just improving the "illegal references checks" to not be over pessimistic. However, this is a hard problem, takes time and it's not likely to happen in the short term.
If this is an issue for you, please file a ticket in sbt/sbt. This is the only way to know the urgency of fixing this issue, if any. For now, the best we can do is to document it.
I'm writing an sbt plugin, and have created a TaskKey that need to get parsed arguments
lazy val getManager = TaskKey[DeployManager]("Deploy manager")
lazy val getCustomConfig = InputKey[String]("Custom config")
...
getCustomConfig := {
spaceDelimited("<arg>").parsed(0)
}
getManager := {
val conf = configResource.evaluated
...
}
but I get this error during compilation:
`parsed` can only be used within an input task macro, such as := or Def.inputTask.
I can't define getManager as InputKey since I later use it's value many times, and for an inputKey the value gets created anew on each evaluation (and I need to use the same instance)
You cannot do what you want in a reasonable way in sbt. (And the type system nicely prevents you from doing that in this case).
Imagine that getManager is a TaskKey that takes parsed arguments (a note aside, the sbt way of naming this would probably be manager, get is implied).
I now decide that, for example, compile depends on getManager. If I type compile in the shell, what arguments should getManager parse?
There is no concept of arguments inside the sbt dependency tree. They are just a shallow (and IMHO somewhat hackish) addition to make for a nicer CLI.
If you want to make getManager configurable, you can add additional settings, getManager depends on and then use set on the command line to change these where necessary.
So in you case:
lazy val configResource = SettingKey[...]("Config resource")
getManager := {
val conf = configResource.value
// ...
}
I am learning to write some more advanced sbt build files, and I came across sbt-proguard's code:
binaryDeps <<= compile in Compile map { _.relations.allBinaryDeps.toSeq },
inputs <<= fullClasspath in Runtime map { _.files },
libraries <<= (binaryDeps, inputs) map { (deps, in) => deps filterNot in.toSet },
outputs <<= artifactPath map { Seq(_) },
I want to know what does <<= mean in this context?
How do I understand the map function at the 3rd line?
The <<= is a method on DefinableSetting (mixed in by TaskKey, InputKey, and SettingKey) which provides a way to initialise a build setting. It's described in the older docs here:
:= assigns a value, overwriting any existing value. <<= uses existing values to initialize a setting.
Essentially, in 0.12 (and current versions, for compatibility) it was a way to define a build setting in terms of some other build setting(s).
As #sjrd points out, in 0.13 a new task setting syntax was introduced allowing you to do this with := instead.
The map in the third line is creating a new settings value by getting only dependency items from binaryDeps which are not already in inputs, i.e. transform these two settings into this new one.
Scopes matters in sbt. And I'm completely OK with it. But there are also delegating rules that allows you build a hierarchical structure of settings. I'd like to use it to bring extra settings to more specific rules.
import sbt._
import Keys._
object TestBuild extends Build {
val sourceExample = settingKey[Seq[String]]("example source for setting dependency")
val targetExample = settingKey[Seq[String]]("example of a dependent setting")
override lazy val settings = super.settings ++ Seq (
sourceExample := Seq("base"),
targetExample := "extended" +: sourceExample.value,
sourceExample in Test += "testing"
)
}
The example gives me unexpected output:
> show compile:sourceExample
[info] List(base)
> show test:sourceExample
[info] List(base, testing)
> show compile:targetExample
[info] List(extended, base)
> show test:targetExample
[info] List(extended, base)
I expect test:targetExample be List(extended, base, testing) not List(extended, base). Once I've get the result I immediately figure out why exactly it works as shown. test:targetExample delegates from *:targetExample the calculated value but not the rule for calculating it in nested scope.
This behavior brings two difficulties for me writing my own plugin. I have extra work to define same rules in every scope as a plugin developer. And I have to memorize scope definitions of internal tasks to use it correctly as user.
How can I overcome this inconvenience? I'd like to introduce settings in call-by-name semantic instead of call-by-value. What tricks may work for it?
P.S. libraryDependencies in Test looks much more concise that using % test.
I should make clear that I perfectly understand that the sbt derives values just as it is described in documentation. It works as the creator intended it to work.
But why should I obey to the rules? I see them completely counter-intuitive. Sbt introduces inheritance semantic that actually works unlike how inheritance used to be defined. When you write
trait A { lazy val x : Int = 5 }
trait B extends A { lazy val y : Int = x * 2}
trait C extends A { override lazy val x : Int = 3 }
you expect (new B with C).y be 6, not 10. Knowing that it would be actually 10 allows you to use this kind of inheritance correctly but leaves your with desire to find more conventional means for implementing inheritance. You may even write your own implementation based on name->value dictionary. And you may proceed further according to the tenth rule of programming.
So I'm searching for a hack that would bring inheritance semantic in accordance with common one. As a start point I may suggest writing command to scan all settings and push them from parents to children explicitly. And than invoke this command automatically each time sbt runs.
But it seems too dirty for me, so I'm curios if there is more graceful way to achieve similar semantic.
The reason for the "incorrect" value is that targetExample depends on sourceExample in Compile scope as in:
targetExample := "extended" +: sourceExample.value
Should it use sourceExample value from Test scope, use in Test to be explicit about your wish as follows:
targetExample := "extended" +: (sourceExample in Test).value
Use inspect to know the dependency chain.
BTW, I strongly advise using build.sbt file for such a simple build definition.
You could have default settings, and reuse it in different configurations as described in Plugins Best Practices - Playing nice with configurations. I believe, this should give you a semantic similar to what you're looking for.
You can define your base settings and reuse it in different configurations.
import sbt._
import Keys._
object TestBuild extends Build {
val sourceExample = settingKey[Seq[String]]("example source for setting dependency")
val targetExample = settingKey[Seq[String]]("example of a dependent setting")
override lazy val settings = super.settings ++
inConfig(Compile)(basePluginSettings) ++
inConfig(Test)(basePluginSettings ++ Seq(
sourceExample += "testing" // we are already "in Test" here
))
lazy val basePluginSettings: Seq[Setting[_]] = Seq (
sourceExample := Seq("base"),
targetExample := "extended" +: sourceExample.value
)
}
PS. Since you're talking about writing your plugin, you may also want to look at the new way of writing sbt plugins, namely AutoPlugin, as the old mechanism is now deprecated.