Why value method cannot be used outside macros? - scala

The error message
`value` can only be used within a task or setting macro, such as :=, +=, ++=, Def.task, or Def.setting.
val x = version.value
^
clearly indicates how to fix the problem, for example, using :=
val x = settingKey[String]("")
x := version.value
The explanation in sbt uses macros heavily states
The value method itself is in fact a macro, one that if you invoke it
outside of the context of another macro, will result in a compile time
error, the exact error message being...
And you can see why, since sbt settings are entirely declarative, you
can’t access the value of a task from the key, it doesn’t make sense
to do that.
however I am confused what is meant by declarative nature of sbt being the reason. For example, intuitively I would think the following vanilla Scala snippet is semantically similar to sbt's
def version: String = ???
lazy val x = s"Hello $version" // ok
trait Foo {
def version: String
val x = version // ok
}
As this is legal, clearly the Scala snippet is not semantically equivalent to the sbt one. I was wondering if someone could elaborate on why value cannot be used outside macros? Is the reason purely syntactic related to macro syntax or am I missing something fundamental about sbt's nature?

As another sentence there says
Defining sbt’s task engine is done by giving sbt a series of settings, each setting declaring a task implementation. sbt then executes those settings in order. Tasks can be declared multiple times by multiple settings, the last one to execute wins.
So at the moment when the line
val x = version.value
would be executed (if it were allowed!), that whole program is still being set up and SBT doesn't know the final definition of version.
In what sense is the program "still being set up"?
SBT's order of actions is, basically (maybe missing something):
All your Scala build code is run.
It contains some settings and tasks definitions, SBT collects those when it encounters (along with ones from core, plugins, etc.)
They are topologically sorted into the task graph, deduplicated ("the last one to execute wins"), etc.
Settings are evaluated.
Now you are allowed to actually run tasks (e.g. from SBT console).
version.value is only available after step 4, but val x = version.value runs on step 1.
Would not lazy evaluation take care of that?
Well, when you write val x = ... there is no lazy evaluation. But lazy val x = ... runs on step 1 too.

Related

sbt illegal dynamic reference in runMain

I'm trying to run a code generator, and passing it the filename to write the output:
resourceGenerators in (proj, Compile) += Def.task {
val file = (resourceManaged in (proj, Compile)).value / "swagger.yaml"
(runMain in (proj, Compile)).toTask(s"api.swagger.SwaggerDump $file").value
Seq(file)
}.value
However, this gives me:
build.sbt:172: error: Illegal dynamic reference: file
(runMain in (proj, Compile)).toTask(s"api.swagger.SwaggerDump $file").value
Your code snippet has two problems:
You use { ... }.value instead of { ... }.taskValue. The type of resource generators is Seq[Task[Seq[File]]] and when you do value, you get Seq[File] not Task[Seq[File]]. That causes a legitimate compile error.
The dynamic variable file is used as the argument of toTask, which the current macro implementation prohibits.
Why static?
Sbt forces task implementations to have static dependencies on other tasks. Otherwise, sbt cannot perform task deduplication and cannot provide correct information in the inspect commands. That means that whichever task evaluation you perform inside a task cannot depend on a variable (a value known only at runtime), as your file in toTask does.
To overcome this limitation, there exists dynamic tasks, whose body allows you to return a task. Every "dynamic dependency" has to be defined inside a dynamic task, and then you can depend on the hoisted up dynamic values in the task that you return.
Dynamic solution
The following Scastie is the correct implementation of your task. I copy-paste the code so that folks can have a quick look, but go to that Scastie to check that it successfully compiles and runs.
resourceGenerators in (proj, Compile) += Def.taskDyn {
val file = (resourceManaged in (proj, Compile)).value / "swagger.yaml"
Def.task {
(runMain in (proj, Compile))
.toTask(s"api.swagger.SwaggerDump $file")
.value
Seq(file)
}
}.taskValue
Discussion
If you had fixed the taskValue error, should your task implementation correctly compile?
In my opinion, yes, but I haven't looked at the internal implementation good enough to assert that your task implementation does not hinder task deduplication and dependency extraction. If it does not, the illegal reference check should disappear.
This is a current limitation of sbt that I would like to get rid of, either by improving the whole macro implementation (hoisting up values and making sure that dependency analysis covers more cases) or by just improving the "illegal references checks" to not be over pessimistic. However, this is a hard problem, takes time and it's not likely to happen in the short term.
If this is an issue for you, please file a ticket in sbt/sbt. This is the only way to know the urgency of fixing this issue, if any. For now, the best we can do is to document it.

Using input args inside a TaskKey

I'm writing an sbt plugin, and have created a TaskKey that need to get parsed arguments
lazy val getManager = TaskKey[DeployManager]("Deploy manager")
lazy val getCustomConfig = InputKey[String]("Custom config")
...
getCustomConfig := {
spaceDelimited("<arg>").parsed(0)
}
getManager := {
val conf = configResource.evaluated
...
}
but I get this error during compilation:
`parsed` can only be used within an input task macro, such as := or Def.inputTask.
I can't define getManager as InputKey since I later use it's value many times, and for an inputKey the value gets created anew on each evaluation (and I need to use the same instance)
You cannot do what you want in a reasonable way in sbt. (And the type system nicely prevents you from doing that in this case).
Imagine that getManager is a TaskKey that takes parsed arguments (a note aside, the sbt way of naming this would probably be manager, get is implied).
I now decide that, for example, compile depends on getManager. If I type compile in the shell, what arguments should getManager parse?
There is no concept of arguments inside the sbt dependency tree. They are just a shallow (and IMHO somewhat hackish) addition to make for a nicer CLI.
If you want to make getManager configurable, you can add additional settings, getManager depends on and then use set on the command line to change these where necessary.
So in you case:
lazy val configResource = SettingKey[...]("Config resource")
getManager := {
val conf = configResource.value
// ...
}

How can I get Scala ToolBox to see REPL definitions?

Back when reflection was still incipient, on the days of Scala 2.10.0 milestones, I asked a question about how could I use it to see the trees of code snippets from REPL. The excellent answer went further than I asked, and showed how they can be used to parse and evaluate trees as well, so I went ahead and tried to use that on a little project I had going on today.
Unfortunately, code parsed and evaluated that way doesn't seem to see any REPL definition:
scala> val x = 1
x: Int = 1
scala> import scala.tools.reflect.ToolBox
import scala.tools.reflect.ToolBox
scala> val tb = scala.reflect.runtime.universe.runtimeMirror(
getClass.getClassLoader).mkToolBox()
tb: scala.tools.reflect.ToolBox[reflect.runtime.universe.type] = ...
scala> tb.eval(tb.parse("x"))
scala.tools.reflect.ToolBoxError: reflective compilation has failed:
not found: value x
Is there a way to get it to recognize definitions made on REPL?
Recently I dug into repl, when trying to make it support type macros, so I'm well equipped to explain why it doesn't work. Getting it to work would be the next step :)
I know that you know that every snippet entered into repl gets wrapped into some boilerplate before being compiled. Therefore that x ends up being a field in a nested-nested-nested object in a package with a weird name.
Apparently, repl keeps track of all defined symbols and then injects the necessary imports along with the boilerplate it generates. Therefore subsequent lines can see that x unqualified. To the contrast, toolboxes simply reuse repl's classloader, but don't do anything about the imports, hence the failure.
A workaround would be to somehow get to an object representing a repl, ask it about defined symbols and then generate corresponding imports into the code that you feed to a toolbox. If you file a ticket, I'll try to code up a workaround after the 2.10.1 code freeze madness ends (supposedly, end of this week).

The difference between scala script and application

What is the difference between a scala script and scala application? Please provide an example
The book I am reading says that a script must always end in a result expression whereas the application ends in a definition. Unfortunately no clear example is shown.
Please help clarify this for me
I think that what the author means is that a regular scala file needs to define a class or an object in order to work/be useful, you can't use top-level expressions (because the entry-points to a compiled file are pre-defined). For example:
println("foo")
object Bar {
// Some code
}
The println statement is invalid in the top-level of a .scala file, because the only logical interpretation would be to run it at compile time, which doesn't really make sense.
Scala scripts in contrast can contain expressions on the top-level, because those are executed when the script is run, which makes sense again. If a Scala script file only contains definitions on the other hand, it would be useless as well, because the script wouldn't know what to do with the definitions. If you'd use the definitions in some way, however, that'd be okay again, e.g.:
object Foo {
def bar = "test"
}
println(Foo.bar)
The latter is valid as a scala script, because the last statement is an expression using the previous definition, but not a definition itself.
Comparison
Features of scripts:
Like applications, scripts get compiled before running. Actually, the compiler translates scripts to applications before compiling, as shown below.
No need to run the compiler yourself - scala does it for you when you run your script.
Feeling is very similar to script languages like bash, python, or ruby - you directly see the results of your edits, and get a very quick debug cycle.
You don't need to provide a main method, as the compiler will add one for you.
Scala scripts tend to be useful for smaller tasks that can be implemented in a single file.
Scala applications on the other hand, are much better suited when your projects start to grow more complex. They allow to split tasks into different files and namespaces, which is important for maintaining clarity.
Example
If you write the following script:
#!/usr/bin/env scala
println("foo")
Scala 2.11.1 compiler will pretend (source on github) you had written:
object Main {
def main(args: Array[String]): Unit =
new AnyRef {
println("foo")
}
}
Well, I always thought this is a Scala script:
$ cat script
#!/usr/bin/scala
!#
println("Hello, World!")
Running with simple:
$ ./script
An application on the other hand has to be compiled to .class and executed explicitly using java runtime.

Is there somewhere a guide to SBT for non-Scala programmers?

Someday, I'd like to learn Scala. What I see about the language from people who like it is very encouraging.
Today, though, is not that day. Today, I'd just like to make some changes to my team's build file. Unfortunately, this build file was put together with SBT, and is nearly incomprehensible.
My main issue is that it appears to me that SBT introduces some huge collection of new operators that do things with strings and lists to create some sort of sbt object. For example, in sbt:
"args4j" % "args4j" % "2.0.12"
Apparently is actually defined; however, I can't even tell what type it is at the scala repl, since at the repl I get the sensible error:
scala> val tstcrap = "args4j" % "args4j" % "2.0.12"
<console>:6: error: value % is not a member of java.lang.String
val tstcrap = "args4j" % "args4j" % "2.0.12"
I get this error even after setting up the classpath to include the sbt-launch.jar file and doing import sbt._.
Likewise, I'm dealing with stuff like this:
val jarSources = (descendents(classesOutput ##, "*") ---
assemblyExclude(classesOutput ##))
What's that ## operator, what's that --- doing, and more importantly what is the type of this expression? Are all these new operators documented somewhere, and is there some way to get a scala repl that's using the same language as is used in the sbt build files?
Looking at this sbt file reminds me of trying to decipher perl without ever reading any of the relevant man pages. (Not a recommended activity)
Update: After looking at the links in the answer below, and looking at other questions and answers tagged sbt, I've come across the major piece of scala knowledge that I was missing: scala allows one to define implicit conversions that will be invoked before methods are resolved. In this case, sbt defines inside the ManagedProject trait, an implicit conversion from String to the private class sbt.GroupID, so that
"a" % "b"
Is really something like
(new GroupID("a")) % "b"
I imagine the resolution order and other rules around implicit conversions must get very complicated; it almost reminds me of the nightmares you can introduce in C++ with operator overloading when done through non-member functions.
Since an SBT build file is a full-fledged Scala source file and relies on some libraries provided by SBT itself, it's difficult to cover SBT well without relying on some familiarity with Scala. I'm not aware of such a guide.
For the specific questions you raise, I think these wiki pages will help:
% operator for strings: http://code.google.com/p/simple-build-tool/wiki/LibraryManagement
## and --- operators: http://code.google.com/p/simple-build-tool/wiki/Paths
If you want to get a Scala REPL running with the SBT libraries available, try this:
$ sbt console-project
Some other useful commands are listed at http://code.google.com/p/simple-build-tool/wiki/RunningSbt .
Update 2016 (5 years later).
This is not a complete guide, but the article "Sbt heiroglyphs and multi-projects explained" from Divan Visagie can help starting to use sbt.
Plus, the sbt documentation is quite complete nowadays, and covers multiple projects in a single build.
The '---' operator is described in the PathFinder (since the 0.2 version).