Someday, I'd like to learn Scala. What I see about the language from people who like it is very encouraging.
Today, though, is not that day. Today, I'd just like to make some changes to my team's build file. Unfortunately, this build file was put together with SBT, and is nearly incomprehensible.
My main issue is that it appears to me that SBT introduces some huge collection of new operators that do things with strings and lists to create some sort of sbt object. For example, in sbt:
"args4j" % "args4j" % "2.0.12"
Apparently is actually defined; however, I can't even tell what type it is at the scala repl, since at the repl I get the sensible error:
scala> val tstcrap = "args4j" % "args4j" % "2.0.12"
<console>:6: error: value % is not a member of java.lang.String
val tstcrap = "args4j" % "args4j" % "2.0.12"
I get this error even after setting up the classpath to include the sbt-launch.jar file and doing import sbt._.
Likewise, I'm dealing with stuff like this:
val jarSources = (descendents(classesOutput ##, "*") ---
assemblyExclude(classesOutput ##))
What's that ## operator, what's that --- doing, and more importantly what is the type of this expression? Are all these new operators documented somewhere, and is there some way to get a scala repl that's using the same language as is used in the sbt build files?
Looking at this sbt file reminds me of trying to decipher perl without ever reading any of the relevant man pages. (Not a recommended activity)
Update: After looking at the links in the answer below, and looking at other questions and answers tagged sbt, I've come across the major piece of scala knowledge that I was missing: scala allows one to define implicit conversions that will be invoked before methods are resolved. In this case, sbt defines inside the ManagedProject trait, an implicit conversion from String to the private class sbt.GroupID, so that
"a" % "b"
Is really something like
(new GroupID("a")) % "b"
I imagine the resolution order and other rules around implicit conversions must get very complicated; it almost reminds me of the nightmares you can introduce in C++ with operator overloading when done through non-member functions.
Since an SBT build file is a full-fledged Scala source file and relies on some libraries provided by SBT itself, it's difficult to cover SBT well without relying on some familiarity with Scala. I'm not aware of such a guide.
For the specific questions you raise, I think these wiki pages will help:
% operator for strings: http://code.google.com/p/simple-build-tool/wiki/LibraryManagement
## and --- operators: http://code.google.com/p/simple-build-tool/wiki/Paths
If you want to get a Scala REPL running with the SBT libraries available, try this:
$ sbt console-project
Some other useful commands are listed at http://code.google.com/p/simple-build-tool/wiki/RunningSbt .
Update 2016 (5 years later).
This is not a complete guide, but the article "Sbt heiroglyphs and multi-projects explained" from Divan Visagie can help starting to use sbt.
Plus, the sbt documentation is quite complete nowadays, and covers multiple projects in a single build.
The '---' operator is described in the PathFinder (since the 0.2 version).
Related
The error message
`value` can only be used within a task or setting macro, such as :=, +=, ++=, Def.task, or Def.setting.
val x = version.value
^
clearly indicates how to fix the problem, for example, using :=
val x = settingKey[String]("")
x := version.value
The explanation in sbt uses macros heavily states
The value method itself is in fact a macro, one that if you invoke it
outside of the context of another macro, will result in a compile time
error, the exact error message being...
And you can see why, since sbt settings are entirely declarative, you
can’t access the value of a task from the key, it doesn’t make sense
to do that.
however I am confused what is meant by declarative nature of sbt being the reason. For example, intuitively I would think the following vanilla Scala snippet is semantically similar to sbt's
def version: String = ???
lazy val x = s"Hello $version" // ok
trait Foo {
def version: String
val x = version // ok
}
As this is legal, clearly the Scala snippet is not semantically equivalent to the sbt one. I was wondering if someone could elaborate on why value cannot be used outside macros? Is the reason purely syntactic related to macro syntax or am I missing something fundamental about sbt's nature?
As another sentence there says
Defining sbt’s task engine is done by giving sbt a series of settings, each setting declaring a task implementation. sbt then executes those settings in order. Tasks can be declared multiple times by multiple settings, the last one to execute wins.
So at the moment when the line
val x = version.value
would be executed (if it were allowed!), that whole program is still being set up and SBT doesn't know the final definition of version.
In what sense is the program "still being set up"?
SBT's order of actions is, basically (maybe missing something):
All your Scala build code is run.
It contains some settings and tasks definitions, SBT collects those when it encounters (along with ones from core, plugins, etc.)
They are topologically sorted into the task graph, deduplicated ("the last one to execute wins"), etc.
Settings are evaluated.
Now you are allowed to actually run tasks (e.g. from SBT console).
version.value is only available after step 4, but val x = version.value runs on step 1.
Would not lazy evaluation take care of that?
Well, when you write val x = ... there is no lazy evaluation. But lazy val x = ... runs on step 1 too.
We have problems when using Scalac -Xfatal-warnings in the following cases:
Implicit vals used by macros internally
Internal vals that macros auto-generate
In both cases, we see Scalac failing to compile because it detects some values are not used, while we know they are (simply, when we remove them, the code doesn't compile anymore)
Although the two might be symptoms of the same problem in Scalac, they boil down to the same issue for us: we need to disable the -Ywarn-unused in Scala 2.11.12
Is there a way to exclude specific class files so they won't be affected by the compiler flag?
As far as I know there is no way of disabling scalac flag for just one file (if you compile your whole project at once by e.g sbt). You can extract class into separate module with different compile flags.
In case of implicit vals used internally in macros, personally I use -Ywarn-macros:after flag, which make these implicits used in macro count as used. (Talking about Scala 2.12.4).
I would like to develop a tool that post-processes a scala program once all the heavy lifting has been completed by the Scala compiler. From what I understand the different phases of the Scala compiler incrementally simplify the program in terms of its syntactic sugars and advanced features like lambdas, closures, pattern-matching etc. However, I notice that what comes out of the so-called cleanup phase - which is the last phase before code-generation - looks like scala but it is not really scala.
Does anyone know personally or can point me to a resource that can help me understand the language that comes out of the cleanup phase ?
To give you an example, in the output of the cleanup phase I see things like:
case <synthetic> val x1: Foo$Bar = l;
case9(){
if (...some condition...)
matchEnd8(scala.Predef.Set().empty())
else
case10()
};
My hypothesis is that this is the result of translating pattern matching but it does not look like valid scala syntax as far as I understand (I am not an experienced Scala developer at all!).
I guess it all comes down to this: is it possible to convert the output of the cleanup phase to valid - compilable - scala code in general ?
In general, at any stage in the scalac compiler (even right after parsing), the internal representation used by the compiler is not valid Scala code anymore. That is essentially because of the existence of labels and gotos, which you discovered.
A structure of the form
labelName(...params){
...
}
is a label definition, and a call of the form
labelName(...args)
is a jump to that label, assigning the ...args to the ...params.
Labels and gotos are used by scalac (and dotc, but with a different representation) to represent while and do..while loops (immediately after parsing), the translation of matches and the tail-recursive-optimized functions.
In general, there is no way to go back from the internal representation to valid Scala code, especially so far in the pipeline as after cleanup.
I see this <<= symbol in lot of SBT code, but I don't what it does.
I tried googling for this symbol as well but I did not get any answers.
Can you please point me to some documentation or an example which clearly explains what does this symbol mean and what does it do?
Further to pfn's comment, this is described in the 0.12 docs under More Kinds of Settings. I guess it was dropped from the 0.13 docs because the same behaviour can now be defined in terms of :=.
Oh, the deep explanation is quite complicated.
Basically, the signature is:
def <<= (app: Initialize[Task[S]]): Setting[Task[S]] = macro std.TaskMacro.itaskAssignPosition[S]
So it involves this macro:
/* Implementations of <<= macro variations for tasks and settings. These just get the source position of the call site.*/
def itaskAssignPosition[T: c.WeakTypeTag](c: Context)(app: c.Expr[Initialize[Task[T]]]): c.Expr[Setting[Task[T]]] =
settingAssignPosition(c)(app)
I already used this kind of operator when dealing with AspectJ compilation:
products in Compile <<= products in Aspectj
Basically, it means: base the code source on the AspectJ source files (generated with a plugin), not the classical ones.
I interpret it as a kind of "replaceAll/erase":
Replace bunch of files to compile by the files involving AspectJ annotations.
In my quest to generate new code in a Scala compiler plugin, I have now created working classes. The next logical step is to put those classes in a new, non-existing package. In Java, a package is basically a directory name, but in Scala a package seems much more complicated. So far I haven't found/recognized an example where a compiler plugin creates a new package.
At my current level of understanding, I would think that I would need to create first a package symbol with:
parentPackage.newPackage(...)
// ...
and than later create a Tree for the package with PackageDef. But PackageDef doesn't take the symbol as parameter, as one would expect, and searching for:
Scala newPackage PackageDef
returned nothing useful. So it seems that I don't need to do those two steps together. Possibly one is done for my by the compiler, but I don't know which one. So far, what I have looks like this:
val newPkg = parentPackage.newPackage(NoPosition, newTermName(name))
newPkg.moduleClass.setInfo(new PackageClassInfoType(new Scope,
newPkg.moduleClass))
newPkg.setInfo(newPkg.moduleClass.tpe)
parentPackage.info.decls.enter(newPkg)
// ...
val newPkgTree = PackageDef(Ident(newPkg.name), List(ClassDef(...)))
I think my answer to your other question should answer this one as well:
How to add a new Class in a Scala Compiler Plugin?