I occasionally play with Scala forks and sometimes need to debug these forks on SBT projects. In general, scalaHome works great, but there are a few things that I'd like to find better ways to achieve.
1) Is it possible to have SBT pick up custom scalac class files produced by the ant quick build rather than jar files emitted by the ant pack build? The latter implies 5-10 seconds of additional delay per build, so it'd be great to avoid it.
2) Even in big projects, problems exhibited by scalac usually manifest themselves when compiling single files. Is there a way to tell sbt to neglect its change tracking heuristics and recompile just a single file? What I would particularly like to prevent is recompilation of the whole world when I recompile scalaHome or change scalac flags.
3) Would it be possible to have sbt hot reload scalac classes coming from scalaHome, when scalaHome gets recompiled? Currently I have to shutdown and restart sbt to apply the changes.
1) No, this would make sbt depend on the details of the Scala build. If Scala were built with sbt, you might be able to depend on Scala as a source dependency or at least this could probably be supported without too many changes.
2) No, see https://github.com/sbt/sbt/issues/604
3) sbt 0.13 should check the last modified times of the jars coming from scalaHome and use a new class loader. It is a bug if it does not.
Related
Currently all our project builds with Maven on Windows. We were not successful with making incrementally compiled code to work in run time (50% of the cases it was failing with some kind of error), so to benefit from warm compiler and (maybe?) properly working incremental compilation we think about moving to SBT. However currently I have only one sprint to work on it, and I'm afraid to put all the eggs in a single basket and try to migrate whole project in a sprint. I need to find a way to make this change gradual, so I could advance one module at a time. So here are the main questions:
How can I include SBT modules in Maven build (or maybe vise versa, having my "parent" in SBT, yet part of the modules still building with Maven)?
How can we still benefit from IDE support (currently IntelliJ 13), like updated indices on changes in pom / Build.scala, task & goals invocation and so on?
Any advises on subject are highly appreciated.
Eventually we did the switch and don't regret it. Writing SBT tasks is easier, because it's plain Scala. Incremental compilation works now (used to fail in Maven with java.lang.InternalError: Enclosing method not found when deployed to JBoss) and build time is significantly faster. Unfortunately we did not find a way to make a gradual switch, so we had to take the risk. Incremental compiled jars still didn't work, yet Typesafe are about to fix this issue in 2.11.6
When I edit a scala file in my Play 2 app, sometimes only a few files are recompiled, but often the entire codebase must be recompiled:
[info] Compiling 1 Scala source to /home/michael/code/superglot/target/scala-2.10/classes...
[success] Compiled in 1s
versus
[info] Compiling 2 Scala sources to /home/michael/code/superglot/target/scala-2.10/classes...
[info] Compiling 52 Scala sources and 1 Java source to /home/michael/code/superglot/target/scala-2.10/classes...
[success] Compiled in 13s
However I see no discernible pattern for when a full recompile is necessary. If I add somewhitespace to a model or controller class it may only compile that file, but doing the same to a comparable file will trigger a recompile.
I would love to have as many of my reloads be closer to 1s, because currently I'm waiting for a full recompile more often than not. I'd gladly refactor my code to make the areas I'm working on faster to reload, but I don't know what I could even do to achieve that. Are frequent recompiles normal for a typical Play 2 app, or is there something anomalous about mine?
In general, if you change the "source API" of a file, dependencies of that file are recompiled. The source API consists of the signatures of non-private methods and types. So, if you have a file that everything depends on, changes to signatures in that file may cause a lot of recompilation. Also, when an ancestor's API changes, all descendents must be recompiled.
You can get some additional information from last compile, such as what triggered other files to be recompiled. (In a multi-module build, last <project-name>/compile) You can
If adding insignificant whitespace causes other files to be recompiled, it is always a bug, often in scalac itself. An example of such a bug is SI-7361 (not that it is useful to anyone besides compiler developers!) that was worked around in sbt here. For these to be fixed, we need a reproducible test case. (Given the effort often involved in that, you might wait for 0.12.4 or 0.13.0 to see if those fix your problem.)
0.13.0 has some improvements that will hopefully reduce what gets invalidated when APIs change.
I think you are going for the slightly wrong approach here. As there is no way to successfully determine which changes will cause the full re-compilation, that path leads nowhere.
For learning purposes perhaps it is helpful to be aware of the compiler's internals to a better extent, but from a productivity perspective there is a much better choice available, and that is jRebel, who have been giving out free Scala licenses for years.
JRebel
From here, you can get a free Scala license within minutes.
Then go ahead and add this to the sbt config. It has to go straight into the sbt file.
Linux/Mac
java -noverify -javaagent:/opt/zt/jrebel/jrebel.jar \
-Xmx1024M -Xss2M -XX:MaxPermSize=512m -XX:+CMSClassUnloadingEnabled -jar \
`dirname $0`/sbt-launch-0.12.jar "$#"
Windows
set SCRIPT_DIR=%~dp0
java -noverify -javaagent:c:/opt/zt/jrebel/jrebel.jar \
-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256m -Xmx1024M -Xss2M \
-jar "%SCRIPT_DIR%\sbt-launch-0.12.jar" %*
At this stage, if you've done everything right, when you re-start the SBT console you will see that the jRebel Agent has been initiated and you will benefit from the real-time redeployment of your changes. When you Save in your IDE, JRebel will reload only the changes you've made.
I've come pretty well along with sbt, the Scala Build Tool. If you only have small problems in the code, it's easy.
Now, after a major feature add, much of my code is broken and I sbt seems to be confused as to how the dependencies are. I could help it, compiling the fundamental modules first, but it does not seem to let me.
It's help system is... notorious.
> help compile
Compiles sources.
Yeah, well. I guessed that.
What I wanted to hear was: how do I compile only - say - src/module/A.scala.
This might not even be possible (hello again, make, never abandoned you!). At least I cannot find any reference on the Internet to applying sbt compile just to a single file.
I'm using sbt from the command line prompt, not an IDE.
UPDATE:
It was my fault. :/ Had split a source file into multiple, but forgot to copy a package clause to each of the new ones. Oooops.
Will keep this open for a while, since compiling just a single file (i.e. something like sbt compile filename) would imho not be a bad thing.
You could define a Multi-Project Build where the files you want to compile separately are encapsulated in a project. According to the docs, the following is then possible:
At the sbt interactive prompt, type projects to list your projects and project to select a current project. When you run a task like compile, it runs on the current project. So you don't necessarily have to compile the root project, you could compile only a subproject.
I just wanted to mention here that I came across sbt-compile-quick-plugin (https://github.com/etsy/sbt-compile-quick-plugin). It does what it says on the tin, just add addSbtPlugin("com.etsy" % "sbt-compile-quick-plugin" % "1.3.0") to your project/plugins.sbt, then you can just start up sbt and run compileQuick /path/to/your/file
(See https://stackoverflow.com/a/46849619/1358677)
I am trying to generate some boilerplate with SBT (tool which is totally new to me). I am using shapeless sbt files as my main reference for the task. I have seen that this project uses code generation from scratch, but my case is slightly different, since I would like to generate some classes from another ones. I pretend to use the new Scala 2.10.0-M4 reflection capabilities for doing so. What basic configuration is needed to have reflection available from a SBT build?
By now, the sbt is unable to find the scala.reflect.runtime.universe package, and I do not know if the problem comes either from the new Scala jar division or from a bad configuration. Besides, my sbt about says:
[info] This is sbt 0.13.0-20120530-052139
[info] The current project is {file:/home/jlg/sandbox/abc/}abc
[info] The current project is built against Scala 2.10.0-SNAPSHOT
[info]
[info] sbt, sbt plugins, and build definitions are using Scala 2.9.2
By the way, does anybody know other projects using SBT to generate source code?
Current SBT releases are based on Scala 2.9, and source code generation runs together with SBT with the same libraries. There are basically two choices:
be extremely bleeding-edge: get an SBT release running on Scala 2.10 (not even the 0.13 branch does), or waiting for it. The biggest problem is not just that you'd have to recompile SBT yourself, it's recompiling every single SBT plugin you'll need for Scala 2.10. In the long-term, this is maybe the best strategy to do what you ask, but it might be a lot of effort for now. However, beware that you cannot use reflection on your compiled code without evil tricks, since code generation is supposed to happen before compilation. If you need to do that, consider instead generating code at compile-time within the program using macros. This excludes SBT and is much more standard, but I'm not sure if you can generate complete classes in this release (this is I think planned for the future).
go with the old: stick with Scala 2.9 and use scalap's capabilities (ScalaSigParser) for compile-time reflection. The problem is that the API is different (not sure how deeply) and not really supported for public use, although various people have been using it for ages. For a project I'm running, a colleague implemented approach and I integrated it within SBT for my project (https://github.com/ps-mr/LinqOnSteroids/); on top of that, I use Scalate to write the templates to use for code generation, which is quite powerful.
See in particular build.sbt, which invokes
project/Generator.scala and project/src/main/scala/ivm/generation/ScalaSigHelpers.scala (some non-fully-generic wrappers for ScalaSigParser). Scalate Templates for generated code are in
src/main/resources, the most relevant here is src/main/resources/WrappedClassInlined.ssp.
Even more stuff is involved, I fear you'll pratically need a checkout and playing with it to see what it does exactly—but feel free to ask questions.
Please note that the code is protected by a BSD license, so you need to keep the original copyright if you copy the code.
Note: all the links (except the license) are to the current HEAD for stability, so that they won't disappear so easily even if the files are moved/removed in future versions.
If you're using 2.10.0-SNAPSHOT, then you should go for scala.reflect.runtime.universe. Take a look at http://dcsobral.blogspot.ch/2012/07/json-serialization-with-reflection-in.html for more information.
I'm currently using IDEA's build mechanism with fsc for developing with Scala. It's still a bit slow and having to (re) start the compilation server is a pain. Many people here are suggesting SBT as a build tool together with IDEA.
What do you consider the pros and cons of each approach?
I tried both and in the end I prefer straight sbt for compiling.
Cons? I really miss being able to click through compile errors and fix the code directly, but... compiling in sbt is just much faster.
The nightly builds of the Idea Scala plugin can vary in quality/performance, but it's been getting better and better lately. The Scala plugin can now flag a number of errors that before I would have had to run compile to catch. (For example, I'm running nightly build 0.4.693 and the new method inspections have already been dead helpful.)
My advice for life with sbt on the command line: start sbt up and leave it running interactively as long as possible to take advantage of everything being loaded and JIT-ed.
sbt left running will fall over eventually but by giving it more memory in your sbt wrapper you can make that happen only rarely.
Here's the sbt launch wrapper that works for me.
java -Xms512M -Xmx1500M -XX:MaxPermSize=512m -jar `dirname $0`/sbt-launch.jar "$#"
My biggest issue with sbt 0.7 is that it frequently goes back and recompiles great swathes of files that seem only tangential to the code I was actually changing. (Even so, still faster than Idea and fsc!)
Good news: sbt 0.9 has some great incremental compile improvements. Unfortunately, the migration path from 0.7 to 0.9 is still in its early days. Mark Harrah's presentation at NEScala is online at http://www.nescala.org/2011/ if you're interested.
Useful plugins
http://github.com/mpeltonen/sbt-idea - easily create and keep your Idea project in sync with your sbt project
http://github.com/orfjackal/idea-sbt-plugin - lets you create run profiles for sbt from within Idea (I found this slower than running sbt at the command line last October - but I see orfjackal is still developing so I should give it another shot)