When I edit a scala file in my Play 2 app, sometimes only a few files are recompiled, but often the entire codebase must be recompiled:
[info] Compiling 1 Scala source to /home/michael/code/superglot/target/scala-2.10/classes...
[success] Compiled in 1s
versus
[info] Compiling 2 Scala sources to /home/michael/code/superglot/target/scala-2.10/classes...
[info] Compiling 52 Scala sources and 1 Java source to /home/michael/code/superglot/target/scala-2.10/classes...
[success] Compiled in 13s
However I see no discernible pattern for when a full recompile is necessary. If I add somewhitespace to a model or controller class it may only compile that file, but doing the same to a comparable file will trigger a recompile.
I would love to have as many of my reloads be closer to 1s, because currently I'm waiting for a full recompile more often than not. I'd gladly refactor my code to make the areas I'm working on faster to reload, but I don't know what I could even do to achieve that. Are frequent recompiles normal for a typical Play 2 app, or is there something anomalous about mine?
In general, if you change the "source API" of a file, dependencies of that file are recompiled. The source API consists of the signatures of non-private methods and types. So, if you have a file that everything depends on, changes to signatures in that file may cause a lot of recompilation. Also, when an ancestor's API changes, all descendents must be recompiled.
You can get some additional information from last compile, such as what triggered other files to be recompiled. (In a multi-module build, last <project-name>/compile) You can
If adding insignificant whitespace causes other files to be recompiled, it is always a bug, often in scalac itself. An example of such a bug is SI-7361 (not that it is useful to anyone besides compiler developers!) that was worked around in sbt here. For these to be fixed, we need a reproducible test case. (Given the effort often involved in that, you might wait for 0.12.4 or 0.13.0 to see if those fix your problem.)
0.13.0 has some improvements that will hopefully reduce what gets invalidated when APIs change.
I think you are going for the slightly wrong approach here. As there is no way to successfully determine which changes will cause the full re-compilation, that path leads nowhere.
For learning purposes perhaps it is helpful to be aware of the compiler's internals to a better extent, but from a productivity perspective there is a much better choice available, and that is jRebel, who have been giving out free Scala licenses for years.
JRebel
From here, you can get a free Scala license within minutes.
Then go ahead and add this to the sbt config. It has to go straight into the sbt file.
Linux/Mac
java -noverify -javaagent:/opt/zt/jrebel/jrebel.jar \
-Xmx1024M -Xss2M -XX:MaxPermSize=512m -XX:+CMSClassUnloadingEnabled -jar \
`dirname $0`/sbt-launch-0.12.jar "$#"
Windows
set SCRIPT_DIR=%~dp0
java -noverify -javaagent:c:/opt/zt/jrebel/jrebel.jar \
-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256m -Xmx1024M -Xss2M \
-jar "%SCRIPT_DIR%\sbt-launch-0.12.jar" %*
At this stage, if you've done everything right, when you re-start the SBT console you will see that the jRebel Agent has been initiated and you will benefit from the real-time redeployment of your changes. When you Save in your IDE, JRebel will reload only the changes you've made.
Related
Currently all our project builds with Maven on Windows. We were not successful with making incrementally compiled code to work in run time (50% of the cases it was failing with some kind of error), so to benefit from warm compiler and (maybe?) properly working incremental compilation we think about moving to SBT. However currently I have only one sprint to work on it, and I'm afraid to put all the eggs in a single basket and try to migrate whole project in a sprint. I need to find a way to make this change gradual, so I could advance one module at a time. So here are the main questions:
How can I include SBT modules in Maven build (or maybe vise versa, having my "parent" in SBT, yet part of the modules still building with Maven)?
How can we still benefit from IDE support (currently IntelliJ 13), like updated indices on changes in pom / Build.scala, task & goals invocation and so on?
Any advises on subject are highly appreciated.
Eventually we did the switch and don't regret it. Writing SBT tasks is easier, because it's plain Scala. Incremental compilation works now (used to fail in Maven with java.lang.InternalError: Enclosing method not found when deployed to JBoss) and build time is significantly faster. Unfortunately we did not find a way to make a gradual switch, so we had to take the risk. Incremental compiled jars still didn't work, yet Typesafe are about to fix this issue in 2.11.6
I'm noticing very slow build times for my Play 2.3 project on our CircleCI server relative to my local build machine. One reason for this is that on every build, SBT prints out the following:
[info] 'compiler-interface' not yet compiled for Scala 2.11.4. Compiling...
then proceeds to spend about a minute compiling the compiler. On my laptop, this happened exactly one time, then never again. My assumption is that on my laptop, the compiler gets cached somewhere so that it doesn't have to be rebuilt again the next time, whereas on the CI server, that cache directory gets blown out.
CircleCI allows you to specify cache directories that won't get destroyed between builds (or more accurately, get destroyed, then rebuilt) so a simple solution for me would be to tell CircleCI to save that directory.
Problem is, I don't know where it is. I'm already saving ~/.sbt and ~/.ivy2 and it hasn't helped. Is the compiler built to a different location that I can cache? Any other ideas on how to make this problem go away short of downgrading my Scala compiler?
I looked on my setup, and it seems that the compiler-interface should indeed be in ~/.sbt or ~/.ivy2.
Your problem is probably coming from how you save these directories. Are you sure you save them for the correct user? You have to save them for the use that launch the sbt command.
#joe: you can do sbt compile:test instead of sbt compile to also get the test dependencies in the cache (as well as ensure the compile interface gets included in the cache).
I occasionally play with Scala forks and sometimes need to debug these forks on SBT projects. In general, scalaHome works great, but there are a few things that I'd like to find better ways to achieve.
1) Is it possible to have SBT pick up custom scalac class files produced by the ant quick build rather than jar files emitted by the ant pack build? The latter implies 5-10 seconds of additional delay per build, so it'd be great to avoid it.
2) Even in big projects, problems exhibited by scalac usually manifest themselves when compiling single files. Is there a way to tell sbt to neglect its change tracking heuristics and recompile just a single file? What I would particularly like to prevent is recompilation of the whole world when I recompile scalaHome or change scalac flags.
3) Would it be possible to have sbt hot reload scalac classes coming from scalaHome, when scalaHome gets recompiled? Currently I have to shutdown and restart sbt to apply the changes.
1) No, this would make sbt depend on the details of the Scala build. If Scala were built with sbt, you might be able to depend on Scala as a source dependency or at least this could probably be supported without too many changes.
2) No, see https://github.com/sbt/sbt/issues/604
3) sbt 0.13 should check the last modified times of the jars coming from scalaHome and use a new class loader. It is a bug if it does not.
I'm trying to package a small application (still learning Scala!) in a "clean way". The goal is to have an executable JAR file. I've done the following:
packaged a JAR using sbt -> will work with
scala -cp myjarfile.jar MyClass
or
java -classpath path\to\scala-library.jar;myjarfile.jar MyClass
but won't work with
java -jar myjarfile.jar
because then scala/ScalaObject cannot be found. And no use adding a classpath on this last one, since the -jar option will ignore the -classpath option. Note that I have added the scala libs in my system CLASSPATH variable, but it seems to be ignored too when -jar is used.
added the scala library contents (by unzipping them first) to the jar created by sbt. This then works (the jar can be double-clicked and launched), but the manipulation is somewhat lengthy, and the resulting file is several megabytes big. Not ideal.
After hours of googling, I can see no way to create a small jar file that will launch when double-clicked and that doesn't involve a painful manipulation. What am I missing? I'm guessing there should be some way to define where the scala libraries are at system level. How do you deal with small applications that you want to be compiled and ready-to-run for efficiency?
Note that while I'm using sbt, I don't have a jar tool at hand (I'm relying on a JRE, not a JDK).
Thanks!
Pierric.
The following setup works for me:
have scala-library.jar in the same folder as the executable jar (and call java from there)
put this into your manifest:
Class-Path: scala-library.jar
Another option is to merge the contents of scala-library.jar into your application jar. The drawback is that this will increase its size. But you can use Proguard to strip unused classes from your final jar. I think there is an easy way of using sbt to package an executable jar using proguard.
Regarding the shrinking using Proguard, you can have a look at this page. It's about Android development; just ignore this part and have a look at the tables of the shrinking results. Some example applications shrink to less than 100kB.
Edit
Maybe you need to refine your question a bit. What are you trying to achieve? Do you want to install the program only on your system or do you want to distribute it?
If all you want is quickly launching a Java application without much impact of the JVM start-up time you can have a look at nailgun.
I'm currently using IDEA's build mechanism with fsc for developing with Scala. It's still a bit slow and having to (re) start the compilation server is a pain. Many people here are suggesting SBT as a build tool together with IDEA.
What do you consider the pros and cons of each approach?
I tried both and in the end I prefer straight sbt for compiling.
Cons? I really miss being able to click through compile errors and fix the code directly, but... compiling in sbt is just much faster.
The nightly builds of the Idea Scala plugin can vary in quality/performance, but it's been getting better and better lately. The Scala plugin can now flag a number of errors that before I would have had to run compile to catch. (For example, I'm running nightly build 0.4.693 and the new method inspections have already been dead helpful.)
My advice for life with sbt on the command line: start sbt up and leave it running interactively as long as possible to take advantage of everything being loaded and JIT-ed.
sbt left running will fall over eventually but by giving it more memory in your sbt wrapper you can make that happen only rarely.
Here's the sbt launch wrapper that works for me.
java -Xms512M -Xmx1500M -XX:MaxPermSize=512m -jar `dirname $0`/sbt-launch.jar "$#"
My biggest issue with sbt 0.7 is that it frequently goes back and recompiles great swathes of files that seem only tangential to the code I was actually changing. (Even so, still faster than Idea and fsc!)
Good news: sbt 0.9 has some great incremental compile improvements. Unfortunately, the migration path from 0.7 to 0.9 is still in its early days. Mark Harrah's presentation at NEScala is online at http://www.nescala.org/2011/ if you're interested.
Useful plugins
http://github.com/mpeltonen/sbt-idea - easily create and keep your Idea project in sync with your sbt project
http://github.com/orfjackal/idea-sbt-plugin - lets you create run profiles for sbt from within Idea (I found this slower than running sbt at the command line last October - but I see orfjackal is still developing so I should give it another shot)