FSC recompiles every time - scala

FSC recompiles my .scala files every time even there is no need - I can compile it twice without editing anything between attempts and it recompiles them!
For example, I have 2 files
Hello.scala
class Hello{
print("hello")
}
And Tokens.scala:
abstract class Token(val str: String, val start: Int, val end: Int)
{override def toString = getClass.getSimpleName + "(" + "[" + start + "-" + end + "]" + str + ")"}
class InputToken(str: String, start: Int, end: Int)
extends Token(str, start, end)
class ParsedToken(str: String, start: Int, end: Int, val invisible: Boolean)
extends Token(str, start, end)
When I ask ant to compile project from scratch I see following output:
ant compile
init:
[mkdir] Created dir: D:\projects\Test\build\classes
[mkdir] Created dir: D:\projects\Test\build\test\classes
compile:
[fsc] Base directory is `D:\projects\Test`
[fsc] Compiling source files: somepackage\Hello.scala, somepackage\Tokens.scala to D:\projects\Test\build\classes
BUILD SUCCESSFUL
Than I don't edit anything and ask ant compile again:
ant compile
init:
[mkdir] Created dir: D:\projects\Test\build\classes
[mkdir] Created dir: D:\projects\Test\build\test\classes
compile:
[fsc] Base directory is `D:\projects\Test`
[fsc] Compiling source files: somepackage\Tokens.scala to D:\projects\Test\build\classes
BUILD SUCCESSFUL
As you can see, fsc acts smart in case of Hello.scala (no recompilation) and acts dumb in case of Tokens.scala. I suggest that the problem is somehow related with inheritance but that is all.
So what is wrong?

Tokens.scala is recompiled because there isn't a class file matching its basename. That is, it doesn't produce a Tokens.class file. When deciding if a source file should be compiled, fsc looks for a classfile with the same basename and if the class file does not exist or the modification time on the source file is later than that of the class file, the source file will be rebuilt. If you can, I suggest that you look into Simple Build Tool, its continuous compile mode accurately tracks source->classfile mapping and won't recompile Tokens.scala
For extra laughs, think about what the compiler might do if you have a different source file that has class Tokens in it.
Although scala allows arbitrary public classes/objects in any source file, there's still quite a bit of tooling that assumes you will somewhat follow the java convention and at least have one class/object in the file with the same name as the source file basename.

I don't like much posting stuff written by others, but I think this question merits a more complete answer that what was strictly asked.
So, first of all, fsc recompiles everything by default, period. It is ant, not fsc, which is leaving Hello.scala out, because the file name matches the class name. It is not leaving Tokens.scala out because there is no class called Tokens compiled -- so, in the absence of a Tokens.class, it recompiled Tokens.scala.
That is the wrong thing to do with Scala. Scala differs in one fundamental aspect from Java in that, because of technical limitations on JVM, a change in a trait requires recompilation of every class, object or instantiation that uses it.
Now, one can fix the ant task to do a smarter thing starting with Scala 2.8. I'm taking this information from blogtrader.net by Caoyuan, of Scala plugin for Netbeans fame. You define the Scala task on the build target like below:
<scalac srcdir="${src.dir}"
destdir="${build.classes.dir}"
classpathref="build.classpath"
force="yes"
addparams="-make:transitive -dependencyfile ${build.dir}/.scala_dependencies"
>
<src path="${basedir}/src1"/>
<!--include name="compile/**/*.scala"/-->
<!--exclude name="forget/**/*.scala"/-->
</scalac>
It tells ant to recompile everything, as ant simply isn't smart enough to figure out what needs to be recompiled or not. It also tells Scala to build a file containing compilation dependencies, and use a transitive dependency algorithm to figure out what needs to be recompiled or not.
You also need to change the init target to include the build directory in the build classpath, as Scala will need that to recompile other classes. It should look like this:
<path id="build.classpath">
<pathelement location="${scala-library.jar}"/>
<pathelement location="${scala-compiler.jar}"/>
<pathelement location="${build.classes.dir}"/>
</path>
For more details, please refer to Caoyuan's blog.

Related

how does scala interpreter execute source file directly without creating class file

I have this scala code
object S extends App{
println("This is trait program")
}
When I execute scala S.scala it executes fine.
Now I want to know how can it execute code without compile and creating of class file.
Scala is a compiled language, and it needs to compile the code and the .class file is needed for execution.
Maybe you are thinking in using the REPL, where you can interactively code: https://www.scala-lang.org/documentation/getting-started.html#run-it-interactively
But, under the hood, the REPL is compiling your code, and executing the compiled .class
The command scala that you are launching is used to launch Scala REPL, and if you provide a file as an argument, it'll execute it will execute the content of the files as if it was bulk pasted in a REPL.
It's true that Scala is a compiled language, but it does not mean that a .class file is necessary. All that the Scala compiler needs to do is generate relevant JVM byte code and call JVM with that byte code. This does not mean that it explicitly has to create a .class file in directory from where you called it. It can do it all using memory and temporary storage and just call JVM with generated byte code.
If you are looking to explicitly generate class files with Scala that you can later execute by calling java manually, you should use Scala compiler CLI (command: scalac).
Please note that Scala compiler has interfaces to check and potentially compile Scala code on the fly, which is very useful for IDEs (checkout IntelliJ and Ensime).
Just call main() on the object (which inherits this method from App):
S.main(Array())
main() expects an Array[String], so you can just provide an empty array.
Scala is a compiled language in terms of source code to java bytecode transition, however some tricks may be taken to make it resemble an interpreted language. A naive implementation is that when run scala myscript.scala, it follows these steps:
scalac Myscript.scala. It generates S.class (which is the entry class that contains main method) and potentially other class files.
scala -cp . S. This runs/interprets from the main entry of the class
file. -cp . specifies the classpath; and S is the entry class without file extension .class. Note that run/interprets means interpreting (java) bytecode (rather than Scala/Java source code), which is done by JVM runtime.
Remove all the temporarily generated class files. This procedure is optional as long as the users are not aware of the temporary files (i.e., transparent to users).
That is to say, scala acts as a driver that may handle 0) initialization 1) compilation(scalac) 2) execute/run (scala) 3) cleanup.
The actual procedures may be different (e.g., due to performance concerns some files are only in memory/cached, or not generated, or not deleted, by using lower-level APIs of scala driver, etc.) but the general idea should be similar.
On Linux machines, you might find some evidences in /tmp folder. For me,
$ tree /tmp
/tmp
├── hsperfdata_hongxu
│   └── 64143
└── scala-develhongxu
├── output-redirects
│   ├── scala-compile-server-err.log
│   └── scala-compile-server-out.log
└── scalac-compile-server-port
└── 34751
4 directories, 4 files
It is also noteworthy that this way of running Scstepsala is not full-fledged. For example, package declarations are not supported.
# MyScript.scala
package hw
object S extends App {
println("Hello, world!")
}
And it emit error:
$ scala Myscript.scala
.../Myscript.scala:1: error: illegal start of definition
package hw
^
one error found
Others have also mentioned the REPL (read–eval–print loop), which is quite similar. Essentially, almost every language can have an (interactive) interpreter. Here is a text from wikipedia:
REPLs can be created to support any language. REPL support for compiled languages is usually achieved by implementing an interpreter on top of a virtual machine which provides an interface to the compiler. Examples of REPLs for compiled languages include CINT (and its successor Cling), Ch, and BeanShell
However interpreted languages (Python, Ruby, etc.) are typically superior due to their dynamic nature and runtime VMs/interpreters.
Additionally, the gap between compiled and interpreted is not that big. And you can see Scala actually has some interpreted features (at least it appears) since it makes you feel that you can execute like a script language.

antlr4 Jar has duplicate classes in different packages - don't know which are referred to by internal code

I am using antlr4.3 (complete) jar.
It has many duplicates in org.antlr.runtime and org.antlr.v4.runtime packages.
In code when I explicitly use 'v4.runtime' - at runtime, classpath picks up 'runtime'.
So I extracted the jar and recreated it without org.antlr.runtime.
But apparently some classes like RecognitionException is now not found.
How should I resolve this other than:
Exploding the latest Jar and specifying org.antlr.v4.runtime BEFORE org.antlr.runtime so that a duplicate class will be picked up from v4.runtime, and if there isn't one in it, it will look at org.antlr.runtime...??
To add to the above, here's the code snippet which gives a problem: the jars are in the classpath.
import org.antlr.v4.runtime.CharStream;
import org.antlr.v4.runtime.ANTLRInputStream;
public class AntlrMain {
public static void main(String[] args) {
System.out.println("Start Hello World");
try {
InputStream is = new FileInputStream(
"/home/ecworkspace/antlrCentral/DSL/mydev.dsl");
org.antlr.runANTLRInputStream input = new ANTLRInputStream(is);
org.antlr.v4.runtime.CharStream cs = (org.antlr.v4.runtime.CharStream) input;
VCFGLexer lexer = new VCFGLexer(cs);
Initially in the ANtlrMain class, I wasn't using explicit
org.antlr.v4.runtime.; but that failed at runtime, with 'CharStream not found'.
Then I changed to include full path of the class
Then changed the ANTLR4 Jar to 'exclude' org.antlr.runtime (it has org.antlr.v4.runtime). That's when the 'RecognitionException not found' error occurred.
The grammar by the way, compiles OK, generating all my VCFG*.java and tokens classes, where VCFG is the grammar name.
UPDATE 1
Keeping in line with suggestions from all - I removed my answer to my own questions and adding it to this original questions.
In antlr-4.2-complete.jar, I see:
/tmp/x/ $ jar -xf antlr-4.2-complete.jar
/tmp/x/ $ ls org/antlr
runtime stringtemplate v4
/tmp/x/ $ ls org/antlr/v4
analysis codegen parse semantics Tool$1UndefChecker.class Tool$OptionArgType.class
automata misc runtime tool Tool.class Tool$Option.class
/tmp/x/ $ ## The 2 runtimes above: org.antlr.runtime and org.antlr.v4.runtime
/tmp/x/ $ ## which to use where, along with same-name classes in
/tmp/x/ $ ## org.antlr and org.antlr.v4
So, in build.xml, I use above jar to:
`
java -jar antlr-4.2-complete grammar.g4 => compiles and gives me
VCFG*.java and VCFG*.tokens
javac -cp "antlr-4.2-complete-jar" VCFG*.java => Succeeds. I have
the VCFG*.class collection.
Then I compile my code AntlrMain.java (which uses AntlrInputStream
etc.), again with the above antlr jar and some 3rd-party Jars
(logging, commons) => successfully.
Finally the RUN of java -cp "antlr-4.2-complete.jar:log4j.jar" -jar
myJar => FAILS on 'CharStream' not found.
UPDATE 2
Adding, based on your response.
I have only recently started posting questions on Stackoverflow. So pointers about whether to respond to my question to provide more info, or to comment to a reply etc. are welcome.
-cp <3rd-party> is -cp "log4j.jar:commonsLang.jar".
By -cp "above-jar" I meant -cp "antlr-4.2-complete.jar.
And if I have not mentioned it, it is an oversight - I have, for every 'java' and 'javac commands, included antlr-4.2-complete.jar.
BUT I see you indicating antlr-runtime-4.2.jar. So there ARE separate antlr-runtime jar and antlr-complete jars.
In the 4 steps below (I am leaving out -cp for convenience, but am including antlr-4.2-complete.jar for 'every' step.
I believe, I should be using the antlr-run-time and antlr-complete jars at different steps:
1 (java MyGrammar.java)
2 (javac MyGrammar*.java)
3. javac MyOwnCode.java
4. Run myCode (java MyCode) ...
which of the two antlr JARs (runtime and complete; and their versions) should I then use, at each of the above 4 steps?
The jar file does not contain duplicate classes. The code generation portion of the ANTLR 4.3 Tool relies on the ANTLR 3.5.2 Runtime library, which is included in the "complete" jar file. While some of the classes have the same name as classes in ANTLR 4, they are not duplicates and cannot be used interchangeably.
#280Z28 / Sam:
I am mortified, but have to admit the simplest answer is most often the correct.
I spent time fleshing out the JAR, making multiple JAR files out of it, include one for compile, one for run and on and on.
The answer is succinctly explain in the ANT build.xml code snippet below: where I produce the 'final' production JAR file, which is the only JAR then included while executing my Main program:
<jar destfile="${p_productionJar}">
<fileset dir="${p_buildDir}" casesensitive="yes">
<include name="**/*.class"/>
</fileset>
<zipfileset includes="**/*.class" src="${p_genCodeJar}"/>
<!-- How did I miss including p_antlrJar earlier?? -->
<zipfileset includes="**/*.class" src="${p_antlrJar}"/>
<zipfileset includes="**/*.class" src="${p_jschJar}"/>
<zipfileset includes="**/*.class" src="${p_log4jJar}"/>
<zipfileset includes="**/*.class" src="${p_commonslangJar}"/>
<manifest>
<attribute name="Main-Class" value="AntlrMain"/>
.....
The production Jar was missing ${p_antlrJar} => which is antlr-4.3-complete.jar!!!!
You did mention this in your answer... but it was a pretty silly mistake to do, and didn't think I had done it...
Thank you.

Buildr: How do I define a project which depends on another project in the same Buildfile?

This seems like a simple task in Buildr, so I must be missing something obvious because I can't make it work. Suppose I have a directory with two files, like so:
test/lib/src/main/scala/MyLib.scala
test/app/src/main/scala/MyApp.scala
MyLib.scala is:
class MyLib {
def hello() { println("Hello!") }
}
And MyApp.scala is:
object MyApp extends App {
val ml = new MyLib
ml.hello()
}
Building these with scalac is straightforward:
$ cd test
$ scalac lib/src/main/scala/MyLib.scala -d target/main/classes
$ scalac app/src/main/scala/MyApp.scala -cp target/main/classes -d target/main/classes
$ cd target/main/classes/
$ scala MyApp
Hello!
However, my naïve attempt to turn this into a Buildfile (in the test folder):
require 'buildr/scala'
lib_layout = Layout.new
lib_layout[:source, :main, :scala] = 'lib/src/main/scala'
app_layout = Layout.new
app_layout[:source, :main, :scala] = 'app/src/main/scala'
define 'mylib', :layout => lib_layout do
end
define 'myapp', :layout => app_layout do
compile.with project('mylib')
end
fails with:
(in /test, development)
Building mylib
Compiling myapp into /test/target/main/classes
/test/app/src/main/scala/MyApp.scala:2: error: not found: type MyLib
val ml = new MyLib
^
one error found
Buildr aborted!
RuntimeError : Failed to compile, see errors above
and if I run buildr --trace it's pretty clear that the reason scalac is failing is because the classpath does not include target/main/classes.
How do I make this happen? I know that separating the two projects may seem contrived, but I have something much more sophisticated in mind, and this example boiled the problem down to its essential components.
The idiomatic way to describe your project with Buildr would be to use sub-projects,
Note: buildfile below goes into the test/ directory.
require 'buildr/scala'
define "my-project" do
define "lib" do
end
define "app" do
compile.with project("lib").compile.target
end
end
The two sub-projects lib and app are automatically mapped to the lib/ and app/ sub-directories and Buildr will automatically look for sources under src/main/scala for each.
Buildr common convention for project layout looks for compiled classes in target/classes and target/test/classes. I see your build is placing classes into target/main/classes:
So you'll want to change your scala params to the expected location, or change the layout to include:
lib_layout[:target, :main, :scala] = 'target/main/classes'
If you also need to change the layout for test classes (I'm guessing you don't), use:
lib_layout[:target, :test, :scala] = ...

How do I install a Scala compiler plugin so that it automatically works with Ant?

I'm writing a Scala compiler plugin that I want to be used whenever scalac is invoked, so I put it in the Scala distribution's plugins directory (misc/scala-devel/plugins) - this works perfectly when I write something simple like scalac HelloWorld.scala, but doesn't work when I try building using an Ant script with an embedded scalac task like:
<scalac srcdir="${src}" destdir="${bin}" classpathref="build.classpath" deprecation="on">
<include name="**/*.scala"/>
</scalac>
I can get it to work by changing the build script to specify the plugins directory, e.g.:
<scalac srcdir="${src}" destdir="${bin}" classpathref="build.classpath" deprecation="on"
addparams="-Xpluginsdir /Users/stuart/Downloads/scala-2.9.2/misc/scala-devel/plugins">
<include name="**/*.scala"/>
</scalac>
However, I need this to work for third-party build scripts that I am unable to change manually, so my simple fix doesn't cut it - any ideas please?
UPDATE:
After a bit more digging, I've found that you can override task definitions using presetdef, e.g.:
<presetdef name="scalac">
<scalac addparams="-Xpluginsdir /Users/stuart/Downloads/scala-2.9.2/misc/scala-devel/plugins"/>
</presetdef>
This doesn't directly solve the problem, but it's potentially interesting - my current line of investigation is whether it's possible to put this (or something like it) in a wrapper Ant script that imports the original one (no idea whether this will work, I don't use Ant that much).

Scalac doesn't find dependent classes

I'm trying to compile the program with 2 simplest classes:
class BaseClass
placed in BaseClass.scala and
class Test extends BaseClass
placed in Test.scala. Issuing command scalac Test.scala fails, cause BaseClass is not found.
I don't want to compile classes one by one or using scalac *.scala.
The same operation in java works: javac Test.java. Where am I wrong?
Let's see first what Java does:
dcs#dcs-132-CK-NF79:~/tmp$ ls *.java
BaseClass.java Test.java
dcs#dcs-132-CK-NF79:~/tmp$ ls *.class
ls: cannot access *.class: No such file or directory
dcs#dcs-132-CK-NF79:~/tmp$ javac -cp . Test.java
dcs#dcs-132-CK-NF79:~/tmp$ ls *.class
BaseClass.class Test.class
So, as you can see, Java actually compiles BaseClass automatically when you do that. Which begs the question: how can it do that? Can does it know what file to compile?
Well, when you write extends BaseClass in Java, you actually know a few things. You know the directory where these files are found, from the package name. It also knows BaseClass is either in the current file, or in a file called BaseClass.java. If you doubt either of these facts, try moving the file from directory or renaming it, and see if Java can compile it.
So, why can't Scala do the same? Because it assumes neither thing! Scala's files can be in any directory, irrespective of the package they declare. In fact, a single Scala file can even declare more than one package, which would make the directory rule impossible. Also, a Scala class can be in any file whatsoever, irrespective of its name.
So, while Java dictates to you what directory the file should be in and what the file is called, and then reaps the benefit by letting you omit filenames from the command line of javac, Scala let you organize your code in whatever way seems best to you, but requires you to tell it where that code is.
Take your pick.
You need to compile BaseClass.scala first:
$ scalac Test.scala
Test.scala:1: error: not found: type BaseClass
class Test extends BaseClass
^
one error found
$ scalac BaseClass.scala
$ scalac Test.scala
$
EDIT So, the question is now why you have to compile the files one by one? Well, because the Scala compiler just doesn't do this kind of dependency handling. Its authors probably expect you to use a build tool like sbt or Maven so that they don't have to bother.