I modified the source code of scala compiler and built it. Now I want to test this compiler. However, many existing scala projects use sbt as build tool. So I wonder if it is possible to replace the official scala compiler used by sbt with the scala compiler built by myself.
See http://www.scala-sbt.org/1.0/docs/Configuring-Scala.html#Using+Scala+from+a+local+directory:
The result of building Scala from source is a Scala home directory <base>/build/pack/ that contains a subdirectory lib/ containing the Scala library, compiler, and other jars. The same directory layout is obtained by downloading and extracting a Scala distribution. Such a Scala home directory may be used as the source for jars by setting scalaHome. For example,
scalaHome := Some(file("/home/user/scala-2.10/"))
If you want to publish the compiler, use #ipoteka's answer.
According to docs it's straightforward:
managedScalaInstance := false
libraryDependencies += "yourPackage" % "yourScalaCompiler" % version
Don't forget to publish-local you compiler first.
Related
The sbt documentation tells me I can suffix the scala version in order to get sbt to pick up the sources when building with that version.
https://www.scala-sbt.org/1.x/docs/Cross-Build.html#Scala-version+specific+source+directory
It doesn't say how I can specify the suffix so that I can match multiple scala versions in one directory. I need one to match both 2.12 and 2.13, and one to match 2.11.
I found a few example of that in a few libraries like zio or cats. The suffixes looked like -2.12+ or -2.12-2.13. When applied to my build though, sbt doesn't seem to find these paths, and compilation fails because some classes are not found. If I separate them with -2.12 and -2.13 it works, but I'd have to duplicate code.
You can try sbt new scala/scala-seed.g8 and rename src/main/scala to src/main/scala-2.12+ and see that it doesn't work, while it works when renamed to src/main/scala-2.13.
Can someone point me to what I'm missing?
I've tried sbt versions 1.3.10 and 1.2.8.
I am porting a small legacy library from scala 2.12 to scala 2.13. sbt version is 1.3.3. The project is flat and relatively simple. scalaVersion declared in the project is 2.13.1.
I am executing clean and compile tasks, and then publish to both local ivy and to the artifactory.
The process seemingly goes fine and creates the artifact with the _2.13 suffix. When this binary gets executed against scala 2.13 runtime, it fails with MethodNotFound exception. Further introspection shows that the artifact was compiled for 2.12 but not for 2.13.
Does anybody have an idea why a different compiler version was used by sbt, and how to fix this problem?
Just had similar issue, sbt was compiling my project to the wrong Scala version, found this question without answers in google.
So, my problem was actually very simple. Turns out you need to start sbt in a project root directory (where build.sbt is located). I was running it from a directory where all my .scala files were located, so it haven't parsed build.sbt and compiled project using default Scala version (2.12 in my case).
I have some main object:
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
object Main {
def main(args: Array[String]) {
val sc = new SparkContext(
new SparkConf().setMaster("local").setAppName("FakeProjectName")
)
}
}
...then I add spark-assembly-1.3.0-hadoop2.4.0.jar to the build path in Eclipse from
Project > Properties... > Java Build Path :
...and this warning appears in the Eclipse console:
More than one scala library found in the build path
(C:/Program Files/Eclipse/Indigo 3.7.2/configuration/org.eclipse.osgi/bundles/246/1/.cp/lib/scala-library.jar,
C:/spark/lib/spark-assembly-1.3.0-hadoop2.4.0.jar).
This is not an optimal configuration, try to limit to one Scala library in the build path.
FakeProjectName Unknown Scala Classpath Problem
Then I remove Scala Library [2.10.2] from the build path, and it still works. Except now this warning appears in the Eclipse console:
The version of scala library found in the build path is different from the one provided by scala IDE:
2.10.4. Expected: 2.10.2. Make sure you know what you are doing.
FakeProjectName Unknown Scala Classpath Problem
Is this a non-issue? Either way, how do I fix it?
This is often a non-issue, especially when the version difference is small, but there are no guarantees...
The problem is (as stated in the warning) that your project has two Scala libraries on the class path. One is explicitly configured as part of the project; this is version 2.10.2 and is shipped with the Scala IDE plugins. The other copy has version 2.10.4 and is included in the Spark jar.
One way to fix the problem is to install a different version of Scala IDE, that ships with 2.10.4. But this is not ideal. As noted here, Scala IDE requires every project to use the same library version:
http://scala-ide.org/docs/current-user-doc/gettingstarted/index.html#choosing-what-version-to-install
A better solution is to clean up the class path by replacing the Spark jar you are using. The one you have is an assembly jar, which means it includes every dependency used in the build that produced it. If you are using sbt or Maven, then you can remove the assembly jar and simply add Spark 1.3.0 and Hadoop 2.4.0 as dependencies of your project. Every other dependency will be pulled in during your build. If you're not using sbt or Maven yet, then perhaps give sbt a spin - it is really easy to set up a build.sbt file with a couple of library dependencies, and sbt has a degree of support for specifying which library version to use.
The easiest solution:
In Eclipse :
1. Project/ (righclick) Properties
2. Go to Scala Compiler
3. click Use Project Settings
4. set Scala Installation to a compatible version. Generally Fixed Scala Installation 2.XX.X (build-in)
5. Rebuild the project.
There are 2 types of Spark JAR files (just by looking at the Name):
- Name includes the word "assembly" and not "core" (has Scala inside)
- Name includes the word "core" and not "assembly" (no Scala inside).
You should include the "core" type in your Build Path via “Add External Jars”
(the version you need) since the Scala IDE already shoves one Scala for you.
Alternatively, you can just take advantage of the SBT and add the following
Dependency (again, pay attention to the versions you need):
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"
Then you should NOT include “forcefully” any spark JAR in the Build Path.
Happy sparking:
Zar
>
I have written a Scala program with Eclipse Scala IDE that uses scala.util.parsing.JSON and I would like to transform it to support Scala 2.11 version. In Scala 2.11 I get an error:
error: object parsing is not a member of package util.
I have found out that the parsing library is not anymore in the util package by default, but has to be downloaded separately here.
I have downloaded that and tried to add it to my Eclipse project as a new source folder, but that didn't work out. The instructions are only for adding it to sbt, but I don't think that is relevant to me if I want to just use it in Eclipse.
Should I try to find a JAR file somewhere?
Should I try to find a JAR file somewhere?
Yes, you should. :)
And specifically, you should use this one (in SBT syntax):
libraryDependencies += "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.2"
The above line should be all you need to add to build.sbt if you're using SBT. If you want to manually add it to your project by downloading it, you can find it on Maven Central.
The scala-parser-combinators library was removed in 2.11 so that people who don't use it don't have to pay a price for having it in the scala runtime library. Consequently, people who do want to use it have to now explicitly include it in their build. Note that the XML library was similarly removed in 2.11 to its own library.
I'm currently using sbt to build and run my scala programs. I'm trying to use sbt.Process to execute system commands. I must be missing something because when I try to import sbt.Process in one of my files in src/ I get this error.
not found: value sbt
[error] import sbt.Process._
So it looks like I can't access the sbt package inside my src/ files. What do I need to do to access it? Thanks.
SBT's environment (v 0.7.x) is only available in your build file or a Plugin.
The easiest way to use sbt.Process library (until 0.9.x which will have Process as an independent library) is to copy (BSD License) Process.scala and ProcessImpl.scala into your project
There are different classpaths for running sbt and compiling your source files.
One classpath is for compilation of files in directory project/build (that one contains sbt jars and usually scala library 2.7.7) and the other one is for building source files of your project (that one contains your dependencies from lib and lib_managed and usually scala library 2.8.*). If you'd like to use sbt.Process in your source files you can do two things:
add sbt jar to lib or lib_managed for it to be available on your project's classpath
use snapshot version of scala 2.9, it would have sbt Process built-in as sys.process package
Wait for Scala 2.9, and then just use it out of scala.sys.process.
sbt package has became an integral part of the Scala standard library since version 2.9
...this API has been included in the Scala standard library for version 2.9.
quoted from sbt wiki
Here's the link (scroll down)
well, in order to use it, all you have to do (assuming you are using sbt for build), is to add in build.sbt file the following line of code: sbtPlugin := true it will add the needed dependencies to your project.
of course, this solution is only to get your imports with sbt package to work. you should refactor your code to use the new package scala.sys.process like Daniel C. Sobral suggested.