I have the following use case. I would like to build the same Scala project for scala 2.10 and 2.12. When doing so I would like to specify some of the dependencies for the 2.10 version as provided whereas I'd like to have those compiled in the jar for 2.12.
I was looking at SBT's docs and found how I can split a build.sbt into separate declarations but those always get mentioned as sub-modules. In my case I'd like to cross-build the whole app - not specific parts of it.
Any hints or resources will be appreciated.
You can assemble the libraryDependencies depending on the Scala version. Simplified example, may not build..:
libraryDependencies := {
CrossVersion.partialVersion(scalaVersion.value) match {
case Some((2, scalaMajor)) if scalaMajor == 12 =>
libraryDependencies.value ++ Seq(
"your.org" %% "your-lib" % "1.0")
case Some((2, scalaMajor)) if scalaMajor == 10 =>
libraryDependencies.value ++ Seq(
"your.org" %% "your-lib" % "1.0" % provided)
case _ => Seq()
}
}
For a full example, see http://github.com/scala/scala-module-dependency-sample
Related
I'm pretty new to scala. While upgrading a multi-module project to Scala 2.13, I found this dependency that is compiled in Scala 2.12 which throws class not found exception during runtime
java.lang.NoClassDefFoundError: scala/collection/mutable/ArrayOps$ofRef
This class is removed in 2.13. It is available only until 2.12. I am looking for a way to enforce v2.12 to compile only this dependency.
I tried to use cross-building but that does not work for a core library, because the dependency url constructed using:
"org.scala-lang" % "scala-library" % scalaVersion.value
looks like
https://repo1.maven.org/maven2/org/scala-lang/scala-library_2.12.15/2.12.15/scala-library_2.12.15-2.12.15.pom
Also, cross-building seems to be the way to allow compiling sub-modules with different scala versions with their compatible dependency versions, not meant for enforcing scala versions on individual dependencies.
Edit 1:
This is the build definition:
root
|
main
|---dependency w/o 2.13 build
|
acceptanceTests
|---dependency w/o 2.13 build
|
(other modules)
The dependency is an internal commons library. This uses the class scala/collection/mutable/ArrayOps during compile time. From scala-lang -> scala-library.
My questions:
Is it even possible to do this? Or is my only option to downgrade to 2.12 as mentioned here
Why 'core' libraries do not follow the url patters of external libraries like:
[organisation]/[module](_[scalaVersion])(_[sbtVersion])/[revision].
`Instead it looks like https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.12.15/scala-library-2.12.15.pom
You can do stuff like this:
libraryDependencies ++= {
CrossVersion.partialVersion(Keys.scalaVersion.value) match {
case Some((2, 12)) => Seq(
"com.typesafe.play" %% "play-json" % "2.6.13"
)
case Some((2, 11)) => Seq(
"com.typesafe.play" %% "play-json" % "2.5.18"
)
case _ => Seq (
"com.typesafe.play" %% "play-json" % "2.4.11"
)
}
}
I am trying to learn the basics of Scala, scalatest, and sbt and I'm following a tutorial. This is my built.sbt file:
name := "demo-hello"
version := "0.1"
scalaVersion := "2.12.6"
libraryDependencies += "org.scalatest" % "scalatest_2.10" % "2.1.0" % "test"
I have a test that looks like this (showing this is probably unnecessary:
package demo
import org.scalatest.FunSuite
class HelloTest extends FunSuite {
test("say hello method works correctly") {
val hello = new Hello
assert(hello.sayHello("Scala") == "Hello, Scala!")
}
}
What should I do from here? I am trying to run the test but I get this error:
Error:(8, 36) can't expand macros compiled by previous versions of Scala
assert(hello.sayHello("Scala") == "Hello, Scala!")
I'm not that familiar with the % symbol btw.
FIX
I changed my build.sbt to this:
name := "demo-hello"
version := "0.1"
scalaVersion := "2.10"
libraryDependencies += "org.scalatest" % "scalatest_2.10" % "2.1.0" % "test"
Remaining questions:
So it seems downgrading to scalaVersion "2.10" worked. Why?
What is an artifact? scalatest is apparently an artifact?
Where is scalaversion 2.10 kept on my machine? It seems I only have scala 2.12. Where in my project folder is version 2.10?
Answering your questions slightly out-of-order:
2 - An "artifact" is something that's built by maven, sbt, or another build system. For Scala or Java, this is almost always a jar file. Each item in libraryDependencies specifies a file in a maven repository (a database of artifacts).
1 - Scala class files are not compatible between minor versions of Scala. When you download a Scala jar from a maven repository, the Scala version is specified as part of the artifact name. The _2.10 in your dependency declares that you wish to use the version of scalatest that's compile for Scala 2.10 - which is why you were getting an error using it in your Scala 2.12 application.
When declaring dependencies on Scala artifacts in sbt, you should always use the %% operator, which automatically appends the appropriate suffix to your artifact, like so:
// This works for any scalaVersion setting.
libraryDependencies += "org.scalatest" %% "scalatest" % "2.1.0" % "test"
3 - sbt handles downloading the appropriate runtime files for the declared version of Scala automatically.
Question
Is .sbt file is a in scala or in sbt proprietary language? Please help to decipher the sbt build definition.
lazy val root = <--- Instance of the SBT Project object? Why "lazy"? Is "root" the reserved keyword for sbt to identify the project in the build.sbt?
(project in file(".")) <--- Is this defining a Project object regarding the current directory having the SBT expected project structure?
.settings( <--- Is this a scala function call of def settings in the Project object?
name := "NQueen",
version := "1.0",
scalaVersion := "2.11.8",
mainClass in Compile := Some("NQueen")
)
libraryDependencies ++= Seq( <--- libraryDependencies is a reserved keyword of type scala.collection.Seq using which sbt downloads and packages them as part of the output jar?
"org.apache.spark" %% "spark-core" % "2.3.0", <--- Concatenating and creating the full library name including version. I suppose I need to look into respective documentations to find out what to specify.
"org.apache.spark" %% "spark-mllib" % "2.3.0"
)
// <--- Please explain what this block does and where I can find the explanations.
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case x => MergeStrategy.firs
}
Resources
Please suggest good resources to understand the design, mechanism, how .sbt works. I looked into the SBT getting started and documents but as Scala definition itself, it is difficult to understand. If it is make, ant, or maven, how things get pieced together and the design/mechanism are so much clear, but need to find good documentations or tutorials for SBT.
References
I looked into the references below trying to understand.
SBT: How to get started using the Build.scala file (instead of build.sbt)
What is the difference between build.sbt and build.scala?
SBT - Build definition
SBT Project object
scala.collection.Seq
SBT Library dependencies
Spark 2.3 Quick Start
sbt can be really difficult for first time users, and it's ok not to fully understand all of the definitions. It will become clearer over time.
let me first simplify you build.sbt. it contains some unnecessary parts, and will be easier to explain without them
name := "NQueen"
version := "1.0"
scalaVersion := "2.11.8"
mainClass in Compile := Some("NQueen")
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.3.0",
"org.apache.spark" %% "spark-mllib" % "2.3.0"
)
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case x => MergeStrategy.firs
}
and for your questions:
Is .sbt file is a in scala or in sbt proprietary language?
well, it's both. you can do most scala operations in an .sbt file. you can import and use external dependencies, write custom code, etc.. but some things you can't do (define classes for example).
It's also might look as a dedicated different language, but in reality, it's just a DSL written in scala (:=, in, %%, % are all function written in scala)
libraryDependencies is a reserved keyword of type scala.collection.Seq using which sbt downloads and packages them as part of the output jar?
libraryDependencies is not a reserved keyword. you can think of it as a way to configure you project.
writing libraryDependencies := Seq(..) you basically setting the value of libraryDependencies.
But you are right about the meaning. it is a list of dependencies that should be downloaded.
Concatenating and creating the full library name including version. I suppose I need to look into respective documentations to find out what to specify.
keep in mind that %% and % are functions. you use those functions to specify what modules should be downloaded and added to the classpath.
you can find many dependencies (and their versions) in mvnrepository.
for example, for spark: https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.3.0
Please explain what this block does and where I can find the explanations.
assemblyMergeStrategy is a setting coming from the sbt-assembly plugin.
that plugin allows you to pack your application into a single jar with all the dependencies.
you can read about the merge strategy here: https://github.com/sbt/sbt-assembly#merge-strategy
I am upgrading one of the my Scala projects to Scala 2.11.7, but it has some dependent projects. They are using 2.10 so it is referring to a lot of dependent libraries with 2.10 Scala version (ex: com.novus:salat-core_2.10:1.9.9). I want to exclude which and all having "_2.10-" instead of writing one by one.
Is this possible?
1: Using scala version and providing sbt to choose correct version with %%
scalaVersion := "2.11.7"
val scalaz = "org.scalaz" %% "scalaz-core" % "7.1.0"
2: To exclude explicitly dependencies which has 2.10 build version use custom methods
// exclude from all with rule which check whether artifact name contains 2.10
def excludeFromAll(items: Seq[ModuleID], group: String, artifact: String) =
items.map(x => if(x.name.contains("_2.10")) x.exclude(group, artifact))
//all your dependencies
val deps = Seq(dependencies) //library Dependencies.
//exlusion
val appDependencies = excludeFromAll(deps, _, _)
I want to build a 2.11 and 2.12 version of my project, so I have something like this in my Build.scala file:
val scalaVer12 = "2.12.1"
val scalaVer = "2.11.8"
lazy val basicSettings = Seq(
// lots of other settings
scalaVersion := scalaVer
)
The fly in the soup is I have a dependency on scala reflection, which is based on the scala version. Before I did this:
val scala_reflect = "org.scala-lang" % "scala-reflect" % Build.scalaVer
How can I modify this dependency line so that sbt will use either the 2.11 or 2.12 dependency based upon the version it's currently building?
lazy val bla = project in file("bla")
.settings(
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % scalaVersion.value
)
)
Never alias dependencies like that, it's clean to have an object to store version numbers, but not more, it's just a smell, especially since deps are often Scala version dependent and you can apply all sorts of rules to them.