why is trouble that Spark issue at sbt in intellij - scala

name := "name"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.0.2"
Log:
[warn] Multiple dependencies with the same organization/name but different versions.
To avoid conflict, pick one version:
[warn] * org.scala-lang:scala-compiler:(2.11.0, 2.11.7)
[warn] * org.scala-lang:scala-library:(2.11.8, 2.11.7)
[warn] * org.scala-lang.modules:scala-parser-combinators_2.11:(1.0.1, 1.0.4)
[warn] * org.scala-lang.modules:scala-xml_2.11:(1.0.2, 1.0.4)

You can exclude the unwanted versions of dependencies by checking dependency tree
or
In dedicated way telling sbt to choose one you want like:
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-compiler" % "2.11.7",
"org.scala-lang.modules" % "scala-parser-combinators_2.11" % "1.0.4",
"org.scala-lang" % "scala-library" % "2.11.7",
"org.scala-lang.modules" % "scala-xml_2.11" % "1.0.4"
)

Related

Scala play with reactivemongo, not working

I am trying to follow along this tutorial and I can't get reactivemongo to work:
This is my error:
UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.reactivemongo#play2-reactivemongo_2.12;0.11.7.play24: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.reactivemongo:play2-reactivemongo_2.12:0.11.7.play24 (/Users/jwan/Desktop/programming/scala_play/rest-api/build.sbt#L12-20)
This is my built.sbt:
name := """rest-api"""
organization := "com.example"
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.12.4"
libraryDependencies += guice
libraryDependencies += "org.scalatestplus.play" %% "scalatestplus-play" % "3.1.2" % Test
libraryDependencies ++= Seq(
jdbc,
cache,
ws,
specs2 % Test,
"org.reactivemongo" %% "play2-reactivemongo" % "0.11.7.play24",
"org.scalatest" % "scalatest_2.11" % "2.2.4" % "test"
)
// Adds additional packages into Twirl
//TwirlKeys.templateImports += "com.example.controllers._"
// Adds additional packages into conf/routes
// play.sbt.routes.RoutesKeys.routesImport += "com.example.binders._"
And this is my application.conf:
# https://www.playframework.com/documentation/latest/Configuration
play.modules.enabled += "play.modules.reactivemongo.ReactiveMongoModule"
mongodb.uri = "mongodb://localhost:27017/getting-started-play-scala"
I can't get sbt run to work. Any ideas?
According to https://mvnrepository.com/artifact/org.reactivemongo/play2-reactivemongo.
Required scala version is 2.11 for reactivemongo 0.11.7.play24.
Try 0.13.0-play26 for scalaVersion := "2.12.4" and play 26.
libraryDependencies += "org.reactivemongo" %% "play2-reactivemongo" % "0.13.0-play26"

How do dependencies work in Scala?

For my existing projects I am migrating from scalaVersion 2.10.5 to 2.11.7 and sbtVersion 1.13.9 also.
Current migrating project is Sales, it have Sales-common, Sales-read, Sales-write modules. It have one dependent project Core. But I don't have Core project code I have only already published artefact jars(I have myapp-core-read_2.10-2.2.33.jar, 2.10 is Scala version).
My project looks like this
Sales
common
build.sbt
read
build.sbt
write
build.sbt
build.sbt
Till now, with scala 2.10.5 everything working fine using in real time also.
After change the version I run sbt clean and sbt update. I am getting Unresolved dependency issue like this:
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: mycompany#myapp-core-read_2.11;2.2.33: not found
[warn] :: mycomapny#myapp-core-write_2.11;2.2.33: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
My Sales buld.sbt is this:
name := "myapp-sales"
organization in ThisBuild := "com.mycompany"
scalaVersion in ThisBuild := "2.11.7"
//crossScalaVersions in Thisq
//Build := List("2.10.5", scalaVersion.value)
//crossPaths := false
scalacOptions in Compile in ThisBuild ++= Seq("-unchecked", "-deprecation", "-encoding", "utf8", "-language:postfixOps", "-language:implicitConVersions")
lazy val common = Project("myapp-sales-common", file("common"))
lazy val read = Project("myapp-sales-read", file("read")).configs(IntegrationTest).settings(Defaults.itSettings: _*).dependsOn(common, write)
lazy val write = Project("myapp-sales-write", file("write")).configs(IntegrationTest).settings(Defaults.itSettings: _*).dependsOn(common)
conflictWarning in ThisBuild := ConflictWarning.disable
parallelExecution in Test in ThisBuild := false
parallelExecution in IntegrationTest in ThisBuild := false
javacOptions in Compile in ThisBuild ++= Seq("-source", "1.6", "-target", "1.6")
//Remove SNAPSHOT check from the release process (for now until Squants gets a release)
releaseProcess := releaseProcess.value.filterNot(_ == ReleaseTransformations.checkSnapshotDependencies)
My read module build.sbt:
libraryDependencies ++= Seq(
"com.mycompany" %% "myapp-core-read" % myappsales.CoreVersion % "compile", // disable using the Scala version in output paths and artifacts,
"com.mycompany" %% "myapp-core-write" % myappsales.CoreVersion % "compile",
"com.mycompany" %% "myapp-registration-common" % myappsales.RegistrationVersion % "compile",
"com.mycompany" %% "myapp-load-common" % myappsales.LoadVersion % "compile",
"com.mycompany" %% "myapp-core-write-test" % myappsales.CoreVersion % "it, test",
"com.mycompany" %% "myapp-core-test" % myappsales.CoreVersion % "it, test"
)
libraryDependencies ++= Seq(
"com.typesafe" % "config" % myappsales.TypeSafeConfigVersion % "compile",
"org.json4s" %% "json4s-native" % Versions.Json4s % "compile",
"io.spray" % "spray-routing" % Versions.Spray % "compile",
"com.typesafe.akka" %% "akka-actor" % Versions.Akka % "compile",
"com.typesafe.akka" %% "akka-remote" % Versions.Akka % "compile"
exclude ("io.netty", "netty")
)
//Assemby settings
test in assembly := {}
assemblyShadeRules in assembly := Seq(
ShadeRule.rename("play.api.libs.iteratee.**" -> "shade.play.api.libs.iteratee.#1")
.inLibrary("play" % "play-iteratees_2.10" % "2.1-RC2")
.inLibrary("org.reactivemongo" % "reactivemongo_2.10" % "0.8.1-SNAPSHOT"),
ShadeRule.rename("scala.concurrent.stm.**" -> "shade.scala.concurrent.stm.#1")
.inLibrary("org.scala-stm" % "scala-stm_2.10.0" % "0.6")
.inLibrary("play" % "play-iteratees_2.10" % "2.1-RC2")
)
assemblyMergeStrategy in assembly := {
case "application.conf" => MergeStrategy.concat
case path => MergeStrategy.defaultMergeStrategy(path)
}
//Make assembly a publishable artifact
artifact in (
Compile, assembly) := {
val art = (artifact in (Compile, assembly)).value
art.copy(`classifier` = Some("assembly"))
}
addArtifact(artifact in (Compile, assembly), assembly)
How can I make it work with existing myapp-core-read_2.10-2.2.33.jar jar?
Important note: Scala 2.10 and 2.11 are not binary compatible so you will have to recompile the modules on which you depend
Answer to question:
This seems to be related to how SBT resolves dependencies. When you declare a dependency like so:
"com.mycompany" %% "myapp-core-read" % myappsales.CoreVersion
The %% part in the declaration means that SBT will automatically append the scala version to the library name, so your dependency becomes
group: com.mycompany
artifactId: myapp-core-read_2.11
version: `your version`
It seems that your myapp-core-read was not compiled with scala 2.11 thus SBT can't find the correct version.
You can avoid this by using % and applying the correct suffix manually, so your dependency would become:
"com.mycompany" % "myapp-core-read_2.10" % myappsales.CoreVersion
Having said that, I think scala 2.10 and 2.11 are not binary compatible so you might have to recompile the myapp-core-read module with 2.11.

Why is Scala looking for SNAPSHOT version?

I mentioned json4s-native 3.3.0 but it refering 3.3.0-SNAPSHOT.
In my project there is 3 modules (common, read and write) each have its scala.sbt.
Before change:
"org.json4s" %% "json4s-native" % Versions.Json4s % "compile",
After change:(Changed all the places in 3 modules)
"org.json4s" %% "json4s-native" % "3.3.0" % "compile",
I run:
sbt
clean
update
Error:
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.json4s#json4s-native_2.10;3.3.0-SNAPSHOT: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.json4s:json4s-native_2.10:3.3.0-SNAPSHOT
My Sales buld.sbt is this:
name := "myapp-sales"
organization in ThisBuild := "com.mycompany"
scalaVersion in ThisBuild := "2.11.7"
//crossScalaVersions in Thisq
//Build := List("2.10.5", scalaVersion.value)
//crossPaths := false
scalacOptions in Compile in ThisBuild ++= Seq("-unchecked", "-deprecation", "-encoding", "utf8", "-language:postfixOps", "-language:implicitConVersions")
lazy val common = Project("myapp-sales-common", file("common"))
lazy val read = Project("myapp-sales-read", file("read")).configs(IntegrationTest).settings(Defaults.itSettings: _*).dependsOn(common, write)
lazy val write = Project("myapp-sales-write", file("write")).configs(IntegrationTest).settings(Defaults.itSettings: _*).dependsOn(common)
conflictWarning in ThisBuild := ConflictWarning.disable
parallelExecution in Test in ThisBuild := false
parallelExecution in IntegrationTest in ThisBuild := false
javacOptions in Compile in ThisBuild ++= Seq("-source", "1.6", "-target", "1.6")
//Remove SNAPSHOT check from the release process (for now until Squants gets a release)
releaseProcess := releaseProcess.value.filterNot(_ == ReleaseTransformations.checkSnapshotDependencies)
My read module build.sbt:
libraryDependencies ++= Seq(
"com.mycompany" %% "myapp-core-read" % myappsales.CoreVersion % "compile", // disable using the Scala version in output paths and artifacts,
"com.mycompany" %% "myapp-core-write" % myappsales.CoreVersion % "compile",
"com.mycompany" %% "myapp-registration-common" % myappsales.RegistrationVersion % "compile",
"com.mycompany" %% "myapp-load-common" % myappsales.LoadVersion % "compile",
"com.mycompany" %% "myapp-core-write-test" % myappsales.CoreVersion % "it, test",
"com.mycompany" %% "myapp-core-test" % myappsales.CoreVersion % "it, test"
)
libraryDependencies ++= Seq(
"com.typesafe" % "config" % myappsales.TypeSafeConfigVersion % "compile",
"org.json4s" %% "json4s-native" % Versions.Json4s % "compile",
"io.spray" % "spray-routing" % Versions.Spray % "compile",
"com.typesafe.akka" %% "akka-actor" % Versions.Akka % "compile",
"com.typesafe.akka" %% "akka-remote" % Versions.Akka % "compile"
exclude ("io.netty", "netty")
)
//Assemby settings
test in assembly := {}
assemblyShadeRules in assembly := Seq(
ShadeRule.rename("play.api.libs.iteratee.**" -> "shade.play.api.libs.iteratee.#1")
.inLibrary("play" % "play-iteratees_2.10" % "2.1-RC2")
.inLibrary("org.reactivemongo" % "reactivemongo_2.10" % "0.8.1-SNAPSHOT"),
ShadeRule.rename("scala.concurrent.stm.**" -> "shade.scala.concurrent.stm.#1")
.inLibrary("org.scala-stm" % "scala-stm_2.10.0" % "0.6")
.inLibrary("play" % "play-iteratees_2.10" % "2.1-RC2")
)
assemblyMergeStrategy in assembly := {
case "application.conf" => MergeStrategy.concat
case path => MergeStrategy.defaultMergeStrategy(path)
}
//Make assembly a publishable artifact
artifact in (
Compile, assembly) := {
val art = (artifact in (Compile, assembly)).value
art.copy(`classifier` = Some("assembly"))
}
addArtifact(artifact in (Compile, assembly), assembly)
Why it still looking for SNAPSHOT version?
There's a fairly standard way of investigating this kind of problems. In your plugins.sbt
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.8.2")
Then go sbt dependencyBrowseGraph, this will open your browser and take you to a page where the entire graph of your dependencies including the evicted one can be browsed. Now figure out where the SNAPSHOT is coming from and get rid of it with an exclusion.

How to force Scala to use a different library version?

After adding
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.0" % "test"
to build.sbt, and refreshed the project, I got this msg.
SBT project import
[warn] Multiple dependencies with the same organization/name but
different versions. To avoid conflict, pick one version:
[warn] * org.scala-lang.modules:scala-xml_2.11:(1.0.5, 1.0.4)
Changing the above to
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "3.0.0" % "test"
exclude("org.scala-lang", "scala-reflect")
exclude("org.scala-lang.modules", "scala-xml_2.11")
)
solves the issue. However, instead of excluding scala-xml_2.11 version 1.0.5 from scalatest, I would like to force the scala compiler to use scala-xml_2.11 version 1.0.5 instead of version 1.0.4. (I researched the versions at https://mvnrepository.com.) Thus I tried substituting scalaVersion := "2.11.8" for
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % "2.11.8",
"org.scala-lang.modules" % "scala-xml_2.11" % "1.0.5"
)
This however results in
SBT project import
[warn] Binary version (2.11) for dependency
org.scala-lang#scala-reflect;2.11.8
[warn] in default#myproject$sources_javadoc_2.10;1.0 differs
from Scala binary version in project (2.10).
[warn] Binary version (2.11) for dependency
org.scala-lang#scala-library;2.11.8
[warn] in default#myproject$sources_javadoc_2.10;1.0 differs
from Scala binary version in project (2.10).
[warn] Multiple dependencies with the same organization/name but
different versions. To avoid conflict, pick one version:
[warn] * org.scala-lang:scala-library:(2.11.8, 2.10.4)
[warn] * org.scala-lang:scala-reflect:(2.11.8, 2.10.4)
[warn] [FAILED ]
com.artima.supersafe#supersafe_2.10.4;1.1.0!supersafe_2.10.4.jar(src):
(0ms)
[warn] ==== local: tried
[warn]
/home/user/.ivy2/local/com.artima.supersafe/supersafe_2.10.4/1.1.0/srcs/supersafe_2.10.4-sources.jar
[warn] ==== activator-local: tried [warn] /Development/Activator/activator-dist-1.3.10/repository/com.artima.supersafe/supersafe_2.1...
(show balloon)
What am I supposed to do?
Edit: What else I tried and did not work:
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % scalaVersion.value,
"org.scala-lang.modules" %% "scala-xml" % "1.0.5"
)
// ScalaTest
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.0" % "test"
[warn] Multiple dependencies with the same organization/name but
different versions. To avoid conflict, pick one version: [warn] *
org.scala-lang.modules:scala-xml_2.11:(1.0.5, 1.0.4)
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % scalaVersion.value,
"org.scala-lang.modules" %% "scala-xml_2.11" % "1.0.5"
)
// ScalaTest
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.0" % "test"
Error:Error while importing SBT project:...[info]
Resolving org.scala-sbt#run;0.13.8 ... [info] Resolving
org.scala-sbt#task-system;0.13.8 ... [info] Resolving
org.scala-sbt#tasks;0.13.8 ... [info] Resolving
org.scala-sbt#tracking;0.13.8 ... [info] Resolving
org.scala-sbt#cache;0.13.8 ... [info] Resolving
org.scala-sbt#testing;0.13.8 ... [info] Resolving
org.scala-sbt#test-agent;0.13.8 ... [info] Resolving
org.scala-sbt#test-interface;1.0 ... [info] Resolving
org.scala-sbt#main-settings;0.13.8 ... [info] Resolving
org.scala-sbt#apply-macro;0.13.8 ... [info] Resolving
org.scala-sbt#command;0.13.8 ... [info] Resolving
org.scala-sbt#logic;0.13.8 ... [info] Resolving
org.scala-sbt#precompiled-2_8_2;0.13.8 ... [info] Resolving
org.scala-sbt#precompiled-2_9_2;0.13.8 ... [info] Resolving
org.scala-sbt#precompiled-2_9_3;0.13.8 ... [trace] Stack trace
suppressed: run 'last *:update' for the full output. [trace] Stack
trace suppressed: run 'last :ssExtractDependencies' for the full
output. [error] (:update) sbt.ResolveException: unresolved
dependency: org.scala-lang.modules#scala-xml_2.11_2.11;1.0.5: not
found [error] (*:ssExtractDependencies) sbt.ResolveException:
unresolved dependency:
org.scala-lang.modules#scala-xml_2.11_2.11;1.0.5: not found [error]
Total time: 4 s, completed 01.10.2016 17:46:55
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % scalaVersion.value,
"org.scala-lang.modules" %% "scala-xml" % "1.0.5"
)
// ScalaTest
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "3.0.0" % "test"
exclude("org.scala-lang", "scala-reflect")
exclude("org.scala-lang.modules", "scala-xml")
)
[warn] Multiple dependencies with the same organization/name but
different versions. To avoid conflict, pick one version: [warn] *
org.scala-lang.modules:scala-xml_2.11:(1.0.5, 1.0.4)
My build.sbt
name := "MyProject"
version := "0.1.0"
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % scalaVersion.value,
"org.scala-lang.modules" %% "scala-xml" % "1.0.5"
)
// ScalaTest
//libraryDependencies += "org.scalactic" %% "scalactic" % "3.0.0"
//libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.0" % "test"
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "3.0.0" % "test"
exclude("org.scala-lang", "scala-reflect")
exclude("org.scala-lang.modules", "scala-xml_2.11")
)
The way to exclude specific transitive dependency is this:
Run sbt evicted to figure out which of the project dependencies is pulling in outdated library, let's assume the problematic library is: com.typesafe.slick.
Add the following exclude (the parentheses are important):
("com.typesafe" %% "slick" % "3.1.1").exclude("org.scala-lang.modules", "scala-xml_2.11")
Add this normally as you would be listing dependencies.
This will prevent sbt from including any version of scala-xml that was coming in as a transitive dependency of Slick.
Don't substitute; you need both scalaVersion and libraryDependencies.
Though use
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % scalaVersion.value,
"org.scala-lang.modules" %% "scala-xml" % "1.0.5"
)
to avoid bugs when you eventually change scalaVersion.
By removing scalaVersion you get default scalaVersion := "2.10.4" (with your version/settings of SBT, at least) but your libraryDependencies still require 2.11.

Why does build break with Akka 2.4-SNAPSHOT missing?

I have this in my build.sbt:
libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.4-SNAPSHOT"
libraryDependencies += "com.typesafe.akka" %% "akka-slf4j" % "2.4-SNAPSHOT"
When I compile it, it says:
::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.typesafe.akka#akka-actor_2.10;2.4-SNAPSHOT: not found
[warn] :: com.typesafe.akka#akka-slf4j_2.10;2.4-SNAPSHOT: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
Why is that? Even in here http://doc.akka.io/docs/akka/snapshot/intro/getting-started.html it says it must exist.
In its directory
sbt-version
[info] 0.12.1
In my home user's directory:
$ sbt sbt-version
[info] Set current project to alex (in build file:/Users/alex/)
[info] 0.13.7
Even after this sbt.version=0.12.4, the error remains.
// name := "my_name123"
version := "0.1"
//scalaVersion := "2.10.4"
resolvers += "Akka Snapshot Repository" at "http://repo.akka.io/snapshots/"
resolvers += "Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/"
//resolvers += "typesafe-snapshots" at "http://repo.typesafe.com/typesafe/snapshots/"
scalacOptions in ThisBuild ++= Seq("-unchecked", "-deprecation", "-optimise") // , "-feature"
libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.4-SNAPSHOT"
libraryDependencies += "com.typesafe.akka" %% "akka-slf4j" % "2.4-SNAPSHOT"
unmanagedResourceDirectories in Compile <+= baseDirectory( _ / "src" / "main" / "scala" )
unmanagedResourceDirectories in Compile <+= baseDirectory( _ / "src" / "main" / "java" )
libraryDependencies += "org.scalatest" %% "scalatest" % "2.2.0" % "test"
You must also add the Akka snapshots resolver to your build.sbt:
resolvers += "Akka Snapshot Repository" at "http://repo.akka.io/snapshots/"