sbt.ResolveException unresolved dependency - scala

Question

I am trying to work on Twitter stream sample project. i am facing problem while defining sbt.
my build.sbt
name := "Tutorial"
version := "0.1.0"
scalaVersion := "2.11.8"
retrieveManaged := true
libraryDependencies ++= Seq(
"org.apache.spark" % "spark-core" % "2.11.0",
"org.apache.spark" % "spark-streaming" % "1.1.0",
"org.apache.spark" % "spark-streaming-twitter" % "1.1.0"
)
Error Log:
[warn] Note: Some unresolved dependencies have extra attributes. Check that these dependencies exist with the requested attributes.
[warn] com.eed3si9n:sbt-assembly:0.9.2 (sbtVersion=0.11.3, scalaVersion=2.11.8)
[warn] com.typesafe.sbteclipse:sbteclipse-plugin:2.2.0 (sbtVersion=0.11.3, scalaVersion=2.11.8)
[warn] com.github.mpeltonen:sbt-idea:1.5.1 (sbtVersion=0.11.3, scalaVersion=2.11.8)
[warn]
[error] {file:/home/muralee1857/scala/workspace/Tutorial/}default-109f4d/*:update: sbt.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.11.8;1.5.1: not found
[error] unresolved dependency: com.eed3si9n#sbt-assembly;0.9.2: not found
[error] unresolved dependency: com.typesafe.sbteclipse#sbteclipse-plugin;2.2.0: not found
[error] unresolved dependency: com.github.mpeltonen#sbt-idea;1.5.1: not found

Answer 1

I would suggest you to define explicitly the packaged versions to the dependency as
libraryDependencies ++= Seq(
"org.apache.spark" % "spark-core_2.10" % "1.1.0",
"org.apache.spark" % "spark-streaming_2.10" % "1.1.0" % "provided",
"org.apache.spark" % "spark-streaming-twitter_2.10" % "1.1.0"
)
You can use %% without defining the packaged version but that will try to download the package with scala version in your system. And sometimes sbt will not find the scala version packaged packages which will create dependency issues.

Answer 2

This should work. note that I am using %% method instead of % here so that right version of the spark library (scala version 2.11) is selected here. ensure you apply the same %% function to others plugins like sbt-assembly, sbt-assembly etc.
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.11.0",
"org.apache.spark" %% "spark-streaming" % "1.1.0",
"org.apache.spark" %% "spark-streaming-twitter" % "1.1.0"
)