I'm trying to create a new Spark application based on Hortonworks tutorial.But the build.sbt keeps throwing the below error when i try to import the necessary modules.
I have SCALA - 2.11.4 AND SBT 0.13 installed.Any help ??
Error:Error while importing SBT project:<br/>...<br/><pre>[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.scala-lang:scala-library:2.11.4 ((sbt.Classpaths) Defaults.scala#L1340)
[warn] +- sparktutorialscala:sparktutorialscala_2.11:1.0
[warn] org.apache.spark:spark-core_2.11:2.1.0 (C:\Users\USERNAME\IdeaProjects\HelloWorld\build.sbt#L7-13)
[warn] +- sparktutorialscala:sparktutorialscala_2.11:1.0
[warn] org.scala-lang:scala-compiler:2.11.4
[warn] +- sparktutorialscala:sparktutorialscala_2.11:1.0
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.11.4: not found
[error] unresolved dependency: org.apache.spark#spark-core_2.11;2.1.0: not found
[error] unresolved dependency: org.scala-lang#scala-compiler;2.11.4: not found
[error] (*:update) sbt.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.11.4: not found
[error] unresolved dependency: org.apache.spark#spark-core_2.11;2.1.0: not found
[error] unresolved dependency: org.scala-lang#scala-compiler;2.11.4: not found
[error] Total time: 66 s, completed 17-Oct-2017 10:49:23
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true</pre><br/>See complete
build.sbt
name := "sparkTutorialScala"
version := "1.0"
scalaVersion := "2.11.4"
libraryDependencies ++= {
val sparkVer = "2.1.0"
Seq(
"org.apache.spark" %% "spark-core" % sparkVer % "provided" withSources()
)
}
Updated Error message :
[error] unresolved dependency: org.apache.spark#spark-core_2.11;2.2.0: public: unable to get resource for org/apache/spark#spark-core_2.11;2.2.0: res=https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/2.2.0/spark-core_2.11-2.2.0.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] unresolved dependency: org.scala-lang#scala-compiler;2.11.8: public: unable to get resource for org/scala-lang#scala-compiler;2.11.8: res=https://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] (*:ssExtractDependencies) sbt.librarymanagement.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.11.8: public: unable to get resource for org/scala-lang#scala-library;2.11.8: res=https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.11.8/scala-library-2.11.8.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] unresolved dependency: org.apache.spark#spark-core_2.11;2.2.0: public: unable to get resource for org/apache/spark#spark-core_2.11;2.2.0: res=https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/2.2.0/spark-core_2.11-2.2.0.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] unresolved dependency: org.scala-lang#scala-compiler;2.11.8: public: unable to get resource for org/scala-lang#scala-compiler;2.11.8: res=https://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] Total time: 9 s, completed 19-Oct-2017 15:11:47
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true</pre><br/>See complete log in <a href="file:/C:/Users/
Well,SBT 1.0 was not compatible with the version of IDEAJ plugin or spark that I was using.Installing SBT 0.13 instead of the latest one resolved the issue.
Related
I am getting the below error on compiling my Twitter streamin Scala program, can some one please suggest to resolve this issue.
[warn] Note: Some unresolved dependencies have extra attributes. Check that these dependencies exist with the requested attributes.
[warn] com.eed3si9n:sbt-assembly:0.14.3 (sbtVersion=0.11.3, scalaVersion=2.11.8)
[warn] com.typesafe.sbteclipse:sbteclipse-plugin:2.2.0 (sbtVersion=0.11.3, scalaVersion=2.11.8)
[warn] com.github.mpeltonen:sbt-idea:1.5.1 (sbtVersion=0.11.3, scalaVersion=2.11.8)
[warn]
[error] {file:/home/muralee1857/scala/workspace/Tutorial/}default-109f4d/*:update: sbt.ResolveException: unresolved dependency: com.eed3si9n#sbt-assembly;0.14.3: not found
[error] unresolved dependency: com.typesafe.sbteclipse#sbteclipse-plugin;2.2.0: not found
[error] unresolved dependency: com.github.mpeltonen#sbt-idea;1.5.1: not found
[error] download failed: org.eclipse.jetty.orbit#javax.transaction;1.1.1.v201105210645!javax.transaction.orbit
[error] download failed: org.eclipse.jetty.orbit#javax.servlet;3.0.0.v201112011016!javax.servlet.orbit
[error] download failed: org.eclipse.jetty.orbit#javax.mail.glassfish;1.4.1.v201005082020!javax.mail.glassfish.orbit
[error] download failed: org.eclipse.jetty.orbit#javax.activation;1.1.0.v201105071233!javax.activation.orbit
[error] Total time: 28 s, completed 21 Jun, 2017 2:07:50 AM
muralee1857#muralee1857-Lenovo-ideapad-300-15ISK:~/scala/workspace/Tutorial$ ^C
muralee1857#muralee1857-Lenovo-ideapad-300-15ISK:~/scala/workspace/Tutorial$
I'm trying to use this code in the link bellow : https://github.com/dmiszkiewicz/sendgrid-scala but i can't download dependency org.miszkiewicz#sendgrid-scala_2.12;0.2-SNAPSHOT, i get this error:
::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.miszkiewicz#sendgrid-scala_2.12;0.2-SNAPSHOT: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.miszkiewicz:sendgrid-scala_2.12:0.2-SNAPSHOT (/Users/aicha/Projets/data-providers/project/FruitionDataProvidersBuild.scala#L62)
[warn] +- com.fruitionsciences:sendgrid-mailer_2.12:0.1
[trace] Stack trace suppressed: run 'last sendgrid-mailer/*:update' for the full output.
[trace] Stack trace suppressed: run 'last sendgrid-mailer/*:ssExtractDependencies' for the full output.
[error] (sendgrid-mailer/*:update) sbt.ResolveException: unresolved dependency: org.miszkiewicz#sendgrid-scala_2.12;0.2-SNAPSHOT: not found
[error] (sendgrid-mailer/*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.miszkiewicz#sendgrid-scala_2.12;0.2-SNAPSHOT: not found
i'm working with scala 2.12, Does anybody have an idea
You should use Scala 2.11 to use this dependency from Sonatype repo. As you may see here, there is build of sendgrid only for Scala 2.11 in Sonatype repo, not for Scala 2.12.
Or maybe, it will be better for you to just use sendgrid-java.
Unable to resolve dependencies while trying to use latest Apache Spark 2.0.2 and Scala 2.11.7 together
I need to use the latest version of spark since it has support for SparkSession. Below configuration working fine for spark 1.6.2. But I want for spark 2.0.*
This is my dependency configuration
scalaVersion := "2.11.7"
val sparkVersion = "2.0.2"
libraryDependencies ++= Seq(
ws ,
"org.apache.spark" %% "spark-core" % sparkVersion,
"org.apache.spark" %% "spark-sql" % sparkVersion,
"org.apache.spark" %% "spark-streaming" % sparkVersion,
"org.mongodb.spark" % "mongo-spark-connector_2.11" % "2.0.0",
specs2 % Test
)
And this is error message:
Error:Error while importing SBT project:
[info] Resolving org.ow2.asm#asm-tree;4.1 ...
[info] Resolving org.ow2.asm#asm-analysis;4.1 ...
[info] Resolving org.ow2.asm#asm-util;4.1 ...
[info] Resolving jline#jline;2.12.1 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.glassfish.hk2#hk2-utils;2.22.2: not found
[warn] :: org.glassfish.hk2#hk2-locator;2.22.2: not found
[warn] :: javax.validation#validation-api;${javax.validation.version}: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: org.glassfish.hk2#hk2-utils;2.22.2: not found
[error] unresolved dependency: org.glassfish.hk2#hk2-locator;2.22.2: not found
[error] unresolved dependency: javax.validation#validation-api;${javax.validation.version}: not found
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.glassfish.hk2#hk2-utils;2.22.2: not found
[error] unresolved dependency: org.glassfish.hk2#hk2-locator;2.22.2: not found
[error] unresolved dependency: javax.validation#validation-api;${javax.validation.version}: not found
[error] Total time: 81 s, completed 3 Dec, 2016 8:51:18 PM
Correct me if anything wrong in applying dependencies. Thank you
There's probably an error in the dependency publication chain. One option would be to exclude the broken dependencies - add exclude("org.glassfish.hk2", "hk2-utils") exclude("org.glassfish.hk2", "hk2-locator") exclude("javax.validation", "validation-api") to whatever Spark dependency that's bringing these in - and explicitly add the working versions of those three dependencies.
Also, I don't see version 2.22.2 for org.glassfish.hk2#hk2-utils or org.glassfish.hk2#hk2-locator.
Excluding at the org level (as suggested in How to exclude commons logging dependency of spring with ivy? ) helped me.
I had to use something like this :
<dependency org="org.glassfish.jersey.core" name="jersey-client" rev="2.3" conf="binaries->default">
<exclude org="org.glassfish.hk2"/>
</dependency>
<dependency org="org.glassfish.hk2" name="hk2-utils" rev="2.2.0-b14" conf="binaries->default"/>
<dependency org="org.glassfish.hk2" name="hk2-locator" rev="2.2.0-b14" conf="binaries->default"/>
<dependency org="org.glassfish.hk2" name="hk2-api" rev="2.2.0-b14" conf="binaries->default"/>
I am getting the below error while building my Play! framework Scala project in Intellij IDEA 15, the following is the error:
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.streamsend#pillar_2.10;1.0.3: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: com.streamsend#pillar_2.10;1.0.3: not found
Please help.
Try to use a newer version:
Replace:
"com.streamsend" % "pillar_2.10" % "1.0.3"
With:
"com.chrisomeara" % "pillar_2.10" % "2.0.1" (It's the same package but rebranded)
I tried setting up a new Play Framework on Scala and MongoDB. I tried all documentation available online, but each time I tried running the project on the terminal I get some obscure error. I have tried to make sense out of it, but all effort prove abortive. I am new Scala and Play Framework developer. Below is the response am getting from the terminal. Thanks
http://repo.typesafe.com/typesafe/releases/reactive_mongo_plugin/reactive_mongo_plugin_2.10/0.0.43/reactive_mongo_plugin_2.10-0.0.43.pom
[info] Resolving com.typesafe.trace#trace-sigar-libs;0.1.1 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: reactive_mongo_plugin#reactive_mongo_plugin_2.10;0.0.43: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: reactive_mongo_plugin#reactive_mongo_plugin_2.10;0.0.43: not found
[error] Total time: 36 s, completed Apr 4, 2015 10:04:19 AM
I guess you try to use the Play plugin, but the dependency doesn't correspond to this lib whose latest artifact is "org.reactivemongo" %% "play2-reactivemongo" % "0.10.5.0.akka23": http://search.maven.org/#search%7Cga%7C1%7Creactivemongo