Google Ads API dependencies with SBT - scala

This is my Build.scala file. I am trying to import some com.google.api-ads library dependencies into my project and I keep getting error stating "Unresolved dependencies path". I can see these libraries being there in the maven repo though.
I have tried out various versions and also tried importing the google-http-client dependency which is a transitive dependency for the ads-lib and dfp-axis libraries.
object Build extends Build {
val commonSettings = Seq(
version := "1.0.0",
organization := "com.collective",
scalaVersion := "2.11.2",
scalacOptions ++= List(
"-encoding", "UTF-8",
"-target:jvm-1.7",
"-feature",
"-unchecked",
"-deprecation",
"-Xlint",
"-Xfatal-warnings"
)
)
val akkaV = "2.3.6"
val sprayV = "1.3.2"
val adsLibVersion = "1.30.0"
lazy val segmentFetcher = Project("segment-fetcher", file("."))
.settings(commonSettings: _*)
.settings(
name := "Segment Fetcher",
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaV,
"com.typesafe.akka" %% "akka-testkit" % akkaV % "test",
"org.specs2" %% "specs2-core" % "2.3.11" % "test",
"io.spray" %% "spray-can" % sprayV,
"io.spray" %% "spray-client" % sprayV,
"io.spray" %% "spray-httpx" % sprayV,
"io.spray" %% "spray-routing" % sprayV,
"io.spray" %% "spray-testkit" % sprayV % "test"
"com.google.api-ads" %% "ads-lib" % adsLibVersion,
"com.google.api-ads" %% "ads-lib-axis" % adsLibVersion,
"com.google.api-ads" %% "dfp-axis" % adsLibVersion
)
)
}
When I run sbt compile, I get the unresolved dependencies error for the com.google.api-ads dependencies
> compile
[info] Updating {file:/Users/anand/IntellijProjects/segments-fetcher/}segment-fetcher...
[info] Resolving com.google.api-ads#ads-lib_2.11;1.30.0 ...
[warn] module not found: com.google.api-ads#ads-lib_2.11;1.30.0
[warn] ==== local: tried
[warn] /Users/anand/.ivy2/local/com.google.api-ads/ads-lib_2.11/1.30.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/google/api-ads/ads-lib_2.11/1.30.0/ads-lib_2.11-1.30.0.pom
[info] Resolving com.google.api-ads#ads-lib-axis_2.11;1.30.0 ...
[warn] module not found: com.google.api-ads#ads-lib-axis_2.11;1.30.0
[warn] ==== local: tried
[warn] /Users/anand/.ivy2/local/com.google.api-ads/ads-lib-axis_2.11/1.30.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/google/api-ads/ads-lib-axis_2.11/1.30.0/ads-lib-axis_2.11-1.30.0.pom
[info] Resolving com.google.api-ads#dfp-axis_2.11;1.30.0 ...
[warn] module not found: com.google.api-ads#dfp-axis_2.11;1.30.0
[warn] ==== local: tried
[warn] /Users/anand/.ivy2/local/com.google.api-ads/dfp-axis_2.11/1.30.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/google/api-ads/dfp-axis_2.11/1.30.0/dfp-axis_2.11- 1.30.0.pom
[info] Resolving jline#jline;2.12 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.google.api-ads#ads-lib_2.11;1.30.0: not found
[warn] :: com.google.api-ads#ads-lib-axis_2.11;1.30.0: not found
[warn] :: com.google.api-ads#dfp-axis_2.11;1.30.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] com.google.api-ads:ads-lib_2.11:1.30.0 (/Users/anand/IntellijProjects/segments- fetcher/project/Build.scala#L29)
[warn] +- com.collective:segment-fetcher_2.11:1.0.0
[warn] com.google.api-ads:ads-lib-axis_2.11:1.30.0 (/Users/anand/IntellijProjects/segments-fetcher/project/Build.scala#L29)
[warn] +- com.collective:segment-fetcher_2.11:1.0.0
[warn] com.google.api-ads:dfp-axis_2.11:1.30.0 (/Users/anand/IntellijProjects/segments-fetcher/project/Build.scala#L29)
[warn] +- com.collective:segment-fetcher_2.11:1.0.0
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: com.google.api-ads#ads-lib_2.11;1.30.0: not found
[error] unresolved dependency: com.google.api-ads#ads-lib-axis_2.11;1.30.0: not found
[error] unresolved dependency: com.google.api-ads#dfp-axis_2.11;1.30.0: not found
[error] Total time: 5 s, completed 29 Oct, 2014 3:31:25 PM

%% is a special SBT operator that suffixes Scala version to artifact that is to be fetched from MAVEN. Try to replace %% with % for Java MAVEN dependencies (that Google is most probable are).
You might noticed that SBT tries to fetch artifact
com.google.api-ads#ads-lib_2.11;1.30.0
but should instead fetch (without 2.11 suffix)
com.google.api-ads#ads-lib;1.30.0

Related

Which version of Sbt and Sbt assembly to use for Spack 2.4.5 and Scala 2.11?

Currently, in AWS EMR Cluster, I am using Spark v2.4.5 which comes with Scala v2.11. So in my project, I want to use Scala v2.11 and corresponding SBT and Sbt-Assembly versions. But I am getting one or the other Version conflicts with all the permutations available on various blogs and Stackoverflow answers.
Here is my dependency files which throws error:
build.sbt
name := "user-view-services"
version := "0.1"
scalaVersion := "2.11.12"
autoScalaLibrary := false
// Major Libraries version
val sparkVersion = "2.4.5"
val kafkaVersion = "2.5.0"
val akkaVersion = "2.6.4"
val confluentRepo = "confluent-repo" at "https://packages.confluent.io/maven/"
lazy val commonDependencies = Seq(
"junit" % "junit" % "4.13" % Test,
"org.mockito" % "mockito-all" % "1.10.19" % Test,
"com.novocode" % "junit-interface" % "0.11" % Test,
"ch.qos.logback" % "logback-classic" % "1.2.3",
"com.typesafe" % "config" % "1.4.0",
"com.opencsv" % "opencsv" % "5.1",
"org.apache.avro" % "avro" % "1.9.2",
"commons-beanutils" % "commons-beanutils" % "1.9.4",
"commons-beanutils" % "commons-beanutils-bean-collections" % "1.8.3",
"org.drools" % "drools-compiler" % "7.41.0.Final",
"com.typesafe.scala-logging" % "scala-logging_2.11" % "3.5.0"
)
lazy val mongoDBDependencies = Seq(
"org.mongodb" % "mongodb-driver-sync" % "4.0.2"
)
lazy val akkaDependencies = Seq(
"com.typesafe.akka" %% "akka-actor-typed" % akkaVersion
)
lazy val kafkaDependencies = Seq(
"org.apache.kafka" % "kafka-clients" % kafkaVersion,
"org.apache.kafka" % "kafka-streams" % kafkaVersion,
"io.confluent" % "kafka-avro-serializer" % "5.4.1"
)
lazy val sparkDependencies = Seq(
"org.apache.spark" %% "spark-core" % sparkVersion,
"org.apache.spark" %% "spark-sql" % sparkVersion,
"org.apache.spark" %% "spark-streaming" % sparkVersion
)
lazy val rulemodule = (project in file("rule-module"))
.enablePlugins(JavaAppPackaging)
.settings(
name := "rule-module",
version := "1.0",
libraryDependencies ++= commonDependencies
)
/**
* to create uberjar - run command
* 1) sbt "sparkjobs / clean"
* 2) sbt "sparkjobs / assembly"
*/
lazy val sparkjobs = (project in file("user-enrich-batch"))
.dependsOn(rulemodule)
.settings(
name := "user-enrich-batch",
version := "1.0",
resolvers ++= Seq(
confluentRepo
),
libraryDependencies ++= commonDependencies,
libraryDependencies ++= sparkDependencies,
libraryDependencies ++= kafkaDependencies,
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.6.7",
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case x => MergeStrategy.first
},
mainClass in assembly := some("com.analytics.user.etl.ETLProcessor"),
assemblyJarName := "user-view-batch-assembly.jar"
)
lazy val userservice = (project in file("user-enrich-service"))
.dependsOn(rulemodule)
.enablePlugins(JavaAppPackaging)
.settings(
name := "user-enrich-service",
version := "1.0",
resolvers ++= Seq(
confluentRepo
),
libraryDependencies ++= commonDependencies,
libraryDependencies ++= kafkaDependencies,
libraryDependencies ++= akkaDependencies,
libraryDependencies ++= mongoDBDependencies
)
build.properties
sbt.version = 0.13.17
plugins.sbt
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.3.4")
scalaVersion := "2.11.12"
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.6")
addSbtPlugin("com.cavorite" % "sbt-avro-1-9" % "1.1.7")
assembly.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.6")
Error:
[info] Loading global plugins from C:\Users\smishra5\.sbt\0.13\plugins
[info] Loading project definition from C:\dataanalytics\Recsys\project
[info] Updating {file:/C:/dataanalytics/Recsys/project/}recsys-build...
[info] Resolving com.eed3si9n#sbt-assembly;0.14.6 ...
[warn] module not found: com.eed3si9n#sbt-assembly;0.14.6
[warn] ==== typesafe-ivy-releases: tried
[warn] https://repo.typesafe.com/typesafe/ivy-releases/com.eed3si9n/sbt-assembly/scala_2.11/sbt_0.13/0.14.6/ivys/ivy.xml
[warn] ==== sbt-plugin-releases: tried
[warn] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.eed3si9n/sbt-assembly/scala_2.11/sbt_0.13/0.14.6/ivys/ivy.xml
[warn] ==== local: tried
[warn] C:\Users\smishra5\.ivy2\local\com.eed3si9n\sbt-assembly\scala_2.11\sbt_0.13\0.14.6\ivys\ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/eed3si9n/sbt-assembly_2.11_0.13/0.14.6/sbt-assembly-0.14.6.pom
[warn] ==== local-preloaded-ivy: tried
[warn] C:\Users\smishra5\.sbt\preloaded\com.eed3si9n\sbt-assembly\0.14.6\ivys\ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:/C:/Users/smishra5/.sbt/preloaded/com/eed3si9n/sbt-assembly_2.11_0.13/0.14.6/sbt-assembly-0.14.6.pom
[info] Resolving com.typesafe.sbt#sbt-native-packager;1.3.4 ...
[warn] module not found: com.typesafe.sbt#sbt-native-packager;1.3.4
[warn] ==== typesafe-ivy-releases: tried
[warn] https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-native-packager/scala_2.11/sbt_0.13/1.3.4/ivys/ivy.xml
[warn] ==== sbt-plugin-releases: tried
[warn] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-native-packager/scala_2.11/sbt_0.13/1.3.4/ivys/ivy.xml
[warn] ==== local: tried
[warn] C:\Users\smishra5\.ivy2\local\com.typesafe.sbt\sbt-native-packager\scala_2.11\sbt_0.13\1.3.4\ivys\ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-native-packager_2.11_0.13/1.3.4/sbt-native-packager-1.3.4.pom
[warn] ==== local-preloaded-ivy: tried
[warn] C:\Users\smishra5\.sbt\preloaded\com.typesafe.sbt\sbt-native-packager\1.3.4\ivys\ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:/C:/Users/smishra5/.sbt/preloaded/com/typesafe/sbt/sbt-native-packager_2.11_0.13/1.3.4/sbt-native-packager-1.3.4.pom
[info] Resolving com.cavorite#sbt-avro-1-9;1.1.7 ...
[warn] module not found: com.cavorite#sbt-avro-1-9;1.1.7
[warn] ==== typesafe-ivy-releases: tried
[warn] https://repo.typesafe.com/typesafe/ivy-releases/com.cavorite/sbt-avro-1-9/scala_2.11/sbt_0.13/1.1.7/ivys/ivy.xml
[warn] ==== sbt-plugin-releases: tried
[warn] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.cavorite/sbt-avro-1-9/scala_2.11/sbt_0.13/1.1.7/ivys/ivy.xml
[warn] ==== local: tried
[warn] C:\Users\smishra5\.ivy2\local\com.cavorite\sbt-avro-1-9\scala_2.11\sbt_0.13\1.1.7\ivys\ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/cavorite/sbt-avro-1-9_2.11_0.13/1.1.7/sbt-avro-1-9-1.1.7.pom
[warn] ==== local-preloaded-ivy: tried
[warn] C:\Users\smishra5\.sbt\preloaded\com.cavorite\sbt-avro-1-9\1.1.7\ivys\ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:/C:/Users/smishra5/.sbt/preloaded/com/cavorite/sbt-avro-1-9_2.11_0.13/1.1.7/sbt-avro-1-9-1.1.7.pom
[info] Resolving jline#jline;2.14.3 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.eed3si9n#sbt-assembly;0.14.6: not found
[warn] :: com.typesafe.sbt#sbt-native-packager;1.3.4: not found
[warn] :: com.cavorite#sbt-avro-1-9;1.1.7: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Some unresolved dependencies have extra attributes. Check that these dependencies exist with the requested attributes.
[warn] com.eed3si9n:sbt-assembly:0.14.6 (scalaVersion=2.11, sbtVersion=0.13)
[warn] com.typesafe.sbt:sbt-native-packager:1.3.4 (scalaVersion=2.11, sbtVersion=0.13)
[warn] com.cavorite:sbt-avro-1-9:1.1.7 (scalaVersion=2.11, sbtVersion=0.13)
[warn]
[warn] Note: Unresolved dependencies path:
[warn] com.eed3si9n:sbt-assembly:0.14.6 (scalaVersion=2.11, sbtVersion=0.13) (C:\dataanalytics\Recsys\project\plugins.sbt#L5-6)
[warn] +- default:recsys-build:0.1-SNAPSHOT (scalaVersion=2.11, sbtVersion=0.13)
[warn] com.typesafe.sbt:sbt-native-packager:1.3.4 (scalaVersion=2.11, sbtVersion=0.13) (C:\dataanalytics\Recsys\project\plugins.sbt#L1-2)
[warn] +- default:recsys-build:0.1-SNAPSHOT (scalaVersion=2.11, sbtVersion=0.13)
[warn] com.cavorite:sbt-avro-1-9:1.1.7 (scalaVersion=2.11, sbtVersion=0.13) (C:\dataanalytics\Recsys\project\plugins.sbt#L7-8)
[warn] +- default:recsys-build:0.1-SNAPSHOT (scalaVersion=2.11, sbtVersion=0.13)
sbt.ResolveException: unresolved dependency: com.eed3si9n#sbt-assembly;0.14.6: not found
unresolved dependency: com.typesafe.sbt#sbt-native-packager;1.3.4: not found
unresolved dependency: com.cavorite#sbt-avro-1-9;1.1.7: not found
at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:320)
at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:191)
at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:168)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156)
at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:133)
at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57)
at sbt.IvySbt$$anon$4.call(Ivy.scala:65)
at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
at xsbt.boot.Using$.withResource(Using.scala:10)
at xsbt.boot.Using$.apply(Using.scala:9)
at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
at xsbt.boot.Locks$.apply0(Locks.scala:31)
at xsbt.boot.Locks$.apply(Locks.scala:28)
at sbt.IvySbt.withDefaultLogger(Ivy.scala:65)
at sbt.IvySbt.withIvy(Ivy.scala:128)
at sbt.IvySbt.withIvy(Ivy.scala:125)
at sbt.IvySbt$Module.withModule(Ivy.scala:156)
at sbt.IvyActions$.updateEither(IvyActions.scala:168)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1555)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1551)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$122.apply(Defaults.scala:1586)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$122.apply(Defaults.scala:1584)
at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:37)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1589)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1583)
at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:60)
at sbt.Classpaths$.cachedUpdate(Defaults.scala:1606)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1533)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1485)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[error] (*:update) sbt.ResolveException: unresolved dependency: com.eed3si9n#sbt-assembly;0.14.6: not found
[error] unresolved dependency: com.typesafe.sbt#sbt-native-packager;1.3.4: not found
[error] unresolved dependency: com.cavorite#sbt-avro-1-9;1.1.7: not found
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? q
Reference: https://github.com/sbt/sbt-assembly/issues/311
It seems that you're mixing up the Scala version used by SBT and Scala version used in your project.
If you need to build project with Scala 2.11, it's just enough to specify in build.sbt
scalaVersion := "2.11.12"
You have to remove scalaVersion := "2.11.12" from plugins.sbt in the project folder as SBT is pre-built with specific Scala version and you cannot change it. For example, 0.13.17 is built with Scala 2.10 and SBT 1.x.x with Scala 2.12.
You're free to use any SBT version you like but I'd suggest to use the latest 1.3.13 and latest versions of corresponding plugins.

SBT unresolved dependecie with Akka

I'm quite new to Scala, Akka and SBT, so this issue is giving me some headache! I'm working with the book Akka in Action, where the author provides examples on Github:
https://github.com/RayRoestenburg/akka-in-action
On a clean clone of the repository, i was trying to have a look at the example in chapter-cluster.
As described in the book, the first thing to do is to start a seed node on the local machine with:
sbt -DHOST=127.0.0.1 -DPORT=2551
I was trying to install another Scala version, run it with root privileges, but without success. In my understanding SBT should take care of all the versioning stuff and download the specified packages.
Error messages goes like that:
[info] Loading global plugins from /home/sfink/.sbt/0.13/plugins
[info] Loading project definition from /home/sfink/IdeaProjects/akka-in-action/chapter-cluster/project
[info] Updating {file:/home/sfink/IdeaProjects/akka-in-action/chapter-cluster/project/}chapter-cluster-build...
[info] Resolving org.scala-sbt#compiler-interface;0.13.1 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.scala-lang#scala-library;2.10.4: configuration not found in org.scala-lang#scala-library;2.10.4: 'master(compile)'. Missing configuration: 'compile'. It was required from org.scalactic#scalactic_2.10;2.2.1 compile
[warn] :: org.scala-lang#scala-reflect;2.10.4: configuration not found in org.scala-lang#scala-reflect;2.10.4: 'master(compile)'. Missing configuration: 'compile'. It was required from org.scalactic#scalactic_2.10;2.2.1 compile
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
sbt.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.10.4: configuration not found in org.scala-lang#scala-library;2.10.4: 'master(compile)'. Missing configuration: 'compile'. It was required from org.scalactic#scalactic_2.10;2.2.1 compile
unresolved dependency: org.scala-lang#scala-reflect;2.10.4: configuration not found in org.scala-lang#scala-reflect;2.10.4: 'master(compile)'. Missing configuration: 'compile'. It was required from org.scalactic#scalactic_2.10;2.2.1 compile
at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:213)
at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122)
at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:104)
at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:51)
at sbt.IvySbt$$anon$3.call(Ivy.scala:60)
at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:98)
at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:81)
at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:102)
at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:102)
at xsbt.boot.Using$.withResource(Using.scala:11)
at xsbt.boot.Using$.apply(Using.scala:10)
at xsbt.boot.Locks$GlobalLock.withFileLock(Locks.scala:102)
at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:62)
at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:52)
at xsbt.boot.Locks$.apply0(Locks.scala:38)
at xsbt.boot.Locks$.apply(Locks.scala:28)
at sbt.IvySbt.withDefaultLogger(Ivy.scala:60)
at sbt.IvySbt.withIvy(Ivy.scala:101)
at sbt.IvySbt.withIvy(Ivy.scala:97)
at sbt.IvySbt$Module.withModule(Ivy.scala:116)
at sbt.IvyActions$.update(IvyActions.scala:121)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1161)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1159)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$73.apply(Defaults.scala:1182)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$73.apply(Defaults.scala:1180)
at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1184)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1179)
at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
at sbt.Classpaths$.cachedUpdate(Defaults.scala:1187)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1152)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1130)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
at sbt.std.Transform$$anon$4.work(System.scala:64)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
at sbt.Execute.work(Execute.scala:244)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
[error] (*:update) sbt.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.10.4: configuration not found in org.scala-lang#scala-library;2.10.4: 'master(compile)'. Missing configuration: 'compile'. It was required from org.scalactic#scalactic_2.10;2.2.1 compile
[error] unresolved dependency: org.scala-lang#scala-reflect;2.10.4: configuration not found in org.scala-lang#scala-reflect;2.10.4: 'master(compile)'. Missing configuration: 'compile'. It was required from org.scalactic#scalactic_2.10;2.2.1 compile
Update 1:
This is the content of the build.sbt file:
name := "words-cluster"
version := "1.0"
scalaVersion := "2.11.7"
organization := "com.manning"
libraryDependencies ++= {
val akkaVersion = "2.4.14"
Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion,
"com.typesafe.akka" %% "akka-remote" % akkaVersion,
"com.typesafe.akka" %% "akka-cluster" % akkaVersion,
"com.typesafe.akka" %% "akka-multi-node-testkit" % akkaVersion % "test",
"com.typesafe.akka" %% "akka-testkit" % akkaVersion % "test",
"org.scalatest" %% "scalatest" % "3.0.0" % "test",
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion,
"ch.qos.logback" % "logback-classic" % "1.0.10"
)
}
// Assembly settings
mainClass in Global := Some("aia.cluster.words.Main")
assemblyJarName in assembly := "words-node.jar"
Any idea what is going wrong?
Try this as your build.sbt:
name := "words-cluster"
version := "1.0"
scalaVersion := "2.11.8"
organization := "com.manning"
libraryDependencies ++= {
val akkaVersion = "2.4.14"
Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion,
"com.typesafe.akka" %% "akka-remote" % akkaVersion,
"com.typesafe.akka" %% "akka-cluster" % akkaVersion,
"com.typesafe.akka" %% "akka-multi-node-testkit" % akkaVersion % "test",
"com.typesafe.akka" %% "akka-testkit" % akkaVersion % "test",
"org.scalatest" %% "scalatest" % "3.0.1" % "test" exclude("org.scala-lang.modules", "scala-xml_2.11"),
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion,
"ch.qos.logback" % "logback-classic" % "1.1.7"
)
}
// Assembly settings
mainClass in Global := Some("aia.cluster.words.Main")
assemblyJarName in assembly := "words-node.jar"
I updated the scala, logging and scalatest versions
Seems like the sbt version i was using was the wrong one! After upgrading to sbt-0.13.15 compilation was running properly.
Thanks for helping!

Plot a dataframe in scala using WISP

I need to plot my dataframe using WISP. I am using Apache Sprak in Scala. However it seems that it can not accept a dataframe.
+-----+--------------------+------------------+
|label| features| prediction|
+-----+--------------------+------------------+
| 101|[1.497846976E9,10...|101.22752534884378|
| 101|[1.497846976E9,10...|101.22752534884378|
| 101|[1.497846976E9,10...|101.22752534884378|
| 101|[1.497846976E9,10...|101.22752534884378|
| 101|[1.497846976E9,10...|101.22752534884378|
+-----+--------------------+------------------+
I also tried to change my datarame to Seq using the following code but, it does't work.
import org.apache.spark.sql.functions.{collect_list, collect_set}
val label=predictions.groupBy($"label").agg(collect_list($"label").alias("label"))
val predicted= predictions.groupBy($"prediction").agg(collect_list($"prediction").alias("prediction"))
line(predicted)
It shows me the following error:
Error:(157, 10) type mismatch;
found : org.apache.spark.sql.DataFrame
(which expands to) org.apache.spark.sql.Dataset[org.apache.spark.sql.Row]
required: com.quantifind.charts.repl.IterablePair[?,?,?,?]
line(predicted)
Is there any trick to plot a dataframe like below using WISP? Thanks in advance.
UPDATE:
According to answer number one, As I try to install vegas libraries, I got an error:
Error:Error while importing SBT project:<br/>...<br/><pre>[warn] ==== MapR Repository: tried
[warn] http://repository.mapr.com/maven/com/github/aishfenton/vegas-spark_2.10_2.11/0.2.0/vegas-spark_2.10_2.11-0.2.0.pom
[info] Resolving org.scala-lang#scala-compiler;2.11.8 ...
[info] Resolving org.scala-lang.modules#scala-xml_2.11;1.0.4 ...
[info] Resolving org.scala-lang.modules#scala-parser-combinators_2.11;1.0.4 ...
[info] Resolving jline#jline;2.12.1 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.github.aishfenton#vegas-spark_2.10_2.11;0.2.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] com.github.aishfenton:vegas-spark_2.10_2.11:0.2.0 (/Users/saeedtkh/Desktop/ML_Alpha/build.sbt#L15-34)
[warn] +- ml:ml_2.11:1.0
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: com.github.aishfenton#vegas-spark_2.10_2.11;0.2.0: not found
[error] (*:update) sbt.ResolveException: unresolved dependency: com.github.aishfenton#vegas-spark_2.10_2.11;0.2.0: not found
[error] Total time: 27 s, completed Sep 22, 2017 1:06:23 PM</pre><br/>See complete log in file:/Users/saeedtkh/Library/Logs/IntelliJIdea2017.1/sbt.last.log
My bulid.sbt file is:
name := "ML"
version := "1.0"
scalaVersion := "2.11.8"
retrieveManaged := true
resolvers += "Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/"
resolvers += "MapR Repository" at "http://repository.mapr.com/maven/"
libraryDependencies ++= Seq(
"co.theasi" %% "plotly" % "0.2.0",
"org.apache.commons" % "commons-csv" % "1.1",
"org.apache.spark" %% "spark-core" % "2.0.2",
"org.apache.spark" %% "spark-core" % "2.0.2",
"org.apache.spark" %% "spark-sql" % "2.0.2",
"org.apache.spark" %% "spark-hive" % "2.0.2",
"org.apache.spark" %% "spark-streaming" % "2.0.2",
"org.apache.spark" %% "spark-mllib" % "2.0.2",
"org.apache.spark" %% "spark-mllib" % "2.0.2",
"org.scalanlp" %% "breeze" % "0.11.2",
"org.scalanlp" %% "breeze-natives" % "0.11.2",
"org.scalanlp" %% "breeze-viz" % "0.11.2",
"com.quantifind" %% "wisp" % "0.0.4"
// https://mvnrepository.com/artifact/com.github.aishfenton/vegas-spark_2.10
//"com.github.aishfenton" %% "vegas-spark_2.10" % "0.2.0"
)
What exactly are you trying to plot? Have you tried using Vegas? Vegas plotting library for Scala
An example for getting histograms of a dataframe column:
Add the following lines to your build file.
"org.vegas-viz" %% "vegas" % "0.3.9",
"org.vegas-viz" %% "vegas-spark" % "0.3.9"
An example code
import vegas._
import vegas.render.WindowRenderer._
import vegas.sparkExt._
val plot = Vegas("approval date").
withDataFrame(castedDf).
mark(Bar).
encodeX("columnName", Quant, bin=Bin(maxbins=20.0), sortOrder=SortOrder.Desc).
show

use squeryl with Play 2

I am new with Play and Scala and I am trying to import squeryl at build.sbt but without success. I am following the Play for Scala book.
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.squeryl#squeryl;0.9.5-6: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.squeryl:squeryl:0.9.5-6 (/home/felipe/workspace-play/products/build.sbt#L9-20)
[warn] +- products:products_2.11:1.0-SNAPSHOT
sbt.ResolveException: unresolved dependency: org.squeryl#squeryl;0.9.5-6: not found
Here is my build.sbt
name := """products"""
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.11.7"
libraryDependencies ++= Seq(
jdbc,
cache,
ws,
evolutions,
"org.squeryl" % "squeryl" % "0.9.5-6",
// "postgresql" % "postgresql" % "9.1-901-1.jdbc4",
"com.typesafe.play" %% "anorm" % "2.4.0",
"net.sf.barcode4j" % "barcode4j" % "2.0",
"org.scalatestplus.play" %% "scalatestplus-play" % "1.5.1" % Test
)
resolvers += "scalaz-bintray" at "http://dl.bintray.com/scalaz/releases"
resolvers += "Typesafe repository" at "http://repo.typesafe.com/typesafe/releases"
resolvers += "Sonatype snapshots" at "http://oss.sonatype.org/content/repositories/snapshots"
resolvers += "Sonatype releases" at "http://oss.sonatype.org/content/repositories/releases"
resolvers += "webjars" at "http://webjars.github.com/m2"
routesGenerator := InjectedRoutesGenerator
I changed for "org.squeryl" % "squeryl_2.9.0-1" % "0.9.4" and it worked.

akka-spray-websocket template build error

I'm having a little problem with these template from TypeSafe, when I try to import it in IntelliJ I get following message error:
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: io.spray#sbt-revolver;0.7.2: not found
[warn] :: com.typesafe.sbt#sbt-aspectj;0.10.1: not found
[warn] :: com.typesafe.sbteclipse#sbteclipse-plugin;2.5.0: not found
[warn] :: com.timushev.sbt#sbt-updates;0.1.7: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Some unresolved dependencies have extra attributes. Check that these dependencies exist with the requested attributes.
[warn] io.spray:sbt-revolver:0.7.2 (sbtVersion=0.13, scalaVersion=2.10)
[warn] com.typesafe.sbt:sbt-aspectj:0.10.1 (sbtVersion=0.13, scalaVersion=2.10)
[warn] com.typesafe.sbteclipse:sbteclipse-plugin:2.5.0 (sbtVersion=0.13, scalaVersion=2.10)
[warn] com.timushev.sbt:sbt-updates:0.1.7 (sbtVersion=0.13, scalaVersion=2.10)
[warn]
[warn] Note: Unresolved dependencies path:
[warn] io.spray:sbt-revolver:0.7.2 (sbtVersion=0.13, scalaVersion=2.10) (C:\Users\jlopesde\akka-spray-websocket\project\plugins.sbt#L1-2)
[warn] +- default:akka-spray-websocket-build:0.1-SNAPSHOT (sbtVersion=0.13, scalaVersion=2.10)
[warn] com.typesafe.sbt:sbt-aspectj:0.10.1 (sbtVersion=0.13, scalaVersion=2.10) (C:\Users\jlopesde\akka-spray-websocket\project\plugins.sbt#L3-4)
[warn] +- default:akka-spray-websocket-build:0.1-SNAPSHOT (sbtVersion=0.13, scalaVersion=2.10)
[warn] com.typesafe.sbteclipse:sbteclipse-plugin:2.5.0 (sbtVersion=0.13, scalaVersion=2.10) (C:\Users\jlopesde\akka-spray-websocket\project\plugins.sbt#L5-6)
[warn] +- default:akka-spray-websocket-build:0.1-SNAPSHOT (sbtVersion=0.13, scalaVersion=2.10)
[warn] com.timushev.sbt:sbt-updates:0.1.7 (sbtVersion=0.13, scalaVersion=2.10) (C:\Users\jlopesde\akka-spray-websocket\project\plugins.sbt#L7-8)
[warn] +- default:akka-spray-websocket-build:0.1-SNAPSHOT (sbtVersion=0.13, scalaVersion=2.10)
sbt.ResolveException: unresolved dependency: io.spray#sbt-revolver;0.7.2: not found
unresolved dependency: com.typesafe.sbt#sbt-aspectj;0.10.1: not found
unresolved dependency: com.typesafe.sbteclipse#sbteclipse-plugin;2.5.0: not found
unresolved dependency: com.timushev.sbt#sbt-updates;0.1.7: not found
[error] (*:update) sbt.ResolveException: unresolved dependency: io.spray#sbt-revolver;0.7.2: not found
[error] unresolved dependency: com.typesafe.sbt#sbt-aspectj;0.10.1: not found
[error] unresolved dependency: com.typesafe.sbteclipse#sbteclipse-plugin;2.5.0: not found
[error] unresolved dependency: com.timushev.sbt#sbt-updates;0.1.7: not found
I know there is a problem on importing it, probably versions of Spray or Scala aren't correct but I don't know how to fix it. Could you provide me some tips for dealing with problems like that?
Thanks
My build.sbt:
organization := "cua.li"
version := "0.4"
scalaVersion := "2.11.5"
libraryDependencies ++= {
val akkaV = "2.3.9"
val sprayV = "1.3.2"
val kamonV = "0.3.5"
Seq(
"com.wandoulabs.akka" %% "spray-websocket" % "0.1.4" withSources() withJavadoc,
"io.spray" %% "spray-json" % "1.3.1" withSources() withJavadoc,
"io.spray" %% "spray-can" % sprayV withSources() withJavadoc,
"io.spray" %% "spray-routing" % sprayV withSources() withJavadoc,
"com.typesafe.akka" %% "akka-actor" % akkaV withSources() withJavadoc,
"com.typesafe.akka" %% "akka-slf4j" % akkaV withSources() withJavadoc,
/*
"org.aspectj" % "aspectjweaver" % "1.8.4" withSources() withJavadoc,
"io.kamon" %% "kamon-core" % kamonV withSources() withJavadoc,
"io.kamon" %% "kamon-spray" % kamonV withSources() withJavadoc,
"io.kamon" %% "kamon-statsd" % kamonV withSources() withJavadoc,
"io.kamon" %% "kamon-log-reporter" % kamonV withSources() withJavadoc,
"io.kamon" %% "kamon-system-metrics" % kamonV withSources() withJavadoc,
"io.kamon" %% "kamon-testkit" % kamonV % "test" withSources() withJavadoc,
// */
"com.typesafe.akka" %% "akka-testkit" % akkaV % "test" withSources() withJavadoc,
"io.spray" %% "spray-testkit" % sprayV % "test" withSources() withJavadoc,
"org.scalatest" %% "scalatest" % "2.2.3" % "test",
"junit" % "junit" % "4.12" % "test",
"org.specs2" %% "specs2" % "2.4.15" % "test",
"ch.qos.logback" % "logback-classic" % "1.1.2"
)
}
scalacOptions ++= Seq("-deprecation", "-encoding", "UTF-8", "-feature", "-target:jvm-1.7", "-unchecked",
"-Ywarn-adapted-args", "-Ywarn-value-discard", "-Xlint")
javacOptions ++= Seq("-Xlint:deprecation", "-Xlint:unchecked", "-source", "1.7", "-target", "1.7", "-g:vars")
doc in Compile <<= target.map(_ / "none")
publishArtifact in (Compile, packageSrc) := false
logBuffered in Test := false
Keys.fork in Test := false
parallelExecution in Test := false
seq(Revolver.settings: _*)
import com.typesafe.sbt.SbtAspectj._
aspectjSettings
fork in run := true
javaOptions <++= AspectjKeys.weaverOptions in Aspectj
Probably missing the repositories to download the dependencies.
Try put this before libraryDependencies:
resolvers ++= Seq(
"Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/",
"Spray repository" at "http://repo.spray.io/",
"Scalaz Bintray Repo" at "http://dl.bintray.com/scalaz/releases"
)
I've had the same issue but only with sbt-revolver. I've managed to solve it by doing following:
My SBT is set up to use my Artifactory instance in order to retrieve dependencies.
1) In Artifactory I added the repository below to the "Remote Repositories" and made it part of the "remote-repos" virtual repository.
http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases
2) On my ~/.sbt/repositories file, I've made sure my Artifactory repository is configured with the correct URL format (please see below). I've taken the format from SBT's documentation.
artifactory-ivy: http://my.artifactory.com:8081/artifactory/repo/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]