Currently I have a simple scala project.
Running sbt compile from the command line succeeds
In Intellij doing Build > Rebuild Project the build will succeed.
However there are several issues with syntax highlighting.
Receive types are not being recognized, context.become(<some Receive def>) is failing (this is due to Receive not being recognized), and lots of other issues that are seemingly random.
Current build.sbt:
scalaVersion := "2.12.1"
resolvers += "Typesafe Repository" at
"http://repo.typesafe.com/typesafe/releases/"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.5.1",
"com.typesafe.akka" %% "akka-http-core" % "10.0.5",
"com.typesafe.akka" %% "akka-http" % "10.0.5",
"com.typesafe.akka" %% "akka-http-spray-json" % "10.0.6",
"com.typesafe.akka" %% "akka-slf4j" % "2.5.1",
"com.typesafe.akka" %% "akka-stream" % "2.5.1",
"ch.qos.logback" % "logback-classic" % "1.1.7",
"com.typesafe.akka" %% "akka-testkit" % "2.5.1" % "test"
)
I've tried invalidating my cache and restarting, didn't work.
I've updated Intellij. I tried wiping my .idea files and re-importing the project.
Any thoughts on how to resolve this?
Related
I'm trying to build an Scalatra application that runs code with spark. I can actually build the fat jar with sbt-assembly and the endpoints work, but when running tests with org.scalatra.test.scalatest._ I get the following error:
*** RUN ABORTED ***
java.lang.NoSuchFieldError: INSTANCE
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.<clinit>(SSLConnectionSocketFactory.java:146)
at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:964)
at org.scalatra.test.HttpComponentsClient$class.createClient(HttpComponentsClient.scala:100)
at my.package.MyServletTests.createClient(MyServletTests.scala:5)
at org.scalatra.test.HttpComponentsClient$class.submit(HttpComponentsClient.scala:63)
at my.package.MyServletTests.submit(MyServletTests.scala:5)
at org.scalatra.test.Client$class.post(Client.scala:62)
at my.package.MyServletTests.post(MyServletTests.scala:5)
at org.scalatra.test.Client$class.post(Client.scala:60)
at my.package.MyServletTests.post(MyServletTests.scala:5)
...
From other sources, this seemed to be an httpclient version error, since both Scalatra and Spark use different versions. These sources suggested the use of Maven Shade Plugin to rename one of these versions. I am, however, using sbt instead of Maven. Even though sbt has a shade functionality, it works when creating the fat jar, and I need this solution during development tests.
Sources are:
Conflict between httpclient version and Apache Spark
Error while trying to use an API. java.lang.NoSuchFieldError: INSTANCE
Is there any way to resolve this kind of conflict using SBT? I'm running Eclipse Scala-IDE and these are my dependencies in built.sbt:
val scalatraVersion = "2.6.5"
// scalatra
libraryDependencies ++= Seq(
"org.scalatra" %% "scalatra" % scalatraVersion,
"org.scalatra" %% "scalatra-scalatest" % scalatraVersion % "test",
"org.scalatra" %% "scalatra-specs2" % scalatraVersion,
"org.scalatra" %% "scalatra-swagger" % scalatraVersion,
"ch.qos.logback" % "logback-classic" % "1.2.3" % "runtime",
"org.eclipse.jetty" % "jetty-webapp" % "9.4.9.v20180320" % "container;compile",
"javax.servlet" % "javax.servlet-api" % "3.1.0" % "provided"
)
// From other projects:
// spark
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.4.0",
"org.apache.spark" %% "spark-mllib" % "2.4.0",
"org.apache.spark" %% "spark-sql" % "2.4.0"
)
// scalatest
libraryDependencies += "org.scalatest" % "scalatest_2.11" % "3.0.5" % "test"
I am trying to set up IntelliJ IDEA with sbt for a Scala project. The external dependencies are specified in my build.sbt and also listed inside the IDE, as can be seen in the screenshot. However, I still get compiler errors that saying the respective symbol cannot be resolved. Can anyone point me into the right direction?
Contents of my build.sbt:
lazy val midas = (project in file("."))
.settings(
name := "test",
mainClass in assembly := Some("core.Service"),
assemblyJarName in assembly := "test.jar",
test in assembly := {},
libraryDependencies ++= Seq(
"org.slf4j" % "slf4j-api" % "1.7.25",
"com.typesafe.akka" %% "akka-actor" % "2.5.13",
"com.typesafe.akka" %% "akka-slf4j" % "2.5.13",
"com.typesafe.akka" %% "akka-remote" % "2.5.13",
"org.scala-lang.modules" %% "scala-xml" % "1.1.0",
"com.typesafe.play" %% "play-json" % "2.6.9",
"com.typesafe.slick" %% "slick" % "3.2.3",
"com.typesafe.scala-logging" %% "scala-logging" % "3.9.0",
"ch.qos.logback" % "logback-classic" % "1.2.3",
"ch.qos.logback" % "logback-core" % "1.2.3",
"com.mchange" % "c3p0" % "0.9.5.2",
"joda-time" % "joda-time" % "2.10",
"org.joda" % "joda-convert" % "2.0.1",
"net.sourceforge.jtds" % "jtds" % "1.3.1"
)
)
Still the dependencies for both, akka and joda-time cannot be resolved inside the IDE. The sbt compile from the command line works fine however.
joda-convert does not include org.joda.time.DateTime - you need "joda-time" % "joda-time" % "2.9.4" or so.
Once you have added this (and fixed your missing Akka import, maybe "com.typesafe.akka" %% "akka-actor" % "2.4.17"), you need to refresh the sbt project.
I press Shift, Shift to bring up "Search Everywhere" and type "sbt". Choose sbt under Tool Windows, then click "Refresh all sbt projects" (blue reload icon) in the sbt window. I close this window because it is not usually useful.
Hope that helps.
I'm trying to build a "hello world"-esque app that uses Spark streaming to stream data from a Kafka broker (this works), filters/processes this data, and pushes it to a (local) web browser using the Scalatra web framework and its supported web sockets functionality from Atmosphere. The Kafka/Spark chunk works independently, and the Scalatra/Atmosphere chunk also works independently. It's when I try to bring the two halves together that I run into issues with library dependencies.
The real question: how do I go about selecting library versions for which Spark will play nice with Scalatra?
A bare bones Scalatra/Atmosphere app works fine as follows:
organization := "com.example"
name := "example app"
version := "0.1.0"
scalaVersion := "2.12.2"
val ScalatraVersion = "2.5.+"
libraryDependencies ++= Seq(
"org.json4s" %% "json4s-jackson" % "3.5.2",
"org.scalatra" %% "scalatra" % ScalatraVersion,
"org.scalatra" %% "scalatra-scalate" % ScalatraVersion,
"org.scalatra" %% "scalatra-specs2" % ScalatraVersion % "test",
"org.scalatra" %% "scalatra-atmosphere" % ScalatraVersion,
"org.eclipse.jetty" % "jetty-webapp" % "9.4.6.v20170531" % "provided",
"javax.servlet" % "javax.servlet-api" % "3.1.0" % "provided"
)
enablePlugins(JettyPlugin)
But if I add new dependencies for Spark and Spark streaming, and knock the Scala version down to 2.11 (required for Spark-Kafka streaming):
organization := "com.example"
name := "example app"
version := "0.1.0"
scalaVersion := "2.11.8"
val ScalatraVersion = "2.5.+"
val SparkVersion = "2.2.0"
libraryDependencies ++= Seq(
"org.json4s" %% "json4s-jackson" % "3.5.2",
"org.scalatra" %% "scalatra" % ScalatraVersion,
"org.scalatra" %% "scalatra-scalate" % ScalatraVersion,
"org.scalatra" %% "scalatra-specs2" % ScalatraVersion % "test",
"org.scalatra" %% "scalatra-atmosphere" % ScalatraVersion,
"org.eclipse.jetty" % "jetty-webapp" % "9.4.6.v20170531" % "provided",
"javax.servlet" % "javax.servlet-api" % "3.1.0" % "provided"
)
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % SparkVersion,
"org.apache.spark" %% "spark-streaming" % SparkVersion,
"org.apache.spark" %% "spark-streaming-kafka-0-8" % SparkVersion
)
enablePlugins(JettyPlugin)
The code compiles, but I get SBT's eviction warning:
[warn] There may be incompatibilities among your library dependencies.
[warn] Here are some of the libraries that were evicted:
[warn] * org.json4s:json4s-jackson_2.11:3.2.11 -> 3.5.3
[warn] Run 'evicted' to see detailed eviction warnings
Then finally, when Jetty tries to run the web server, it fails with this error:
WARN:oejuc.AbstractLifeCycle:main: FAILED org.eclipse.jetty.annotations.ServletContainerInitializersStarter#53fb3dab: java.lang.NoClassDefFoundError: com/sun/jersey/spi/inject/InjectableProvider
java.lang.NoClassDefFoundError: com/sun/jersey/spi/inject/InjectableProvider
How do I get to the bottom of this? I'm new to the Scala world, and the intricacies of dependencies are blowing my mind.
One way to remove the eviction warning is to add the library dependency with the required version using dependencyOverrides
try to add the following in your SBT file and re-build the application
dependencyOverrides += "org.json4s" % "json4s-jackson_2.11" % "3.5.3"
Check SBT documentation here
I'm trying to import an akka dependency to my project using sbt.
The akka modules I need are akka-actor and akka-remote. The curious thing is that akka-actor has no problems importing, but the remote module appears as an unknown artifact.
I'm using IntelliJ and scala 2.12.1. Does someone have this problem or can help me in any way?
Try leave a blank line between the two libraryDependencies:
libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.5.0"
libraryDependencies += "com.typesafe.akka" %% "akka-remote" % "2.5.0"
Or keep them in a Seq:
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.5.0",
"com.typesafe.akka" %% "akka-remote" % "2.5.0"
)
I have tried many things but cant seem to get this to work.
I have the following in my build.sbt
name := "MyTestApp"
version := "0.1-SNAPSHOT"
scalaVersion := "2.10.3"
resolvers ++= Seq("spray repo" at "http://repo.spray.io",
"Typesafe Releases" at "http://repo.typesafe.com/typesafe/releases/",
"Scalaz Bintray Repo" at "http://dl.bintray.com/scalaz/releases",
"Plugin Releases" at
"http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases")
libraryDependencies ++= Seq(
"Io.spray" %% "spray-routing" % "1.3.2",
"io.spray" %% "spray-can" % "1.3.2",
"com.typesafe.akka" %% "akka-actor" % "2.2.3",
"com.typesafe.akka" %% "akka-slf4j" % "2.2.3",
"org.slf4j" % "slf4j-simple" % "1.6.4",
"io.spray" % "sbt-revolver" % "0.7.2"
)
I have also added the following to my plugins.sbt file
addSbtPlugin("io.spray" % "sbt-revolver" % "0.7.2")
I am using scala 2.10 with sbt version 0.13.5
My build log seems to indicate it is trying to resolve at the following location
==== Plugin Releases: tried
[warn] http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/io/spray/sbt-revolver/0.7.2/sbt-revolver-0.7.2.pom
Browsing to the following perhaps it would resolve if it was looking in
http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/io.spray/sbt-revolver/scala_2.10/sbt_0.13/0.7.2/
Any ideas as to why io.spray has turned into io/spray in the url
Any help deeply appreciated
Use this:
Resolver.url("Plugin Releases", url("http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/"))(Resolver.ivyStylePatterns)
instead of:
"Plugin Releases" at "http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases"
This sets the correct URL pattern for this repository.