I'm trying to import an akka dependency to my project using sbt.
The akka modules I need are akka-actor and akka-remote. The curious thing is that akka-actor has no problems importing, but the remote module appears as an unknown artifact.
I'm using IntelliJ and scala 2.12.1. Does someone have this problem or can help me in any way?
Try leave a blank line between the two libraryDependencies:
libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.5.0"
libraryDependencies += "com.typesafe.akka" %% "akka-remote" % "2.5.0"
Or keep them in a Seq:
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.5.0",
"com.typesafe.akka" %% "akka-remote" % "2.5.0"
)
Related
Getting started with "Testing Akka Actors"
I think there is something wrong with my "akka-testkit" library-dependency. I copied it from Lightbend Testing Classic Actors
build.sbt
version := "0.1.0-SNAPSHOT"
scalaVersion := "2.12.7"
val akkaVersion = "2.5.13"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"org.scalatest" %% "scalatest" % "3.0.5",
"com.typesafe.akka" %% "akka-testkit" % akkaVersion % Test
)
.scala
package part3testing
import akka.actor.ActorSystem
import akka.testkit.TestKit
class BasicSpec extends TestKit(ActorSystem("BasicSpec")){
}
Marking a dependency as % Test means that only code in the test directories (by default, src/test) will depend on it. Main application code (by default in src/main) does not depend on test-scope dependencies; the benefit of this is that the test dependencies aren't needed for distributing/deploying the built software so don't get included or need to be provided.
I'm new to Scala/Spark, so please be easy on me :)
I'm trying to run an EMR cluster on AWS, running the jar file I packed with sbt package.
When I run the code locally, it is working perfectly fine, but when I'm running it in the AWS EMR cluster, I'm getting an error:
ERROR Client: Application diagnostics message: User class threw exception: java.lang.NoClassDefFoundError: upickle/core/Types$Writer
From what I understand, this error originates in the dependencies of the scala/spark versions.
So I'm using Scala 2.12 with spark 3.0.1, and in AWS I'm using emr-6.2.0.
Here's my build.sbt:
scalaVersion := "2.12.14"
libraryDependencies += "com.amazonaws" % "aws-java-sdk" % "1.11.792"
libraryDependencies += "com.amazonaws" % "aws-java-sdk-core" % "1.11.792"
libraryDependencies += "org.apache.hadoop" % "hadoop-aws" % "3.3.0"
libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "3.3.0"
libraryDependencies += "org.apache.hadoop" % "hadoop-client" % "3.3.0"
libraryDependencies += "org.apache.spark" %% "spark-core" % "3.0.1"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "3.0.1"
libraryDependencies += "com.lihaoyi" %% "upickle" % "1.4.1"
libraryDependencies += "com.lihaoyi" %% "ujson" % "1.4.1"
What am I missing?
Thanks!
If you use sbt package, the generated jar will contain only the code of your project, but not dependencies. You need to use sbt assembly to generate so-called uberjar, that will include dependencies as well.
But in your cases, it's recommended to mark Spark and Hadoop (and maybe AWS) dependencies as Provided - they should be already included into the EMR runtime. Use something like this:
libraryDependencies += "org.apache.spark" %% "spark-core" % "3.0.1" % Provided
I am trying to set up IntelliJ IDEA with sbt for a Scala project. The external dependencies are specified in my build.sbt and also listed inside the IDE, as can be seen in the screenshot. However, I still get compiler errors that saying the respective symbol cannot be resolved. Can anyone point me into the right direction?
Contents of my build.sbt:
lazy val midas = (project in file("."))
.settings(
name := "test",
mainClass in assembly := Some("core.Service"),
assemblyJarName in assembly := "test.jar",
test in assembly := {},
libraryDependencies ++= Seq(
"org.slf4j" % "slf4j-api" % "1.7.25",
"com.typesafe.akka" %% "akka-actor" % "2.5.13",
"com.typesafe.akka" %% "akka-slf4j" % "2.5.13",
"com.typesafe.akka" %% "akka-remote" % "2.5.13",
"org.scala-lang.modules" %% "scala-xml" % "1.1.0",
"com.typesafe.play" %% "play-json" % "2.6.9",
"com.typesafe.slick" %% "slick" % "3.2.3",
"com.typesafe.scala-logging" %% "scala-logging" % "3.9.0",
"ch.qos.logback" % "logback-classic" % "1.2.3",
"ch.qos.logback" % "logback-core" % "1.2.3",
"com.mchange" % "c3p0" % "0.9.5.2",
"joda-time" % "joda-time" % "2.10",
"org.joda" % "joda-convert" % "2.0.1",
"net.sourceforge.jtds" % "jtds" % "1.3.1"
)
)
Still the dependencies for both, akka and joda-time cannot be resolved inside the IDE. The sbt compile from the command line works fine however.
joda-convert does not include org.joda.time.DateTime - you need "joda-time" % "joda-time" % "2.9.4" or so.
Once you have added this (and fixed your missing Akka import, maybe "com.typesafe.akka" %% "akka-actor" % "2.4.17"), you need to refresh the sbt project.
I press Shift, Shift to bring up "Search Everywhere" and type "sbt". Choose sbt under Tool Windows, then click "Refresh all sbt projects" (blue reload icon) in the sbt window. I close this window because it is not usually useful.
Hope that helps.
Currently I have a simple scala project.
Running sbt compile from the command line succeeds
In Intellij doing Build > Rebuild Project the build will succeed.
However there are several issues with syntax highlighting.
Receive types are not being recognized, context.become(<some Receive def>) is failing (this is due to Receive not being recognized), and lots of other issues that are seemingly random.
Current build.sbt:
scalaVersion := "2.12.1"
resolvers += "Typesafe Repository" at
"http://repo.typesafe.com/typesafe/releases/"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.5.1",
"com.typesafe.akka" %% "akka-http-core" % "10.0.5",
"com.typesafe.akka" %% "akka-http" % "10.0.5",
"com.typesafe.akka" %% "akka-http-spray-json" % "10.0.6",
"com.typesafe.akka" %% "akka-slf4j" % "2.5.1",
"com.typesafe.akka" %% "akka-stream" % "2.5.1",
"ch.qos.logback" % "logback-classic" % "1.1.7",
"com.typesafe.akka" %% "akka-testkit" % "2.5.1" % "test"
)
I've tried invalidating my cache and restarting, didn't work.
I've updated Intellij. I tried wiping my .idea files and re-importing the project.
Any thoughts on how to resolve this?
I have the following build.sbt file:
name := "akkaHttp"
version := "1.0"
scalaVersion := "2.12.0"
resolvers ++= Seq("Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/",
Resolver.bintrayRepo("hseeberger", "maven"))
libraryDependencies ++= {
val AkkaVersion = "2.3.9"
val AkkaHttpVersion = "2.0.1"
val Json4sVersion = "3.2.11"
Seq(
"com.typesafe.akka" %% "akka-slf4j" % AkkaVersion,
"com.typesafe.akka" %% "akka-http-experimental" % AkkaHttpVersion,
"ch.qos.logback" % "logback-classic" % "1.1.2",
"org.json4s" %% "json4s-native" % Json4sVersion,
"org.json4s" %% "json4s-ext" % Json4sVersion,
"de.heikoseeberger" %% "akka-http-json4s" % "1.4.2"
)
}
In it, these dependencies are not met.
Error:Unresolved dependencies:
com.typesafe.akka#akka-slf4j_2.12;2.3.9: not found
com.typesafe.akka#akka-http-experimental_2.12;2.0.1: not found
org.json4s#json4s-native_2.12;3.2.11: not found
org.json4s#json4s-ext_2.12;3.2.11: not found
de.heikoseeberger#akka-http-json4s_2.12;1.4.2: not found
So what should I add to it such that the imports work?
Notice the _2.12 appended to the artifacts? Based on your scalaVersion, SBT tried to download the dependencies built against Scala 2.12.x, but couldn't find any. Try using Scala version 2.11.8.
Different versions of Scala can be binary incompatible, so libraries are cross built against those different versions and published to the repositories. Looks like it hasn't yet happened for those libraries above.
Note that Scala 2.12 is indeed not binary compatible with Scala 2.11.