Getting started with "Testing Akka Actors"
I think there is something wrong with my "akka-testkit" library-dependency. I copied it from Lightbend Testing Classic Actors
build.sbt
version := "0.1.0-SNAPSHOT"
scalaVersion := "2.12.7"
val akkaVersion = "2.5.13"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"org.scalatest" %% "scalatest" % "3.0.5",
"com.typesafe.akka" %% "akka-testkit" % akkaVersion % Test
)
.scala
package part3testing
import akka.actor.ActorSystem
import akka.testkit.TestKit
class BasicSpec extends TestKit(ActorSystem("BasicSpec")){
}
Marking a dependency as % Test means that only code in the test directories (by default, src/test) will depend on it. Main application code (by default in src/main) does not depend on test-scope dependencies; the benefit of this is that the test dependencies aren't needed for distributing/deploying the built software so don't get included or need to be provided.
Related
I am new to scala. I am trying to create a scala project in IntellIj and adding a test class.
I am using the below 2 dependencies in sbt.
libraryDependencies += ("org.scalactic" %% "scalactic" % "3.0.8")
// https://mvnrepository.com/artifact/org.scalatest/scalatest
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.8" % Test
But I am unable to use the class FunSuite in the test class 'ProcessCSVTest.scala' for testing as it is giving a compilation error.
Although I can see the dependencies in the external library in my IntellIj
Build.sbt file
name := "CSVParser"
version := "0.1"
scalaVersion := "2.13.0"
libraryDependencies += ("org.scalactic" %% "scalactic" % "3.0.8")
// https://mvnrepository.com/artifact/org.scalatest/scalatest
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.8" % Test
The entire code can be found here - https://github.com/practice09/CSVParser
Can anyone please tell me where I am doing wrong?
One issue is the test ProcessCSVTest.scala is under main sources which means ScalaTest needs to be on the main classpath, however in build.sbt ScalaTest dependency is scoped to the Test classpath
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.8" % Test
So if you remove Test scope like so
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.8"
then ScalaTest will end up on the main classpath and we can add the following import
import org.scalatest.FunSuite
However my suggestion is to move the tests out of the main sources and put them under src/test/scala/, and then scope the dependency under Test like before
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.8" % Test
The following command builds an example with correct project structure expected by sbt
sbt new scala/scala-seed.g8
so try exploring how it is setup and fit your project to match.
Because FunSuite is in a package, you need to add
import org.scalatest.FunSuite
If you put the cursor on it and press Alt+Enter, you should get a suggestion to fix it.
I am not able to import spark mllib libraries in Intellij for Spark scala project. I am getting a resolution exception.
Below is my sbt.build
name := "ML_Spark"
version := "0.1"
scalaVersion := "2.11.12"
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.2.1"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.2.1"
libraryDependencies += "org.apache.spark" %% "spark-mllib" % "2.2.1" % "runtime"
I tried to copy/paste the same build.sbt file you provided and i got the following error :
[error] [/Users/pc/testsbt/build.sbt]:3: ';' expected but string literal found.
Actually, the build.sbt is invalid :
intellij error
Having the version and the Scala version in different lines solved the problem for me :
name := "ML_Spark"
version := "0.1"
scalaVersion := "2.11.12"
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.2.1"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.2.1"
libraryDependencies += "org.apache.spark" %% "spark-mllib" % "2.2.1" % "runtime"
I am not sure that this is the problem you're facing (can you please share the exception you had ?), it might be a problem with the repositories you specified under the .sbt folder in your home directory.
I have met the same problem before. To solve it, I just used the compiled version of mllib instead of the runtime one. Here is my conf:
name := "SparkDemo"
version := "0.1"
scalaVersion := "2.11.12"
// https://mvnrepository.com/artifact/org.apache.spark/spark-core
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-mllib
libraryDependencies += "org.apache.spark" %% "spark-mllib" % "2.3.0"
I had a similar issue, but I found a workaround. Namely, you have to add the spark-mllib jar file to your project manually. Indeed, despite my build.sbt file was
name := "example_project"
version := "0.1"
scalaVersion := "2.12.10"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "3.0.0",
"org.apache.spark" %% "spark-sql" % "3.0.0",
"org.apache.spark" %% "spark-mllib" % "3.0.0" % "runtime"
)
I wasn't able to import the spark library with
import org.apache.spark.sql._
import org.apache.spark.sql.types._
import org.apache.spark.ml._
The solution that worked for me was to add the jar file manually. Specifically,
Download the jar file of the ml library you need (e.g. for spark 3 use https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.12/3.0.0 ).
Follow this link to add the jar file to your intelliJ project: Correct way to add external jars (lib/*.jar) to an IntelliJ IDEA project
Add also the mlib-local jar (https://mvnrepository.com/artifact/org.apache.spark/spark-mllib-local)
If, for some reason, you compile again the build.sbt you need to re-import the jar file again.
I'm trying to import an akka dependency to my project using sbt.
The akka modules I need are akka-actor and akka-remote. The curious thing is that akka-actor has no problems importing, but the remote module appears as an unknown artifact.
I'm using IntelliJ and scala 2.12.1. Does someone have this problem or can help me in any way?
Try leave a blank line between the two libraryDependencies:
libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.5.0"
libraryDependencies += "com.typesafe.akka" %% "akka-remote" % "2.5.0"
Or keep them in a Seq:
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.5.0",
"com.typesafe.akka" %% "akka-remote" % "2.5.0"
)
I have the following build.sbt file:
name := "akkaHttp"
version := "1.0"
scalaVersion := "2.12.0"
resolvers ++= Seq("Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/",
Resolver.bintrayRepo("hseeberger", "maven"))
libraryDependencies ++= {
val AkkaVersion = "2.3.9"
val AkkaHttpVersion = "2.0.1"
val Json4sVersion = "3.2.11"
Seq(
"com.typesafe.akka" %% "akka-slf4j" % AkkaVersion,
"com.typesafe.akka" %% "akka-http-experimental" % AkkaHttpVersion,
"ch.qos.logback" % "logback-classic" % "1.1.2",
"org.json4s" %% "json4s-native" % Json4sVersion,
"org.json4s" %% "json4s-ext" % Json4sVersion,
"de.heikoseeberger" %% "akka-http-json4s" % "1.4.2"
)
}
In it, these dependencies are not met.
Error:Unresolved dependencies:
com.typesafe.akka#akka-slf4j_2.12;2.3.9: not found
com.typesafe.akka#akka-http-experimental_2.12;2.0.1: not found
org.json4s#json4s-native_2.12;3.2.11: not found
org.json4s#json4s-ext_2.12;3.2.11: not found
de.heikoseeberger#akka-http-json4s_2.12;1.4.2: not found
So what should I add to it such that the imports work?
Notice the _2.12 appended to the artifacts? Based on your scalaVersion, SBT tried to download the dependencies built against Scala 2.12.x, but couldn't find any. Try using Scala version 2.11.8.
Different versions of Scala can be binary incompatible, so libraries are cross built against those different versions and published to the repositories. Looks like it hasn't yet happened for those libraries above.
Note that Scala 2.12 is indeed not binary compatible with Scala 2.11.
I've just added ScalaTest to build.sbt so it now looks as follows:
name := "appname"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.4.1",
"org.scalatest" % "scalatest_2.11" % "2.2.4" % "test"
)
After that I am receiving the warning message:
SBT project import
[warn] Multiple dependencies with the same organization/name but different versions. To avoid conflict, pick one version:
[warn] * org.scala-lang:scala-reflect:(2.11.2, 2.11.7)
[warn] * org.scala-lang.modules:scala-xml_2.11:(1.0.2, 1.0.4)
I also tried changing the line concerning ScalaTest into:
"org.scalatest" %% "scalatest" % "2.2.4" % "test"
but the warning still remains the same as above.
How could I deal with this issue since I haven't anywhere written "reflect" or "xml" in my project. I am using the newest version of both Akka and ScalaTest and Scala version 2.11.
The solution might be to add explicitly one of proposed versions by SBT. All warnings pass away when libraryDependencies is:
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.4.1",
"org.scalatest" %% "scalatest" % "2.2.4" % "test",
"org.scala-lang" % "scala-reflect" % "2.11.7",
"org.scala-lang.modules" %% "scala-xml" % "1.0.4"
)
In your particular case this is related to the ISSUE 1933 and you can ignore it for now. You can also explicitly specify version of the dependency you need, to silence warnings.
The problem is fixed with sbt 0.13.12