Kindly consider if there's something very stupid I am missing, I am very new to this.
Creating this example on Eclipse (Scala-ide). When I create the CorrelationJobTest.scala class, there's a problem in importing classes from specs2 package.
import org.specs2.runner.JUnitRunner
The error I am getting is pretty obvious,
object specs2 is not a member of package org
Even after creating a build.sbt file, adding the dependencies, and doing a sbt clean, the error is still showing.
Below is my build.sbt
name := "SparkCorrelation"
organization := "com.xyz"
version := "0.1"
/* scala versions and options */
scalaVersion := "2.10.4"
// These options will be used for *all* versions.
javacOptions ++= Seq("-Xlint:unchecked", "-Xlint:deprecation")
scalacOptions ++= Seq("-feature", "-unchecked", "-deprecation")
/* dependencies */
libraryDependencies ++= Seq("org.specs2" %% "specs2-core" % "3.7" % "test")
scalacOptions in Test ++= Seq("-Yrangepos")
EDIT:
I faced this problem with apache.spark packages too. Then I included the jar spark-assembly-1.6.0-hadoop-2.6.0.jar and it got solved. Isn't there a library for specs2 that I can import in Eclipse?
Got it working. Here's what I did:
Deleted the project from eclipse
Did sbt eclipse
Imported the project into eclipse again.
Added dependencies in build.sbt
Did sbt clean
The problem was that the project was directly created in eclipse and was not able to link to sbt build file.
You need to add the junit module to your dependencies
libraryDependencies ++= Seq(
"org.specs2" %% "specs2-core" % "3.7" % "test",
"org.specs2" %% "specs2-junit" % "3.7" % "test")
Related
I made a dependency of Spark which worked in my first project. But when I try to make a new project with Spark, my SBT does not import the external jars of org.apache.spark. Therefore IntelliJ Idea gives the error that it "cannot resolve symbol".
I already tried to make a new project from scratch and use auto-import but none works. When I try to compile I get the messages that "object apache is not a member of package org". My build.sbt looks like this:
name := "hello"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies += "org.apache.spark" % "spark-parent_2.10" % "1.4.1"
I have the impression that there might be something wrong with my SBT settings, although it already worked one time. And except for the external libraries everything is the same...
I also tried to import the pom.xml file of my spark dependency but that also doesn't work.
Thank you in advance!
This worked for me->
name := "ProjectName"
version := "0.1"
scalaVersion := "2.11.11"
libraryDependencies ++= Seq(
"org.apache.spark" % "spark-core_2.11" % "2.2.0",
"org.apache.spark" % "spark-sql_2.11" % "2.2.0",
"org.apache.spark" % "spark-mllib_2.10" % "1.1.0"
)
I use
scalaVersion := "2.11.7"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.4.1"
in my build.sbt and it works for me.
I had a similar problem. It seems the reason was that the build.sbt file was specifying the wrong version of scala.
If you run spark-shell it'll say at some point the scala version used by Spark, e.g.
Using Scala version 2.11.8
Then I edited the line in the build.sbt file to point to that version and it worked.
Currently spark-cassandra-connector compatible with Scala 2.10 and 2.11.
It worked for me when I updated the scala version of my project like below:
ThisBuild / scalaVersion := "2.11.12"
and I updated my dependency like:
libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector" % "2.4.0",
If you use "%%", sbt will add your project’s binary Scala version to the artifact name.
From sbt run:
sbt> reload
sbt> compile
Your library dependecy conflicts with with the scala version you're using, you need to use 2.11 for it to work. The correct dependency would be:
scalaVersion := "2.11.7"
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "1.4.1"
note that you need to change spark_parent to spark_core
name := "SparkLearning"
version := "0.1"
scalaVersion := "2.12.3"
// additional libraries
libraryDependencies += "org.apache.spark" % "spark-streaming_2.10" % "1.4.1"
I am creating a new Play Scala project within Eclipse + Scala IDE.
The original build.sbt file is:
name := """portal"""
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.11.1"
libraryDependencies ++= Seq(
jdbc,
cache,
ws
)
I've edited it to include some more dependencies:
libraryDependencies ++= Seq(
jdbc,
javaEbean,
cache,
ws,
"org.postgresql" % "postgresql" % "9.3-1100-jdbc4",
"org.scalatestplus" %% "play" % "1.1.0" % "test"
)
I can't figure out why SBT won't include Ebean, postgresql, nor scalatest in my classpath. Any help?
It happened that I needed to run the command
activator eclipse
again, and refresh the IDE.
I have an existing project in IntelliJ. I have ensured the project conforms to the Maven Standard Directory layout. I have used g8 to install sbt. I have been able to run sbt successfully and am now trying to write some tests using scalatest.
My classSpec.scala cannot see the classes within the rest of the project.
Given the File Structure is as follows:
~/projects/scala-project/build.sbt
~/projects/scala-project/project/build.properties
~/projects/scala-project/src/main/scala/class.scala
~/projects/scala-project/src/test/scala/classSpec.scala
Within the build.sbt file I have:
name := "Scala Project"
organization := "com.examples"
version := "0.1.0-SNAPSHOT"
scalaVersion := "2.11.2"
crossScalaVersions := Seq("2.10.4", "2.11.2")
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "2.2.1",
"org.scalacheck" %% "scalacheck" % "1.11.5"
)
initialCommands := "import com.example.scalaproject._"
Does anyone know if this is the issue?
I am new to scala and am currently trying to setup IntelliJ IDEA 13.1 with the Scala plugin. It has support for SBT. I have simply followed the basic tutorial for creating a new project for SBT here: http://confluence.jetbrains.com/display/IntelliJIDEA/Getting+Started+with+SBT
Currently my build.sbt file is:
name := "scalasandpit"
version := "1.0"
scalaVersion := "2.10"
libraryDependencies += "org.scalatest" % "scalatest_2.10" % "2.1.0" % "test"
autoAPIMappings := true
This pulls down various jar binaries, but no sources and no javadoc. I wondered if there is a way to have both sources and javadoc work with IntelliJ and SBT. I think I'm missing something.
There seem to be two issues: getting sbt to pull down sources and docs, and then getting Idea to show them to you. To solve the former problem see the sbt documentation -- about half way down there's a section called "Download Sources" which tells you what to add to your build.sbt:
libraryDependencies +=
"org.scalatest" % "scalatest_2.10" % "2.1.0" % "test" withSources() withJavadoc()
I am building a simple Scala project with SBT 0.11.
All the code files are in ~/MyProject/src/main/scala/
~/MyProject/build.sbt is the following
name := "MyProject"
version := "1.0"
scalaVersion := "2.9.1"
libraryDependencies ++= Seq(
"mysql" % "mysql-connector-java" % "5.1.+",
"c3p0" % "c3p0" % "0.9.1.2",
"org.apache.commons" % "commons-lang3" % "3.0.1",
"commons-lang" % "commons-lang" % "2.6",
"javassist" % "javassist" % "3.12.1.GA"
)
~/MyProject/project/Build.scala is the following
import sbt._
object MyProjectBuild extends Build {
lazy val MyProject = Project("MyProject", file("."))
}
This seems to work almost fine. The project does compile and run. The project name is set correctly (if I don't use Build.scala, then the name seems to appear something like "default-????", despite it being specified in build.sbt).
But the problem is that dependencies do not seem to work - update command doesn't download anything. How to fix this? Do I need to specify my dependencies in Build.scala rather than in build.sbt in this case?
Is it possible that you've already retrieved the project dependencies, but don't realize it because they are stored in the Ivy cache? You can view the managed classpath from the SBT console with the command
show managed-classpath
Recent versions of SBT do not store the managed dependencies in the project directory, unless the project is configured to do so. If you want, you can add the following to your build.sbt file:
retrieveManaged := true
This will create a ~/MyProject/lib_managed/ directory and contents.