I am new to scala and am currently trying to setup IntelliJ IDEA 13.1 with the Scala plugin. It has support for SBT. I have simply followed the basic tutorial for creating a new project for SBT here: http://confluence.jetbrains.com/display/IntelliJIDEA/Getting+Started+with+SBT
Currently my build.sbt file is:
name := "scalasandpit"
version := "1.0"
scalaVersion := "2.10"
libraryDependencies += "org.scalatest" % "scalatest_2.10" % "2.1.0" % "test"
autoAPIMappings := true
This pulls down various jar binaries, but no sources and no javadoc. I wondered if there is a way to have both sources and javadoc work with IntelliJ and SBT. I think I'm missing something.
There seem to be two issues: getting sbt to pull down sources and docs, and then getting Idea to show them to you. To solve the former problem see the sbt documentation -- about half way down there's a section called "Download Sources" which tells you what to add to your build.sbt:
libraryDependencies +=
"org.scalatest" % "scalatest_2.10" % "2.1.0" % "test" withSources() withJavadoc()
Related
I am trying to setup IntelliJ for spark 2.11 but it is very daunting and after days I have not been able to compile a simple instruction such as with "spark.read.format" which is not found in main core and sql spark libraries.
I have seen a few posts on the subject but with none resolved. Does anyone have some experience with perhaps a working sample program I can start with?
Could it be that it would be easier with Eclipse?
Many thanks in advance for your answers,
EZ
build project in Intellij using with scala 2.11 and sbt 0.13: then ensure that your plugins.sbt contains as below:
logLevel := Level.Warn
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.13.0")
then your build.sbt must contain as below:
scalaVersion := "2.11.8"
val sparkVersion = "2.1.0"
libraryDependencies += "org.apache.spark" %% "spark-core" % sparkVersion %"provided"
libraryDependencies += "org.apache.spark" %% "spark-sql" % sparkVersion %"provided"
Then write your code, click Terminal in Intellij and type sbt assembly: you can ship that jar to remote cluster, otherwise run from Intelij locally, let me know how it goes
Kindly consider if there's something very stupid I am missing, I am very new to this.
Creating this example on Eclipse (Scala-ide). When I create the CorrelationJobTest.scala class, there's a problem in importing classes from specs2 package.
import org.specs2.runner.JUnitRunner
The error I am getting is pretty obvious,
object specs2 is not a member of package org
Even after creating a build.sbt file, adding the dependencies, and doing a sbt clean, the error is still showing.
Below is my build.sbt
name := "SparkCorrelation"
organization := "com.xyz"
version := "0.1"
/* scala versions and options */
scalaVersion := "2.10.4"
// These options will be used for *all* versions.
javacOptions ++= Seq("-Xlint:unchecked", "-Xlint:deprecation")
scalacOptions ++= Seq("-feature", "-unchecked", "-deprecation")
/* dependencies */
libraryDependencies ++= Seq("org.specs2" %% "specs2-core" % "3.7" % "test")
scalacOptions in Test ++= Seq("-Yrangepos")
EDIT:
I faced this problem with apache.spark packages too. Then I included the jar spark-assembly-1.6.0-hadoop-2.6.0.jar and it got solved. Isn't there a library for specs2 that I can import in Eclipse?
Got it working. Here's what I did:
Deleted the project from eclipse
Did sbt eclipse
Imported the project into eclipse again.
Added dependencies in build.sbt
Did sbt clean
The problem was that the project was directly created in eclipse and was not able to link to sbt build file.
You need to add the junit module to your dependencies
libraryDependencies ++= Seq(
"org.specs2" %% "specs2-core" % "3.7" % "test",
"org.specs2" %% "specs2-junit" % "3.7" % "test")
I have a spark streaming Scala project which uses Apache NiFi receiver. The projects runs fine under Eclipse/Scala IDE and now I want to package it for deployment now.
When I add it as
libraryDependencies += "org.apache.nifi" %% "nifi-spark-receiver" % "0.3.0"
sbt assumes it's a Scala library and tries to resolve it.
How doe I add NiFi receiver and all it's dependencies to project's SBT file?
Also, is it possible to pint dependencies to local directories instead of sbt trying to resolve?
Thanks in advance.
Here is my sbt file contents:
name := "NiFi Spark Test"
version := "1.0"
scalaVersion := "2.10.5"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.2" % "provided"
libraryDependencies += "org.apache.nifi" %% "nifi-spark-receiver" % "0.3.0"
libraryDependencies += "org.apache.nifi" %% "nifi-spark-receiver" % "0.3.0"
Double % are used for adding scala version as suffix to the maven artefact. It is required because different scala compiler versions produces incompatible bytecode. If you are would like to use java library from maven, then you should use single % character
libraryDependencies += "org.apache.nifi" % "nifi-spark-receiver" % "0.3.0"
I also found that I can put libraries the project depends on into the lib folder and they will be picked up during assembly.
I am new at scala. And for start I want to use Intellij 13.1.5 IDE.
However IDE can't attach sources. Here is how it looks for AnyVal:
Search at internet can't find any source.
I tried Attach sources and attach unpacked scala archive. It doesn't work either.
UPDATE:
Here is sbt configuration:
name := "scalatest-selenium"
version := "1.0"
scalaVersion := "2.11.1"
libraryDependencies ++= Seq(
"net.sourceforge.htmlunit" % "htmlunit" % "2.14",
"org.seleniumhq.selenium" % "selenium-java" % "2.42.2",
"org.scalacheck" % "scalacheck_2.10" % "1.11.4" % "test",
"org.scalatest" % "scalatest_2.11" % "2.2.0" % "test"
)
testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-u", "target/test-reports")
How to solve this trouble?
I get rid of this trouble at the following way:
removed the .sbt directory in your Home Folder.
When you run sbt again, the new folder is created in the correct format and the error goes away.
build.sbt file:
name := "Bag"
version := "0.7.252"
scalaVersion := "2.11.3"
libraryDependencies ++= Seq(
"org.scalatest" % "scalatest_2.11" % "2.1.3" % "test",
"org.scala-lang.modules" %% "scala-swing" % "1.0.1",
"org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.2"
)
The project compile properly with 2.11.2, but on 2.11.3 I get this.
Scala 2.11.3 is not yet officially released, although as you have witnessed, the artifact was already pushed to Maven Central.
I think that version will be "pulled" because of a binary incompatibility bug introduced in collections. See SI-8899 and SI-8900. Stick to 2.11.2 until a new version (2.11.4?) will be announced.
I'm not sure I understand what is going on in your case, reading the pastbin, but I suggest you open another ticket unless it clearly stems from either of these two issues.
Scala 2.11.3 is not officially released and not recommend. If you want to read the whole story see this.
Scala 2.11.4 should be used instead.