I am a noob to Spark and Intellij.I want to run Spark using Scala
I initially installed Scala 2.12 and created the SBT accordingly.Then I got a NoSuchMethod runtime error on
sc = new SparkContext(conf)
From the solution posted NoSuchMethodError when using Sparka and IntelliJ I used Scala version 2.11.3 while creating the project and used the SBT
version := "0.1"
scalaVersion := "2.11.3"
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.0.2"
I am now getting the error
Error:scalac: No 'scala-library*.jar' in Scala compiler classpath in Scala SDK SBT: org.scala-lang:scala-library:2.11.3:jar
This is the library on the External libraries section
I tried creating the project from scratch and Cache Invalidate/Restart option.Same result
I also tried downloading via Maven through File -> Project Structure.Only found spark-core 2.10.Showed the same NoSuchMethod Error
Can anyone identify the problem?
It clearly says that, It could not find scala-library*.jar file
So go to
"Project Structure -> Modules"
And see these jar files
scala-compiler.jar, scala-library.jar, scala-reflect.jar
If they are absent from the Modules add them manually or Reinstall the scala and scala plugin.
Hope this should work!
Related
I am trying to load an xml file using scala-xml_2.12-1.0.6.jar but it gives me NullPointerEexception while loading
Following is my line of code to load xml
import scala.xml.XML
val xml = XML.loadFile("sample.xml")
I have decompiled this jar and is method is present in that jar but for some reasons it is unable to find it in code.
I have Scala 2.13.1 on my system but for this project I am using scala 2.12.1 and it is mentioned in mu built.sbt
scalaVersion := "2.12.1"
I have following dependency in my built.sbt for this xml package
libraryDependencies += "org.scala-lang.modules" %% "scala-xml" % "1.0.6"
If I copy and paste the same code to Scala interactive shell( scala 2.13.1) I get following error
import scala.xml.XML
^
error: object xml is not a member of package scala
did you mean Nil?
Can anyone please identify what am i doing wrong?
Thanks in advance.
I'm not sure how you are loading up the Scala REPL, but as mentioned in "How to use third party libraries with Scala REPL?", you should be launching the the REPL from SBT with sbt console. Your .sbt files will also need to be in scope.
The Scala REPL independently does not deal with .sbt files. They are 2 different tools.
Alternatively you could also install Ammonite-REPL which supports Magic Imports. You will be able to import Maven dependencies with import $ivy.
Scala 2.12 comes with scala-xml and you can remove that dependency as long as you run REPL with sbt console and not your native Scala REPL which is already # Scala 2.13. Otherwise you can also switch to Scala 2.13 for your SBT project.
I'm trying to create jar file from scala file - which will be used for Scalatron bot (http://scalatron.github.io/)
The problem is that if I create jar file using new scala versions 2.10 - 2.13 (these are the only one's that InteliJJ allows me to use) scalatron bot doesn't work.
However I found out that downgrading to scala 2.9.2 fixes the problem (Getting Scalatron to Work (trouble with opcode)).
How to downgrade to 2.9.2 on Intellij?
SBT is fully integrated with IntelliJ Idea, so you can create a SBT project in IntlliJ and make up the build.sbt file by yourself and put in it the scala version you want, compile, package and execute with SBT
name := "scalatron"
version := "0.1"
scalaVersion := "2.9.2"
libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.0-SNAP3" % Test
I have a CLI app which compile only to 2.11 (because of some internal dependency).
I want to package this app as a sbt plugin. This sbt plugin run the app by forking the JVM, running separately with its own classpath to avoid Scala library conflict.
Obviously I need to download the scala 2.11 app with all its dependencies and I am using a custom Configuration for it. My issue is that when I try to list the dependencies it comes with the scala library configured by the project.
Specific code is here : https://github.com/thibaultdelor/CliAppSbtPlugin/blob/master/plugin/src/main/scala/com/thibaultdelor/MyWrapperPlugin.scala#L33
autoScalaLibrary in CliConfig := false,
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-library" % "2.11.12" % CliConfig,
"com.thibaultdelor" % "mycli_2.11" % "0.0.1" % CliConfig
)
val dependencies = (update in CliConfig).value.select(configurationFilter(CliConfig.name))
Here, if the project has the scala version 2.12, dependencies will contains scala-library 2.12 instead of what 2.11 as I would like.
Any help welcome, I am stuck. The sample project is on github and has a failing test case for it.
I'm having problems with my project which I think are due to intellij pulling in a load of scala 2.10 libraries when it created/if I reimport my project.
How can I configure it to either read the version of scala from build.sbt, or manually configure it to compile with scala 2.11.7? I'm running intellij IDEA 14 with the latest updates and it always uses scala 2.10.4.
Make sure you have your Scala Version set in your build.sbt
scalaVersion := "2.11.7"
The rest works seemlessly. Enable "Auto-Import" in IDEA when creating the project.
The 2.10 libraries are used by SBT itself. If you check the Dependencies tab for your modules, you shouldn't see them. If you do, you likely have an error in libraryDependencies in build.sbt.
I am playing with the spray template project and Scala 2.9.0.1.
I want to work with Eclipse and so I've added the following lines to the build.sbt file
of the spray template project.
resolvers += {
val typesafeRepoUrl = new java.net.URL("http://repo.typesafe.com/typesafe/releases")
val pattern = Patterns(false, " [organisation]/[module]/[sbtversion]/[revision]/[type]s/[module](-[classifier])-[revision]. [ext]")
Resolver.url("Typesafe Repository", typesafeRepoUrl)(pattern)
}
libraryDependencies <<= (libraryDependencies, sbtVersion) { (deps, version) =>
deps :+ ("com.typesafe.sbteclipse" %% "sbteclipse" % "1.3-RC2" extra("sbtversion" -> version))
}
I always get this error:
com.typesafe.sbteclipse#sbteclipse_2.9.0-1;1.3-RC2: not found
Of course the directory at the typesafe server does not exist. There is only:
http://repo.typesafe.com/typesafe/releases/com.typesafe.sbteclipse/sbteclipse_2.8.1/
but sbt is trying to get:
http://repo.typesafe.com/typesafe/releases/com.typesafe.sbteclipse/sbteclipse_2.9.0-1/0.10.1/1.3-RC2/jars/sbteclipse_2.9.0-1-1.3-RC2.jar
I figured out that 2.8.1 is the scala version sbt uses internaly. But my
sbt version (0.10) uses the scala version set in the build.sbt (scalaVersion := "2.9.0-1") to build the url to download sbteclipse.
Does anyone know how to set this up correctly? How do I tell the build.sbt file that I want to use Scala 2.9 but to look for the sbteclipse plugin at the correct URL?
Are there any plans to include something like sbteclipse to the sbt standard distribution? That would be more than welcome and probably help a lot of beginners with Scala, sbt and Eclipse.
Claus
try some thing like "com.typesafe.sbteclipse" % "sbteclipse_2.8.1" % "1.3-RC2....."
using "%%" will automicly add scala version to the library name. using "%" will not.
There is also a shorter way to get sbteclipse into use. Create a file ~/.sbt/plugins/plugins.sbt with this content:
addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "2.1.0")