I am using Scala on Eclipse Luna and trying to connect to Cassandra. My code is showing the error apache object is not a member of package org on the following line:
import org.apache.spark.SparkConf
I already imported the Scala and Spark libraries into the project. Does someone know how can I make my program import Spark libraries?
Related
Am trying to add some imports , but when I add
import org.apache.spark.streaming.kafka.kafkautils its showing the below error
object kafka is not a member of the object org.apache.spark.streaming.kafka.kafkautils
working on eclipse with
scala ide 4.7
version 2.11.11,
spark-2.3.0-bin-hadoop2.7 jar files,
kafka 2.11 jars,
spark-streaming-kafka-0-10_2.11-2.3.0 jar
If you are using spark-streaming-kafka-0-10_2.11-2.3.0 jar then the KafkaUtils is available in org.apache.spark.streaming.kafka010 this package.
So import
import org.apache.spark.streaming.kafka010.KafkaUtils
and not
import org.apache.spark.streaming.kafka.kafkautils
Hope this hepls!
As shown in image, its giving error when i am importing the Spark packages. Please help. When i hover there, it shows "object apache is not a member of package org".
I searched on this error, it shows spark jars has not been imported. So, i imported "spark-assembly-1.4.1-hadoop2.2.0.jar" too. But still same error.Below is what i actually want to run:
import org.apache.spark.{SparkConf, SparkContext}
object ABC {
def main(args: Array[String]){
//Scala Main Method
println("Spark Configuration")
val conf = new SparkConf()
conf.setAppName("My First Spark Scala Application")
conf.setMaster("spark://ip-10-237-224-94:7077")
println("Creating Spark Context")
}
}
Adding spark-core jar in your classpath should resolve your issue. Also if you are using some build tools like Maven or Gradle (if not then you should because spark-core has lot many dependencies and you would keep getting such problem for different jars), try to use Eclipse task provided by these tools to properly set classpath in your project.
I was also receiving the same error, in my case it was compatibility issue. As Spark 2.2.1 is not compatible with Scala 2.12(it is compatible with 2.11.8) and my IDE was supporting Scala 2.12.3.
I resolved my error by
1) Importing the jar files from the basic folder of Spark. During the installation of Spark in our C drive we have a folder named Spark which contains Jars folder in it. In this folder one can get all the basic jar files.
Goto to Eclipse right click on the project -> properties-> Java Build Path. Under 'library' category we will get an option of ADD EXTERNAL JARs.. Select this option and import all the jar files of 'jars folder'. click on Apply.
2) Again goto properties -> Scala Compiler ->Scala Installation -> Latest 2.11 bundle (dynamic)*
*before selecting this option one should check the compatibility of SPARK and SCALA.
The problem is Scala is NOT backward compatible. Hence each Spark module is complied against specific Scala library. But when we run from eclipse, we have one SCALA VERSION which was used to compile and create the spark Dependency Jar which we add to the build path, and SECOND SCALA VERSION is there as the eclipse run time environment. Both may conflict.
This is a hard reality, although, we wish Scala to be ,backward compatible. Or at least a complied jar file created could be backward compatible.
Hence, the recommendation is , use Maven or similar where dependency version can be managed.
If you are doing this in the context of Scala within a Jupyter Notebook, you'll get this error. You have to install the Apache Toree kernel:
https://github.com/apache/incubator-toree
and create your notebooks with that kernel.
You also have to start the Jupyter Notebook with:
pyspark
I have spark 1.6.0 installed and would like to import it in Scala 2.11. I can use spark-shell, which has org.apache in its path. How do I put it in my system installation of scala's path?
Does anybody have any experience with trying to get swing to work in eclipse, I can import it fine into scala using the command line interface, but when i try and use it in an eclipse scala project I get the following error:
import scala.swing._
"object swing is not a member of package scala"
Any help would be much appreciated.
If you are missing the library (as Rüdiger said, it is a separate dependency now), you can find how to add it to your build system here (for Scala 2.11):
http://search.maven.org/#artifactdetails|org.scala-lang.modules|scala-swing_2.11|1.0.1|bundle
Look under "Dependency Information", e.g. for sbt if you use that.
I am trying out Lift Framework, however I am unable to get it working properly with eclipse.
I get the following error:
import org.eclipse.jetty.server.handler.ContextHandler
import org.eclipse.jetty.server.nio.SelectChannelConnector
import org.eclipse.jetty.server.{Handler, Server}
import org.eclipse.jetty.webapp.WebAppContext
object eclipse is not a member of package org on resource: RunWebApp.scala
Your classpath is incomplete. How do you import the project in Eclipse? Are you using Sbt and one of the plugins for exporting an Eclipse project?
Check out this tutorial on the Scala IDE website: Setup and use Lift framework 2.4 in Scala IDE 2.0.