Error: Could not find or load main class while trying to run spark from eclipse(scala ide 4.3) - eclipse

i have build the project using sbt and imported into eclipse . while trying to execute from run configuration i cannot find my main class and on running the application its prompted with the following error : could not find the main class.i have installed spark-hadoop version 1.4 and scala version 2.10.6 in the local machine also changed the scala compiler version to 2.10.6 in scala ide . The same error is produced while trying build spark from eclipse using maven . Please advice..

Check if Your main class in on build path.
Project -> Properties -> Java Build Path -> Source tab
If not add appropriate folder(s).

Add the folder where classes are generated to build path -> configure build path -> java build path -> source -> "Default Output Folder" text box.

Check if you have .scala files included in compilation path.
Even if you have the source folder included, it may be possible that .scala files are not being picked up.
The following might help, I was facing the same issue and it solved the problem for me:
build path -> configure build path -> java build path -> source -> add **/*.scala in the Included section.

Related

How to export Scala Spark project to jar

Im working on Scala/Spark project,i would like to export my project to jar file and run it into spark via spark-submit.
I tried this solution :
File -> Project Structure -> Artifacts -> + -> Jar -> From modules with dependencies -> Selected Main Class after browsing -> selected extract to the target jar -> Directory for META-INF automatically gets populated -> OK -> Apply -> OK -> Build -> Build Artifacts -> Build.
But i didn't find my main class on the jar file so i can't run it.
The basic Idea that you can follow:
As you are working on Scala
You can use sbt as your build management system to add all the dependencies to your project
You can use sbt assembly pluggin to build fat jar
Export this fat jar into your cluster to submit the spark jobs.
pls use Google to get more details...
or you can use this project https://github.com/khodeprasad/spark-scala-examples to start with and integrate sbt assembly plugin to create fat jars by following their documentation https://github.com/sbt/sbt-assembly

sbt Project name must be valid Scala identifier error while creating new project

I wanted to create a new project in scala in intellij. I have chosen Lightbend Project Starter and then Akka Quickstart Scala. On finish I got this error:
sbt Project name must be valid Scala identifier
The name of my project is "akka-demo" but "akkademo" and default "untitled" don't work either. Empty catalog is created. Same with other templates. How can I overcome this?
Env:
Intellij idea ultimate 18.3.4
sbt 1.2.6
As mentioned, akkademo should indeed work.
It sounds like an IntelliJ problem.
If a project directory was created, you can try to remove the .idea directory and reimport the project.
Furthermore, you can download the Akka Quickstart Scala project from Lightbend's website, and start the sbt shell in this directory.
Try to run and compile to make sure your environment is set up correctly.
If everything works with the sbt shell but not in IntelliJ, then open:
IntelliJ -> Preferences -> Build, Execution, Deployment -> Build Tools -> sbt and check the Use sbt shell for imports and for builds.
Then go to the sbt tab in IntelliJ and refresh.

I couldn't build Jhipster sample app

I'm trying to run Jhipster sample app in my Eclipse kepler . I've downloaded the project from here and I've imported it as existing maven project but it couldn't be build . I got this exception :
The type java.util.Optional cannot be resolved. It is indirectly referenced from required .class files AccountResource.java
and many "Undefined CSS file" errors
how can I run this project in my Eclipse ?
Normally you should create your application using the yeoman generator. Install yeoman and execute yo jhipster from your command line.
a)For the java exception basically you need to remove the old version of java from your build path and add support for java 8.
You need to follow these steps
1) Right-click on project » Properties » Java Build Path
2) Select Libraries tab
3) Find the JRE System Library and remove it
4) Click Add Library... button at right side » Add the JRE System Library(Workspace default JRE)
Do not forget to set the Java jdk compiler compliance level to 1.8 (Preference -> Java -> compiler )
b) The jhipster site mentions about the configuration settings(css,javascript and html etc)for eclipse to avoid errors in eclipe. https://jhipster.github.io/configuring_ide_eclipse.html

build paths eclipse java j2ee

I have imported a project (I am very new at this) and I get the following errors:
Project cannot be built until build path errors are resolved
Project FST is missing required library: 'C:program Files/Apache Group/Tomcat 4.1/common/lib/servlet.jar'
Project FST is missing required library: 'C:program Files/Apache Group/Tomcat 4.1/common/lib/struts.jar'
The project cannot be built until build path errors are resolved
Unbound classpath variable: 'TOMCAT_HOME/common/lib/jasper-runtime.jar' in project
Unbound classpath variable: 'TOMCAT_HOME' in project FST
I create a variable called TOMCAT_HOME and give it the proper directory ,but Also, we should change the project classpath to use TOMCAT_HOME rather than the absolute path.
i dont know how to do it (change the project classpath and the absolute path)
thanks !
To answer somewhat indirectly, if you configure your project build with something like Maven or Gradle, so that you can successfully build the project using the corresponding command-line tool, then it should be quite straightforward to import the project into eclipse using the Maven or Gradle eclipse plugin. I think doing so will be worth whatever trouble it causes you in the short run--just take care to make your project structure conform to the usual project structure that Maven expects or you'll be asking for trouble (it should be no problem to do so for greenfield work).

Access configuration resources in Scala IDE

Some of my colleagues use Eclipse 3.7.2 and Scala IDE 2.1 for development. I want to use typesafe's config module for applicaton configuration. I want to use the convention based default configuration location. According to the examples and documentation, the default config can be found at the following path relative to project root
/src/main/resources/application.conf
But when I run my project using Scala IDE's Scala Application loader, the SimpleConfig type is unable to load any configuration values set in this file. An alternative is to pass in a config-file system property via sbt, but I don't want to have to explicitly set this path somewhere. Can anyone point out what I'm doing wrong?
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.foo.dataservices.MyServer.main(MyServer.scala)
Caused by: com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'bar'
at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:115)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:138)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:150)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:155)
at com.typesafe.config.impl.SimpleConfig.getConfigNumber(SimpleConfig.java:170)
at com.typesafe.config.impl.SimpleConfig.getInt(SimpleConfig.java:181)
You need to add your resources folder to your java build path:
Right Click on your project in the Project Explorer
properties -> click Java Build Path -> select "Source" tab
click "Add Folder..." and add your src/main/resources folder
update: if you are using the sbt eclipse plugin you can configure it to automatically add the resources folder to the classpath :
(from: sbteclipse docs)
EclipseKeys.createSrc := EclipseCreateSrc.Default + EclipseCreateSrc.Resource