Building Scala Object, Class Not Found - scala

I am trying to build my Scala Project with Maven, I have used this sample POM as a guide line:
http://ryancompton.net/2014/05/19/sample-pomxml-to-build-scala--jar-with-dependenciesjar/
But I am getting this:
Exception in thread "main" java.lang.ClassNotFoundException: com.organization.XXX.dataProfiling
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:195)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:122)
Process finished with exit code 1
A colleague has managed to build the code fine with SBT but our organization uses Maven as corporate standard. This happens have it builds the jar with maven clean package in IntelliJ's

There are three things to check.
Check if you have defined the "mainClass" manifest in your POM, like the following. Your built fat jar needs to know which class is the main class.
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.5.5</version>
<configuration>
<archive>
<manifest>
<mainClass>org.sample.App</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</project>
Make sure your main class has same package name as specified in the "mainClass" manifest in POM.
For example, if your main class is specified as "org.sample.App" in POM, your main class should have package "org.sample" and class name of "App".
Ensure your main class defines the "main" method, like the following.
object HelloWorld {
def main(args: Array[String]) {
println("Hello, world!")
}
}

Related

Running Scala Maven Project

I am a scala beginner, and was running a starter project on Maven and using IntelliJ as IDE.
This is the link to project on github which I am using Github project and I compiled the project against OpenJDK8.
The HelloJava class runs successfully, however, when I try running the HelloScala class I come across the following error:
java -cp scala-maven-example-1.0.0-SNAPSHOT.jar com.jesperdj.example.HelloScala
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: scala/Function0
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: scala.Function0
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
Scala has its own runtime libraries above JVM. A compiled Scala .class has imported some classes from Scala runtime libraries. When you try to run a Scala .class file, you need to append the Scala runtime to the classpath.
If you are running inside IntelliJ IDEA, the Scala Plugin will automatically do this, but when you run java from command line, you should do this yourself.
If you are using Maven, then you can add a <plugin>. From Scala Docs -> Scala with Maven -> Creating a Jar:
<build>
...
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.your-package.MainClass</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
After adding this, mvn package will also create [artifactId]-[version]-jar-with-dependencies.jar under target. Note: this will also copy the Scala library into your Jar. This is normal. Be careful that your dependencies use the same version of Scala, or you will quickly end up with a massive Jar.
To run your programs on Intellij you need to install scala sdk along with JDK.
Once you have scala SDK in your Intellij classpath you are good to go with scala coding.
Tick the checkbox 'include dependencies with "Provided" scope' under: 'Run\Debug Configurations'.
(Helps if yours is a maven project and you added the 'scala-library' dependency scope as 'Provided')
'Run\Debug Configurations'

GsonBuilder cannot be found in JavaFX + Maven project's runnable jar

Background
I'm developing a Maven project + JavaFX in Eclipse.
My application uses Gson for serialization.
Of course, when you haven't included Gson.jar then Eclipse will complaint that GsonBuilder (or any part of gson library for that matter) cannot be found.
I included the library through Maven dependency
After this, Eclipse is cool and I can import GsonBuilder (or any part of gson library for that matter).
Everything is good.
And mvn clean install works with no error.
The Problem:
I go to target directory and execute java -jar myapplication.jar
Exception in thread "JavaFX Application Thread" java.lang.NoClassDefFoundError: com/google/gson/GsonBuilder
at app.MyApplication.<clinit>(MyApplication.java:68)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at com.sun.javafx.application.LauncherImpl.lambda$launchApplicationWithArgs$153(LauncherImpl.java:352)
at com.sun.javafx.application.LauncherImpl$$Lambda$46/683287027.run(Unknown Source)
at com.sun.javafx.application.PlatformImpl.lambda$runAndWait$172(PlatformImpl.java:326)
at com.sun.javafx.application.PlatformImpl$$Lambda$47/254413710.run(Unknown Source)
at com.sun.javafx.application.PlatformImpl.lambda$null$170(PlatformImpl.java:295)
at com.sun.javafx.application.PlatformImpl$$Lambda$49/1171144110.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.application.PlatformImpl.lambda$runLater$171(PlatformImpl.java:294)
at com.sun.javafx.application.PlatformImpl$$Lambda$48/553264065.run(Unknown Source)
at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
Caused by: java.lang.ClassNotFoundException: com.google.gson.GsonBuilder
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 13 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:767)
Caused by: java.lang.NullPointerException
at com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:383)
at com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:328)
... 5 more
So, can you please help me out here? What's happening and how can I fix this?
Why is it not detected? I mean, the compiler compiles without error. So, gson must be there.
I also tried to exclude and reinclude the library. No result.
#dunni is right.
I used maven-jar-plugin instead maven-assembly-plugin.
After adding my pom.xml with
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>my.main.class</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
Then, execute:
mvn clean compile assembly:single
mvn install
there is an executable jar with postfix jar-with-dependencies in target directory.
Afterwards, executing java -jar target/myapp-jar-with-dependencies.jar works like a charm.

How to run a spark example program in Intellij IDEA

First on the command line from the root of the downloaded spark project I ran
mvn package
It was successful.
Then an intellij project was created by importing the spark pom.xml.
In the IDE the example class appears fine: all of the libraries are found. This can be viewed in the screenshot.
However , when attempting to run the main() a ClassNotFoundException on SparkContext occurs.
Why can Intellij not simply load and run this maven based scala program? And what can be done as a workaround?
As one can see below, the SparkContext is looking fine in the IDE: but then is not found when attempting to run:
The test was run by right clicking inside main():
.. and selecting Run GroupByTest
It gives
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/SparkContext
at org.apache.spark.examples.GroupByTest$.main(GroupByTest.scala:36)
at org.apache.spark.examples.GroupByTest.main(GroupByTest.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkContext
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
Here is the run configuration:
Spark lib isn't your class_path.
Execute sbt/sbt assembly,
and after include "/assembly/target/scala-$SCALA_VERSION/spark-assembly*hadoop*-deps.jar" to your project.
This may help IntelliJ-Runtime-error-tt11383. Change module dependencies from provide to compile. This works for me.
You need to add the spark dependency. If you are using maven just add these lines to your pom.xml:
<dependencies>
...
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
...
</dependencies>
This way you'll have the dependency for compiling and testing purposes but not in the "jar-with-dependencies" artifact.
But if you want to execute the whole application in an standalone cluster running in your intellij you can add a maven profile to add the dependency with compile scope. Just like this:
<properties>
<scala.binary.version>2.11</scala.binary.version>
<spark.version>1.2.1</spark.version>
<spark.scope>provided</spark.scope>
</properties>
<profiles>
<profile>
<id>local</id>
<properties>
<spark.scope>compile</spark.scope>
</properties>
<dependencies>
<!--<dependency>-->
<!--<groupId>org.apache.hadoop</groupId>-->
<!--<artifactId>hadoop-common</artifactId>-->
<!--<version>2.6.0</version>-->
<!--</dependency>-->
<!--<dependency>-->
<!--<groupId>com.hadoop.gplcompression</groupId>-->
<!--<artifactId>hadoop-gpl-compression</artifactId>-->
<!--<version>0.1.0</version>-->
<!--</dependency>-->
<dependency>
<groupId>com.hadoop.gplcompression</groupId>
<artifactId>hadoop-lzo</artifactId>
<version>0.4.19</version>
</dependency>
</dependencies>
<activation>
<activeByDefault>false</activeByDefault>
<property>
<name>env</name>
<value>local</value>
</property>
</activation>
</profile>
</profiles>
<dependencies>
<!-- SPARK DEPENDENCIES -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
<scope>${spark.scope}</scope>
</dependency>
</dependencies>
I also added an option to my application to start a local cluster if --local is passed:
private def sparkContext(appName: String, isLocal:Boolean): SparkContext = {
val sparkConf = new SparkConf().setAppName(appName)
if (isLocal) {
sparkConf.setMaster("local")
}
new SparkContext(sparkConf)
}
Finally you have to enable "local" profile in Intellij in order to get proper dependencies. Just go to "Maven Projects" tab and enable the profile.

Error running Maven Project in Eclipse using jdk1.5

OS: Ubuntu 12.10
Eclipse: 4.2
Java: jdk1.5.0_22 and jdk1.7
Apache Maven: 3.0.4
Maven home: /usr/share/maven
m2e: 1.2.020120903-1050
I have created a very simple "Hello, World" Maven Project in Eclipse, in order to test it to migrate our own project (which uses 1.5). My pom.xml is as follows:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>br.com.contmatic</groupId>
<artifactId>maven-test</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>MavenTest</name>
<description>teste do maven</description>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
I have set J2SE-1.5 (jdk1.5.22) in my project build path.
Yet, trying to run Maven Clean or Maven Install from within Eclipse, I get the following error:
Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version number in .class file
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClassFromSelf(ClassRealm.java:386)
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:42)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
at org.apache.maven.cli.MavenCli.container(MavenCli.java:375)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:191)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:592)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Googling around, I found out that this happens when the jre version used to compile is different than the one used to run. Changing everything to 1.7 (both pom.xml and build path) fixes the errors, which suggest that it either compiles or runs using java 1.7.
But I can't seem to find the error in my configuration.
when running
mvn clean install
in the terminal, maven builds the project without errors.
What could I be doing wrong in Eclipse?
Turns out some of my Maven dependencies weren't compiled with 1.5, which caused the problem. It wasn't happening in the console because I had already set java 1.7 as default.
Fortunately, I was able to build my project using 1.7 and specifying 1.5 to run compile and execute. Here's how:
in Eclipse>Run>Run Configurations>Maven Build>New, I had this configuration:
Main Tab
Base Directory:
${project_loc}
Goals:
clean install
I just chose the JRE tab and chose and Alternate JRE, using my 1.7 SDK installation.
Next, I had to include 1.5 as source and target for the maven-compiler-plugin my pom.xml, as such:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<encoding>ISO-8859-1</encoding>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
And it works.

Maven dependency given but still one class not found on execution

I have added maven dependencies in pom.xml for cdk but still i get one error of class not found when executing the jar file.
dyna218-128:spark4vs laeeqahmed$ java -cp target/spark4vs-1.0-SNAPSHOT.jar se.uu.farmbio.spark4vs.RunPrediction
Exception in thread "main" java.lang.NoClassDefFoundError: org/openscience/cdk/interfaces/IAtomContainer
Caused by: java.lang.ClassNotFoundException: org.openscience.cdk.interfaces.IAtomContainer
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
POM.XML is as under
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
<dependency><!-- SVM depedency -->
<groupId>tw.edu.ntu.csie</groupId>
<artifactId>libsvm</artifactId>
<version>3.1</version>
</dependency>
<dependency>
<groupId>org.openscience.cdk</groupId>
<artifactId>cdk</artifactId>
<version>1.4.7</version>
</dependency>
<repositories>
<repository>
<id>3rdparty</id>
<url>https://maven.ch.cam.ac.uk/content/repositories/thirdparty/</url>
</repository>
</repositories>
Maven dependencies are for building project. Maven Jar Plugin is not using it when JAR id packaged. So you cant run without additional work.
Hovewer there are many solution for this. For example you can use Maven One Jar Plugin and package all dependencies into JAR - but this is not always usable
http://onejar-maven-plugin.googlecode.com/svn/mavensite/usage.html
You can create archive with jar-dependencies
http://maven.apache.org/plugins/maven-assembly-plugin/descriptor-refs.html#jar-with-dependencies
You can merge all jars into one with Maven Shade Plugin
http://maven.apache.org/plugins/maven-shade-plugin/