Why is this Scala Maven POM file isn't good? - scala

Question

I got a new Spark app I'm writing with Scala on Maven, and I just found out I can't even run a "Hello World" for some reason. While compilation works fine trying to run the jar itself ends up with an error.
EDIT: Removed <scope>provided</scope> from dependencies, getting another error.
This is the POM file:
<modelVersion>4.0.0</modelVersion>
<groupId>com.app</groupId>
<artifactId>deviceScore</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<name>Device Score</name>
<properties>
<spark.version>1.6.2</spark.version>
<app.main.class>com.app.deviceScore.App</app.main.class>
</properties>
<profiles>
<profile>
<id>scala-2.10</id>
<properties>
<scala.version>2.10.6</scala.version>
<scala.binary.version>2.10</scala.binary.version>
</properties>
</profile>
<profile>
<id>scala-2.11</id>
<properties>
<scala.version>2.11.8</scala.version>
<scala.binary.version>2.11</scala.binary.version>
</properties>
</profile>
</profiles>
<repositories>
<repository>
<id>maven-repo</id>
<name>Maven Repository</name>
<url>http://repo1.maven.apache.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>s3.release</id>
<url>s3://clojure-deps/releases</url>
</repository>
<repository>
<id>apache-repo</id>
<name>Apache release repo</name>
<url>https://github.com/adatao/mvnrepos/tree/master/releases/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-Tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</pluginRepository>
<pluginRepository>
<id>protoc-plugin</id>
<url>http://sergei-ivanov.github.com/maven-protoc-plugin/repo/releases/</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-compiler</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_${scala.binary.version}</artifactId>
<version>2.2.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.10.11</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.0</version>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-compiler</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-reflect</artifactId>
<version>${scala.version}</version>
</dependency>
</dependencies>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
<jvmArgs>
<jvmArg>-Xms256m</jvmArg>
<jvmArg>-Xmx2048m</jvmArg>
</jvmArgs>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>${app.main.class}</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
The App is as simple as:
package com.app.deviceScore
object App {
def main(args: Array[String]) = {
println ("Hi!")
}
}
When I run mvn clean package -P scala-2.11 the build goes fine, but when I try to run the jar I get using java -jar target/deviceScore-1.0.0.jar i get:
Error: Could not find or load main class com.app.deviceScore.App
When trying scala target/deviceScore-1.0.0.jar I get:
java.lang.ClassNotFoundException: com.app.deviceScore.App
at scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:63)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:101)
at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:22)
at scala.tools.nsc.JarRunner$.run(MainGenericRunner.scala:13)
at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
at scala.tools.nsc.JarRunner$.runJar(MainGenericRunner.scala:25)
at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:69)
at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:87)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:98)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
What am I missing? What else is supposed to be included in the POM file in order to simply run the jar?

Answer 1

You've defined scala-library (and other Scala dependencies) as provided - which means they don't get packaged in the jar, but are rather expected to be provided at runtime externally, and yet they are not - hence these classes are missing at runtime.
If you remove the <scope>provided</scope> from all dependencies (perhaps scala-compiler can remain provided), this should work.
EDIT:
Per your update - not sure why you get that specific error (main class should be found), but seems like there's more to be fixed here: when you use maven's maven-jar-plugin, it builds the jar without including its dependencies, which means you still won't have Scala's classes available.
Instead you can use maven's maven-assembly-plugin which can create a "fat jar", with its dependencies included. Do that by replacing the jar plugin with:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.6</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>${app.main.class}</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>assemble-all</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
Then, run:
> mvn clean package -P scala-2.11
> java -jar target/deviceScore-1.0.0-jar-with-dependencies.jar
Hi!

Answer 2

To run a Scala program packaged in a runnable jar file, use the scala command instead of java -jar:
scala target/deviceScore-1.0.0.jar
If you want to run it with java -jar, then you must make sure that the Scala library is included in the classpath. By setting the scope of the Scala library to provided, you excluded it from the runtime classpath.

Answer 3

Got it to work. In the plugins section, I used these three:
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>${app.main.class}</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>${app.main.class}</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
The first one is different than the scala-maven-plugin used in the previous pom file. The second plugin makes the jar executable when using scala target/deviceScore-1.0.0.jar, and the third as mentioned by Tzach Zohar creates a fat jar which is executable with java -jar target/deviceScore-1.0.0-jar-with-dependencies.jar