Create executable jar file using maven on Scala - scala

Question

I have been trying to create a simple JAR file from a scala script with maven and pass it to spark-submit. For that, I am using the following version of maven
> mvn -version
Apache Maven 3.6.3
Maven home: /usr/share/maven
Java version: 1.8.0_312, vendor: Private Build, runtime: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.4.0-100-generic", arch: "amd64", family: "unix"
To start, I create the working directory with the following command
mvn archetype:generate -DgroupId=com.xom -DartifactId=workflow -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
After creating the directory with maven, I add the MainTest.scala file into src/main/java/com/xom/. this Scala file is the demo code in spark quickstart
package com.xom
import org.apache.spark.sql.SparkSession
object MainTest {
def main(args: Array[String]) {
val logFile = "/opt/spark/README.md" // Should be some file on your system
val spark = SparkSession.builder.appName("Simple Application").config("spark.master", "local").getOrCreate()
val logData = spark.read.textFile(logFile).cache()
val numAs = logData.filter(line => line.contains("a")).count()
val numBs = logData.filter(line => line.contains("b")).count()
println(s"Lines with a: $numAs, Lines with b: $numBs")
spark.stop()
}
}
According to maven website, in order to create an executable JAR file, I replaced the default <build> </build> in pom.xml for the one provided in the previous link, passing the path of my class.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.xom.MainTest</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Then I run
mvn clean install
which creates the jar file. When I try to access it with java -jar, I get the following error message:
Error: Could not find or load main class com.xom.MainTest
Caused by: java.lang.ClassNotFoundException: com.xom.MainTest
I cannot launch it with spark-submit neither (I get Error: Failed to load class MainTest.).
I am really new to this world of scala & maven, and I have searched everywhere for a work around, but so far I was not able to figure out why this is happening. Most of the solutions mention changing some configs in the pom.xml, but everything I have tried did not work.