Maven Build Works, During Deployment Gets Class Not Found - eclipse

Using Eclipse Mars.2 4.5.2
The Maven build is successful. During deployment I get the following error:
java.lang.ClassNotFoundException: com.authy.AuthyApiClient from [Module \"deployment.mavenFlowChartDB5.war:main\" from Service Module Loader]"}}
In the Maven dependency directory I see the related dependency with the class underneath the dependency entry.
Libraries>Maven Dependencies>authy-java-1.1.0.jar>com.auth.AuthyApiCLient
Why doesn't the Deployment see this missing class?
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany</groupId>
<artifactId>mavenFlowChartDB5</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>war</packaging>
<name>mavenFlowChartDB5</name>
<properties>
<endorsed.dir>${project.build.directory}/endorsed</endorsed.dir>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.primefaces</groupId>
<artifactId>primefaces</artifactId>
<version>6.0</version>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.1</version>
</dependency>
<dependency>
<groupId>org.eclipse.persistence</groupId>
<artifactId>eclipselink</artifactId>
<version>2.5.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.eclipse.persistence</groupId>
<artifactId>org.eclipse.persistence.jpa.modelgen.processor</artifactId>
<version>2.5.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-web-api</artifactId>
<version>7.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.2-1002-jdbc4</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.8.2</version>
</dependency>
<dependency>
<groupId>org.jboss.ejb3</groupId>
<artifactId>jboss-ejb3-ext-api</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>com.authy</groupId>
<artifactId>authy-java</artifactId>
<version>1.1.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<compilerArguments>
<endorseddirs>${endorsed.dir}</endorseddirs>
</compilerArguments>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.3</version>
<configuration>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>${endorsed.dir}</outputDirectory>
<silent>true</silent>
<artifactItems>
<artifactItem>
<groupId>javax</groupId>
<artifactId>javaee-endorsed-api</artifactId>
<version>7.0</version>
<type>jar</type>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<url>http://repository.primefaces.org/</url>
<id>PrimeFaces-maven-lib</id>
<layout>default</layout>
<name>Repository for library PrimeFaces-maven-lib</name>
</repository>
<repository>
<id>jboss-public-repository-group</id>
<name>JBoss Public Maven Repository Group</name>
<url>https://repository.jboss.org/nexus/content/groups/public-jboss/</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>jboss-public-repository-group</id>
<name>JBoss Public Maven Repository Group</name>
<url>https://repository.jboss.org/nexus/content/groups/public-jboss/</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</snapshots>
</pluginRepository>
</pluginRepositories>
</project>
This is the updated pom.xml. I still get the same error. I did notice that the maven-compiler-plugin is repeated a few times. I added this part.
<manifest>
<addClasspath>true</addClasspath>
</manifest>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
</manifest>
</archive>
<source>1.8</source>
<target>1.8</target>
<compilerArguments>
<endorseddirs>${endorsed.dir}</endorseddirs>
</compilerArguments>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.3</version>
<configuration>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>${endorsed.dir}</outputDirectory>
<silent>true</silent>
<artifactItems>
<artifactItem>
<groupId>javax</groupId>
<artifactId>javaee-endorsed-api</artifactId>
<version>7.0</version>
<type>jar</type>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

This is probably because you are not adding classpath entry in manifest for mavenFlowChartDB5 war. Could you try adding that using this documentation.

Related

spark kafka streaming code not working on livy

I've been working in some scala codes to run spark streaming for kafka data acquisition. Once updated all dependencies and adding new ones to the compiled jar, I am able to to run it with spark-submit with the code spark-submit /codigos/codigos/adquisicion.jar. But when running it via livy2 I'm getting an exception when creating stream in:
val stream = KafkaUtils.createDirectStream[String, String](
ssc,
PreferConsistent,
Subscribe[String, String](topics, kafkaParams)
)
Code used to run it via livy2 is:
uri<-"http://192.168.0.28:8999/batches"
jsonBody2 = paste0('{ "file": "/user/bigdata/adquisicion.jar",
"numExecutors":1,
"executorCores":3,
"name": "Adquisición Datos",
"args": [""],
"className": "codigos.adquisicionDatos" }')
request <- httr::POST(url = uri, body = jsonBody2, httr::add_headers(.headers = c('Content-Type' = 'application/json', 'X-Requested-by'='user')))
And received exception is the following one:
18/11/06 08:01:00 ERROR ApplicationMaster: User class threw exception: java.lang.NoClassDefFoundError: org/apache/kafka/clients/consumer/Consumer
java.lang.NoClassDefFoundError: org/apache/kafka/clients/consumer/Consumer
at org.apache.spark.streaming.kafka010.ConsumerStrategies$.Subscribe(ConsumerStrategy.scala:256)
at codigos.adquisicionDatos$.main(adquisicionDatos.scala:251)
at codigos.adquisicionDatos.main(adquisicionDatos.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:646)
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.consumer.Consumer
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 8 more
EDIT:
POM file:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>codigos</groupId>
<artifactId>adquisicionDatos</artifactId>
<version>1.2</version>
<packaging>jar</packaging>
<name>Adquisicion Datos Kafka</name>
<properties>
<app.main.class>adquisicionDatos.SparkKMeansApp</app.main.class>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<spark.core.version>2.3.2</spark.core.version>
<!--hbase.version>0.98.7-hadoop1</hbase.version-->
<hbase.version>1.0.1</hbase.version>
<slf4j.version>1.7.5</slf4j.version>
<guava.version>13.0.1</guava.version>
<gson.version>2.2.4</gson.version>
<spark.version>2.3.2</spark.version>
</properties>
<repositories>
<repository>
<id>scalanlp.org</id>
<name>ScalaNLP Maven2 Repository</name>
<url>http://repo.scalanlp.org/repo</url>
</repository>
<repository>
<id>apache release</id>
<url>https://repository.apache.org/content/repositories/releases/</url>
</repository>
<repository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</repository>
<repository>
<id>cloudera.repo</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
<name>Cloudera Repositories</name>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.sameersingh.scalaplot/scalaplot -->
<dependency>
<groupId>org.sameersingh.scalaplot</groupId>
<artifactId>scalaplot</artifactId>
<version>0.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/javax.mail/javax.mail-api -->
<dependency>
<groupId>javax.mail</groupId>
<artifactId>javax.mail-api</artifactId>
<version>1.5.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>${spark.version}</version>
</dependency>
<!--
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
<version>${spark.version}</version>
</dependency>
-->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
<version>${spark.version}</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>0.10.2.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>${spark.version}</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.datastax.spark/spark-cassandra-connector_2.11 -->
<dependency>
<groupId>com.datastax.spark</groupId>
<artifactId>spark-cassandra-connector_2.11</artifactId>
<version>${spark.version}</version>
<!--<version>2.0.5</version>-->
</dependency>
<!-- https://mvnrepository.com/artifact/org.postgresql/postgresql -->
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.1.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients -->
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.10.2.1</version>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<finalName>${project.artifactId}-${project.version}-jar-with-dependencies</finalName>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>codigos.adquisicionDatos</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
I compile it in server and then I copy it to docker (via shared volume cp ~/adquisicionDatos/target/adquisicionDatos1.2-jar-with-dependencies.jar /mnt/codigos/adquisicion.jar) and upload it to HDFS using incrond script to automatically upload files when updated (working tested).
If running with spark-submit under root user everything is working fine but using livy or running under livy user is not working and throwing above's error.

Error: Could not find or load main class scala Ecllipse IDE

I have clone the project through my GitHub repository.
Whenever trying to run, it is giving the error:
Could not find main class.
I have my pom.xml as:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>Scalable ML</artifactId>
<groupId>org.big-mint</groupId>
<version>0.0.1-SNAPSHOT</version>
<name>Scalable ML</name>
<description>
Scalable Machine learning algorithms to process big data.
</description>
<inceptionYear>2017</inceptionYear>
<licenses>
<license>
<name>GNU GENERAL PUBLIC LICENSE, Version 3</name>
<url>http://www.gnu.org/licenses/gpl-3.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<scala.version>2.11.8</scala.version>
<scala.binary.version>2.11</scala.binary.version>
<spark.version>2.1.1</spark.version>
<sansa.version>0.2.0</sansa.version>
<jena.version>3.1.1</jena.version>
</properties>
<dependencies>
<!-- Scala -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<!-- Apache Spark Core -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
</dependency>
<!-- Apache Spark SQL -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
</dependency>
<!-- Apache JENA 3.x -->
<dependency>
<groupId>org.apache.jena</groupId>
<artifactId>apache-jena-libs</artifactId>
<type>pom</type>
<version>${jena.version}</version>
</dependency>
<!-- SANSA RDF -->
<dependency>
<groupId>net.sansa-stack</groupId>
<artifactId>sansa-rdf-spark-bundle_${scala.binary.version}</artifactId>
<version>${sansa.version}</version>
</dependency>
<!-- SANSA OWL -->
<dependency>
<groupId>net.sansa-stack</groupId>
<artifactId>sansa-owl-spark_${scala.binary.version}</artifactId>
<version>${sansa.version}</version>
</dependency>
<!-- SANSA Inference -->
<!-- <dependency> <groupId>${project.groupId}</groupId> <artifactId>sansa-inference-parent_${scala.binary.version}</artifactId>
<version>${sansa.version}</version> </dependency> -->
<dependency>
<groupId>net.sansa-stack</groupId>
<artifactId>sansa-inference-spark_${scala.binary.version}</artifactId>
<version>${sansa.version}</version>
</dependency>
<!-- SANSA Querying -->
<dependency>
<groupId>net.sansa-stack</groupId>
<artifactId>sansa-query-spark-bundle_${scala.binary.version}</artifactId>
<version>${sansa.version}</version>
</dependency>
<!-- SANSA ML -->
<dependency>
<groupId>net.sansa-stack</groupId>
<artifactId>sansa-ml-spark_${scala.binary.version}</artifactId>
<version>${sansa.version}</version>
</dependency>
<!-- Test -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_${scala.binary.version}</artifactId>
<version>2.2.6</version>
<scope>test</scope>
</dependency>
<!-- Logging -->
<dependency>
<groupId>com.typesafe.scala-logging</groupId>
<artifactId>scala-logging-slf4j_${scala.binary.version}</artifactId>
<version>2.1.2</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.1</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
<configuration>
<args>
<!--<arg>-make:transitive</arg> -->
<arg>-dependencyfile</arg>
<arg>${project.build.directory}/.scala_dependencies</arg>
</args>
</configuration>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
<recompileMode>incremental</recompileMode>
</configuration>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes>
<exclude>asm:asm</exclude>
<exclude>com.clearspring.analytics:stream</exclude>
<exclude>com.esotericsoftware:kryo*</exclude>
<exclude>com.esotericsoftware:minlog</exclude>
<exclude>com.fasterxml.jackson.core:jackson*</exclude>
<exclude>com.fasterxml.jackson.module:jackson-module*</exclude>
<exclude>com.google.code.findbugs:jsr305</exclude>
<exclude>com.google.code.gson:gson</exclude>
<exclude>com.google.inject.extensions:guice-servlet</exclude>
<exclude>com.google.guava:guava</exclude>
<exclude>com.google.protobuf:protobuf-java</exclude>
<exclude>com.jcraft:jsch</exclude>
<exclude>com.ning:compress-lzf</exclude>
<exclude>com.sun.jersey:jersey-*</exclude>
<exclude>com.sun.jersey.contribs:jersey-guice</exclude>
<exclude>com.sun.xml.bind:jaxb-impl</exclude>
<exclude>com.thoughtworks.paranamer:paranamer</exclude>
<exclude>com.twitter:chill*</exclude>
<exclude>com.univocity:univocity-parsers</exclude>
<exclude>commons-beanutils:commons-beanutils*</exclude>
<exclude>commons-cli:commons-cli</exclude>
<exclude>commons-codec:commons-codec</exclude>
<exclude>commons-collections:commons-collections</exclude>
<exclude>commons-configuration:commons-configuration</exclude>
<exclude>commons-digester:commons-digester</exclude>
<exclude>commons-httpclient:commons-httpclient</exclude>
<exclude>commons-io:commons-io</exclude>
<exclude>commons-lang:commons-lang</exclude>
<exclude>commons-logging:commons-logging</exclude>
<exclude>commons-net:commons-net</exclude>
<exclude>io.dropwizard.metrics:metrics*</exclude>
<exckude>io.netty:netty*</exckude>
<exclude>javax.activation:activation</exclude>
<exclude>javax.annotation:javax.annotation-api</exclude>
<exclude>javax.servlet:javax.servlet-api</exclude>
<exclude>javax.servlet.jsp:jsp-api</exclude>
<exclude>javax.servlet:servlet-api</exclude>
<exclude>javax.validation:validation-api</exclude>
<exclude>javax.ws.rs:javax.ws.rs-api</exclude>
<exclude>javax.xml.bind:jaxb-api</exclude>
<exclude>javax.xml.stream:stax-api</exclude>
<exclude>jdk.tools:jdk.tools</exclude>
<exclude>net.java.dev.jets3t:jets3t</exclude>
<exclude>net.jpountz.lz4:lz4</exclude>
<exclude>net.razorvine:pyrolite</exclude>
<exclude>net.sf.py4j:py4j</exclude>
<exclude>org.antlr:antlr4-runtime</exclude>
<exclude>org.apache.avro:avro*</exclude>
<exclude>org.apache.commons:commons-lang3</exclude>
<exclude>org.apache.commons:commons-math3</exclude>
<exclude>org.apache.commons:commons-compress</exclude>
<exclude>org.apache.curator:curator*</exclude>
<exclude>org.apache.directory.api:*</exclude>
<exclude>org.apache.directory.server:*</exclude>
<exclude>org.apache.hadoop:*</exclude>
<exclude>org.apache.htrace:htrace-core</exclude>
<exclude>org.apache.httpcomponents:*</exclude>
<exclude>org.apache.ivy:ivy</exclude>
<exclude>org.apache.mesos:mesos</exclude>
<exclude>org.apache.parquet:parquet*</exclude>
<exclude>org.apache.spark:*</exclude>
<exclude>org.apache.xbean:xbean-asm5-shaded</exclude>
<exclude>org.apache.zookeeper:zookeeper</exclude>
<exclude>org.codehaus.jackson:jackson-*</exclude>
<exclude>org.codehaus.janino:*</exclude>
<exclude>org.codehaus.jettison:jettison</exclude>
<exclude>org.fusesource.leveldbjni:leveldbjni-all</exclude>
<exckude>org.glassfish.hk2*</exckude>
<exclude>org.glassfish.jersey*</exclude>
<exclude>org.javassist:javassist</exclude>
<exclude>org.json4s:json4s*</exclude>
<exclude>org.mortbay.jetty:jetty*</exclude>
<exclude>org.objenesis:objenesis</exclude>
<exclude>org.roaringbitmap:RoaringBitmap</exclude>
<exclude>org.scala-lang:*</exclude>
<exclude>org.slf4j:jul-to-slf4j</exclude>
<exclude>org.slf4j:jcl-over-slf4j</exclude>
<exclude>org.spark-project.spark:unused</exclude>
<exclude>org.xerial.snappy:snappy-java</exclude>
<exclude>oro:oro</exclude>
<exclude>xmlenc:xmlenc</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
</execution>
</executions>
</plugin>
<!-- disable surefire -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7</version>
<configuration>
<useFile>false</useFile>
<disableXmlReport>true</disableXmlReport>
<!-- If you have classpath issue like NoDefClassError,... -->
<!-- useManifestOnlyJar>false</useManifestOnlyJar -->
<includes>
<include>**/*Test.*</include>
<include>**/*Suite.*</include>
</includes>
<skipTests>true</skipTests>
</configuration>
</plugin>
<!-- enable scalatest -->
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
<filereports>WDF TestSuite.txt</filereports>
</configuration>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>oss-sonatype</id>
<name>oss-sonatype</name>
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
<repository>
<id>apache-snapshot</id>
<name>Apache repository (snapshots)</name>
<url>https://repository.apache.org/content/repositories/snapshots/</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
<repository>
<id>maven.aksw.internal</id>
<name>AKSW Release Repository</name>
<url>http://maven.aksw.org/archiva/repository/internal</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>maven.aksw.snapshots</id>
<name>AKSW Snapshot Repository</name>
<url>http://maven.aksw.org/archiva/repository/snapshots</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
Have several maven and other dependencies.
I Don't know why it is not loading the main class.
I have tried many solutions as mentioned by others i.e. in run configuration giving complete class name and etc. But nothing worked.
try to add this property :
<properties>
<jobMainClass>package.mainclass</jobMainClass>
</properties>
If it doesn't work, give me the command that you use to run your jarfile.

I am facing error when using scala eclipse maven GUI

I am facing error while using scala project in eclipse and maven, This is the project which I build from scratch , and it was working fine at that time.But now when I open it after a month it does show errors as shown in screenshots, even if i do mvn package it generates jar file as expected , But I see all my class files showing error at each line as shown in screen shot, what may be the problem?
I am using scala version: 2.10.6 and attached is below pom file.
scala.ide.org :
provider : scala-ide.org
plugin-name : scala2.10 jar
version ; 4.4.1201605041056
pligin-id: org.scala-ide.scala210.jars
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.blogspot.whileonefork</groupId>
<artifactId>mvn-scala-trial1</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>Sample Project for Blog</name>
<url>http://com.blogspot.whileonefork</url>
<properties>
<spark.version>1.6.0</spark.version>
</properties>
<!-- Notify Maven it can download things from here -->
<repositories>
<repository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>scala</id>
<name>Scala Tools</name>
<url>http://scala-tools.org/repo-releases/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
<dependencies>
<!-- Scala version is very important. Luckily the plugin warns you if you don't specify:
[WARNING] you don't define org.scala-lang:scala-library as a dependency of the project -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.6</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.scalatest/scalatest_2.10 -->
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.10</artifactId>
<version>2.2.6</version>
</dependency>
<!-- MySQL -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.37</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.10</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.10</artifactId>
<version>${spark.version}</version>
</dependency>
<!-- SPARK CSV -->
<dependency>
<groupId>com.databricks</groupId>
<artifactId>spark-csv_2.10</artifactId>
<version>1.4.0</version>
</dependency>
</dependencies>
<build>
<!-- add the maven-scala-plugin to the toolchain -->
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.14.2</version>
</plugin>
<!-- adding here -->
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.1</version>
<configuration>
<mainClass>com.asharma.mainjob.main_job</mainClass>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>assemble-all</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<!-- This plugin creates a single jar with all
the dependencies that are not already supplied by
the Spark runtime (scope 'provided'). -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4</version>
</plugin>
<plugin>
<groupId>com.frugalmechanic</groupId>
<artifactId>scala-optparse</artifactId>
<version>1.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>fully.qualified.MainClass</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<!--This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself. -->
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.0.0</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>
org.scala-tools
</groupId>
<artifactId>
maven-scala-plugin
</artifactId>
<versionRange>
[2.15.2,)
</versionRange>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</pluginExecutionFilter>
<action>
<ignore></ignore>
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<forkMode>pertest</forkMode>
<argLine>-Xms256m -Xmx512m</argLine>
<testFailureIgnore>false</testFailureIgnore>
<skip>false</skip>
<includes>
<include>**/*IntegrationTestSuite.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>

java.lang.NoClassDefFoundError when running Scala-Maven project

Trying to run a new project for the first time. mvn install goes smoothly, but when I try to run it, I get:
Exception in thread "main" java.lang.NoClassDefFoundError: scala/Tuple2
Now, something is terribly wrong if it can't even find the Scala basics. To be honest, it doesn't recognize anything outside of the code itself.
I''m attaching my POM file, hope someone can help..
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.foo</groupId>
<artifactId>bar</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<name>Foo Bar</name>
<properties>
<app.main.class>com.foo.main</app.main.class>
<spark.version>1.6.1</spark.version>
<scala.version>2.10.4</scala.version>
<scala.binary.version>2.10</scala.binary.version>
</properties>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</pluginRepository>
</pluginRepositories>
<repositories>
<repository>
<id>maven-repo</id>
<name>Maven Repository</name>
<url>http://repo1.maven.apache.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>apache-repo</id>
<name>Apache release repo</name>
<url>https://github.com/adatao/mvnrepos/tree/master/releases/</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<extensions>
<extension>
<groupId>org.springframework.build</groupId>
<artifactId>aws-maven</artifactId>
<version>5.0.0.RELEASE</version>
</extension>
</extensions>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>com.foo.main</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<jvmArgs>
<jvmArg>-Xms256m</jvmArg>
<jvmArg>-Xmx2048m</jvmArg>
</jvmArgs>
</configuration>
</plugin>
</plugins>
</build>
</project>

Generate WSDL and Copy in Maven Projects

I can generate WSDL directly from service GUI as picture below.
But In the Maven project I don't see such "Generate and Copy WSDL...". My question is How to enable it in Maven Project ?
Is there anything that I am missing. I am also attaching my pom.xml.
Thank you.
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.avalant</groupId>
<artifactId>OTPMaven</artifactId>
<version>1.0</version>
<packaging>war</packaging>
<name>OTPMaven</name>
<properties>
<endorsed.dir>${project.build.directory}/endorsed</endorsed.dir>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.1</version>
</dependency>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.8</version>
</dependency>
<dependency>
<groupId>com.sun.mail</groupId>
<artifactId>javax.mail</artifactId>
<version>1.5.0</version>
</dependency>
<dependency>
<groupId>org.codehaus.mojo</groupId>
<artifactId>jaxws-maven-plugin</artifactId>
<version>1.12</version>
</dependency>
<dependency>
<groupId>com.sun.xml.ws</groupId>
<artifactId>jaxws-tools</artifactId>
<version>2.2.7</version>
</dependency>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-web-api</artifactId>
<version>6.0</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
<compilerArguments>
<endorseddirs>${endorsed.dir}</endorseddirs>
</compilerArguments>
<showDeprecation>false</showDeprecation>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.1.1</version>
<configuration>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.1</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>${endorsed.dir}</outputDirectory>
<silent>true</silent>
<artifactItems>
<artifactItem>
<groupId>javax</groupId>
<artifactId>javaee-endorsed-api</artifactId>
<version>6.0</version>
<type>jar</type>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
As I have not got answer to this question. I have found temporary solution.
I am using command line. JDK's wsgen command to generate required output.
#echo off
set WSGEN="C:\Java\jdk1.6.0_39\bin\wsgen.exe"
set J1="C:\Java\jdk1.6.0_39\lib\tools.jar"
set J2="C:\Java\glassfish-3.1.2.2\glassfish\modules\webservices-osgi.jar"
set J3="C:\Java\glassfish-3.1.2.2\glassfish\modules\endorsed\webservices-api-osgi.jar"
set J4="C:\Java\glassfish-3.1.2.2\glassfish\modules\jaxb-osgi.jar"
set J5="C:\Java\glassfish-3.1.2.2\glassfish\modules\endorsed\jaxb-api-osgi.jar"
set J6="C:\Java\glassfish-3.1.2.2\glassfish\modules\javax.ejb.jar"
set J7="D:\NetBeansProjects\OTP\target\OTP-1.0\WEB-INF\lib\commons-lang3-3.1.jar"
set J8="D:\NetBeansProjects\OTP\target\OTP-1.0\WEB-INF\lib\commons-codec-1.8.jar"
set OUTPUT_DIR="D:\NetBeansProjects\OTP"
#echo on
%WSGEN% -classpath %J1%;%OUTPUT_DIR%\target\classes;%J2%;%J3%;%J4%;%J5%;%J6%;%J7%;%J8%; -d %OUTPUT_DIR%\jax-ws -Xendorsed -keep -wsdl -r %OUTPUT_DIR%\jax-ws -s %OUTPUT_DIR%\jax-ws -verbose com.avalant.ws.GenerateOTPWS