error: java.lang.NoSuchMethodError: 'scala.tools.nsc.reporters.Reporter scala.tools.nsc.Global.reporter()' in scoverage-maven-plugin - scala

I am trying to get some reports with the code coverage information for my project. I found that scoverage-maven-plugin produce xml files covering the % of the different modules that the unit test cases have covered so far.
So I tried to add that plugin in my pom, but I am getting the following error:
[ERROR] error: java.lang.NoSuchMethodError: 'scala.tools.nsc.reporters.Reporter scala.tools.nsc.Global.reporter()'
[INFO] at scoverage.ScoverageInstrumentationComponent$$anon$1.run(plugin.scala:115)
[INFO] at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1514)
[INFO] at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1498)
[INFO] at scala.tools.nsc.Global$Run.compileSources(Global.scala:1491)
[INFO] at scala.tools.nsc.Global$Run.compile(Global.scala:1620)
[INFO] at scala.tools.nsc.Driver.doCompile(Driver.scala:47)
[INFO] at scala.tools.nsc.MainClass.doCompile(Main.scala:32)
[INFO] at scala.tools.nsc.Driver.process(Driver.scala:67)
[INFO] at scala.tools.nsc.Driver.main(Driver.scala:80)
[INFO] at scala.tools.nsc.Main.main(Main.scala)
[INFO] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
[INFO] at java.base/java.lang.reflect.Method.invoke(Method.java:577)
[INFO] at scala_maven_executions.MainHelper.runMain(MainHelper.java:164)
[INFO] at scala_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
[INFO] java.lang.reflect.InvocationTargetException
[INFO] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:119)
[INFO] at java.base/java.lang.reflect.Method.invoke(Method.java:577)
[INFO] at scala_maven_executions.MainHelper.runMain(MainHelper.java:164)
[INFO] at scala_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
[ERROR] Caused by: java.lang.NoSuchMethodError: 'scala.tools.nsc.reporters.Reporter scala.tools.nsc.Global.reporter()'
[INFO] at scoverage.ScoverageInstrumentationComponent$$anon$1.run(plugin.scala:115)
[INFO] at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1514)
[INFO] at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1498)
[INFO] at scala.tools.nsc.Global$Run.compileSources(Global.scala:1491)
[INFO] at scala.tools.nsc.Global$Run.compile(Global.scala:1620)
[INFO] at scala.tools.nsc.Driver.doCompile(Driver.scala:47)
[INFO] at scala.tools.nsc.MainClass.doCompile(Main.scala:32)
[INFO] at scala.tools.nsc.Driver.process(Driver.scala:67)
[INFO] at scala.tools.nsc.Driver.main(Driver.scala:80)
[INFO] at scala.tools.nsc.Main.main(Main.scala)
[INFO] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
[INFO] ... 3 more
Not able to find from where I should get scala.tools.nsc.reporters.Reporter scala.tools.nsc.Global.reporter()
The full pom is:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>mygroupid</groupId>
<artifactId>myartifact</artifactId>
<version>1.0.0</version>
<name>myproject</name>
<properties>
<app.build.version>1.0.0</app.build.version>
<encoding>UTF-8</encoding>
<scala.version>2.12.14</scala.version>
<scala.compat.version>2.12</scala.compat.version>
<scala.binary.version>2.12</scala.binary.version>
<spark.version>3.2.1</spark.version>
<scope.value>provided</scope.value>
<scoverage.plugin.version>1.3.0</scoverage.plugin.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
<scope>${scope.value}</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
<scope>${scope.value}</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
<scope>${scope.value}</scope>
</dependency>
<dependency>
<groupId>io.delta</groupId>
<artifactId>delta-core_2.12</artifactId>
<version>2.0.0</version>
<scope>${scope.value}</scope>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-streams</artifactId>
<version>3.1.2</version>
<scope>${scope.value}</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql-kafka-0-10_${scala.binary.version}</artifactId>
<version>${spark.version}</version>
<scope>${scope.value}</scope>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.12</artifactId>
<version>3.2.14</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<finalName>myproject-${app.build.version}</finalName>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.scoverage</groupId>
<artifactId>scoverage-maven-plugin</artifactId>
<version>${scoverage.plugin.version}</version>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
<highlighting>true</highlighting>
<aggregate>true</aggregate>
</configuration>
<executions>
<execution>
<goals>
<goal>report</goal>
</goals>
<phase>test</phase>
</execution>
</executions>
</plugin>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.2</version>
<executions>
<execution>
<id>Scaladoc</id>
<goals>
<goal>doc</goal>
</goals>
<phase>prepare-package</phase>
<configuration>
<args>
<arg>-no-link-warnings</arg>
</args>
</configuration>
</execution>
<execution>
<id>attach-javadocs</id>
<goals>
<goal>doc-jar</goal>
</goals>
</execution>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
<configuration>
<args>
<arg>-dependencyfile</arg>
<arg>${project.build.directory}/.scala_dependencies</arg>
</args>
<jvmArgs>
<jvmArg>-Xss4m</jvmArg>
<jvmArg>-Xms512m</jvmArg>
<jvmArg>-Xmx4096m</jvmArg>
</jvmArgs>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
<filereports>WDF TestSuite.txt</filereports>
<htmlreporters>${project.build.directory}/html/scalatest</htmlreporters>
</configuration>
<executions>
<execution>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Edits after #Dmytro feedback:
I added in the pom the following dependencies:
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-compiler</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.scoverage</groupId>
<artifactId>scalac-scoverage-plugin_2.12</artifactId>
<version>1.4.1</version>
</dependency>
And changed the scope to
<scope.value>compile</scope.value>
But I am still getting the error: java.lang.NoSuchMethodError: 'scala.tools.nsc.reporters.Reporter scala.tools.nsc.Global.reporter()'

Could you try to add
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-compiler</artifactId>
<!--<version>2.12.14</version>-->
<version>${scala.version}</version>
</dependency>
?
https://mvnrepository.com/artifact/org.scala-lang/scala-compiler
Does this change anything for you?
scoverage-maven-plugin 1.3.0 depends on scalac-scoverage-plugin 1.3.0
https://github.com/scoverage/scoverage-maven-plugin/blob/scoverage-maven-plugin-1.3.0/pom.xml#L93-L101
scalac-scoverage-plugin 1.3.0 depends on scala-compiler 2.12.0
https://index.scala-lang.org/scoverage/scalac-scoverage-plugin/artifacts/scalac-scoverage-plugin/1.3.0
https://github.com/scoverage/scalac-scoverage-plugin/blob/v1.3.0/build.sbt#L89
https://github.com/scoverage/scalac-scoverage-plugin/blob/v1.3.0/build.sbt#L15
scala-compiler is marked as provided in the dependencies of scalac-scoverage-plugin so should be added manually when needed.
The method scala.tools.nsc.Global.reporter() returns Reporter in scala-compiler 2.12.0 https://github.com/scala/scala/blob/v2.12.0/src/compiler/scala/tools/nsc/Global.scala#L1581 (it seems in 2.12.0 there is no scala.tools.nsc.Global.reporter(), it appears in 2.12.5). In 2.12.14 the signature is different, the method returns FilteringReporter https://github.com/scala/scala/blob/v2.12.14/src/compiler/scala/tools/nsc/Global.scala#L1748 https://github.com/scala/scala/blob/v2.12.14/src/compiler/scala/tools/nsc/Global.scala#L90 So the issue seems to be in the combination of versions scalac-scoverage-plugin 1.3.0 + scala-compiler 2.12.14.
Try to either upgrade scoverage-plugin or downgrade Scala version so that the signature of the method is the same (returning either Reporter both times or FilteringReporter both times). The signature changed in 2.12.12 -> 2.12.13 (Reporter -> FilteringReporter) https://github.com/scala/scala/blob/v2.12.12/src/compiler/scala/tools/nsc/Global.scala#L90 https://github.com/scala/scala/blob/v2.12.13/src/compiler/scala/tools/nsc/Global.scala#L90 If you're on scoverage-plugin 1.4.1 can you change Scala to 2.12.12?
Try then to downgrade the scala version:
<properties>
<scala.version>2.12.12</scala.version>
...
</properties>
in combination with
<dependency>
<groupId>org.scoverage</groupId>
<artifactId>scalac-scoverage-plugin_2.12</artifactId>
<version>1.4.1</version>
</dependency>

Related

Exception While Using net.alchim31.maven:scala-maven-plugin

I have a scala-maven-spark project.
Here is my pom.xml file
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
<scala.version>2.11.8</scala.version>
</properties>
<dependencies>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.11</artifactId>
<version>3.0.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.4.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>28.1-jre</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api-scala_2.11</artifactId>
<version>11.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.17.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.17.0</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.3.3</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
<filereports>WDF TestSuite.txt</filereports>
</configuration>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<executions>
<!-- Run shade goal on package phase -->
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<finalName>${project.artifactId}-${project.version}</finalName>
<relocations>
<relocation>
<pattern>com.google.protobuf</pattern>
<shadedPattern>shaded.com.google.protobuf</shadedPattern>
</relocation>
<relocation>
<pattern>com.google.common</pattern>
<shadedPattern>shaded.com.google.common</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
When I run mvn clean package I am getting the below error
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.3.3:compile (default) on project decisionengine-data-ingestion-maven: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
If I remove the net.alchim31.maven plugin the build completes successfully.
Error details
[ERROR] error: scala.reflect.internal.MissingRequirementError: object java.lang.Object in compiler mirror not found.
[ERROR] at scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:17)
[ERROR] at scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:18)
[ERROR] at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:53)
[E
If you have scala source code you need
to add the scala-maven-plugin (please use a more recent version see https://search.maven.org/search?q=a:scala-maven-plugin, today 4.5.6 is the latest, why using 3.3.3 from 2018-06 ?) to compile your code
to have the scala-library as an explicit dependency
And remove
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
It's better to let the plugin detects the scala version from dependencies.

Scala IDE, import project from Maven and proper Scala bundle version selection

This is my Maven pom.xml for Scala Spark project:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example.spark</groupId>
<artifactId>poc</artifactId>
<version>0.0.1-SNAPSHOT</version>
<properties>
<java.source.version>1.8</java.source.version>
<java.target.version>1.8</java.target.version>
<spark.version>2.3.2</spark.version>
<scala-maven-plugin.version>3.4.6</scala-maven-plugin.version>
<maven-compiler-plugin.version>2.0.2</maven-compiler-plugin.version>
<maven-assembly-plugin.version>2.4</maven-assembly-plugin.version>
</properties>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.11.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.11</artifactId>
<version>${spark.version}</version>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>${scala-maven-plugin.version}</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<!-- mixed scala/java compile -->
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>${java.source.version}</source>
<target>${java.target.version}</target>
</configuration>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- for fatjar -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>${maven-assembly-plugin.version}</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>assemble-all</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>com.example.Application</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
</project>
Everything works fine except one thing - after the import of the project (as existing Maven project) into Scala Eclipse IDE in order to properly compile the project I have to manually go to project Properties in my Scala Eclipse IDE -> Scala Compiler and select Latest 2.11 bundle (dynamic) because by default Latest 2.12 bundle (dynamic) is selected.
Please advise how to properly configure my pom.xml in order to tell Scala Eclipse IDE to automatically use Latest 2.11 bundle (dynamic) after the import without any manual intervention. Thanks!
Use this property block in your pom
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
<scala.version>2.11.8</scala.version>
<spark.scala.version>2.11</spark.scala.version>
<spark.version>2.2.0</spark.version>
</properties>
also add this block in your dependency
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
<scope>provided</scope>
</dependency>
You will get a compiler of Scala version 2.11 in Scala IDE

Artifact has not been packaged yet MDEP-98

When compiling one of the Patient Matching algorithm files -> Choicemaker
i found this error
I have tried few solutions from
Artifact has not been packaged yet
i still haven't found the solution
Kindly please Help
THE ERROR IS AS FOLLOWS
Artifact has not been packaged yet. When used on reactor artifact,
unpack should be executed after packaging: see MDEP-98.
(org.apache.maven.plugins:maven-dependency-plugin:2.10:unpack-dependencies:
e2-standard-platform:pre-integration- ....
The ERROR IS SHOWN NEXT TO tag
This is the POM file
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
>
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.choicemaker.cm</groupId>
<artifactId>choicemaker-base</artifactId>
<version>2.7.1-SNAPSHOT</version>
<relativePath>../../choicemaker-base/pom.xml</relativePath>
</parent>
<groupId>com.choicemaker.cmit</groupId>
<artifactId>cm-analyzer-ce-it</artifactId>
<name>Integration Test: ChoiceMaker Analyzer, Part 1</name>
<description><![CDATA[
Integration test for the standard-E2 version of CM Analyzer, Community Edition.
]]></description>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<eclipse.application.dir>${project.build.testOutputDirectory}/eclipse.application.dir</eclipse.application.dir>
<eclipse.application.examples.dir>${eclipse.application.dir}/examples</eclipse.application.examples.dir>
<eclipse.application.plugins.dir>${eclipse.application.dir}/plugins</eclipse.application.plugins.dir>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<compilerVersion>1.7</compilerVersion>
<source>1.7</source>
<target>1.7</target>
<compilerArgument>-g</compilerArgument>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution> ---------> ERROR
<id>plugins</id>
<phase>pre-integration-test</phase>
<goals>
<goal>unpack-dependencies</goal>
</goals>
<configuration>
<includeArtifactIds>cm-analyzer-ce</includeArtifactIds>
<excludeTransitive>true</excludeTransitive>
<outputDirectory>${eclipse.application.dir}</outputDirectory>
</configuration>
</execution>
<execution> ---------> ERROR
<id>e2-standard-platform</id>
<phase>pre-integration-test</phase>
<goals>
<goal>unpack-dependencies</goal>
</goals>
<configuration>
<classifier>eclipse2prj</classifier>
<includeArtifactIds>com.choicemaker.e2.std,com.choicemaker.cm.modelmaker.std</includeArtifactIds>
<excludeTransitive>true</excludeTransitive>
<outputDirectory>${eclipse.application.plugins.dir}</outputDirectory>
</configuration>
</execution>
<execution> ----------> ERROR
<id>examples</id>
<phase>pre-integration-test</phase>
<goals>
<goal>unpack-dependencies</goal>
</goals>
<configuration>
<includeArtifactIds>simple_person_matching</includeArtifactIds>
<excludeTransitive>true</excludeTransitive>
<outputDirectory>${eclipse.application.examples.dir}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.choicemaker.e2</groupId>
<artifactId>com.choicemaker.e2</artifactId>
</dependency>
<dependency>
<groupId>com.choicemaker.e2it</groupId>
<artifactId>com.choicemaker.e2it</artifactId>
</dependency>
<dependency>
<groupId>com.choicemaker.e2</groupId>
<artifactId>com.choicemaker.e2.std</artifactId>
</dependency>
<dependency>
<groupId>com.choicemaker.e2</groupId>
<artifactId>com.choicemaker.e2.std</artifactId>
<classifier>eclipse2prj</classifier>
<type>zip</type>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.choicemaker.cm</groupId>
<artifactId>cm-analyzer-ce</artifactId>
<classifier>distrib</classifier>
<type>zip</type>
</dependency>
<dependency>
<groupId>com.choicemaker.cm</groupId>
<artifactId>simple_person_matching</artifactId>
</dependency>
<dependency>
<groupId>com.choicemaker.cm</groupId>
<artifactId>com.choicemaker.cm.modelmaker</artifactId>
</dependency>
<dependency>
<groupId>com.choicemaker.cm</groupId>
<artifactId>com.choicemaker.cm.modelmaker.std</artifactId>
<classifier>eclipse2prj</classifier>
<type>zip</type>
</dependency>
<dependency>
<groupId>com.choicemaker.cm</groupId>
<artifactId>org.eclipse.core.boot</artifactId>
</dependency>
<dependency>
<groupId>com.choicemaker.cm</groupId>
<artifactId>org.eclipse.core.launcher</artifactId>
</dependency>
</dependencies>
</project>

java.lang.NoClassDefFoundError: com/typesafe/config/ConfigFactory when packaging Scala project with Maven

This question is a continuation of this thread.
The problem that I have refers to packaging Scala Spark project using Maven.
When I run this command:
spark-submit --name 28 --master local[2] --class org.test.consumer.TestRunner \
/usr/tests/test1/target/test_service-1.0-SNAPSHOT.jar \
$arg1 $arg2 $arg3 $arg4 $arg5
..., I get the following error:
Exception in thread "main" java.lang.NoClassDefFoundError: com/typesafe/config/ConfigFactory
at org.test.consumer.kafka.KafkaConsumer.<init>(KafkaConsumer.scala:38)
at org.test.consumer.TestRunner$.main(TestRunner.scala:19)
at org.test.consumer.TestRunner.main(TestRunner.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:729)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: com.typesafe.config.ConfigFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 12 more
This is my current pom.xml that considers the recommendations from the thread mentioned above (I checked that generated jar contains scala classes):
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.test.consumer</groupId>
<artifactId>test_service</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<scala.version>2.11.8</scala.version>
</properties>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>1.6.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka_2.11</artifactId>
<version>1.6.2</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
<version>2.7.5</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.7.5</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.7.5</version>
</dependency>
<dependency>
<groupId>org.sedis</groupId>
<artifactId>sedis_2.11</artifactId>
<version>1.2.2</version>
</dependency>
<dependency>
<groupId>com.lambdaworks</groupId>
<artifactId>jacks_2.11</artifactId>
<version>2.3.3</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>1.6.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib-local_2.11</artifactId>
<version>2.0.0</version>
</dependency>
<dependency>
<groupId>com.github.nscala-time</groupId>
<artifactId>nscala-time_2.11</artifactId>
<version>2.12.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<!-- Configure maven-compiler-plugin to use the desired Java version -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<!-- Use build-helper-maven-plugin to add Scala source and test source directories -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.10</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/main/scala</source>
</sources>
</configuration>
</execution>
<execution>
<id>add-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/test/scala</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<!-- Use scala-maven-plugin for Scala support -->
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.2</version>
<executions>
<execution>
<goals>
<!-- Need to specify this explicitly, otherwise plugin won't be called when doing e.g. mvn compile -->
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Add this to your pom.xml.
<dependency>
<groupId>com.typesafe</groupId>
<artifactId>config</artifactId>
<version>1.3.1</version>
</dependency>
Note that you don't have to add this jar if it is already present.
Looks like the jar should be an executable jar with all your dependencies packaged within the jar file.
You will have to use the maven assembly plugin to do that.
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>org.sample.App</mainClass> // put your main class here.
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
This plugin will create an additional jar (apart from the regular jar) with the name appended with -jar-with-dependencies. Use this jar instead of using the original jar.

Maven compile replies 'No sources to compile' for scala project

I have the following (single) scala class
[ 14253 Oct 30 8:44] ./pom.xml
[ 9083 Oct 30 8:30] ./scaladem.iml
[ 102 Oct 29 19:21] ./src
[ 102 Oct 29 19:21] ./src/main
[ 102 Oct 29 19:21] ./src/main/scala
[ 102 Oct 29 19:21] ./src/main/scala/com
[ 102 Oct 29 19:21] ./src/main/scala/com/blazedb
[ 102 Oct 30 8:30] ./src/main/scala/com/blazedb/scalademo
[ 4646 Oct 30 8:30] ./src/main/scala/com/blazedb/scalademo/SDemo.scala
Here is the applicable section of the pom
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.1.6</version>
<configuration>
<recompileMode>incremental</recompileMode>
<javacArgs>
<javacArg>-Xlint:unchecked</javacArg>
<javacArg>-Xlint:deprecation</javacArg>
</javacArgs>
</configuration>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
When we run
mvn compile
We get (notice the 'no sources' ..)
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building SDemo 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) # scalademo ---
[INFO] No sources to compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
Update: when running the following command
$mvn scala:compile -DdisplayCmd=true
The compilation succeeds.
From suggestion of #badtrumpet I have added the explicit as shown below
<sourceDirectory>src/main/scala</sourceDirectory>
And this works even by mvn compile. But that would be an issue for mixed java/scala projects.
Here's an example (simple) pom.xml which I use as a bit of a boilerplate for Scala compilation and build using Maven:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>badtrumpet</groupId>
<artifactId>blog</artifactId>
<version>1.0-SNAPSHOT</version>
<inceptionYear>2013</inceptionYear>
<packaging>jar</packaging>
<properties>
<scala.version>2.10.2</scala.version>
<commons.codec.version>1.8</commons.codec.version>
<grizzled.version>1.0.1</grizzled.version>
<slf4j-log4j12.version>1.7.5</slf4j-log4j12.version>
</properties>
<repositories>
<repository>
<id>Sonatype repository</id>
<name>Sonatype's Maven repository</name>
<url>http://oss.sonatype.org/content/groups/public</url>
</repository>
<repository>
<id>scala-tools.org</id>
<name>Scala-Tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</repository>
<repository>
<id>milestone.repo.springsource.org</id>
<name>repo.springsource.org-milestone</name>
<url>https://repo.springsource.org/libs-milestone</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-Tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>${commons.codec.version}</version>
</dependency>
<dependency>
<groupId>org.clapper</groupId>
<artifactId>grizzled-slf4j_2.10</artifactId>
<version>${grizzled.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j-log4j12.version}</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
</plugins>
</build>
<reporting>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
</plugins>
</reporting>
To add extra source folders to your maven project you can use the build-helper-maven-plugin.
So to add src/main/scala as a source folder you would configure the following in your pom:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>${basedir}/src/main/scala</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
For a standard maven project this would mean that now both src/main/java and src/main/scala are considered source directories by maven.
Have you followed the configuration steps in the maven scala plugin homepage?
http://davidb.github.io/scala-maven-plugin/usage.html
Looks as though you have bound the Scala compilation to the process-resources phase to me. The output about no sources is from the regular maven compiler plugin that compiles Java.
To mix Java and scala, just ensure your source tree is something like:
src
main
-- java
-- scala
-- resources
test
-- java
-- scala
-- resources
etc...basically a sane Maven directory structure and then configure your build section like this:
<build>
<sourceDirectory>src/main/</sourceDirectory>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</build>
For some reason the official document doesn't work for me, I have to add the following
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
so my tag under tag looks like this:
<build>
<!-- To define the plugin version in your parent POM -->
<pluginManagement>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.0</version>
</plugin>
</plugins>
</pluginManagement>
<!-- To use the plugin goals in your POM or parent POM -->
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>