I have a scala project that depends on 2.11.8. Hence that is the version I am declaring as a dependency in my project. I have configured the maven-scala-plugin as follows.
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-compiler</artifactId>
<version>2.11.8</version>
</dependency>
</dependencies>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<sourceDir>src/main/scala</sourceDir>
<testSourceDir>src/test/scala</testSourceDir>
<scalaVersion>${scala.version}</scalaVersion>
<jvmArgs>
<jvmArg>-Xms64m</jvmArg>
<jvmArg>-Xmx1024m</jvmArg>
</jvmArgs>
</configuration>
</plugin>
However, when I build my project I receive:
[ERROR] Failed to execute goal org.scala-tools:maven-scala-plugin:2.11:compile (default) on project myproject: wrap: org.apache.maven.project.InvalidProjectModelException: 1 problem was encountered while building the effective model
[ERROR] [FATAL] Non-parseable POM /Users/myuser/.m2/repository/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.pom: entity reference name can not contain character =' (position: START_TAG seen ....uk/main?ParticipantID=nmum6rqpq5q6gkm3933n3nf9s76onu6r&FailedURI=... #1:265) # line 1, column 265
Let me know if there is any additional information that would help resolve this issue.
Thanks in advance!
Dependencies for scala in .m2 were corrupt - cleaned them out and did a clean install for everything. All fixed!
Related
We are currently using Maven to build a Scala Application. Now we are trying to convert this project into Gradle. I have 3/4 main classes in this project and I want to build a jar with dependencies that include all the main classes and execute this jar with spark-submit by calling any of the classes.
I'm new to Gradle and facing issues with Gradle. Could some one help me.
Contents of pom.xml :
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org-groupId</groupId>
<artifactId>project-name</artifactId>
<version>3.0-SNAPSHOT</version>
<properties>
<scala.version>2.11.12</scala.version>
<scala.compat.version>2.11</scala.compat.version>
<scala.test.version>3.0.4</scala.test.version>
<project.type>application</project.type>
</properties>
<dependencies>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.42.Final</version>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_${scala.compat.version}</artifactId>
<version>3.0.4</version>
<scope>test</scope>
</dependency>
</dependencies>
<repositories>
<repository>
<id>maven-repo</id>
<name>Maven central repository</name>
<url>https://repo1.maven.org/maven2</url>
</repository>
</repositories>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
</configuration>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>4.3.0</version>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<release>11</release>
</configuration>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.scalastyle</groupId>
<artifactId>scalastyle-maven-plugin</artifactId>
<version>1.0.0</version>
<configuration>
<verbose>false</verbose>
<failOnViolation>true</failOnViolation>
<includeTestSourceDirectory>true</includeTestSourceDirectory>
<failOnWarning>false</failOnWarning>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<configLocation>${project.basedir}/src/main/resources/plugin/scalastyle_config.xml</configLocation>
<outputFile>${project.build.directory}/scalastyle-output.xml</outputFile>
<outputEncoding>UTF-8</outputEncoding>
</configuration>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Contents of build.gradle:
apply plugin: 'java'
apply plugin: 'scala'
apply plugin: 'idea'
apply plugin: 'eclipse'
apply plugin: 'maven'
apply plugin: 'maven-publish'
repositories {
mavenLocal()
mavenCentral()
maven {
url = uri('https://repo1.maven.org/maven2')
}
}
dependencies {
// Some internal dependencies
implementation 'com.google.code.gson:gson:2.8.2'
implementation 'io.netty:netty-all:4.1.42.Final'
testImplementation "org.scalatest:scalatest_2.11:$scalaTestVersion"
}
tasks.withType(JavaCompile) {
options.encoding = 'UTF-8'
}
Contents of gradle.properties :
# Description
# Versions
version = 4.0
scalaVersion=2.11.12
scalaMajorVersion=2.11
scalaTestVersion=3.0.4
sparkVersion=2.4.6
sourceCompatibility = '1.8'
I have tried ./gradlew clean assemble and ./gradlew clean build.
Both of them building a regular jar but not dependencies jar.
Also, I'm trying to execute this in IntelliJ since Jar is not building, but with IntelliJ getting the below error.
21:36:59: Executing task 'mainclass1.main()'...
FAILURE: Build failed with an exception.
* Where:
Initialization script '/private/var/folders/sq/npjk1mkn7lgfm57mf9g_3rrh0000gn/T/mainclass1_main__.gradle' line: 4
* What went wrong:
A problem occurred configuring root project 'project-name'.
> Could not create task ':mainclass1.main()'.
> Unnecessarily replacing a task that does not exist is not supported. Use create() or register() directly instead. You attempted to replace a task named 'mainclass1.main()', but there is no existing task with that name.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.4.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 567ms
Cause: invalid type code: FA
21:37:00: Task execution finished 'mainclass1.main()'.
While converting from Maven pom.xml to Gradle, I didn't included and build plugins in Gradle. What are the plugins I can use to build a the jar in Gradle.
It looks like you want a fat jar. This will contain all the dependencies that allow you to run the application.
Gradle does not have a straightforward out-of-the-box solution because its aim is to keep things simple.
You can do this with by setting the following on your build.gradle file:
jar {
from {
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
}
Alternatively, you can use this plugin, by setting the following on the build.gradle:
plugins {
id 'com.github.johnrengelman.shadow' version '5.2.0' // check version compatibiltiy
}
and the running the task gradlew shadowJar.
On another note, did I understand you have 3/4 main classes? I don't know SPARK, so maybe it allows for several main classes, but outside of SPARK, your manifest file must set which is the main class. There can be only one.
This will allows you to run java -jar your-jar.jar. The following SO question might be relevant: Multiple runnable classes inside JAR, how to run them?
To set your main class on the Manifest using gradle:
jar {
manifest.attributes["Main-Class"] = "<your-main-class-full-path>"
}
Updating an existing maven project from Java 8 to 11, and I'm struggling to find the right combination of dependency and plugin versions.
One of the modules uses the play framework, with its associated dependencies, here's what they are set to now:
java: 11.0.5
scala: 2.12.8
sbt: 0.13.17
sbt-compiler-maven-plugin: 1.0.0
sbtrun-maven-plugin: 1.0.1
play: 2.6.21
play2-maven-plugin: 1.0.0-rc5
play2-provider-play26: 1.0.0-rc5
akka: 2.5.27
Which is giving me the following build error:
[ERROR] Failed to execute goal com.google.code.play2-maven-plugin:play2-maven-plugin:1.0.0-rc5:enhance (default-play2-enhance) on project services: Execution default-play2-enhance of goal com.google.code.play2-maven-plugin:play2-maven-plugin:1.0.0-rc5:enhance failed: An API incompatibility was encountered while executing com.google.code.play2-maven-plugin:play2-maven-plugin:1.0.0-rc5:enhance: java.lang.NoSuchMethodError: 'scala.collection.mutable.ArrayOps scala.Predef$.byteArrayOps(byte[])'
[ERROR] -----------------------------------------------------
[ERROR] realm = extension>com.google.code.play2-maven-plugin:play2-maven-plugin:1.0.0-rc5
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
...
...
...
[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import from realm ClassRealm[maven.api, parent: null]]
Note: I am hoping to keep akka to version 2.5.x, as 2.6.x requires a lot of code changes, but will go to a higher version if necessary
<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>groupId</groupId>
<artifactId>artifactId</artifactId>
<version>version</version>
</parent>
<artifactId>artifactId</artifactId>
<packaging>play2</packaging>
<name>name</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<repositories>
<repository>
<id>typesafe</id>
<url>http://repo.typesafe.com/typesafe/releases/</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.typesafe.play</groupId>
<artifactId>play_2.12</artifactId>
</dependency>
<dependency>
<groupId>com.typesafe.play</groupId>
<artifactId>play-test_2.12</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.typesafe.play</groupId>
<artifactId>play-java_2.12</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-joda</artifactId>
</dependency>
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-stream_2.12</artifactId>
</dependency>
<dependency>
<groupId>org.easytesting</groupId>
<artifactId>fest-assert-core</artifactId>
</dependency>
</dependencies>
<build>
<!-- Play source directory -->
<sourceDirectory>${basedir}/app</sourceDirectory>
<testSourceDirectory>${basedir}/test</testSourceDirectory>
<resources>
<resource>
<directory>${basedir}/conf</directory>
</resource>
<resource>
<directory>${basedir}/public</directory>
<targetPath>public</targetPath>
</resource>
<resource>
<directory>${basedir}/target/sbt/web/public/main</directory>
<targetPath>public</targetPath>
</resource>
</resources>
<plugins>
<plugin>
<groupId>com.google.code.play2-maven-plugin</groupId>
<artifactId>play2-maven-plugin</artifactId>
<version>1.0.0-rc5</version>
<extensions>true</extensions>
<configuration>
<mainLang>java</mainLang>
</configuration>
<dependencies>
<dependency>
<groupId>com.google.code.play2-maven-plugin</groupId>
<artifactId>play2-provider-play27</artifactId>
<version>1.0.0-rc5</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>default-play2-enhance</id>
<goals>
<goal>enhance</goal>
</goals>
</execution>
<execution>
<phase>package</phase>
<goals>
<goal>dist</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.google.code.sbt-compiler-maven-plugin</groupId>
<artifactId>sbt-compiler-maven-plugin</artifactId>
<version>1.0.0</version>
</plugin>
<plugin>
<groupId>com.google.code.sbtrun-maven-plugin</groupId>
<artifactId>sbtrun-maven-plugin</artifactId>
<version>1.0.1</version>
<executions>
<execution>
<id>compile-assets</id>
<phase>generate-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<args>web-assets:assets</args>
<jvmArgs>-Dscala.version=2.12.8</jvmArgs>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>default-jar</id>
<configuration>
<archive>
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<execution>
<id>attach-artifacts</id>
<phase>package</phase>
<goals>
<goal>attach-artifact</goal>
</goals>
</execution>
</executions>
<configuration>
<!-- <!– These empty tags keep IntelliJ quiet –> -->
<name />
<regex />
<source />
<value />
<fileSet />
<artifacts>
<artifact>
<file>${project.build.directory}/service-${project.version}-dist.zip</file>
<type>zip</type>
<classifier>dist</classifier>
</artifact>
</artifacts>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>eclipse</id> <!-- for M2Eclipse only -->
<build>
<directory>${project.basedir}/target-eclipse</directory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<skipMain>true</skipMain>
<skip>true</skip>
<source>11</source>
<target>11</target>
</configuration>
<executions>
<execution>
<id>default-compile</id>
<goals><goal>compile</goal></goals>
</execution>
<execution>
<id>default-testCompile</id>
<goals><goal>testCompile</goal></goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
Edit:
Updated SBT and Scala version, added redacted pom
As per scala documentation
https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html
Your SBT version is not compatible with Java 11. I would recommend upgrading to sbt.version=1.2.3
Akka version you are using should be good if you don't want to upgrade to higher.
Update:
I can see you updated sbt version to 0.13.17 and you tried with higher, but you should forget that sbt 0.13 and stick with sbt 1.x since sbt 0.13 is using scala 2.10, and sbt 1.x scala 2.12.
I've also noticed that you don't have version tag for maven-compiler-plugin. If you are using maven 3, it should fail to build because of missing this tag, and since it's not, I guess you are using maven 2 which should be upgraded. Also you should add 3.8.0 version tag to maven-compiler-plugin.
Check this guide to see how to properly configure this plugin for java 11. You should also check other instructions to see if everything is properly configured.
I have looked around from various questions on stackoverflow, but I have not found the answer that solve my purpose.
I want to import a properties file in a pom.xml; my purpose is to replace the <properties> section with the properties loaded from the external file.
Each property refers to the version of a particular maven dependency.
I have tried the properties-maven-plugin, but the properties are not solved and the project is not built.
I'm looking for a way that preserve the standard build of Eclipse, and also the mvn install goal of Maven.
As an example, I want that this section:
<properties>
<dependency1.version>1.0.0</dependency1.version>
</properties>
will be replaced with the property declared in a dependency.properties file, like this one:
dependency1.version=1.0.0
How can I implement my pom.xml in order to obtain this behaviour?
Thanks in advance.
EDIT
This is what I have tried:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>${basedir}/dependencies.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>generate-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<echo>my version: ${dependency1.version}</echo>
</tasks>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Well, if I do a simple mvn install, in maven console it prints:
[INFO] Executing tasks
[echo] my version: 1.0.0
[INFO] Executed tasks
So, the plugin solve the property, but when I try to import the dependency with this:
<dependencies>
<dependency>
<groupId>dependency1</groupId>
<artifactId>dependency1</artifactId>
<version>${dependency1.version}</version>
</dependency>
</dependencies>
I obtain this error:
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[ERROR] 'dependencies.dependency.version' for dependency1:dependency1:jar must be a valid version but is '${dependency1.version}'. # line 21, column 13
#
The property is solved in the maven-antrun-plugin, but if it is used in the dependencies section, it doesn't work.
I'm testing to convert some files using Avro, I had added dependency as the guide of avro 1.7.7 suggest.
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>1.7.7</version>
</dependency>
As well as the Avro Maven plugin (for performing code generation):
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>1.7.7</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>schema</goal>
</goals>
<configuration>
<sourceDirectory>${project.basedir}/src/main/avro/</sourceDirectory>
<outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
But this guide uses shell command to compile the avro schema user.avsc. I want to ask how to compile it in the eclipse IDE.
As #Tunaki has suggested, I "Run as... > Maven install".
Additionally, I updated added "target/generated-sources" to the Build Path so I can reference the auto-generated files.
While working with scalatest I ran into a weird problem. I have a maven project with multiple modules. If I execute mvn test directly in the module. It works without problems, but if I do it in the root folder it complains about missing packages (dependencies) while compiling.
My configuration looks like following:
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_${scala.shortversion}</artifactId>
<version>3.0.0-SNAP2</version>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
</dependency>
plugin configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
<filereports>WDF TestSuite.txt</filereports>
</configuration>
<executions>
<execution>
<id>scala-test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
If I remove from maven-scala-plugin the goals it'll compile but scalatest will not find the test sources and exit with No tests were executed.:
<plugin>
<version>2.15.2</version>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
Any ideas on what I am doing wrong?!
Cheers
Hard to say without seeing all your poms, but generally speaking:
Define in root Pom all your dependencies in dependencyManagement section, your plugins in pluginManagement section with their configuration. In child poms define only those plugins and dependencies you need.
Keep in mind that xxxManagement sections only build shared definition, kind of root configuration. You still have to define in plugins and dependencies sections what you need but you can omit version and configuration elements.
This will allow all your child poms to run tests in all modules.