I have a maven project with both Java and Scala components, but when I use maven-shade-plugin, it relocates package names for both Java and Scala files, but ONLY renames packages inside Java files, Scala files still contain the older package names, what am I missing?
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<!--<minimizeJar>true</minimizeJar>-->
<artifactSet>
<includes>
<include>ml.dmlc:xgboost4j-spark</include>
<include>ml.dmlc:xgboost4j</include>
</includes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<relocations>
<relocation>
<pattern>ml.dmlc.xgboost4j</pattern>
<shadedPattern>ml.dmlc.xgboost4j.shaded</shadedPattern>
</relocation>
</relocations>
<transformers>
</transformers>
</configuration>
</execution>
</executions>
</plugin>```
Sadly, I believe that Maven is intended to have this functionality but, currently (Dec 2020), it does not.
This can be seen with this bug ticket:
https://issues.apache.org/jira/browse/MSHADE-345
workaround
I have personally done a silly workaround for this. I make a new empty mvn project that has the dependency:
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>28.0-jre</version>
</dependency>
And the plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>com.google.</pattern>
<shadedPattern>shader.com.google.</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
Then in the project with code that requires a low version of guava and a new version of guava, include the empty project as a dependency.
<dependency>
<groupId>com.yoursite.yourwork</groupId>
<artifactId>shader</artifactId>
<version>0.0.1</version>
</dependency>
Then in a .scala file you import the new (version 28) guava classes like this:
import shader.com.google.common.graph.Network
Why this works
Since the error only occurs in scala projects where you refer to your own class that uses the dependency, or as said in the question "Scala files still contain the older package names", shading a project that does not refer to its own dependencies bypasses the bug.
Yes, it does. Choose any build version you want and import the library into your Scala project.
Related
I am getting above exception in my web application running in Tomcat when packaged all my dependencies including spring-data-jpa.jar in a single jar using maven-shaded-plugin and put under WEB-INF/lib directory.
Problem dis-appears if I package the spring-data-jpa.jar directly into WEB-INF/lib along with my shaded jar?
NOTE: I will be running the same package as AWS Lambda hence I need to create a shaded jar.
To help others, the problem was that multiple spring-*.jar files META-INF/spring.handlers files which overwrites each other while running the maven-shade-plugin.
To resolve use <transformers> in the plugin configuration. My final plugin configuration looks like as follows;
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<!-- Remove signatures from transitive dependencies and append spring handlers and schemas -->
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.handlers</resource>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.schemas</resource>
</transformer>
</transformers>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
Above will merge all handlers in one single file in final jar. Enjoy :-)
I'm trying to use the Maven shade plugin (according to the tutorial here) to create a "fat jar" from my project.
I'm working on my project in eclipse and when I look at the (huge) resulting fat jar, I see that it contains a lot (possibly all) of classes from the Eclipse IDE code itself.
Why is it doing that and how to prevent it from doing it?
I've tried just listing a bunch of directories in the <exclude> <filters>, but the Eclipse JDT jars also have some files in jars "root" folders and its not easy to list all of these as well.
The shade <plugin> part of the pom.xml file currently looks like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>${maven.shade.version}</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<createDependencyReducedPom>true</createDependencyReducedPom>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>vertx-unit*/**</exclude>
<exclude>org/apache/derby/**</exclude>
<exclude>javax/annotation/**</exclude>
<exclude>com/google/googlejavaformat/**</exclude>
<exclude>org/eclipse/**</exclude>
<exclude>jdtCompilerAdapter.jar</exclude>
<exclude>ant_tasks/**</exclude>
<exclude>META-INF/services/org.osgi.framework.launch.FrameworkFactory</exclude>
<exclude>about_files/**</exclude>
<exclude>org/osgi/**</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<manifestEntries>
<Main-Class>io.vertx.core.Starter</Main-Class>
<Main-Verticle>io.thesphere.service.App</Main-Verticle>
</manifestEntries>
</transformer>
</transformers>
<artifactSet />
<outputFile>${project.build.directory}/${project.artifactId}-${project.version}-fat.jar</outputFile>
</configuration>
</execution>
</executions>
</plugin>
Due to some quirks in some dependencies, I'm having trouble with sbt-assembly, and have been told that people working with Java and have had good results with Maven's shade plugin.
How can I use Maven's shade plugin for Scala / sbt?
You can add the following to your POM
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.group.id.Launcher1</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
Can you tell me, if this is possible, how to export a Scala application to a normal runnable JAR that can run directly in the JVM ?
Thanks
It is perfectly possible, see for instance this: running a maven scala project. Since Scala compiles to Java bytecode, JVM is not even aware of the underlying implementation language.
In essence after compiling Scala sources using scalac you will get a bunch of .class files which you can later package into a JAR. Then you can simply run them using:
$ java -cp "your.jar:scala-library.jar" com.example.Main
Note that you must include scala-library.jar on the CLASSPATH (currently it is almost 9 MiB...) and specify class containing main method.
If you use sbt to build you can use one of the one-jar plugins. They will put all dependencys into one big jar file (inclusive all the scala.jar files). This means that you only need one jar file and don't have to manage all the dependencys.
As an example with sbt-assembly (mostly copied from https://github.com/sbt/sbt-assembly):
project/plugins.sbt:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "X.X.X")
build.sbt:
import AssemblyKeys._ // put this at the top of the file
seq(assemblySettings: _*)
then you can generate the jar with:
sbt assembly
As an alternative to Fabian's answer, if you're using Maven, you can use the assembly-plugin. Something like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>package-jar-with-dependencies</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<appendAssemblyId>true</appendAssemblyId>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifestEntries>
<SplashScreen-Image>splash.png</SplashScreen-Image>
</manifestEntries>
<manifest>
<mainClass>se.aptly.epm.main.PrognosisApp</mainClass>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
That will package all your deps up, including scala-library.jar (if it's in your deps), but will do so flattened with all classes unpacked. This is because runnable jar's cannot out of the box use code in jars in the jar.
To make that work (which is nicer), use http://code.google.com/p/onejar-maven-plugin/, I think it's a Maven mojo wrapper to one-jar: http://one-jar.sourceforge.net/
There is also an sbt-plugin for one-jar:
https://github.com/sbt/sbt-onejar
In order to package a swing application in a runnable jar, the solution that worked for me was to export my project as a normal jar file (non executable) and update the jar's manifest to:
add scala-library.jar scala-swing.jar packages to the path
indicate the main class
You can find the Manifest file inside the jar (that you can open with 7z for example) at the following path:
META-INF/MANIFEST.MF
Add the following lines at the end of the manifest:
Main-Class: myPackage.myMainClass
Class-Path: scala-library.jar scala-swing.jar
Now your jar should execute properly when clicking on it.
NOTE: You can find more information about manifest customizing here:
http://docs.oracle.com/javase/tutorial/deployment/jar/manifestindex.html.
Hers is my solution, maven -->create scala runnable jar.
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<execution>
<id>scala-compile-first</id>
<goals>
<goal>compile</goal>
</goals>
<configuration>
<includes>
<include>**/*.scala</include>
</includes>
</configuration>
</execution>
<execution>
<id>scala-test-compile</id>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>xxx.xxx.xxx.Main</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>reference.conf</resource>
</transformer>
</transformers>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
I am writing a Maven plugin which needs to check if a certain project
dependency has javadocs and sources available... and if so, would like
to download them and archive them on a server.
I cannot find out how to check if the javadocs and source are available
or how to access them if they are.
Any help would be appreciated.
You can reference additional artifacts by adding the classifier tag to a dependency. The classifier is the additional part of the artifact's name in the repository, e.g junit-4.5-sources.jar
So to directly declare a dependency on the junit sources jar you can specify it as follows:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.5</version>
<classifier>sources</classifier>
<scope>test</scope>
</dependency>
If you want to download all the dependency sources, use the maven-dependency-plugin's copy-dependencies goal specifying the classifier sources. The following example defines two executions, one for sources and one for javadocs.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>sources</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<classifier>sources</classifier>
<failOnMissingClassifierArtifact>false</failOnMissingClassifierArtifact>
<outputDirectory>${project.build.directory}/sources</outputDirectory>
</configuration>
</execution>
<execution>
<id>javadocs</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<classifier>javadoc</classifier>
<failOnMissingClassifierArtifact>false</failOnMissingClassifierArtifact>
<outputDirectory>${project.build.directory}/javadocs</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
If you want to package all the downloaded artifacts into a zip, you can use the maven-assembly-plugin to create an archive of the project. The example below are the contents of an assembly descriptor file to include the sources and javadocs directories:
<assembly>
<id>project</id>
<formats>
<format>zip</format>
</formats>
<fileSets>
<fileSet>
<directory>${project.basedir}</directory>
<useDefaultExcludes>true</useDefaultExcludes>
<includes>
<include>${project.build.directory}/sources</include>
<include>${project.build.directory}/javadocs</include>
</includes>
</fileSet>
</fileSets>
</assembly>
To reference the assembly, add a plugin configuration to your pom. This assumes the above contents have been put in src/main/assembly/sources.xml (make sure it is defined after the dependency configuration above):
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2-beta-4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>src/main/assembly/sources.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>