I'm trying to use the Maven shade plugin (according to the tutorial here) to create a "fat jar" from my project.
I'm working on my project in eclipse and when I look at the (huge) resulting fat jar, I see that it contains a lot (possibly all) of classes from the Eclipse IDE code itself.
Why is it doing that and how to prevent it from doing it?
I've tried just listing a bunch of directories in the <exclude> <filters>, but the Eclipse JDT jars also have some files in jars "root" folders and its not easy to list all of these as well.
The shade <plugin> part of the pom.xml file currently looks like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>${maven.shade.version}</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<createDependencyReducedPom>true</createDependencyReducedPom>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>vertx-unit*/**</exclude>
<exclude>org/apache/derby/**</exclude>
<exclude>javax/annotation/**</exclude>
<exclude>com/google/googlejavaformat/**</exclude>
<exclude>org/eclipse/**</exclude>
<exclude>jdtCompilerAdapter.jar</exclude>
<exclude>ant_tasks/**</exclude>
<exclude>META-INF/services/org.osgi.framework.launch.FrameworkFactory</exclude>
<exclude>about_files/**</exclude>
<exclude>org/osgi/**</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<manifestEntries>
<Main-Class>io.vertx.core.Starter</Main-Class>
<Main-Verticle>io.thesphere.service.App</Main-Verticle>
</manifestEntries>
</transformer>
</transformers>
<artifactSet />
<outputFile>${project.build.directory}/${project.artifactId}-${project.version}-fat.jar</outputFile>
</configuration>
</execution>
</executions>
</plugin>
Related
I have a maven project with both Java and Scala components, but when I use maven-shade-plugin, it relocates package names for both Java and Scala files, but ONLY renames packages inside Java files, Scala files still contain the older package names, what am I missing?
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<!--<minimizeJar>true</minimizeJar>-->
<artifactSet>
<includes>
<include>ml.dmlc:xgboost4j-spark</include>
<include>ml.dmlc:xgboost4j</include>
</includes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<relocations>
<relocation>
<pattern>ml.dmlc.xgboost4j</pattern>
<shadedPattern>ml.dmlc.xgboost4j.shaded</shadedPattern>
</relocation>
</relocations>
<transformers>
</transformers>
</configuration>
</execution>
</executions>
</plugin>```
Sadly, I believe that Maven is intended to have this functionality but, currently (Dec 2020), it does not.
This can be seen with this bug ticket:
https://issues.apache.org/jira/browse/MSHADE-345
workaround
I have personally done a silly workaround for this. I make a new empty mvn project that has the dependency:
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>28.0-jre</version>
</dependency>
And the plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>com.google.</pattern>
<shadedPattern>shader.com.google.</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
Then in the project with code that requires a low version of guava and a new version of guava, include the empty project as a dependency.
<dependency>
<groupId>com.yoursite.yourwork</groupId>
<artifactId>shader</artifactId>
<version>0.0.1</version>
</dependency>
Then in a .scala file you import the new (version 28) guava classes like this:
import shader.com.google.common.graph.Network
Why this works
Since the error only occurs in scala projects where you refer to your own class that uses the dependency, or as said in the question "Scala files still contain the older package names", shading a project that does not refer to its own dependencies bypasses the bug.
Yes, it does. Choose any build version you want and import the library into your Scala project.
I am getting above exception in my web application running in Tomcat when packaged all my dependencies including spring-data-jpa.jar in a single jar using maven-shaded-plugin and put under WEB-INF/lib directory.
Problem dis-appears if I package the spring-data-jpa.jar directly into WEB-INF/lib along with my shaded jar?
NOTE: I will be running the same package as AWS Lambda hence I need to create a shaded jar.
To help others, the problem was that multiple spring-*.jar files META-INF/spring.handlers files which overwrites each other while running the maven-shade-plugin.
To resolve use <transformers> in the plugin configuration. My final plugin configuration looks like as follows;
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<!-- Remove signatures from transitive dependencies and append spring handlers and schemas -->
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.handlers</resource>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.schemas</resource>
</transformer>
</transformers>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
Above will merge all handlers in one single file in final jar. Enjoy :-)
Following this Guide I ran the command
mvn assembly:assembly
and got the Build Failure of
Error reading assemblies: No assembly descriptors found.
I've looked at numerous questions on this, but to no avail.
From this post, I created a .xml with this inside:
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>jar-with-dependencies</id>
<formats>
<format>jar</format>
</formats>
<dependencySets>
<dependencySet>
<scope>runtime</scope>
<unpack>true</unpack>
<unpackOptions>
<excludes>
<exclude>**/LICENSE*</exclude>
<exclude>**/README*</exclude>
</excludes>
</unpackOptions>
</dependencySet>
</dependencySets>
<fileSets>
<fileSet>
<directory>${project.build.outputDirectory}</directory>
<outputDirectory>${project.build.outputDirectory}</outputDirectory>
</fileSet>
<fileSet>
<directory>src/main/resources/META-INF/services</directory>
<outputDirectory>META-INF/services</outputDirectory>
</fileSet>
</fileSets>
</assembly>
and included this in the pom.xml:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2-beta-5</version>
<configuration>
<descriptors>
<descriptor>jar-with-dependencies.xml</descriptor>
</descriptors>
</configuration>
</plugin>
but still no luck.
I'm pretty new to this as you can probably tell, how can I get this running?
~~EDIT~~
In the pom.xml I changed
<descriptor>jar-with-dependencies.xml</descriptor>
To
<descriptor>src/main/assembly/jar-with-dependencies.xml</descriptor>
~~EDIT 2~~
pom.xml now contains this:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2-beta-5</version>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>src/main/assembly/jar-with-dependencies.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
~~EDIT 3~~
This pom.xml now works for me:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2-beta-5</version>
<configuration>
<descriptors>
<descriptor>src/main/assembly/jar-with-dependencies.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
For this to work, you need to create the file jar-with-dependencies.xml in src/main/assembly/ and this XML:
<descriptors>
<descriptor>src/main/assembly/jar-with-dependencies.xml</descriptor>
</descriptors>
i.e. you need to specify the path to the file and the convention is to put the files into src/main/assembly/.
To use the ones provided by Maven, you need to use the descriptorRef element instead (wrapped in a descriptorRefs).
Also don't put the descriptor inside of the execution element or mvn assembly:assembly can't find it anymore (since you specifically moved it to the mvn package target).
[EDIT] I followed the tutorial myself and there is an important point which you might have missed: You need to select the correct archetype. In my case, that was 5 but the order can change. So read the whole list and look for the string openimaj-quickstart-archetype or things will break.
Due to some quirks in some dependencies, I'm having trouble with sbt-assembly, and have been told that people working with Java and have had good results with Maven's shade plugin.
How can I use Maven's shade plugin for Scala / sbt?
You can add the following to your POM
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.group.id.Launcher1</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
Can you tell me, if this is possible, how to export a Scala application to a normal runnable JAR that can run directly in the JVM ?
Thanks
It is perfectly possible, see for instance this: running a maven scala project. Since Scala compiles to Java bytecode, JVM is not even aware of the underlying implementation language.
In essence after compiling Scala sources using scalac you will get a bunch of .class files which you can later package into a JAR. Then you can simply run them using:
$ java -cp "your.jar:scala-library.jar" com.example.Main
Note that you must include scala-library.jar on the CLASSPATH (currently it is almost 9 MiB...) and specify class containing main method.
If you use sbt to build you can use one of the one-jar plugins. They will put all dependencys into one big jar file (inclusive all the scala.jar files). This means that you only need one jar file and don't have to manage all the dependencys.
As an example with sbt-assembly (mostly copied from https://github.com/sbt/sbt-assembly):
project/plugins.sbt:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "X.X.X")
build.sbt:
import AssemblyKeys._ // put this at the top of the file
seq(assemblySettings: _*)
then you can generate the jar with:
sbt assembly
As an alternative to Fabian's answer, if you're using Maven, you can use the assembly-plugin. Something like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>package-jar-with-dependencies</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<appendAssemblyId>true</appendAssemblyId>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifestEntries>
<SplashScreen-Image>splash.png</SplashScreen-Image>
</manifestEntries>
<manifest>
<mainClass>se.aptly.epm.main.PrognosisApp</mainClass>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
That will package all your deps up, including scala-library.jar (if it's in your deps), but will do so flattened with all classes unpacked. This is because runnable jar's cannot out of the box use code in jars in the jar.
To make that work (which is nicer), use http://code.google.com/p/onejar-maven-plugin/, I think it's a Maven mojo wrapper to one-jar: http://one-jar.sourceforge.net/
There is also an sbt-plugin for one-jar:
https://github.com/sbt/sbt-onejar
In order to package a swing application in a runnable jar, the solution that worked for me was to export my project as a normal jar file (non executable) and update the jar's manifest to:
add scala-library.jar scala-swing.jar packages to the path
indicate the main class
You can find the Manifest file inside the jar (that you can open with 7z for example) at the following path:
META-INF/MANIFEST.MF
Add the following lines at the end of the manifest:
Main-Class: myPackage.myMainClass
Class-Path: scala-library.jar scala-swing.jar
Now your jar should execute properly when clicking on it.
NOTE: You can find more information about manifest customizing here:
http://docs.oracle.com/javase/tutorial/deployment/jar/manifestindex.html.
Hers is my solution, maven -->create scala runnable jar.
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<execution>
<id>scala-compile-first</id>
<goals>
<goal>compile</goal>
</goals>
<configuration>
<includes>
<include>**/*.scala</include>
</includes>
</configuration>
</execution>
<execution>
<id>scala-test-compile</id>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>xxx.xxx.xxx.Main</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>reference.conf</resource>
</transformer>
</transformers>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>