I have a Scala Spark application for which I want to create jar with dependencies using maven.
I am using maven-assembly-plugin
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
but when I am running my app with Spark-submit command, I am still getting java.lang.NoClassDefFoundError. When I am running it with --jars option and provide path for dependencies jar, It is running fine. Please let me know if I am missing something. Thanks
The configuration looks good.
Try to delete previous application jars and run ‘mvn package’ command to create a new one.
Related
I am trying to executing a powershell script that writes to a file during a maven build.
I am invoking the build with mvn clean install through Eclipse IDE.
This is the plugin in my pom.xml:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>${project.basedir}/.my-file.ps1</executable>
</configuration>
</plugin>
</plugins>
The powershell script is a hidden file so that's why I have a . in front of the file name.
However, the plugin is not executing and I am following the instructions in the official documentation.
You are running mvn clean install which will go through various build phases, but your exec plugin execution is not attached to any phase. You'll have to either:
Attach your execution to a phase by adding the <phase> element to your execution, for example to attach it to the pre-integration-test phase:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<id>my-exec</id>
<phase>pre-integration-test</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>${project.basedir}/.my-file.ps1</executable>
</configuration>
</plugin>
Or call specifically the exec goal using mvn exec:exec command.
If you are not familiar with the Maven lifecycle and various phases of a build, read the Introduction to the Build Lifecycle guide, or specifically the Plugins part to learn more about plugins executions and phases attachment.
I tried to look around a lot to solve this problem but I can't really solve it.
In my scala project i'm trying to build a fatjar with all the dependencies in it, so here is my pom:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.my.project.start.CommandStarter</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
So i'm pointing to my main class, and in the manifest , inside the jar , i can see that the main class is there too..
But when i run my jar by launching : scala myjar.jar i have a
java.lang.ClassNotFoundException
Any suggestion?
Thank you!
I have an applet that uses several external libs. The project requires the applet JAR is signed because I perform disk operations. Another requirement is that all libs are included in the applet jar . My first attempt working this way was to include all the JARs of the libraries in a local directory of the Eclipse project and include them in the Eclipse project. After that I exported the entire project as a non-executable JAR, getting run most libs. But some libraries are still not referenced and I can not run my application via applet completely. Is there any more appropriate way to use libs inside a signed applet JAR?
If you are using maven then you can extract all classes needed. The maven plugin will do this for you. It can also sign it if you have an jks file. Here is some setup for maven.
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.1</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>attached</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>sign</goal>
</goals>
</execution>
</executions>
<configuration>
<keystore>${basedir}/the jks.jks</keystore>
<alias>the alias</alias>
<storepass>the store pass</storepass>
<signedjar>${project.build.directory}/signed/${project.build.finalName}.jar</signedjar>
<verify>true</verify>
<jarPath>${project.build.directory}/${project.build.finalName}-jar-with-dependencies.${project.packaging}</jarPath>
</configuration>
</plugin>
</plugins>
I have a workspace with multiple projects. All are Maven projects. The target directory of one of the projects contains a batch file after it is built. Now, I need one of the other projects in the workspace to run this batch file. So, I want to get the path to the current workspace programmatically without introducing new dependencies to accomplish this. Does anybody know of a way to do this?
Edit 1: I have a parent Maven project in the workspace. One of its children's target directory contians the batch file. A different child of the parent (which is a testing project) needs to run the batch file. I can use the Maven basedir variable to get the batch file which isn't pretty and doesn't work if I am running individual tests with Eclipse. So I'd like to avoid that solution.
The problem you'll have is that projects in Eclipse aren't necessarily stored in the workspace directory; they could be anywhere on the file system. This means that simply knowing where the workspace is won't necessarily help you find the batch file.
For example: my workspace is $HOME/workspace, but all my projects (the working copies) are in $HOME/code/project. Being able to determine the workspace isn't very helpful. Projects can exist outside the workspace, and still appear in Eclipse by using File -> Import.
Your best bet is probably to 'attach' the batch file to the build using the attach-artifact goal of build-helper-maven-plugin. There's an example of how to do that here.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>attach-artifacts</id>
<phase>package</phase>
<goals>
<goal>attach-artifact</goal>
</goals>
<configuration>
<artifacts>
<artifact>
<file>script.bat</file>
<type>bat</type>
</artifact>
</artifacts>
</configuration>
</execution>
</executions>
</plugin>
Then, your other project can use the copy goal of maven-dependency-plugin to resolve the script into its own directory and run it from there.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.5.1</version>
<executions>
<execution>
<id>copy</id>
<phase>package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>...</groupId>
<artifactId>...</artifactId>
<version>...</version>
<type>bat</type>
<overWrite>true</overWrite>
</artifactItem>
</artifactItems>
<outputDirectory>${project.build.directory}/scripts</outputDirectory>
<overWriteSnapshots>true</overWriteSnapshots>
</configuration>
</execution>
</executions>
</plugin>
How about the maven-antrun-plugin? It's not pretty, but it gets the job done:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id><!-- insert an id --></id>
<phase><!-- insert a maven lifecycle phase --></phase>
<configuration>
<tasks>
<exec
dir="${basedir}"
executable="${basedir}/src/main/sh/your-script.sh"
failonerror="true">
<arg line="arg1 arg2 arg3" />
</exec>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
Ref: http://maven.apache.org/plugins/maven-antrun-plugin/usage.html
Can you tell me, if this is possible, how to export a Scala application to a normal runnable JAR that can run directly in the JVM ?
Thanks
It is perfectly possible, see for instance this: running a maven scala project. Since Scala compiles to Java bytecode, JVM is not even aware of the underlying implementation language.
In essence after compiling Scala sources using scalac you will get a bunch of .class files which you can later package into a JAR. Then you can simply run them using:
$ java -cp "your.jar:scala-library.jar" com.example.Main
Note that you must include scala-library.jar on the CLASSPATH (currently it is almost 9 MiB...) and specify class containing main method.
If you use sbt to build you can use one of the one-jar plugins. They will put all dependencys into one big jar file (inclusive all the scala.jar files). This means that you only need one jar file and don't have to manage all the dependencys.
As an example with sbt-assembly (mostly copied from https://github.com/sbt/sbt-assembly):
project/plugins.sbt:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "X.X.X")
build.sbt:
import AssemblyKeys._ // put this at the top of the file
seq(assemblySettings: _*)
then you can generate the jar with:
sbt assembly
As an alternative to Fabian's answer, if you're using Maven, you can use the assembly-plugin. Something like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>package-jar-with-dependencies</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<appendAssemblyId>true</appendAssemblyId>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifestEntries>
<SplashScreen-Image>splash.png</SplashScreen-Image>
</manifestEntries>
<manifest>
<mainClass>se.aptly.epm.main.PrognosisApp</mainClass>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
That will package all your deps up, including scala-library.jar (if it's in your deps), but will do so flattened with all classes unpacked. This is because runnable jar's cannot out of the box use code in jars in the jar.
To make that work (which is nicer), use http://code.google.com/p/onejar-maven-plugin/, I think it's a Maven mojo wrapper to one-jar: http://one-jar.sourceforge.net/
There is also an sbt-plugin for one-jar:
https://github.com/sbt/sbt-onejar
In order to package a swing application in a runnable jar, the solution that worked for me was to export my project as a normal jar file (non executable) and update the jar's manifest to:
add scala-library.jar scala-swing.jar packages to the path
indicate the main class
You can find the Manifest file inside the jar (that you can open with 7z for example) at the following path:
META-INF/MANIFEST.MF
Add the following lines at the end of the manifest:
Main-Class: myPackage.myMainClass
Class-Path: scala-library.jar scala-swing.jar
Now your jar should execute properly when clicking on it.
NOTE: You can find more information about manifest customizing here:
http://docs.oracle.com/javase/tutorial/deployment/jar/manifestindex.html.
Hers is my solution, maven -->create scala runnable jar.
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<execution>
<id>scala-compile-first</id>
<goals>
<goal>compile</goal>
</goals>
<configuration>
<includes>
<include>**/*.scala</include>
</includes>
</configuration>
</execution>
<execution>
<id>scala-test-compile</id>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>xxx.xxx.xxx.Main</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>reference.conf</resource>
</transformer>
</transformers>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>