Deployment Concepts: Packaging JAR Dependencies, When & Why - deployment

So I'm relatively new to Java EE and I am having a tough time understanding when, where and why Java deployment files are packaged with their dependencies.
Say I build my project into myapp.jar, and it depends on fizz.jar, buzz.jar and JODA (joda-time-2.0.jar).
I've heard that the default classloader doesn't package jars inside of other jars, so I have to assume that if I called a jar task from Ant, then the default classloader would get invoked and myapp.jar would be created without those 3 dependencies in it.
Is this because the mentality is to deploy main-less jars in containers or other systems that will provide its requirements at runtime? If not, then how does myapp.jar ever run correctly?
What about executable jars? To met, these must be different than main-less jars, because they are meant to be standalone units, right? That means they would need all of their dependencies packaged with them, right?
Last but not least, what about jars that depend on jars that depend on jars...etc. (i.e., dependencygraphs that are huge)?
I guess all of these questions can be summed up as follows:
Is the idea behind a non-executable jar that it will be ran in such a way that it will know what classpath(s) to look on for its dependencies at runtime? (And thus doesn't need to be packaged with its dependencies)?
Is the idea behind an executable jar that it is a standalone unit and should be pacakged with its dependencies?
If my assertion to Question #1 above is correct, how does such classpath configuration take place? Are these settings that are stored inside the jar (such as in the manifest)? Else, how would a JRE know where to search for a particular jars dependencies at runtime?
Answers to these questions will actually clarify quite a lot of hangups I have with Java fundamentals, and so any input/help here will be appreciated enormously! Thanks

Jars do not know about other jars (unless assisted by some tools like Maven). The inter dependency of the jars is purely resolved by the Classloaders. I strongly recommend to have some idea about classloaders.
To address your questions,
Is the idea behind a non-executable jar that it will be ran in such a way that it will know what classpath(s) to look on for its dependencies at runtime? (And thus doesn't need to be packaged with its dependencies)?
NO. As mentioned, it's the classloader which looks the classpath and the jars mentioned therein. The jars do not have any information about other jars.
Is the idea behind an executable jar that it is a standalone unit and should be packaged with its dependencies?
NO. A classloader loads the standalone executable jars at the start of execution. If it needs other dependency jars it'll look into the classpath for those jars.
If my assertion to Question #1 above is correct, how does such classpath configuration take place? Are these settings that are stored inside the jar (such as in the manifest)? Else, how would a JRE know where to search for a particular jars dependencies at runtime?
For standalone jar (executable jar), the classloader looks for the classpath variable OR classpath passed while invoking the application.
For other type of application (WAR, EAR), There are predefined places/folders where the dependencies should be placed in order to get picked up. This is standardized by specs.
In a nutshell, it's the classloader which is pulling all the threads. There is standard places where it looks for all the dependent jars. This link nicely describes how the classloaders in standalone application and in a deployed (in some container) works.

JAR files are a way to package togher complex java application. Jar application are easy to move between diffrent machines and operating system.
I think the right way to use Jars is not tu put everything (every dependencies) into a single jar.
For example if your application uses a jar libryra (for example jdbc) to access a database you should not put the jdbc jar into your jar.
You had better to build a jar file with only your .class file.
Of course your code needs the jdbc jar to work properly. Here comes to explain how the virtual machine searches for extarnal classes:
-it first searches in the directories that contain the classes that come standard with J2SE (the path depends on your installation)
-it searches in the directories specified by the classpath (a classpath is either an environment variable or an option of the java command)
for example:
java -jar -c /your/path/ yourApp.jar
will run your application and will search th classes your application refers to in the directory /your/path/ so if you have external jars you can put them in that directory.
For detailed documentation: http://download.oracle.com/javase/tutorial/deployment/jar/index.html

Related

Adding Dependency Management to an Existing Java Project

I'm working on upgrading a legacy Java project to be compatible with jboss wildfly. As part of that process, I'm replacing our old system of managing dependencies (manually scanning for jars in a folder) with an automated system.
My first thought was to use maven, which worked well initially. The maven plugin for eclipse was able to scan my project and create a pom with most of the required dependencies. That works fine for compiling and running with eclipse, but production deployment uses an ant build script. I looked into maven-ant-resolver ( https://maven.apache.org/resolver-ant-tasks/index.html ) but as far as I can tell that project doesn't have a way to add dependencies to the classpath, the best it can do is bundle them into a jar.
The other option I looked at was Ivy. It seems better suited to integration with ant. Unfortunately, the tooling for ivy seems primitive compared to maven. From what I can tell, there is no option to generate the dependency file (ivy.xml) from an existing project. With the number of dependencies I'm dealing with, especially from jboss, creating the dependency xml from scratch is not a realistic option.
What are my options for solving this problem? Is there a way to do what I want with maven or ivy that I'm not seeing? Is there another dependency management tool out there that offers all the features I need?
The maven-assembly-pluginis what i can recommend for likely usecases. Not sure if it suits you though.
In a nutshell:
You can pack folders, jars, resources, dependencies, whatever into a jar for production deployment. This jar is packaged with the, from maven-assembly-plugin internally used and thus not needed to be referenced explicitly, maven-archiver-plugin which also stores a MANIFEST.MF with the classpath in it (not by default but with few codes of tweaking).
Useful to know though: Maven allows you to quite easily create own Plugins that completely do what you want. If its just a file with the stored classpath, this could be a clean solution.

multi-project sbt build - package all dependent JARs in one directory

I have a multi-project SBT build: some projects are dependent on each other, some are dependent on third-party JARs, and there's a "main" project which depends on everything .
When I sbt package it, I get one JAR in each target/ directory.
What I want to achieve is getting all relevant JARs (mine and external) is one directory. Very similar to the way you package a WAR with Maven.
(And to clarify - I'm not interested in an assembled "FAT JAR" that contains all the dependencies in a single file. Just one directory with all JARs in it)
Im not 100% sure about the suprobject dependencies but I think SBT native packager should help you do something like that, and will also provide a start-script for windows and unixes:
http://www.scala-sbt.org/sbt-native-packager/GettingStartedApplications/MyFirstProject.html
I would recommend sbt-pack for creating self-contained JARs:
https://github.com/xerial/sbt-pack
I use it and haven't seen a glitch so far.
It also generates both OS X/Linux as well as Windows .bat entry scripts for the main classes/objects you choose.

Using JavaCompiler with Classpath referencing jars within ear

I am working on a project in which an enterprise archive (ear) deployed on a JBoss server needs to compile (and run) a class dynamically. I am using the JavaCompiler class to do this - the complication comes from the fact that the class being compiled has references to some of the classes contained within the ejb jar within the ear.
This is not a problem when the deployed ear is 'exploded' on deployment, so it is just a directory rather than an archive - in this case I am able to specify the required jar in the -classpath option of the compiler, and compilation works fine. Unfortunately due to constraints of the systems I am working with, it is not an acceptable solution to deploy these ears 'exploded', and the compiler seems not to be able to 'see' the required jar when it's wrapped up in an archive.
Given that the dynamic compilation is taking place from the ear in question, and therefore the system's class loader has access to the contents of the required jar, is there any way I can tell the compiler to just use the classes as loaded by the system class loader?
I appreciate this is something of a wordy question, but any help would be appreciated.
Thanks
It seems that there is no simple way to have the JavaCompiler load dependencies of compiled code from a ClassLoader. However, one could implement JavaFileManager directly and redirect the operations for the StandardLocation.CLASS_PATH using resource lookups on the context ClassLoader (getResource(<class/resource name>)). This would withdraw the limitation of StandardJavaFileManager directly operating on Files.
Someone already seems to have prototypically implemented that approach:
http://atamur.blogspot.de/2009/10/using-built-in-javacompiler-with-custom.html

How to add JAR libraries to WAR project without facing java.lang.ClassNotFoundException? Classpath vs Build Path vs /WEB-INF/lib

How should I add JAR libraries to a WAR project in Eclipse without facing java.lang.ClassNotFoundException or java.lang.NoClassDefFoundError?
The CLASSPATH environment variable does not seem to work. In some cases we add JAR files to the Build Path property of Eclipse project to make the code compile. We sometimes need to put JAR files inside /WEB-INF/lib folder of the Java EE web application to make the code to run on classes inside that JAR.
I do not exactly understand why CLASSPATH does not work and in which cases we should add JARs to Build Path and when exactly those JARs should be placed in /WEB-INF/lib.
The CLASSPATH environment variable is only used by the java.exe command and even then only when the command is invoked without any of the -cp, -classpath, -jar arguments. The CLASSPATH environment variable is ignored by IDEs like Eclipse, Netbeans and IDEA. See also java.lang.ClassNotFoundException in spite of using CLASSPATH environment variable.
The Build Path is only for libraries which are required to get the project's code to compile. Manually placing JAR in /WEB-INF/lib, or setting the Deployment Assembly, or letting an external build system like Maven place the <dependency> as JAR in /WEB-INF/lib of produced WAR during the build, is only for libraries which are required to get the code to deploy and run on the target environment too. Do note that you're not supposed to create subfolders in /WEB-INF/lib. The JARs have to be placed in the root.
Some libraries are already provided by the target JEE server or servletcontainer, such as JSP, Servlet, EL, etc. So you do not need put JARs of those libraries in /WEB-INF/lib. Moreover, it would only cause classloading trouble. It's sufficient to (indirectly) specify them in Build Path only. In Eclipse, you normally do that by setting the Targeted Runtime accordingly. It will automatically end up in Build Path. You do not need to manually add them to Build Path. See also How do I import the javax.servlet / jakarta.servlet API in my Eclipse project?
Other libraries, usually 3rd party ones like Apache Commons, JDBC drivers and JEE libraries which are not provided by the target servletcontainer (e.g. Tomcat doesn't support many JEE libraries out the box such as JSF, JSTL, CDI, JPA, EJB, etc), need to end up in /WEB-INF/lib. You can just copy and paste the physical JAR files in there. You do not necessarily need to specify it in Build Path. Only perhaps when you already have it as User Library, but you should then use Deployment assembly setting for this instead. See also ClassNotFoundException when using User Libraries in Eclipse build path.
In case you're using Maven, then you need to make absolutely sure that you mark libraries as <scope>provided</scope> if those are already provided by the target runtime, such as JEE, Servlet, EL, etc in case you deploy to WildFly, TomEE, etc. This way they won't end up in /WEB-INF/lib of produced WAR (and potentially cause conflicts with server-bundled libraries), but they will end up in Eclipse's Build Path (and get the project's code to compile). See also How to properly install and configure JSF libraries via Maven?
Those JARs in the build path are referenced for the build (compile) process only. If you export your Web Application they are not included in the final WAR (give it a try).
If you need the JARs at runtime you must place them in WEB-INF/lib or the server classpath. Placing your JARs in the server classpath does only make sense if several WARs share a common code base and have the need to access shared objects (e.g. a Singleton).
If you are using Maven:
Open the project properties, and under Deployment Assembly click Add...
Then select Java Build Path Entries and select Maven Dependencies
Resolved by setting permissions.
Had related issue using PySpark and Oracle jdbc. The error does not state that the file cannot be accessed, just that the class cannot be loaded.
So if anyone still struggles, check the permissions. Some might find it obvious tho'.
I want to give the answer for the folowing link question ClassNotFoundException oracle.jdbc.driver.OracleDriver only in servlet, using Eclipse
Ans: In Myeclipse go to Server-->left click on Myeclipse Tomcat7-->Configure Server Connector-->(Expand)Myeclipse Tomcat7--> Paths-->Prepend to classpath-->Add jar (add oracle14 jar)-->ok

Scala: Creating a small executable Jar relying on external Scala libraries

I'm trying to package a small application (still learning Scala!) in a "clean way". The goal is to have an executable JAR file. I've done the following:
packaged a JAR using sbt -> will work with
scala -cp myjarfile.jar MyClass
or
java -classpath path\to\scala-library.jar;myjarfile.jar MyClass
but won't work with
java -jar myjarfile.jar
because then scala/ScalaObject cannot be found. And no use adding a classpath on this last one, since the -jar option will ignore the -classpath option. Note that I have added the scala libs in my system CLASSPATH variable, but it seems to be ignored too when -jar is used.
added the scala library contents (by unzipping them first) to the jar created by sbt. This then works (the jar can be double-clicked and launched), but the manipulation is somewhat lengthy, and the resulting file is several megabytes big. Not ideal.
After hours of googling, I can see no way to create a small jar file that will launch when double-clicked and that doesn't involve a painful manipulation. What am I missing? I'm guessing there should be some way to define where the scala libraries are at system level. How do you deal with small applications that you want to be compiled and ready-to-run for efficiency?
Note that while I'm using sbt, I don't have a jar tool at hand (I'm relying on a JRE, not a JDK).
Thanks!
Pierric.
The following setup works for me:
have scala-library.jar in the same folder as the executable jar (and call java from there)
put this into your manifest:
Class-Path: scala-library.jar
Another option is to merge the contents of scala-library.jar into your application jar. The drawback is that this will increase its size. But you can use Proguard to strip unused classes from your final jar. I think there is an easy way of using sbt to package an executable jar using proguard.
Regarding the shrinking using Proguard, you can have a look at this page. It's about Android development; just ignore this part and have a look at the tables of the shrinking results. Some example applications shrink to less than 100kB.
Edit
Maybe you need to refine your question a bit. What are you trying to achieve? Do you want to install the program only on your system or do you want to distribute it?
If all you want is quickly launching a Java application without much impact of the JVM start-up time you can have a look at nailgun.