Let me explain first what I mean by necessary libs. I'm creating my first project using the Google App Engine for Java with the official Google Maven Plugin, the main problem that have Maven as a packaging solution (or maybe the Java development as a whole) is that if the dependency tree grows too much, the release process may be harder.
Let me illustrate it with an example. Let's start with the Jackson JSON library (it's a good starting point since it has no parent dependencies), now someone makes a JSON-RPC library and uses Jackson for the JSON serialization/deserialization. Imagine that this library not just provides a JSON-RPC client implementation, but also a server, that means that the POM of this lib will add some Java EE related libraries such us Jetty as dependencies.
Probably the guidelines say that the application should be either divided into modules or mark the server related deps as optional, but you know that many people don't follow the standards.
Now someone need a JSON-RPC client for his/her project, call it Project X, and uses the lib mentioned above, at compile time there will be no problems, Maven will successfully download the required libs and the application will compile fine, but the problem comes when that person wants to release the application. Which dependencies should be distributed along with the package (in a lib folder for example)?
Actually that's something that happened to me, I wasn't too much familiar with Maven so I used the Eclipse Runnable Jar Exporter, that produced jar file with all the maven libs copied to a lb subfolder, so the workaround that I did then was to just delete the libs that looked unnecessary and then tested if the application was still working. If there are classes that are not executed, as far as I know they are not loaded by the ClassLoader so they could be omitted and are unnecessary
I can't use the same trick now since the scenario is much more complex, we are talking of a Java Web Application, not a desktop application like the other one, and the library that I want to include is a Liquid Template Engine, which uses the ANTLR framework to generate the parsers plus Jackson for the JSON handler and Jsoup for HTML parsing.
Which libs should be packaged inside the WEB-INF/lib folder? I'm sure that I will need Jackson for JSON parsing but I'm not so sure about Jsoup, and what about ANTLR, it is necessary or is used just at compile time?
Update: I think I need to re-formulate my question, actually what I want is to determine which dependencies are really necessary for the application, and package those into the app WEB-INF/lib folder
Solution: It seems that the POM file that is packaged in the WAR file of the web app is used once the app is in the Google App Engine production environment to retrieve the necessary dependencies, and probably the appengine:update goal only packages those dependencies that can't be retrieved from the maven central repo, so there is no need to worry about that.
Thanks to David to point this.
You should check Maven's dependency scopes. Here's an extract from the documentation :
There are 6 scopes available:
compile This is the default scope, used if none is specified. Compile
dependencies are available in all classpaths of a project.
Furthermore, those dependencies are propagated to dependent projects.
provided This is much like compile, but indicates you expect the JDK
or a container to provide the dependency at runtime. For example, when
building a web application for the Java Enterprise Edition, you would
set the dependency on the Servlet API and related Java EE APIs to
scope provided because the web container provides those classes. This
scope is only available on the compilation and test classpath, and is
not transitive.
runtime This scope indicates that the dependency is
not required for compilation, but is for execution. It is in the
runtime and test classpaths, but not the compile classpath.
test This
scope indicates that the dependency is not required for normal use of
the application, and is only available for the test compilation and
execution phases.
system This scope is similar to provided except that
you have to provide the JAR which contains it explicitly. The artifact
is always available and is not looked up in a repository.
import (only
available in Maven 2.0.9 or later) This scope is only used on a
dependency of type pom in the section. It
indicates that the specified POM should be replaced with the
dependencies in that POM's section. Since they
are replaced, dependencies with a scope of import do not actually
participate in limiting the transitivity of a dependency.
So in a Maven project, the developer indicates which dependencies should be bundled in the application and which should not.
Basically there are two cases here :
If you're building a web application (WAR or EAR format) and want to deploy it, or if you're building an actual runnable jar, then you will need to bundle it with all the dependencies with scope compile and runtime.
If you're building a library, then you do not package any dependency with your library. Instead you include the pom.xml so that others know what dependency your library requires. For Maven to know how to find the associated POM for a given jar, the best and most common solution is to deploy the library to a Maven repository. Repos have a directory structure that helps Maven find the right version of a library, and find the POM that indicates the required dependencies.
Depending on wether your library is open source or not, you will be able to be hosted for free by some repositories such as Sonatype (complete list here). But you can also setup your own repository either by installing a dedicated software such as Nexus or by configuring a Github project as the repo, as is explained on this blog.
You can exclude any transitive dependency.
For your case, to remove jetty from this json-rpc-library, you need:
<dependency>
<groupId>com.somecomp</groupId>
<artifactId>jsonrpclib</artifactId>
<version>1.0</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
</exclusion>
</exclusions>
</dependency>
See docs: http://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html
Related
I was using eclipse for building my project. Now I am configuring the project to Maven project. All the libs were manually downloaded to lib folder. If I do a Maven build, Maven is unable to map those libs. There are lots of libs, I don't want to manually place them in pom.xml. What is the best way out?
I have read few answers, first declare local repo and then add the respective dependencies. But again I don't want to add all the dependencies manually.
Maven: best way of linking custom external JAR to my project?
1.) Install locally
There are two scopes which are interesting for you. If you declare a scope "runtime" (which is the default scope) and have a local repository configured, maven will try to download the file from your local repository.
2.) System dependency
If you just don't want maven to manage dependencies, you can (although shan't) use the system scope like this:
<dependency>
...
<scope>system</scope>
<systemPath>/path/to/dependency.jar</systemPath>
</dependency>
The downsides are:
the system scope is deprecated and might not work in future versions of maven.
you need to check in a jar file, which is not a good idea.
it may not be portable (e.g. architecture dependent libraries).
I have started reading and trying maven since yesterday. But its making me go crazy.
I am java developer but never came across ant or maven before.
I want to know what exactly happens with the dependency tag in POM.xml file?
Lets say, I am using camel framework and want to use camel core jars.
If one of my class file contains following line:
CamelContext context = new DefaultCamelContext();
so what exactly I need to do after that?
Do I need to include the jars myself in the class path or dependency tag will download the jar files over internet for me?
If the case is former, what dependency tag will do? & where should I place my jar files? I mean is there any specific location on my hard drive? and
if the case is lateral then during compile time I get error "cannot be resolved to a type"
And the imports are to be specified or not?
I know the question might sound silly but I am not able to find its answer.
I have tried googling alot, it didn't help me still.
Any help would be greatful, even help on maven topics which I might come across in near future would be appreciated.
Thanks in advance.
Solved. Please check https://stackoverflow.com/a/20094312/1121208 for help
dependency tag will download the jar specified in the dependency tag for you if available. Otherwise will raise a pom.xml error - could not found dependency..
Imports have nothing to do with maven. They will appear when you will you another class in your class/java file. So if you import in build path the jar by yourserf or if you put it there with maven, you will have the import.
Are you using eclipse or any other ide ?
First of all, Maven is a build tool. It doesn't run your app. It builds it. So, at runtime, the classpath needs to be set like for any oter application yo would have built with something else.
When you build an app, you depend on external libraries. The dependencies mechanism of Maven simply lets you declare wwhich libraries your ap needs. When you build your app, Maven downloads these libraries from a central repository (or sevaral ones), and stores them in a local repository on your hard drive. These jars are automatically added to the build classpath by Maven. At runtime though, depending on the kind of ap you're building, you'll have to copy or embed those jars in order to create a runnable application.
The rules of Java don't change just you build them with Maven. Meven uses the stadard Java compiler (javac). And of course, if you want to use a class by its simple name, you'll have to add an import statement for this class.
I think that, before using Maven, you should try to compile and run a simple application depending on an external library without using any IDE. You would then understand better all the steps that are required to build and run an app, the concept of build and runtime classpath, etc.
Finally got what I needed to know
Sharing it for others who may stuck up in same situation
Does dependency tag download the jar specified?
maven dependency tag actually downloads the jar files you specify in the dependency tag. It downloads and save it under .m2/repositories(local repository) folder on your hard drive (along with some information like last updated, etc)
Local repository is shared among all your projects
from where it downloads?
It downloads the jar from the central repositories. There central repositories contain almost all the open source jar files one needs in a project. It downloads based on information you provide in groupid, artifactid, etc.
http://repo1.maven.org/maven/
http://mvnrepository.com/
can be checked for correct groupid, etc
Once these jar files are downloaded, they are automatically added to the classpath and are available in your project for use.
If the jar files you are searching for, are not available in the central repository, maven may throw error, in that case you can download it manually and let maven know about it.
Without maven you need to put jars into lib folder.
With maven you specify as declaration inside <dependencies>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
and when you do mvn package, maven will download required jars on your PC.
With Eclipse and m2e (maven eclipse integration) you can do that all not leaving Eclipse,
and even get sources for used libraries automatically.
Read http://maven.apache.org/ It worth it.
We're trying to manage a project made of multiple Eclipse plug-ins.
One plug-in has a dependency to a 3rd party plug-in. It imports a class from a library named bpmn2. This library is a jar file and NOT included in the Maven remote repository.
Locally, in Eclipse, we have the library checked out into the workspace and referenced in the classpath of the Eclipse project. The plug-in manifest doesn't explicitly state the dependency to bpmn2. Which works locally...
We try to use Jenkins for continuous integration. The Multi-Plug-in-Project is managed using Maven and multiple POM files, using the Maven tycho plug-in.
The problem is that Maven doesn't care for the locally present library bpmn2 (of course). So we thought that using Maven install:install to install bpmn2 to the local Maven repository
./mvn install:install-file -Dfile=/home/someUser/bpmn2/org.eclipse.bpmn2_0.7.0.201111021300.jar -DgroupId=org.eclipse.bpmn2 -DartifactId=bpmn2 -Dversion=0.7.0 -Dpackaging=jar -DlocalRepositoryPath=/var/lib/jenkins/localRep/
and adapting the corresponding POM with a dependency entry for the library
<dependencies>
<dependency>
<groupId>org.eclipse.bpmn2</groupId>
<artifactId>bpmn2</artifactId>
<version>0.7.0</version>
<type>jar</type>
</dependency>
</dependencies>
would work. But it didn't.
The output of Maven is:
[ERROR] Cannot resolve project dependencies:
[ERROR] Software being installed: modeltype.bpmn2 1.0.0.qualifier
[ERROR] Missing requirement: modeltype.bpmn2 1.0.0.qualifier requires 'bundle org.eclipse.bpmn2 0.7.0' but it could not be found
[ERROR]
The question is:
How could we better integrate the 3rd party library?
The local classpath reference is not the optimal solution for the greater picture, I think. Should everyone who develops for the project install the bpmn2 library and only use the dependency in the manifest?
And what are we doing wrong with Maven? The local repository is
/var/lib/jenkins/localRep
and after installing the library to the repository, it seemed that the created dir structure was okay.
/org/eclipse/bpmn2/bpmn2/0.7.0/bpmn2-0.7.0.jar
Can somebody help?
The simplest solution would be to set up a repository server for your company and proxy all your calls through that server.
I use Nexus from Sonatype.
It acts as a proxy when you need to download artifacts from remote locations, like maven central, but it also has an ability to setup repositories to store non-publcicly distributed artifacts.
The whole setup process is very well documented here -> http://www.sonatype.com/books/nexus-book/reference/.
Note, that open-source edition of the product is very good and is enough for your purposes.
Just to be fair, there is a competing product called Artifactory. You may read about it here -> http://www.jfrog.com/products.php
I had a similar scenario where I need to include a third party library (not loaded in maven repository) in Maven Web project.
Following approach helped us make the code portable.
Solution:
Created the "lib" directory under WEB_INF as below:
<<Project_Base_Dir>>/src/main/webapp/WEB-INF/lib
Copied the third party JARs to LIB directory.
Updated POM.xml to use the SYSTEMPATH as below:
<dependency>
<groupId>GROUP_ID</groupId>
<artifactId>ARTIFACT_ID</artifactId>
<version>VERSION</version>
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/<<JAR_FILENAME>></systemPath>
<scope>system</scope>
</dependency>
If you can't use Nexus, you can use scope system :
Dependency Scopes
system: This dependency is required in some phase of your
project's lifecycle, but is
system-specific. Use of this scope
is discouraged: This is considered an
"advanced" kind of feature and should
only be used when you truly understand
all the ramifications of its use,
which can be extremely hard if not
actually impossible to quantify.
This scope by definition renders your
build non-portable. It may be
necessary in certain edge cases. The
system scope includes the
<systemPath> element which points to
the physical location of this
dependency on the local machine. It is
thus used to refer to some artifact
expected to be present on the given
local machine an not in a repository;
and whose path may vary
machine-to-machine. The systemPath
element can refer to environment
variables in its path: ${JAVA_HOME}
for instance.
You can use it with
Be careful, using this is probably making your build not portable.
I have a library that I wrote in Scala that uses Bouncy Castle and has a whole bunch of dependencies. When I roll a jar, I can either roll a "fat" jar that has all the dependencies (including scala), which weighs in around 19 MB, or I can roll a skinny jar, which doesn't have dependencies, but is only a few hundred KB.
The problem is that I need to include the Bouncy Castle classes/jar with my library, because if its not on the classpath at runtime, all kinds of exceptions get thrown.
So, I think the ideal situation is if there is some way that I can get either Maven or SBT to include some but not all dependencies in the jar that gets rolled. Some dependencies are needed at compile-time, but not at run time, such as the Scala standard libraries. Is there some way to get that to happen?
Thanks!
I would try out the sbt proguard plugin from https://github.com/nuttycom/sbt-proguard-plugin . It should be able to weed out the classes that are not in use.
If it is sufficient to explicitly define which dependencies should be added (one the artifact-level, i.e., single JARs), you can define an assembly (in case of a single project) or an additional assembly project (in case of a multi-module project). Assembly descriptors can explicitly exclude/include artifacts from the dependencies.
Here is some good documentation on this topic (section 8.5.4), here is the official documentation.
Note that you can include all artifacts that belong to one group by using the wildcard notation in dependecySets, e.g. hibernate:*:jar would include all JAR files belonging to the hibernate group.
Covering maven...
Because you declare your project to be dependent upon bouncy castle in your maven pom, anybody using maven to depend upon your library will by default pull in bouncy castle as a transitive dependency.
You should set the appropriate scope on your dependencies, e.g. compile for stuff needed at compile and runtime, test for dependencies only needed in testing and provided for stuff you expect to be provided by the environment.
Whether your library's dependencies are packaged into dependent projects when they are built is a question of how those are projects configured and setting the scopes will influence the default behaviour.
For example, jar type packaging by default does not include dependencies, whereas war will include those in compile scope (but not test or provided). The design aim here was to have packaging plugins behave in the most commonly required way without needing configuration, but of course packaging plugins in maven can be configured to have different behaviour if needed. The plugins themselves which do packaging are well documented at the apache maven site.
If users of your library are unlikely to be using maven to build their projects, an option is to use the shade plugin which will allow you to produce an "uber-jar" which contains all the dependencies you wish. You can configure particular includes or excludes.
This can be a problematic way to deliver, for example where your library includes dependencies which version clash with the direct dependencies of projects using it, i.e. they use a different version of the same libraries yours does.
However if you can it is best that you leave this to maven to manage so that projects using your library can decide whether they want your dependencies or to specify particular versions giving them more flexibility. This is the idiomatic approach.
For more information on dependencies and scopes in maven, see the reference guide published by Sonatype.
I'm not a scala guy, but I have played around with assembling stuff in Java + Maven.
Have you tried looking into creating your own assembly descriptor for the assembly plugin? https://maven.apache.org/plugins/maven-assembly-plugin/assembly.html
You can copy / paste the jar-with-dependencies descriptor then just add some excludes to your < dependencySet >. I'm not a Maven expert, but you should be able to configure it so different profiles kick off different assembly builds.
EDIT: Ack, didn't see my HTML got hidden
I used Maven before in quite a few projects but they we already configured by someone else. I quickly understood the main concepts (groups, artifacts and versions for jars, local and remote repositories) so I assumed that if I'm asked I'll be able to set up a new project from scratch easily.
That turned out not to be the case when I deal with multiple modules which depend on one another. I poked around maven docs but they are either way too concise or way too technical (sometimes I get a feeling that Maven people wrote it for themselves and not for Maven users). So I'm asking for community help.
Here's a typical scenario:
Three repositories:
My local repo at <User>/.m2
My company repo at http://maven.mycompany.com/repository
Maven Central at http://repo1.maven.org/maven2/
I'm building a normal Java EE project consisting of one EJB jar with a corresponding client jar, a web module and a JPA module (I like keeping my entities and all db-connectivity separated from business logic). That gives me 5 project in my IDE:
myapp-ear
myapp-ejb
myapp-ejb-client
myapp-jpa
myapp-web
myapp-jpa is used by myapp-ejb and myapp-web. myapp-ejb-client contains business interfaces for my EJBs.
How should I set all that up? I suspect that I need some kind of parent project but I'm not sure how should I organize inter-project dependency resolution. For example: currently when I specify
<dependency>
<groupId>com.mycompany.myapp</groupId>
<artifactId>myapp-jpa</artifactId>
<version>0.1-SNAPSHOT</version>
</dependency>
in myapp-ejb's pom.xml maven tries to search for it in my repository and tells me that the .jar can't be found. Even if I build and install all app components into my local repository one by one Maven always fails to build the ear file itself :(
I'd love to get it working under Eclipse or NetBeans.
I think you're on the right path. You need a parent project to pull it all together. In the parent project (and here is the important part) you need your dependencies laid out NOT in the dependency section of your parent pom, rather, you need them in your dependencyManagement section. Then, in your child pom's you can declare dependencies that are in the parent.
The only other gotcha (that I can think of) is that when you check the project using a version control system in Eclipse you need to remember to check the parent project out "as Maven project" otherwise the m2Eclipse plugin tends to freak out and not resolve things properly.
I hope this helps.