Maven 2 project versions, dependency versions, and perpetual releasing - version-control

Scenario: The system has number of components, each with its own POM. There are some long dependency chains (A depends on B depends on C, etc.). I want each "non-developer-desktop" build to be a potential release candidate -- if it passes QA, we will deploy it without rebuilding. In other words, I never want to build SNAPSHOT versions as part of my regularly scheduled builds, only versions like 1.3.0.5, 1.3.0.6, etc. I also want to enable the developers to work on multiple components at once.
To forestall some expected suggestions: the Maven Release Plugin doesn't help me ... unless there's some magic way I can have my dependency versions not be SNAPSHOTs in the POMs but still let the developers work on more than one component at once?
How should we manage the project and dependency versions in all of our our POMs? Right now it's just SNAPSHOTs everywhere, which makes things simple for the developers (they started with SNAPSHOTs and never concerned themselves with anything else). But it's worrying at deployment time (builds with SNAPSHOT dependencies are not well-defined, and not reproducible).
I'd like to avoid excessive juggling at build time, but right now I don't see any way around it.
Let's say I have components util, libj, libk, APP, and UI, with dependencies as follows:
libj -> util (libj depends on util)
libk -> util
APP -> libj
UI -> libj, libk
I have development teams working on APP and UI, and they will sometimes need to make changes/additions to some of the dependencies (even util) in order to enable their current work. How should the checked-in POM and dependency versions for each component look?
Edit: I updated the title to refer to Maven 2 instead of 2.0, since it became apparent that I would need to work with 2.1 or better in order to best resolve this.

Try to layout your builds such that modules that need to be developed together are released together. This will let the maven release plugin do most of the work for you.
For dependencies that really should have a separate lifecyle...because they change infrequently or are shared by multiple projects, you want to handle those differently. They way I do it is to keep the dependencies at the last release version until a change actually requires an upgrade to the next snapshot. In this way when you go to release the product, you will find all the things that may also be released simply by following the snapshot trails.
I find it also helpful to keep the external dependency versions specified as properties in the top pom of my project. This makes it easy to see at a glance what needs to be released. Look for an example of the Nexus pom.

This is something I find very difficult with maven and internal projects; you have two version control systems (maven's, which, quite frankly, isn't very good) and your source code control's (which, assuming it's CVS or better, supports real workflow).
Here's how we do it:
report --depends on--> core
web --depends on--> core
We use the maven release plug:
report's pom, during development, would have a SNAPSHOT version matching what's in core's pom. I do a mvn clean install in core, then I can see those changes in report in my local environment.
When I do a release of report, I must first release core via the maven release plug. When I I use it on core, it asks me to set the version of core to release (i.e. remove the -SNAPSHOT), to which I say yes, and the resulting released artifact doesn't depend on a SNAPSHOT release. When the release plugin is done, the report pom now depends on the next SNAPSHOT release of core (though you can override this during mvn release:prepare if you want).
The devs on web then get an email that a new version of core is available and they can choose to use it if they wish. Our policy is that they should update before releasing.

This is really the same problem as someone needing to branch code they are working on. You need some form of identifier to be able to get at the specific version until you merge back to the mainline development of that component.
The dependency classifier may be something worth looking into. You'll still need to have developers that will properly identify their modified code.

we work with a "Super Parent".pom-project, in it the versions are defined as properties and in every child.project, the versions are set with those properties
in addition the real projects have no version set, instead they inherit it from their parent pom-project (incl. groupId etc.)
with this set up the versions are only defined in some parent pom-projects, thus the needed setup is minimized
e.g. this structure
super parent pom-project (all 3rd party versions as properties defined, e.g. 2.5.6 spring-framework)
---- persistence-parent pom-project, inherits all from super parent
------ persistence-foobar jar project, inherits all from persistence-parent
---- util-parent
etc.
only versions you see are the ones inside the parent relations
example:
<modelVersion>4.0.0</modelVersion>
<artifactId>foo-bar-foo</artifactId>
<packaging>jar</packaging>
<name>whatever</name>
<description>...</description>
<parent>
<groupId>org.foo.bar</groupId>
<artifactId>persistence-parent</artifactId>
<version>1.0-SNAPSHOT</version>
</parent>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>foo-bar</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>foo-bar</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>org.springframework.jdbc</artifactId>
<version>${parent.spring.version}</version>
</dependency>
it works this way, because inside the pom you can use already defined values like version, even if the version is set via inheritance
we combine this with module setups inside the parent projects to make the builds easier

Related

How to package only the necessary libs in Google App Engine Project(Java)?

Let me explain first what I mean by necessary libs. I'm creating my first project using the Google App Engine for Java with the official Google Maven Plugin, the main problem that have Maven as a packaging solution (or maybe the Java development as a whole) is that if the dependency tree grows too much, the release process may be harder.
Let me illustrate it with an example. Let's start with the Jackson JSON library (it's a good starting point since it has no parent dependencies), now someone makes a JSON-RPC library and uses Jackson for the JSON serialization/deserialization. Imagine that this library not just provides a JSON-RPC client implementation, but also a server, that means that the POM of this lib will add some Java EE related libraries such us Jetty as dependencies.
Probably the guidelines say that the application should be either divided into modules or mark the server related deps as optional, but you know that many people don't follow the standards.
Now someone need a JSON-RPC client for his/her project, call it Project X, and uses the lib mentioned above, at compile time there will be no problems, Maven will successfully download the required libs and the application will compile fine, but the problem comes when that person wants to release the application. Which dependencies should be distributed along with the package (in a lib folder for example)?
Actually that's something that happened to me, I wasn't too much familiar with Maven so I used the Eclipse Runnable Jar Exporter, that produced jar file with all the maven libs copied to a lb subfolder, so the workaround that I did then was to just delete the libs that looked unnecessary and then tested if the application was still working. If there are classes that are not executed, as far as I know they are not loaded by the ClassLoader so they could be omitted and are unnecessary
I can't use the same trick now since the scenario is much more complex, we are talking of a Java Web Application, not a desktop application like the other one, and the library that I want to include is a Liquid Template Engine, which uses the ANTLR framework to generate the parsers plus Jackson for the JSON handler and Jsoup for HTML parsing.
Which libs should be packaged inside the WEB-INF/lib folder? I'm sure that I will need Jackson for JSON parsing but I'm not so sure about Jsoup, and what about ANTLR, it is necessary or is used just at compile time?
Update: I think I need to re-formulate my question, actually what I want is to determine which dependencies are really necessary for the application, and package those into the app WEB-INF/lib folder
Solution: It seems that the POM file that is packaged in the WAR file of the web app is used once the app is in the Google App Engine production environment to retrieve the necessary dependencies, and probably the appengine:update goal only packages those dependencies that can't be retrieved from the maven central repo, so there is no need to worry about that.
Thanks to David to point this.
You should check Maven's dependency scopes. Here's an extract from the documentation :
There are 6 scopes available:
compile This is the default scope, used if none is specified. Compile
dependencies are available in all classpaths of a project.
Furthermore, those dependencies are propagated to dependent projects.
provided This is much like compile, but indicates you expect the JDK
or a container to provide the dependency at runtime. For example, when
building a web application for the Java Enterprise Edition, you would
set the dependency on the Servlet API and related Java EE APIs to
scope provided because the web container provides those classes. This
scope is only available on the compilation and test classpath, and is
not transitive.
runtime This scope indicates that the dependency is
not required for compilation, but is for execution. It is in the
runtime and test classpaths, but not the compile classpath.
test This
scope indicates that the dependency is not required for normal use of
the application, and is only available for the test compilation and
execution phases.
system This scope is similar to provided except that
you have to provide the JAR which contains it explicitly. The artifact
is always available and is not looked up in a repository.
import (only
available in Maven 2.0.9 or later) This scope is only used on a
dependency of type pom in the section. It
indicates that the specified POM should be replaced with the
dependencies in that POM's section. Since they
are replaced, dependencies with a scope of import do not actually
participate in limiting the transitivity of a dependency.
So in a Maven project, the developer indicates which dependencies should be bundled in the application and which should not.
Basically there are two cases here :
If you're building a web application (WAR or EAR format) and want to deploy it, or if you're building an actual runnable jar, then you will need to bundle it with all the dependencies with scope compile and runtime.
If you're building a library, then you do not package any dependency with your library. Instead you include the pom.xml so that others know what dependency your library requires. For Maven to know how to find the associated POM for a given jar, the best and most common solution is to deploy the library to a Maven repository. Repos have a directory structure that helps Maven find the right version of a library, and find the POM that indicates the required dependencies.
Depending on wether your library is open source or not, you will be able to be hosted for free by some repositories such as Sonatype (complete list here). But you can also setup your own repository either by installing a dedicated software such as Nexus or by configuring a Github project as the repo, as is explained on this blog.
You can exclude any transitive dependency.
For your case, to remove jetty from this json-rpc-library, you need:
<dependency>
<groupId>com.somecomp</groupId>
<artifactId>jsonrpclib</artifactId>
<version>1.0</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
</exclusion>
</exclusions>
</dependency>
See docs: http://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html

Maven development time and release time

I'm starting working with Eclipse + Maven (2.2.1) and I have some question about it:
1- I have an internal framework MyFramework that is dependency in another project, something like that:
<dependency>
<groupId>com.mycompany</groupId>
<artifactId>MyFramework</artifactId>
<version>[7.0.0,),[LATEST]</version>
</dependency>
The version is a range because I want to work with my workspace framework version, or with the latest (in case I don't have the project in workspace).
My project is building fine, but when I try to make a relase (maven-release-plugin 2.3.2) I got:
NullPointerException: version of MyFramework is null
Is that a bug or am I doing something wrong?
2- Suppose that I have a version 7.0.4-SNAPSHOT and I want to roll the second version number instead of third like that:
7.0.4-SNAPSHOT -> 7.1.0-SNAPSHOT or 7.1.4-SNAPSHOT (whatever)
Can I do it with maven-release-plugin?
3- I'm starting with Maven, so I don't need to care about migrations, should I start with Maven 3 instead of Maven 2?
Please, I need help with it.
Thanks in advanced.
Answers:
This appears to be a known issue: http://jira.codehaus.org/browse/MRELEASE-318
Yes, every time you perform a release you can override the default version number. If you release using the command line, you will be prompted to select the next version number (with a sensible default displayed).
Use Maven 3. Read this blog post for a list of reasons why it is superior.

How to configure maven or eclipse in order to use the RELEASE constant within versions?

All our projects are built using maven.
we have centralized some of our main configuration within a super pom.
In order to always have an update version of this super pom (without having to modify the version), we have used the following syntax :
<parent>
<groupId>my.organization</groupId>
<artifactId>superPom</artifactId>
<version>RELEASE</version>
</parent>
The problem is that Maven Eclipse plugin (m2e) doesn't understand this syntax (the RELEASE constant is not resolved).
So, our Eclipse users can't use built-in compilation.
What do you suggest to overcome this problem ?
By the way, we have tried several options from a maven point of view (especially those described here), but the version.RELEASE is the easiest for everybody (except those who are using Eclipse).
EDIT:
Our projects sources are split within multiple SVN repositories.
This super pom is an independent project. It is retrieved through our Nexus server.
You are trying to go into the wrong direction. A release in maven is a particular version like 1.0.0 and it indicates that you have a defined state of that artifact. In your case you super pom has a particular state. If you are trying to define the version to "RELEASE" you are saying my release is always the same but in reallity it's not true.
Usually such a super pom will change over the time lets say today you have defined some particular dependency versions in it (dependencyManagemet). And tomorrow you change those definition. Now the 1.000.000$ questions which state of the super pom is used in a build which has been done today? Ok in that simple scenario you can answer the question but if you have changed the super pom sometime yesterday you can't answer the question accurately.
Furthermore if you try to recreate an artifact of let's say last week you can't say which exact state of super pom has been used at that particular time cause you have no indicator which gives you the chance to see it.
And that's the reason why you need real versions like 1.0.0 or 1.1.0 etc.
I can strongly recomment to use real versions like 1.0.0 etc. but NOT things like "RELEASE" that will creep in the Maven system with its corrdinate group, artifact and version.
Version ranges and expansion indeed do not work for parent artifacts.
Someone advised to invoke the version plugin instead :
mvn versions:update-parent
which does not cover exactly your need, but I am afraid there is no better workaround. Other ideas : using a SNAPSHOT parent pom (not very satisfactory I admit). See also Maven2 cannot find parent from relative path.

maven: automatic upgrade version in all dependent projects

Before the question I'd like to describe the methodology I use.
I have a lot of projects under version control folder, some of them multi maven projects, some of them standalone bundles, some of them maven plugins or archetypes. All jars are snapshot (currently we can not use release artifacts). So for example application A1 depended on bundle B, which depended on utility C, another application A2 directly depended on utility C. When I change code in C I need to update it's version and then update B and A2, then A1. It is really annoying to update all those poms once at week. So I'm looking for some automatic solution that can handle it for me (like if C has new version all depended modules have to be updated).
Does any body have idea?
Thanks in advance
P.S. I thought to make a MOJO which can handle this, but I faced with some difficulty since not all projects has common parent project ...
Sounds like something the versions plugin can handle... http://mojohaus.org/versions-maven-plugin/
This is considered a bad practice, but if you deploy your projects using -DupdateReleaseInfo=true (or with the release plugin), then you can set the dependency version to RELEASE
<dependency>
<groupId>some.groupid</groupId>
<artifactId>some.artifactid</artifactId>
<version>RELEASE</version>
</dependency>
and you will always get the latest release version
If you're using SNAPSHOT-s only, you could always go for parent based projects. Define the versions in a parent and make the children extend it. You can also choose to use versions such as RELEASE or LATEST.
Consider using a Continuous integration engine to watch all projects and build them when changed.
If you use Jenkins you can set it up to provide the built maven artifacts as a Maven repository, which you can then use in your own Maven configuration.
This should be enough - the snapshot mechanism handles the rest.

Can someone explain Maven 2 (or 3) dependency resolution to me?

I used Maven before in quite a few projects but they we already configured by someone else. I quickly understood the main concepts (groups, artifacts and versions for jars, local and remote repositories) so I assumed that if I'm asked I'll be able to set up a new project from scratch easily.
That turned out not to be the case when I deal with multiple modules which depend on one another. I poked around maven docs but they are either way too concise or way too technical (sometimes I get a feeling that Maven people wrote it for themselves and not for Maven users). So I'm asking for community help.
Here's a typical scenario:
Three repositories:
My local repo at <User>/.m2
My company repo at http://maven.mycompany.com/repository
Maven Central at http://repo1.maven.org/maven2/
I'm building a normal Java EE project consisting of one EJB jar with a corresponding client jar, a web module and a JPA module (I like keeping my entities and all db-connectivity separated from business logic). That gives me 5 project in my IDE:
myapp-ear
myapp-ejb
myapp-ejb-client
myapp-jpa
myapp-web
myapp-jpa is used by myapp-ejb and myapp-web. myapp-ejb-client contains business interfaces for my EJBs.
How should I set all that up? I suspect that I need some kind of parent project but I'm not sure how should I organize inter-project dependency resolution. For example: currently when I specify
<dependency>
<groupId>com.mycompany.myapp</groupId>
<artifactId>myapp-jpa</artifactId>
<version>0.1-SNAPSHOT</version>
</dependency>
in myapp-ejb's pom.xml maven tries to search for it in my repository and tells me that the .jar can't be found. Even if I build and install all app components into my local repository one by one Maven always fails to build the ear file itself :(
I'd love to get it working under Eclipse or NetBeans.
I think you're on the right path. You need a parent project to pull it all together. In the parent project (and here is the important part) you need your dependencies laid out NOT in the dependency section of your parent pom, rather, you need them in your dependencyManagement section. Then, in your child pom's you can declare dependencies that are in the parent.
The only other gotcha (that I can think of) is that when you check the project using a version control system in Eclipse you need to remember to check the parent project out "as Maven project" otherwise the m2Eclipse plugin tends to freak out and not resolve things properly.
I hope this helps.