Running Storm Starter project in Maven / Eclipse - eclipse

I am new to eclipse and maven. I have been trying for two days to run the storm starter project and am hitting a number of problems.
I have a couple of questions of which I will explain my attempts below.
How can I import this project in to eclipse so that it runs and retrieves all required dependencies.
How can I start a storm project from scratch in eclipse. When I start a new maven project and add storm dependency, it only adds storm, and not all of its dependencies.
What I have tried:
I have the latest m2eclipse plugin installed. When I choose import existing maven project (I change the m2-pom.xml to pom.xml I dont know why it is named m2-pom) everything looks ok, but when I do maven install, in maven build, i get the error
clojure-maven-plugin:1.3.8:compile(1 errors)
No marketplace entries found to handle clojure-maven-plugin:1.3.8:compile in Eclipse. Please see Help for more information.
I have tried maven 2 and 3, I have tried creating a new java project then adding a maven configuration and running that.
I have got to the point where it looks like all dependencies have been downloaded, but in my project, all backtype types are still underlined in red and it when I hover over them, it does say import..
I also had another attempt when it said "import backtype.storm..." but when I cliked it the dependency still didnt resolve.
The code I am trying to run is:
https://github.com/nathanmarz/storm-starter
Please help

#girlcoder: I had a similar issue while playing with the "storm-starter" project (i.e. backtype types not recognized despite importing them). I used IntelliJ IDEA at that time, but the fix I found may apply to your case as well:
In the pom.xml file (I also renamed m2-pom.xml as pom.xml), go to the section that defines the dependency on Storm and change its scope to "compile":
<dependency>
<groupId>storm</groupId>
<artifactId>storm</artifactId>
<version>0.9.0.1</version>
<!-- keep storm out of the jar-with-dependencies -->
<!--<scope>provided</scope>-->
<scope>compile</scope>
</dependency>
I used the "provided" scope when deploying the topologies to an actual Storm cluster, but I had to change the scope to "compile" to submit a topology to a LocalCluster. I hope this will help you solve the problem.
UPDATE: I am now trying to package storm-starter in Eclipse 4.3. It turned out that the pom.xml file is not compatible with the m2e eclipse plugin. Besides the modification I suggested above, I was able to compile the project only after changing the pom.xml as described here: https://github.com/nathanmarz/storm-starter/issues/23.

Related

Eclipse: The project was not built since its build path is incomplete. Cannot find guava Predicate

I am using new Eclipse Oxygen. My project setup includes Maven. One of the Maven dependencies is Guava:
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>20.0</version>
</dependency>
There are 2 problems I am facing that do not allow me to build the project:
1.
The project was not built since its build path is incomplete. Cannot find the class file for com.google.common.base.Predicate. Fix the build path then try building this project
2.
The type com.google.common.base.Predicate cannot be resolved. It is indirectly referenced from required .class files [...] Java Problem
Here is what I tried to solve it:
I have made clean & build of my project; I have closed and then reopened it; I have removed it from Eclipse and then re-imported and made clean & build.
Nothing seems to help.
Anyone has any ideas? This is super annoying.
The problem was not related to Eclipse, but it was a maven issue.
In order to fix it I had to clear ~/.m2/repository folder and run Maven->Update Project in Eclipse.
Eclipse by default uses embedded Maven. It is possible, that the guava dependency got corrupted in the embedded Maven's .m2 folder. Try to delete it from there.
Also try to change the Maven in Eclipse to use the same Maven as you use in the command line in:
Preferences->Maven->Installations

NocloassDefFound Error in Eclipse

I get a java.lang.NoClassDefFoundError: org/apache/http/impl/conn/PoolingHttpClientConnectionManager when trying to build a maven project in eclipse. Where as it builds successfully on command line
I get the above mentioned class through the following dependency defined in the pom
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.4</version>
</dependency>
--I tried, deleting the local repo and running the build with -U option but doesnt seem to fix the problem.
--When I look at the java build path in eclipse, it shows certain libraries missing. I found newer versions of these jars in the local repo and added them in the eclipse java build path. But the build still fails for the same error
Note: I have one multimodule project in my eclipse workspace.
--I tried updating the project and verified if eclipse was using teh same settings.xml and local repo that the command line is.
--I am running a clean install
--Cleaning project also doesnt fix it
--I looked at the referrenced libraries and there is a class org/apache/http/impl/conn/PoolingHttpClientConnectionManager present. Dont know why it doesnt find it
If project builds on command line it should build in Eclipse as well. Reasons why it may not be the case include:
settings.xml used by Eclipse is different, so Maven does not have access to same repos. This can be changed in preferences.
Workspace resolution is picking code in dependent projects that have local changes and thus build fails
Eclipse maven configuration is not up-to-date that can be fixed by using Maven->Update project .. option
Try that if it fails please clarify if you have one project in Eclipse or multiple and what errors exactly do you see.
Also note that sometimes maven build from command line will delete target folder and will cause Eclipse to run build in parallel re-using same target folder and resulting in inconsistent state.
Try Adding dependency for HttpCore library, the class exists there too
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.4.3</version>
So the problem was that I had two versions of http-client library. I excluded one of them from the pom

Maven integration with IDEs

I thought I understood Maven as I've worked on a few projects using it, but I seem to be having trouble creating my own project with it, so can someone correct me wherever I may be wrong?
Assuming I create a new Maven project (we'll say I did it through Eclipse or IntelliJ):
If I add a dependency to my POM file, assuming I add a public repository where that dependency can be found, shouldn't the IDE download the specified JAR and make it so that it is recognized when I use it in my code? Currently, my projects do not recognize any classes that should be found in JARs via my POM dependencies.
Assuming #1 does work, how can I determine via maven which transient dependencies I have? For example, I'm using classes from Pentaho Data Integration, and some of the plugins for it reference things like org.mozilla.javascript.*. Could maven automatically tell me that without me having to run the code, see it fail, and manually fix the dependency? There will be hundreds of those instances here, so manual fixing really isn't viable.
Here are my IntelliJ two cents:
1 - Adding a dependency in pom.xml of your project
Should indeed download the depended jar(s). You may need to approve "Import Changes" dialog if it pops in, or enable auto import.
2 - Seeing transitive dependencies
It can be achieved via Maven Dependencies Diagram - unfortunately only in IntelliJ Ultimate edition. You can use Maven Dependencies Plugin to see the dependencies tree in your favorite CLI.
Question 1: Adding a dependency
In Eclipse, depending on how you created the project, you should be able to add dependencies that are automatically recognized using the maven context menu.
Note that you should have created the project using the eclipse maven plugin so that it has maven nature.
To add dependencies/plugins from a remote repository, you can search in the resulting UI for a dependency if you know the artifactId or groupId. The plugin will pull up the deps whether the repo url is specified in the pom.xml or not.
After adding a dependency to the POM, the IDE will start downloading it and all transient dependencies as soon as you save the file.
If something goes wrong, you can try to "Update Project" from the context menu.
Question 2: Determining transitive dependencies
Transient dependencies are visible in the "Dependency Hierarchy" tab of the POM editor.
I usually default to the command line because it allows much more flexibility and functionality when tracking the dependency graph.
I am sorry but I have not worked with IntelliJ

Errors when importing Mahout's maven resource to eclipse

I'm trying to import Mahout's maven resource to eclipse, but i meet these errors:
I have tried google but it doesn't help.
Eclipse : Helios Service Release 1
Maven : External Maven 3.
Edit: If I try to play with latest stable release ( Mahout 0.7 ) and not the latest snapshot that the official site recommends for downloading, I experience the same error.
Follow step-by-step this link: https://cwiki.apache.org/MAHOUT/buildingmahout.html
After doing this and re-importing mahout, these errors you noticed go away.
Notice: Before compiling the code in the right way (as stated at the link above), I have followed the instructions from here:
http://domengrabec.wordpress.com/2010/12/14/installing-maven-on-eclipse/
Q: If anyone could help me with this related issue:
Include classes of Mahout math libraries of both core & math folders - Eclipse - Hadoop , I would be grateful.
I think what you need is to build mahout in a different way,
first install m2e plugin for eclipse, then you can create maven project. You can make the run configuration (package) after that to make sure everything is fine.
The important part is that you have the pom.xml in the project so you have to add the dependencies of mahout that you want.
You can get the dependencies from searching on google " name of the dependency in mahout api".
i solved it by manually applying the patch in https://issues.apache.org/jira/browse/MAHOUT-1043 to 0.7 sources (changes pom.xml files and src/conf/driver.classes.conf )
just one note, do not remove the groupid in distribution/pom.xml
launch an external mvn eclipse:clean and update the projects. All errors are gone when workspace is rebuilt

In PlayN, how can I upgrade to PlayN version 1.3.1 without getting "no lwjgl in java.library.path" error?

Update
The reason I couldn't update properly was I didn't have game's jar file properly installed. That was fixed by first running this on the command line:
mvn install:install-file -DgroupId=com.mygame -DartifactId=mygame-core -Dversion=0.9-beta -Dpackaging=jar -Dfile=/home/klenwell/projects/mygame/playn/mygame/java/target/mygame-java-0.9-beta.jar
Then, in Eclipse, I right-clicked java folder > Maven > Test. This updated all my dependencies including lwjgl.
Otherwise, see thomasbroad's answer.
Original Question
I've consulted the following authorities on this topic and applied their various recommendations:
http://lwjgl.org/installation.php
http://www.lwjgl.org/wiki/index.php?title=Setting_Up_LWJGL_with_Maven
http://www.lwjgl.org/wiki/index.php?title=Setting_Up_LWJGL_with_Eclipse
https://stackoverflow.com/a/11065814/1093087
https://groups.google.com/forum/?fromgroups#!topic/playn/E2t7gNh4ab0
https://groups.google.com/d/msg/playn/3D4JZ1Dbm-E/HmPHofS2fqgJ
Part of the problem is I'm not sure whether I should be following the Maven instructions or the Eclipse instructions. I've tried both and everything in between.
I'm working in Ubuntu 12.04 64-bit on Eclipse using Maven. Everything was working fine until I changed the PlayN version in my POM file from 1.2 to 1.3.1 and introduced the lwjgl dependency.
What I would have expected to have worked:
1) Download lwjgl (latest version 2.8.4) and test per lwjgl.org instructions (this was successful)
2) Update the appropriate POM files
2A) Add lwjgl.version as a property to mygame/pom.xml
<properties>
<playn.version>1.3.1</playn.version>
<lwjgl.version>2.8.4</lwjgl.version>
</properties>
2B) Add lwjgl as a dependency to mygame/mygame-java/pom.xml
<dependency>
<groupId>org.lwjgl.lwjgl</groupId>
<artifactId>lwjgl</artifactId>
<version>${lwjgl.version}</version>
</dependency>
3) Right-click mygame-java project in Eclipse > Run As > Run Configurations > Arguments and add either -Djava.library.path=target/natives or -Djava.library.path=/absolute/path/to/my/lwjgl/lib to VM arguments. (I've tried both variations.)
I can see the dependencies listed under my java project folder in Eclipse. And when I follow the lwjgl.org guide for Eclipse, I can see the jar listed under the java project folder. But I still get the error.
After two hours of hair-wrenching frustration, I hope somebody can either point out my fatal oversight or provide explicit instructions that will help me get back to work. Thanks.
I upgraded my project from 1.2 to 1.3 a while ago. I only updated my pom.xml files to refer to the new version of PlayN . I did not manually install lwjgl and neither did I add the lwjgl dependency as that one is implicitly pulled in by PlayN
Manually apply the diffs found here to your project, to your poms.
The key part being the piece in the java pom that tells Maven to unpack native dependencies. This part was actually added in the previous diff, so you may want to look there as well.
Now your IDE will most likely tell you that your project is not up-to-date with your poms, update your project, this should pull in the lwjgl dependency and set it up correctly.
I believe you should add the -Djava.library.path arguments to the VM Arguments, not to the Program Arguments in your runtime configuration.
The other -using maven- option is described on the Getting Started page, it worth noting, that the default embedded Maven is buggy for development, you should download and use at least Maven 3.0.3 as it is mentioned on the page (but I guess you already do this).
I just started a new project fresh with PlayN 1.4 and encountered this issue again in Eclipse. Resolved it as follows:
Right click mygame-java > Run As > Run Configurations...
Then added the following to VM Arguments under Arguments tab:
-Djava.library.path=target/natives
Then successfully ran as application.
At one point, I also ran the mvn install:install-file... command mentioned in question but I'm not sure whether that was necessary or not.