I am trying to use eclipse (Kepler) for building, running MapReduce (ver 2), using maven plugin.
I am able to successfully build the project, and i see that maven created jar file as final output, suppose that jar is -> mapreducedemo.jar.
Now, my question is how to run this jar using eclipse?
I tried it on command prompt and it works fine, like this:
--> $ hadoop jar mapreducedemo.jar MainDriver input output.
The thing is hadoop is shell script and it sets all the env variables internally, and required jars.
How can we run this mapreducedemo.jar using eclipse?
Any answers would be great help.
Thanks,
Vipin
You should be able to just run MainDriver as a java application from within Eclipse. Maven will make sure you have all your dependencies and MainDriver, once it configures the job will submit it for execution, just as it does when you run the hadoop jar command.
Related
I'm working on an Eclipse microprofile application, and I would like to create a run configuration that executes these 3 commands in order:
mvn clean package
mvn package
java -jar target/myApp.jar
How can I do this? I have this:
Where and how can I set what I want?
I'm not sure that from the m2 plugin, you be able to accomplish the task of running multiple maven goals within a single run configuration.
However, as a workaround, you can add a batch file within your maven project directory with the following entries
call cls
%cd%
call mvn clean install
call java -jar target/myApp.jar
Running the above script can help you with running multiple maven commands in a synchronized manner.
First, you have to create a Maven Build launch configuration for each of the first two commands. The third command (java -jar target/myApp.jar) can be done with an External Tools Configuration of the type Program.
Then, create a Launch Group with all these three launch configurations.
I want to run a simple example Beam Program using Apache Spark runner.
1) I was to able to compile the program in my local successfully.
2) I want to push the JAR file to QA box where Maven is not installed.
3) I see the example with Maven command which compiles and executes the examples program.
4) Could you please tell me the steps to run the code without installing Maven.
5) spark-submit command runs fine.
6) Do you want me to put all the dependent JAR files one by one in
/opt/mapr/spark/spark-2.1.0/jars directory to execute the program
Thanks.
You can do this by following the instructions in Beam Spark Runner documentation.
These instructions demonstrate how to Submit your application by doing the following:
Packaging your application, with dependencies, as an uber jar.
Submit your packaged application using spark-submit.
Is it possible to start a Map Reduce job on a remote cluster with the Eclipse Run Dialog (F11)?
Currently I have to run it with the External Tool Chain Dialog and Maven.
Note: To execute it on a local cluster is no big deal with the Run Dialog. But for a remote connection it's mandatory to have a compiled JAR. Otherwise you get a ClassNotFoundException (also if Jar-By-Class is set)
Our current Setup is:
Spring-Data-Hadoop 1.0.0
STS - Springsource Toolsuite
Maven
CDH4
This we set on our applicationContext.xml (this is what you specify in the *-site.xml on a vanilla hadoop)
<hdp:configuration id="hadoopConfiguration">
fs.defaultFS=hdfs://carolin.ixcloud.net:8020
mapred.job.tracker=michaela.ixcloud.net:8021
</hdp:configuration>
Is there a way to tell Eclipse it should build a JAR when the Run Dialog is executed.
I do not know if it builds a new jar (may be you must extract a jar to a folder), adding "Run Configurations->Classpath" your jar clears the problem "ClassNotFoundException".
I'm looking for a solution that will force Eclipse to automatically clean a project before I run it (I'm talking about running a project using just Eclipse- no Maven, no Ant). For building I already have a Maven configuration, but sometimes I run the build directly from Eclipse as well and this is when I need that cleaning.
Shouldn't it be possible to have Maven and Eclipse use different class folders, e.g. /target for the Maven build and /bin for the Eclipse internal Java compiler? If so, you should be able to have 2 different launch configurations running the code from 2 different locations.
Second alternative: You can create a small Ant script to clear the target directory. That Ant script can be run from inside Eclipse, so a workaround is running the Ant launch configuration first and your Java launch configuration afterwards. To make this a one-step process, please install the launch groups feature from Eclipse CDT (you only need that small feature, not the whole CDT!), then you can create a "batch" like launch configuration from the other two launch configurations. Now everything is inside Eclipse with a single launch configuration!
After doing some work on setting the eclipse build path, I would like to use what I did (which is in .classpath) to launch a version of my project, where I have only the jar and which has no dependencies compiled in, with the same classpath. Is this possible using the normal java command? Is it possible on the command line, at all? Do I need eclipse to use .classpath?
If you want to use the build path settings you made in eclipse, you should run your program from eclipse using the "Run As" command. However, this will not run the program from the jar, but from your compiled classes.
If you plan on running the jar file from a command line, you will need to set the classpath using the "-cp" option. I am not aware of any way to pass the eclipse .classpath file to the jvm.