how to run a class from java jar file on Hadoop? - eclipse

I have a jar file exported from Eclipse (Runnable JAR>>Copy required libraries into a sub folder).
In java if you set the main class in the manifest.xml you can run:
java -jar MyTest.jar arguments
if you want to run another main class in the jar file or if you didn't set a main class in the manifest.xml you can run:
java -cp MyTest.jar MyOtherMainClass arguments
In Hadoop if main class is set in manifest.xml you can run:
hadoop jar MyTest.jar arguments
If you type:
hadoop jar MyTest.jar MyOtherMainClass arguments
It will consider MyOtherMainClass as argument (not as a class to run) in the "args" array of the original main class of jar.
Now if you want to run another main class in the jar file what will you type?
I expect something similar to:
hadoop java -cp MyTest.jar MyOtherMainClass arguments
but that gives:
Error: Could not find or load main class java
Notice: if I remove "hadoop" from "hadoop java -cp MyTest.jar MyOtherMainClass arguments" it will launch normally

The problem comes from Eclipse forcing you to set the main class in the jar file and hence preventing you to run the class that you want. All you have to do is remove the main class from the manifest.xml file of the jar file and run:
hadoop jar MyTest.jar MyOtherMainClass arguments
Take a look here:
http://www.roman10.net/2012/07/26/specify-the-main-class-in-a-jar-file-for-hadoop/
I typed the same text in case the url deleted:
Hadoop support execution of jar file. For an executable jar file in normal java execution, one can specify the main class in the command line, as covered in my previous post: switch between main classes in a jar file.
However, the rules are a bit different for executable jar file running with hadoop. Basically the following rules hold (I tested on Hadoop 1.0.3),
If a jar file contains a main class specified in its manifest file, hadoop will take the main class even if the command specify another main class. This is different from normal java execution where we can specify a main class to overwrite the one in the manifest file.
If a jar file does not contain a main class in manifest file, hadoop allows us to specify the main class.
At eclipse, when one export a project as runnable jar file, it always ask for a main class at Launch configuration.
The main class selected will be put in the manifest file. Below is the content of the META-INF/MANIFEST.MF file in my helloworld project where the main class is set to HelloWorld.
Manifest-Version: 1.0
Class-Path: .
Main-Class: hello.HelloWorld
One can browse the jar file using a file extractor, open the manifest file using a file editor, and simply delete the last line to remove the main class configuration, and save the changes to the jar file when prompted. This will create a runnable jar file without main class.
The modified jar file can then be used in Hadoop with user supplied main class configuration, as shown in the sample command below,
$ hadoop jar hello.jar hello.HelloWorld

Related

How to specify a different main class when running a scala jar on the command line?

I have a scala project with 2 objects that extend App. I have specified one as the main class in build.sbt. I'm using assembly to build a fat jar.
How can I execute the non-default main class when running the jar on the command line? I.e. if com.example.app1 is specified as the main class in build.sbt, how could I run com.example.app2 from the command line using the jar (assuming that also extends App)?
This will be in a production environment where I won't have sbt.
You run the default main class (from the jar's manifest) like this:
java -jar assembly.jar
And you would run a different main class like this:
java -cp assembly.jar com.example.app2

Intellij class path setting when build a jar for maven project

when running the jar in command line, throw this exception:
java.lang.ClassNotFoundException
When building jar in intellij, we need set the manifest properties, like main class, class path, where could I find the right setting for this? it is not in the manifest file

How to create a groovy jar with eclipse/ggts?

So I create a groovy jar but I'm not able to run it.
I'm using HelloWorld.groovy as an example
class HelloWorld {
static main(args) {
println("Hello World");
}
}
I save it as a jar like I would with any java file via export in eclipse/ggts:
right clicking groovy file in project explorer
click export
select jar File
specify jar file name
specify HelloWorld as the main class in the Manisfest file
keep all the defaults selected
click finish and create the jar
Now I try to run HelloWorld.groovy via java -jar HelloWorld.jar on my command line.
I get:
Exception in thread "main" java.lang.NoClassDefFoundError: groovy/lang/GroovyObject
When I export it also allows me to export .classpath into my jar.
Shouldn't that include my groovy-all.jar that my GGTS is using?
Am I missing something?
I've been successful with using GroovyWrapper to create groovy jars. I want to see if it's possible with just my IDE.
I think that in eclipse you export an executable jar and that gives the option to package the dependencies?

Trouble running jar file created by Eclipse

I exported my Eclipse project as a runnable Jar file, added a manifest, as well as the appropriate class files with the command:
jar cfm JarFile.jar manifest.txt *.class
However, when I try to run the jar file with
java -jar JarFile.jar
I get the error that it "Could not find or load main class" etc. etc.
The structure of my manifest.txt file looks like this:
Main-Class: EclipseProjectName.src.packagename.mainclassname
(With a carriage return at the end)
Is something wrong with my manifest file? If not, what may be the reason that the main class cannot be found?
Thank you!
The manifest file extension should be .mf i.e manifest.mf. See here for jar file specification
Suggest to use fat jar eclipse plugin for exporting java projects as runnable jars.
Edit
Correct the content of manifest.txt as shown below
Main-Class: EclipseProjectName.src.packagename.mainclassname
project name and src are not required. Refer this
The entry in the META-INF/MANIFEST.MF entry in the finished jar file should be
Main-Class: packagename.mainclassname
and should correspond exactly to
/packagename/mainclassname.class
in your jar file (or jar file_s_ if you use Class-Path too).

How can scala see classes from files in command line mode?

Say, I have an eclipse project located in D:\DanHenderson\scala\MyProject. MyProject has 2 directories: src and bin. In src I have scala code itself, it might be some classes or an application. So how can I use those classes or run whole application from command line? I.e., there is a file with class A, and I want to have a possibility to type val a = new A in command line. How can I do this?
By command line I assume you mean the Scala REPL. Add the directory of your class files to the classpath when starting the Scala REPL:
scala -classpath "path to some dir where your .class files reside"
Note that if your project relies on some libraries then those need to be on the classpath as well.