What version of H2O gen-model maven library supports XGBoost MOJO's? - pojo

I am trying to use XGBoost MOJO created and downloaded from H2O in runtime using java, but I am getting errors while compiling. I have tries multiple different versions of dependencies but couldn't get through it.
Dependencies :
<dependency>
<groupId>ai.h2o</groupId>
<artifactId>h2o-genmodel</artifactId>
<version>3.22.1.6</version>
</dependency>
<!-- https://mvnrepository.com/artifact/ai.h2o/h2o-genmodel-ext-xgboost -->
<dependency>
<groupId>ai.h2o</groupId>
<artifactId>h2o-genmodel-ext-xgboost</artifactId>
<version>3.24.0.1</version>
<type>pom</type>
</dependency>
Main.java
import hex.genmodel.easy.EasyPredictModelWrapper;
import hex.genmodel.easy.RowData;
import hex.genmodel.easy.prediction.BinomialModelPrediction;
public class Main {
public static void main(String[] args) throws Exception {
EasyPredictModelWrapper model = new EasyPredictModelWrapper(
MojoModel.load("/Users/p0g0085/Downloads/grid_5dd10c7e_297d_42eb_b422_56c687ba85df_model_18.zip"));
RowData row = new RowData();
row.put("brand", "Great Value");
row.put("product_type", "Cheeses");
row.put("duration", "21.0");
row.put("quantity", "1.0");
row.put("ghs_frequency", "11.3714");
BinomialModelPrediction p = model.predictBinomial(row);
System.out.println("Has penetrated the prostatic capsule (1=yes; 0=no): " + p.label);
System.out.print("Class probabilities: ");
for (int i = 0; i < p.classProbabilities.length; i++) {
if (i > 0) {
System.out.print(",");
}
System.out.print(p.classProbabilities[i]);
}
System.out.println("");
}
}
Error :
Exception in thread "main" java.lang.IllegalStateException: Algorithm `XGBoost` is not supported by this version of h2o-genmodel. If you are using an algorithm implemented in an extension, be sure to include a jar dependency of the extension (eg.: ai.h2o:h2o-genmodel-ext-xgboost)
at hex.genmodel.ModelMojoFactory.getMojoReader(ModelMojoFactory.java:102)
at hex.genmodel.ModelMojoReader.readFrom(ModelMojoReader.java:31)
at hex.genmodel.MojoModel.load(MojoModel.java:37)
at Main.main(Main.java:9)

It took me a while to figure out. Below dependency worked for me.
<dependency>
<groupId>ai.h2o</groupId>
<artifactId>h2o-genmodel-ext-xgboost</artifactId>
<version>3.22.0.3</version>
</dependency>

Related

java.util.NoSuchElementException while executing junit Testrunner

Error I am getting is:
java.util.NoSuchElementException while running junit test. Here is full error from console in eclipse:
java.util.NoSuchElementException
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:970)
at java.base/java.util.Collections.max(Collections.java:713)
at io.cucumber.core.feature.FeatureParser.parseResource(FeatureParser.java:45)
at java.base/java.util.function.BiFunction.lambda$andThen$0(BiFunction.java:70)
at io.cucumber.core.resource.ResourceScanner.lambda$processResource$1(ResourceScanner.java:79)
at io.cucumber.core.resource.PathScanner$ResourceFileVisitor.visitFile(PathScanner.java:75)
at io.cucumber.core.resource.PathScanner$ResourceFileVisitor.visitFile(PathScanner.java:60)
at java.base/java.nio.file.Files.walkFileTree(Files.java:2811)
at io.cucumber.core.resource.PathScanner.findResourcesForPath(PathScanner.java:53)
at io.cucumber.core.resource.PathScanner.findResourcesForUri(PathScanner.java:31)
at io.cucumber.core.resource.ResourceScanner.findResourcesForUri(ResourceScanner.java:61)
at io.cucumber.core.resource.ResourceScanner.scanForResourcesUri(ResourceScanner.java:134)
at io.cucumber.core.runtime.FeaturePathFeatureSupplier.loadFeatures(FeaturePathFeatureSupplier.java:62)
at io.cucumber.core.runtime.FeaturePathFeatureSupplier.get(FeaturePathFeatureSupplier.java:45)
at io.cucumber.junit.Cucumber.<init>(Cucumber.java:156)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:37)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at org.junit.internal.requests.ClassRequest.createRunner(ClassRequest.java:28)
at org.junit.internal.requests.MemoizingRequest.getRunner(MemoizingRequest.java:19)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createUnfilteredTest(JUnit4TestLoader.java:90)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createTest(JUnit4TestLoader.java:76)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.loadTests(JUnit4TestLoader.java:49)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:513)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Library I am using:
My Testrunner script is:
package TestRunner;
import org.junit.runner.RunWith;
import org.openqa.selenium.NoSuchElementException;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
#RunWith(Cucumber.class)
#CucumberOptions(features="C:\\Users\\Ashok Kumar\\eclipse-workspace\\CucumberWithSelenium\\Features",glue={"StepDefinition"})
public class Runner {
}
My steps script:
package StepDefinition;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.NoSuchElementException;
//import io.cucumber.java.*;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
//import cucumber.api.junit.Cucumber;
public class Steps {
WebDriver driver = null;
#Given("^I am on Facebook login page$")
/*
public void goToFacebook() {
WebDriver driver = new ChromeDriver();
driver.get("https://www.facebook.com");
driver.manage().timeouts().pageLoadTimeout(30, TimeUnit.SECONDS);
} */
#Given("^Open the Chrome and launch the application$")
public void open_the_Chrome_and_launch_the_application() throws Throwable{
System.out.println("This Step open the Firefox and launch the application.");
}
#When("^Enter the Username and Password$")
public void enter_the_Username_and_Password() throws Throwable
{
System.out.println("This step enter the Username and Password on the login page.");
}
#Then("^Reset the credential$")
public void Reset_the_credential() throws Throwable
{
System.out.println("This step click on the Reset button.");
}
}
I have searched all previous posts in google on java.util.NoSuchElementException. But still I didn't come to webdriver as you have seen above. I did comment it as I have come across this error. I have tried even with WebDriver code above, still I am getting the error. Please help me. I am unable to edit java.base/java.util.ArrayList$Itr.next using next I have seen in google.
Thanks God. I found a solution or work around. Here is what I had done.
Had converted the project to Maven project.
I have mentioned latest dependencies
Here is my POM.
4.0.0
CucumberWithSelenium
CucumberWithSelenium
0.0.1-SNAPSHOT
src
maven-compiler-plugin
3.8.1
17
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>7.2.3</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>7.2.3</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-jvm-deps</artifactId>
<version>1.0.6</version>
</dependency>
<dependency>
<groupId>net.masterthought</groupId>
<artifactId>cucumber-reporting</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>gherkin</artifactId>
<version>22.0.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.1.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/info.cukes/cucumber-picocontainer -->
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-picocontainer</artifactId>
<version>7.2.3</version>
</dependency>
</dependencies>

How to solve java.lang.ClassNotFoundException in appium java?

I have to perform login activity in appium for android application. But I am getting class not found error in my script. My application is opened in but after opening application login not performed.
public class AppiumTest2 {
AppiumDriver driver;
public void setup() throws Exception {
DesiredCapabilities Capabilities = new DesiredCapabilities();
Capabilities.setCapability("deviceName", "codeblaze");
Capabilities.setCapability("platforVersion", "7.0");
Capabilities.setCapability("platformName", "Adnroid");
Capabilities.setCapability("appPackage", "package name");
Capabilities.setCapability("appActivity", "activity name");
driver = new AndroidDriver(new URL("http://0.0.0.0:4723/wd/hub"), Capabilities);
}
public void tearDown() throws Exception {
driver.quit();
}
public void LogInWithInvalidEmail() {
//WebElement emailLoginButton = driver.findElement(By.id("lllogin"));
//emailLoginButton.click();
WebElement emailTextField = (new WebDriverWait(driver,60)).until(ExpectedConditions.presenceOfElementLocated(By.id("etusername")));
emailTextField.sendKeys("Invalid Email");
WebElement passwordTextField = driver.findElement(By.id("etpassword"));
passwordTextField.sendKeys("Random Password");
WebElement loginButton = driver.findElement(By.id("lllogin"));
loginButton.click();
}
}
Actual Result: I am getting below error:
java.lang.ClassNotFoundException: org.apache.commons.lang3.StringUtils
Expected Result: above mention error should not come and login activity should perform in application.
I was facing the same issue, the RCA was I was using appium-java Client from Reference libraries and Selenium-server jar from Maven Dependency section of my dev tool(Eclipse). That is causing sync in issue when driver is calling Java client to interact with Appium jar.
When I imported java client in my POM this issue resolved for me.
<dependencies>
<!-- http://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-server -->
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-server</artifactId>
<version>3.141.59</version>
</dependency>
<dependency>
<groupId>io.appium</groupId>
<artifactId>java-client</artifactId>
<version>7.4.1</version>
</dependency>
</dependencies>

Cucumber - Content Assistance in Feature file not working

So, I've been having a tough time getting the dependencies down with Cucumber/Scala integration. I finally have a simple step definition running, but when I press control + space bar, the list of step definitions do not show up in my feature file. However, when I run the feature file, it runs successfully.
Test Runner
package CucumberTest
import cucumber.api.CucumberOptions
import cucumber.api.junit.Cucumber
import org.junit.runner.RunWith
#RunWith(classOf[Cucumber])
#CucumberOptions(
features = Array("Feature")
,glue= Array("stepDefinition")
,plugin = Array ("pretty", "html:target/cucumber/html")
)
class TestRunner {
def main(args: Array[String]): Unit = {
println("Hi")
}
}
Step Definition file
package stepDefinition
import cucumber.api.scala.{ ScalaDsl, EN }
class Test_Steps extends ScalaDsl with EN{
Given("""^this pre condition$""") { () =>
println("YOOOOOOOOO!!!")
}
When("""^I do this$""") { () =>
//// Write code here that turns the phrase above into concrete actions
}
Then("""^I can verify that$""") { () =>
//// Write code here that turns the phrase above into concrete actions
}
Then("""^I can also verify that$""") { () =>
//// Write code here that turns the phrase above into concrete actions
}
This is what my feature looks like. "this pre condition" is highlighted in yellow, indicating that the feature file is not finding the glue code.
When I hover my mouse over the Given statement, I get this message
Step 'this pre condition' does not have a matching glue code
But when I run it, I get this as the output.
Scala Console output
Since YOOOOOOOOO!!! printed in the console, it's seeing my glue code and running successfully, but I don't get a list of step definitions and the phrase "this pre condition" is highlighted yellow.
Does anyone know what the issue could be?
Here are some dependencies relating to cucumber/scala
<!-- https://mvnrepository.com/artifact/info.cukes/cucumber-java -->
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-java</artifactId>
<version>1.2.4</version>
</dependency>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-scala_2.11</artifactId>
<version>1.2.4</version>
</dependency>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-junit</artifactId>
<version>1.2.4</version>
</dependency>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>gherkin</artifactId>
<version>2.12.2</version>
</dependency>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-core</artifactId>
<version>1.2.4</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-nop</artifactId>
<version>1.7.12</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.11.8</version>
</dependency>
So I think the issue was a combination on having dependency mismatches as well as not adding Junit to my project path.
This is what my Test_Steps class looks like now. I imported libraries from Cucumber java api.
package stepDefinition
//import org.slf4j.LoggerFactory
import cucumber.api.java.en.Given;
import cucumber.api.scala._
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;
import cucumber.api.java8._
import cucumber.api.scala.{ ScalaDsl, EN }
import cucumber.runtime.java.StepDefAnnotation
#StepDefAnnotation
class Test_Steps extends ScalaDsl with EN {
//this works
#Given("""^this pre condition$""")
def this_pre_condition() = {
println("Hello")
}
#When("""^blah condition$""")
def when_condition() = {
println("In the when statement -- ")
}
}
My feature file's content assistance feature works now.
Junit output
Console output

No injection source found for a parameter of type public javax.ws.rs.core.Response - Jersey - MultiPartFeature

I'm creating a web service with Jersey and Jetty Embedded with no web.xml file. It is very simple, It receive a binary file by a POST from a HTML form. It seems I didn't register the MultiPart Feature properly because When I try to use it with HTML form I get this error :
*
WARNING: No injection source found for a parameter of type public
javax.ws.rs.core.Response
org.multipart.demo.ReceiveFile.postMsg(java.io.InputStream,org.glassfish.jersey.media.multipart.FormDataContentDisposition)
throws java.lang.Exception at index 0. 2016-02-09
21:49:59.916:WARN:/:qtp1364335809-16: unavailable
org.glassfish.jersey.server.model.ModelValidationException: Validation
of the application resource model has failed during application
initialization.|[[FATAL] No injection source found for a parameter of
type public javax.ws.rs.core.Response
org.multipart.demo.ReceiveFile.postMsg(java.io.InputStream,org.glassfish.jersey.media.multipart.FormDataContentDisposition)
throws java.lang.Exception at index 0.;
source='ResourceMethod{httpMethod=POST,
consumedTypes=[multipart/form-data], producedTypes=[text/plain],
suspended=false, suspendTimeout=0,
I was looking for the solution for weeks, I have read all question related to this error on StackOverflow, for instance:
MULTIPART_FORM_DATA: No injection source found for a parameter of type public javax.ws.rs.core.Response
Jersey 2 injection source for multipart formdata
They didn't help me because Im not using web.xml
I have 3 classes
- ReceiveFile.class (try to receive the POST)
- resourceConfig.class (try to register the MultiPart feature)
- JettyServer.class (create server instance)
ReceiveFile.class
package org.multipart.demo;
import java.io.InputStream;
import javax.ws.rs.ApplicationPath;
import javax.ws.rs.Consumes;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import org.glassfish.jersey.media.multipart.FormDataContentDisposition;
import org.glassfish.jersey.media.multipart.FormDataParam;
#Path("/resources")
public class ReceiveFile
{
#POST
#Path("/fileUpload")
#Produces("text/plain")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response postMsg (
#FormDataParam("file") InputStream stream,
#FormDataParam("file") FormDataContentDisposition fileDetail) throws Exception {
Response.Status respStatus = Response.Status.OK;
return Response.status(respStatus).build();
}
}
resourceConfig.class
package org.multipart.demo;
import java.util.HashSet;
import java.util.Set;
import javax.ws.rs.ApplicationPath;
import javax.ws.rs.core.Application;
import org.glassfish.jersey.media.multipart.MultiPart;
import org.glassfish.jersey.media.multipart.MultiPartFeature;
import org.glassfish.jersey.server.ResourceConfig;
/**
* Registers the components to be used by the JAX-RS application
*
*/
#ApplicationPath("/resources/fileUpload")
public class resourceConfig extends ResourceConfig {
/**
* Register JAX-RS application components.
*/
public resourceConfig(){
register(ReceiveFile.class);
register(JettyServer.class);
register(MultiPartFeature.class);
//packages("org.glassfish.jersey.media", "com.mypackage.providers");
}
}
JettyServer.class
package org.multipart.demo;
import org.eclipse.jetty.server.Handler;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.handler.DefaultHandler;
import org.eclipse.jetty.server.handler.HandlerList;
import org.eclipse.jetty.server.handler.ResourceHandler;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
import org.glassfish.jersey.media.multipart.MultiPartFeature;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.server.ServerProperties;
public class JettyServer
{
// private static final Logger LOGGER = Logger.getLogger(UploadFile.class.getName());
public static void main(String[] args) throws Exception
{
ResourceConfig config = new ResourceConfig();
config.packages("org.multipart.demo");
Server jettyServer = new Server(8080);
ResourceHandler resource_handler = new ResourceHandler();
// Configure the ResourceHandler. Setting the resource base indicates where the files should be served out of.
// In this example it is the current directory but it can be configured to anything that the jvm has access to.
resource_handler.setDirectoriesListed(true);
resource_handler.setWelcomeFiles(new String[]{ "./index.html" , "./html/FileUpload.html" });
resource_handler.setResourceBase(".");
//Jersey ServletContextHandler
final ResourceConfig resourceConfig = new ResourceConfig(ReceiveFile.class);
resourceConfig.register(MultiPartFeature.class);
ServletContextHandler servletContextHandler = new ServletContextHandler(ServletContextHandler.SESSIONS);
ServletHolder jerseyServlet = servletContextHandler.addServlet(org.glassfish.jersey.servlet.ServletContainer.class, "/*" );
jerseyServlet.setInitParameter(ServerProperties.PROVIDER_PACKAGES, "org.multipart.demo");
// Add the ResourceHandler to the server.
HandlerList handlers = new HandlerList();
handlers.setHandlers(new Handler[] { resource_handler, servletContextHandler, new DefaultHandler() });
jettyServer.setHandler(handlers);
try {
jettyServer.start();
jettyServer.join();
} finally {
jettyServer.destroy();
}
}
}
the pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org</groupId>
<artifactId>multipart.demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>multipart.demo</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
<version>9.2.3.v20140905</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-servlet</artifactId>
<version>9.2.3.v20140905</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-server</artifactId>
<version>2.7</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet-core</artifactId>
<version>2.7</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-jetty-http</artifactId>
<version>2.7</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-moxy</artifactId>
<version>2.7</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-multipart</artifactId>
<version>2.7</version>
</dependency>
<dependency>
<groupId>org.jvnet.mimepull</groupId>
<artifactId>mimepull</artifactId>
<version>1.9.6</version>
</dependency>
</dependencies>
</project>
Thanks in advance!
I See three different ResourceConfigs in your codebase, but none of them are actually used for the application. So the MultiPartFeature is never register, which is what is causing the error. You have a few options on how to use a ResourceConfig in your case.
You can instantiate the ServletContainer, passing in the ResourceConfig instance. Unfortunately, there is no ServletContextHolder#addServlet(Servlet) method, but there is a ServletContextHolder#addServlet(ServletHolder) method, so we need to wrap the ServletContainer in a ServletHolder
ServletHolder jerseyServlet = new ServletHolder(new ServletContainer(resourceConfig));
servletContextHolder.addServlet(jerseyServlet, "/*");
With the above option, you can use a local instance or a subclass, but if you only have a subclass, like your first bit of code, then you add a servlet init param, that specifies the ResourceConfig subclass.
ServletContextHandler servletContextHandler = new ServletContextHandler(ServletContextHandler.SESSIONS);
ServletHolder jerseyServlet = servletContextHandler.addServlet(org.glassfish.jersey.servlet.ServletContainer.class, "/*" );
jerseyServlet.setInitParameter(ServerProperties.PROVIDER_PACKAGES, "org.multipart.demo");
jerseyServlet.setInitParameter(ServletProperties.JAXRS_APPLICATION_CLASS, resourceConfig.class.getCanonicalName());
Notice the last call where I set the application class name.
Without using a ResourceConfig, you could just register the MulitPartFeature with an init param
jerseyServlet.setInitParameter(ServerProperties.PROVIDER_CLASSNAMES, MultiPartFeature.class.getCanonicalName());

hadoop 2.6 cluster cannot be initialized. Successfully run with local jars, but not maven dependency

I'm trying to debug wordcount sample using apache hadoop 2.6.0.I create the project in eclipse. My first try was configure the build path and include all the hadoop jar files (extracted from hadoop folder) in the buildpath. I can successfully run the word count and get the result. Then my second try is to make this project a 'maven' project and using pom.xml to specify needed hadoop jars (and remove local jars in buildpath). Here comes the problem. This time exception throws as follows:
Exception in thread "main" java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at WordCount.main(WordCount.java:59)
My wordcount code is pretty simple and classic wordcount.
import java.io.IOException;
import java.util.StringTokenizer;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
public class WordCount {
public static class TokenizerMapper
extends Mapper<Object, Text, Text, IntWritable>{
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
public void map(Object key, Text value, Context context
) throws IOException, InterruptedException {
StringTokenizer itr = new StringTokenizer(value.toString());
while (itr.hasMoreTokens()) {
word.set(itr.nextToken());
context.write(word, one);
}
}
}
public static class IntSumReducer
extends Reducer<Text,IntWritable,Text,IntWritable> {
private IntWritable result = new IntWritable();
public void reduce(Text key, Iterable<IntWritable> values,
Context context
) throws IOException, InterruptedException {
int sum = 0;
for (IntWritable val : values) {
sum += val.get();
}
result.set(sum);
context.write(key, result);
}
}
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "word count");
job.setJarByClass(WordCount.class);
job.setMapperClass(TokenizerMapper.class);
job.setCombinerClass(IntSumReducer.class);
job.setReducerClass(IntSumReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path("/home/jsun/share/wc/input"));
FileOutputFormat.setOutputPath(job, new Path("/home/jsun/share/wc/output"));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}
And the pom.xml for maven:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/1/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>wordcount2</groupId>
<artifactId>wordcount2</artifactId>
<version>0.0.1-SNAPSHOT</version>
<repositories>
<repository>
<id>apache</id>
<url>http://central.maven.org/maven2/</url>
</repository>
</repositories>
<build>
<sourceDirectory>src</sourceDirectory>
<resources>
<resource>
<directory>src</directory>
<excludes>
<exclude>**/*.java</exclude>
</excludes>
</resource>
</resources>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>2.6.0</version>
<type>jar</type>
</dependency>
</dependencies>
</project>
What is the difference using local hadoop jars and using maven dependencies?
Is that a problem of cluster or the wordcount or using maven?
Thanks in advance.
please check this Link
i had the same issue and i don't have hadoop installed on my machine. you can't run the program without installation. i think it looks for some environment variables to run hadoop commands.
Hope this helps