I am trying to run a simple Hadoop Map reduce program on eclipse in windows. Iam getting the following exception.
Exception in thread "main" java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:121)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:83)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:76)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1188)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1184)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1183)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1212)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1236)
at com.hadoop.mapreduce.WordCountDriverClass.main(WordCountDriverClass.java:41)
These are the jar files I have added in the project.
com.google.guava_1.6.0.jar
commons-configuration-1.7.jar
commons-lang-2.6.jar
commons-logging-1.1.3.jar
commons.collections-3.2.1.jar
guava-13.0.1.jar
hadoop-annotations-2.7.2.jar
hadoop-auth-2.6.0.jar
hadoop-common-2.3.0.jar
hadoop-common.jar
hadoop-mapreduce-client-core-2.0.2-alpha.jar
hadoop-mapreduce-client-core-2.7.2.jar
hadoop-mapreduce-client-jobclient-2.2.0.jar
hadoop-test-1.2.1.jar
log4j-1.2.17.jar
slf4j-api-1.7.7.jar
slf4j-simple-1.6.1.jar
I have added those jar files after checking the exception messages in the console. But I couldn't understand this exception.
Can anyone please help me fixing this.
This is my driver class.
Configuration conf = new Configuration();
// Creating a job
Job job = Job.getInstance(conf,"WordCountDriverClass");
job.setJarByClass(WordCountDriverClass.class);
job.setMapperClass(WordCountMapper.class);
job.setReducerClass(WordCountReducer.class);
job.setNumReduceTasks(2);
job.setInputFormatClass(KeyValueTextInputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path("inputfiles"));
FileOutputFormat.setOutputPath(job, new Path("outputfiles"));
job.waitForCompletion(true);
Looks like you are running the wordcount example, what it requires are 1.2.1 hadoop-core and 2.2.0 hadoop-common. If you use Maven instead, the config should be as simple as
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.2.0</version>
</dependency>
Related
I am trying to connect to AWS Aurora (Postgre) cluster from spring boot app using following properties
spring.datasource.driver.class.name = org.postgre.jdbc.Driver
spring.datasource.url=jdbc:postgresql://a2-xxxxx.cluster-xxxxxxxxxx.us-east-1.rds.amazonaws.com:5432
spring.datasource.username=username
spring.datasource.password=password
but I am getting following error -
*Caused by: java.lang.RuntimeException: Driver org.postgresql.Driver claims to not accept jdbcUrl, jdbc:postgresql://a2-xxxxx.cluster-xxxxxxxxxx.us-east-1.rds.amazonaws.com:5432
at com.zaxxer.hikari.util.DriverDataSource.<init>(DriverDataSource.java:110) ~[HikariCP-4.0.3.jar:na]
at com.zaxxer.hikari.pool.PoolBase.initializeDataSource(PoolBase.java:331) ~[HikariCP-4.0.3.jar:na]
at com.zaxxer.hikari.pool.PoolBase.<init>(PoolBase.java:114) ~[HikariCP-4.0.3.jar:na]*
Following are my maven pom.xml contents -
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>
You should read the jdbc documentation.
If the host is specified, the url needs a / at the end. If the database you are connecting to has a different name than the user, you also have to put the name of the database after the /. Even if they are the same, it's a good idea to include the db name because that default behavior is a bit confusing.
I faced a problem with camel REST routes when I switched from camel version 3.5.0 to version 3.8.0.
In the version 3.8.0 all routes from the REST service are generated twice! So at start of the camel context the exception is thrown:
Failed to start route route7 because of Multiple consumers for the same endpoint is not allowed: jetty:http://0.0.0.0:6060/start?httpMethodRestrict=GET
In detail:
I have a simple RouteBuilder to generate REST routes:
#Component
public class RestInterface extends RouteBuilder {
#Override
public void configure() throws Exception {
restConfiguration()
.component("jetty")
.host(urlInfo.getHost())
.port(urlInfo.getPort())
.bindingMode(RestBindingMode.json)
.dataFormatProperty("prettyPrint", "true")
.clientRequestValidation(true);
rest("/")
.get("/start").outType(RepositoryPackageInfo.class).to("direct:start")
.get("/stop").outType(RepositoryPackageInfo.class).to("direct:stop");
// ...
}
}
When I start this code with camel 3.8.0 I get following console output:
...
jetty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 11.0.4+11
Started o.e.j.s.ServletContextHandler#5f65e0c0{/,null,AVAILABLE}
Started ServerConnector#27b7204{HTTP/1.1, (http/1.1)}{0.0.0.0:6060}
Started #4095ms
Error starting CamelContext (camel-1) due to exception thrown: Failed to start route route7 because of Multiple consumers for the same endpoint is not allowed: jetty:http://0.0.0.0:6060/start?httpMethodRestrict=GET
org.apache.camel.FailedToStartRouteException: Failed to start route route7 because of Multiple consumers for the same endpoint is not allowed: jetty:http://0.0.0.0:6060/start?httpMethodRestrict=GET
...
Apache Camel 3.8.0 (camel-1) shutting down (timeout:45s)
Stopped ServerConnector#27b7204{HTTP/1.1, (http/1.1)}{0.0.0.0:6060}
Stopped o.e.j.s.ServletContextHandler#5f65e0c0{/,null,UNAVAILABLE}
Routes shutdown summary (total:40 stopped:40) <-------------- total 40 routes!!!
...
Stopped route8 (rest://get:/:/stop)
Stopped route7 (rest://get:/:/start)
Stopped route2 (rest://get:/:/stop)
Stopped route1 (rest://get:/:/start) <-------------- the routes "stop" and "start" are doubled!!!
...
If I run the same code with camel 3.5.0, the application starts without exceptions, and camel has only 38 routes:
etty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 11.0.4+11
Started o.e.j.s.ServletContextHandler#6cbb6c41{/,null,AVAILABLE}
Started ServerConnector#42172065{HTTP/1.1, (http/1.1)}{0.0.0.0:6060}
Started #4034ms
Route: route1 started and consuming from: jetty:http://0.0.0.0:6060/start
Route: route2 started and consuming from: jetty:http://0.0.0.0:6060/stop
...
Total 38 routes, of which 38 are started <-------------- total 38routes!!!
...
Completed initialization in 16 ms
Thank you in advance. I hope somebody can help me with this issue.
For completeness, here is a part of my pom.xml
<!-- Camel -->
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-rest</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-jetty</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-jackson</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-zipfile</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-zipfile-starter</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-bean-starter</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-management</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-test-spring</artifactId>
<version>${camel.version}</version>
<scope>test</scope>
</dependency>
Check if configure method on your RestInterface RouteBuilder is getting called more than once. Multiple calls cause duplicate RouteDefinitions to be added to the RouteBuilder instance which can lead to this error.
One way to solve this is to use boolean value in your RouteBuilder to check if configure method has already been called. Then use this value to early out or to throw an exception.
boolean isConfigured = false;
#Override
public void configure() throws Exception {
if(isConfigured){
return;
}
// Configure routes
isConfigured = true;
}
Encountered similar problem with Camel 2.x running in Karaf + OSGi where modifying config file lead to same error.
Basically configuring CamelContext service caused it to restart while the RouteBuilder services it referenced kept their state leading to situation where their configure method got called the second time leading to same error.
I have a Spring Boot MVC project. It works fine with h2 database and also fine in local postgre database with following application.properties configuration
spring.datasource.url=myUrl
spring.datasource.username=myUsername
spring.datasource.password=myPassword
But when I am trying to deploy it in Heroku it is getting this following error:
Caused by: org.hibernate.HibernateException: Access to
DialectResolutionInfo cannot be null when 'hibernate.dialect' not set
If I use this configuration
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.datasource.driverClassName=org.postgresql.Driver
spring.datasource.url=herokuPostgreSqlDbUrl
spring.datasource.username=herokuUsername
spring.datasource.password=mherokuPassword
I am getting this when try to run my project from STS and failed deployment.
Caused by: java.sql.SQLException: Driver:org.postgresql.Driver#17a3dff6
returned null for URL:myUrl
My pom is :
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.4-1206-jdbc42</version>
</dependency>
Can any body help ?
You need a proper JDBC url. See: https://springframework.guru/configuring-spring-boot-for-postgresql/
I'm trying to set up my Spring MVC app to send email, but i think i'm doing something wrong in the configuration
In My AppConfig i instantiate the Bean:
#Bean
public JavaMailSender mailSender() {
JavaMailSenderImpl mailSender = new JavaMailSenderImpl();
mailSender.setDefaultEncoding("UTF-8");
return mailSender;
}
Then in my application.properties I have:
mail.port= 465
mail.host= smtp.gmail.com
mail.username= username
mail.password= password
emailTo = info#beingsmart.it
emailSubject = ALERT - ePark Server
mail.smtp.auth = true
mail.smtp.starttls.enable = true
mail.properties.mail.smtp.socketFactory.class=javax.net.ssl.SSLSocketFactory
mail.properties.mail.smtp.socketFactory.fallback = false
When I try to send the email i'm getting
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.mail.javamail.JavaMailSender]: Factory method 'mailSender' threw exception; nested exception is java.lang.NoClassDefFoundError: javax/mail/AuthenticationFailedException
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
... 49 more
Caused by: java.lang.NoClassDefFoundError: javax/mail/AuthenticationFailedException
at it.besmart.config.AppConfig.mailSender(AppConfig.java:128)
at it.besmart.config.AppConfig$$EnhancerBySpringCGLIB$$b2a09c0.CGLIB$mailSender$7(<generated>)
at it.besmart.config.AppConfig$$EnhancerBySpringCGLIB$$b2a09c0$$FastClassBySpringCGLIB$$dd65f050.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:318)
at it.besmart.config.AppConfig$$EnhancerBySpringCGLIB$$b2a09c0.mailSender(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
... 50 more
Caused by: java.lang.ClassNotFoundException: javax.mail.AuthenticationFailedException
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1285)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
... 61 more
EDIT
I added javax mail to my pom
<dependency>
<groupId>javax.mail</groupId>
<artifactId>javax.mail-api</artifactId>
<version>1.5.6</version>
</dependency>
Now I have a different exception when trying to send the email
java.lang.NoClassDefFoundError: com/sun/mail/util/MessageRemovedIOException
It looks like an authentication error, but i'm thinking that JavaMailSender does not takes the setting in my application.properties, could it be?
Thanks
It seems to be missing dependency rather than anything else. If using spring boot have dependency like:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-mail</artifactId>
</dependency>
If you don't I suggest you look inside that POM and figure out what are the missing pieces.
In your particular case it seems you need:
<dependency>
<groupId>com.sun.mail</groupId>
<artifactId>javax.mail</artifactId>
<version>1.5.5</version>
</dependency>
Generic advice:
ClassNotFoundException means class which you need is not on classpath. It's usually a missing dependency
Google to find which jar contains the missing class
Find the library it on maven central
Add to dependencies
im getting following Error
**ERROR:**
2014-10-07 16:33:46,692 ERROR [stderr] (default task-1) javax.naming.NamingException:
JBAS011843: **Failed instantiate** **InitialContextFactory
org.jboss.naming.remote.client.InitialContextFactory** from classloader
ModuleClassLoader for Module "deployment.wildfly8.1.ear.wildfly8.1-war.war:main"
from Service Module Loader [Root exception is java.lang.ClassNotFoundException:
org.jboss.naming.remote.client.InitialContextFactory from [Module
"deployment.wildfly8.1.ear.wildfly8.1-war.war:main" from Service Module Loader]]
Code:
final Properties env = new Properties();
env.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
env.put(Context.PROVIDER_URL, "127.0.0.1");
//OR env.put(Context.PROVIDER_URL, "http-remoting://127.0.0.1");
env.put(Context.SECURITY_PRINCIPAL, "admin");
env.put(Context.SECURITY_CREDENTIALS, "password");
context = new InitialContext(env);
Try adding a jboss-deployment-structure.xml to the META-INF of the top level .ear file. In the jboss-deployment-structure.xml add a dependency for org.jboss.remote-naming and org.jboss.ejb-client.
Here's some documentation on jboss-deployment-strucure.xml: Class Loading in WildFly
Other useful links:
EJB invocations from a remote server instance
ejb-multi-server: EJB Communication Across Servers
You'll need to include in your war/WEB-INF/lib directory all dependencies of a remote client:
<dependency>
<groupId>org.wildfly</groupId>
<artifactId>wildfly-jms-client-bom</artifactId>
<version>${version.wildfly}</version>
<type>pom</type>
</dependency>