We are working on jar updates and removing unwanted jars from the project. As you can see it's a very old project and many have added unwanted jars based on their requirment.
I am getting below error from past week, even if I revert my code to past production code. I have removed below jars from the project as we are having SCA scan errors on these. There are no compile issue but getting below exception on runtime.
from WEB-INF/lib
jackcess-1.2.10-SNAPSHOT.jar
gson-1.6.jar
xmlbeans-2.3.0.jar
below from a folder, not used in the project.
jsp-api-2.1-6.0.0.jar
jsr311-api-1.1.1.jar
json-20090211.jar
servlet-2.3.jar
Help would be much appreciated...
07:50:24,097 INFO [org.primefaces.webapp.PostConstructApplicationEventListener] (ServerService Thread Pool -- 99) Running on PrimeFaces 8.0
07:50:24,167 INFO [com.sun.jersey.api.core.servlet.WebAppResourceConfig] (ServerService Thread Pool -- 99) Scanning for root resource and provider classes in the Web app resource paths:
/WEB-INF/lib
/WEB-INF/classes
07:50:24,842 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 99) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./CactusServer: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./CactusServer: java.lang.IllegalArgumentException
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
at java.lang.Thread.run(Thread.java:750)
at org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: java.lang.IllegalArgumentException
at jersey.repackaged.org.objectweb.asm.ClassReader.<init>(ClassReader.java:170)
at jersey.repackaged.org.objectweb.asm.ClassReader.<init>(ClassReader.java:153)
at jersey.repackaged.org.objectweb.asm.ClassReader.<init>(ClassReader.java:424)
at com.sun.jersey.spi.scanning.AnnotationScannerListener.onProcess(AnnotationScannerListener.java:138)
at com.sun.jersey.core.spi.scanning.JarFileScanner.scan(JarFileScanner.java:97)
at com.sun.jersey.spi.scanning.servlet.WebAppResourcesScanner$1.f(WebAppResourcesScanner.java:94)
at com.sun.jersey.core.util.Closing.f(Closing.java:71)
at com.sun.jersey.spi.scanning.servlet.WebAppResourcesScanner.scan(WebAppResourcesScanner.java:92)
at com.sun.jersey.spi.scanning.servlet.WebAppResourcesScanner.scan(WebAppResourcesScanner.java:79)
at com.sun.jersey.api.core.ScanningResourceConfig.init(ScanningResourceConfig.java:80)
at com.sun.jersey.api.core.servlet.WebAppResourceConfig.init(WebAppResourceConfig.java:102)
at com.sun.jersey.api.core.servlet.WebAppResourceConfig.<init>(WebAppResourceConfig.java:89)
at com.sun.jersey.api.core.servlet.WebAppResourceConfig.<init>(WebAppResourceConfig.java:74)
at com.sun.jersey.spi.container.servlet.WebComponent.getWebAppResourceConfig(WebComponent.java:668)
at com.sun.jersey.spi.container.servlet.ServletContainer.getDefaultResourceConfig(ServletContainer.java:435)
at com.sun.jersey.spi.container.servlet.ServletContainer.getDefaultResourceConfig(ServletContainer.java:602)
at com.sun.jersey.spi.container.servlet.WebServletConfig.getDefaultResourceConfig(WebServletConfig.java:87)
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:699)
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:674)
at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:205)
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:394)
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:577)
at javax.servlet.GenericServlet.init(GenericServlet.java:244)
at io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:117)
at org.wildfly.extension.undertow.security.RunAsLifecycleInterceptor.init(RunAsLifecycleInterceptor.java:78)
at io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:103)
at io.undertow.servlet.core.ManagedServlet$DefaultInstanceStrategy.start(ManagedServlet.java:300)
at io.undertow.servlet.core.ManagedServlet.createServlet(ManagedServlet.java:140)
at io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:584)
at io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:555)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514)
at io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:597)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:97)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:78)
... 8 more
07:50:24,868 ERROR [org.jboss.as.controller.management-operation] (DeploymentScanner-threads - 2) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "CactusServer.war")]) - failure description: {
"WFLYCTL0080: Failed services" => {"jboss.undertow.deployment.default-server.default-host./CactusServer" => "java.lang.IllegalArgumentException
Caused by: java.lang.IllegalArgumentException"},
"WFLYCTL0288: One or more services were unable to start due to one or more indirect dependencies not being available." => {
"Services that were unable to start:" => ["jboss.deployment.unit.\"CactusServer.war\".deploymentCompleteService"],
"Services that may be the cause:" => [
"jboss.clustering.web.route.default-server",
"jboss.iiop-openjdk.poa-service.rootpoa",
"jboss.txn.service.remote",
"jboss.xts.handlers",
"org.wildfly.clustering.cache.default-service-provider-registry.ejb",
"org.wildfly.clustering.cache.default-service-provider-registry.web",
"org.wildfly.clustering.cache.group.ejb.passivation",
"org.wildfly.clustering.cache.group.hibernate.entity",
"org.wildfly.clustering.cache.group.hibernate.local-query",
"org.wildfly.clustering.cache.group.hibernate.timestamps",
"org.wildfly.clustering.cache.group.server.client-mappings",
"org.wildfly.clustering.cache.group.server.default",
"org.wildfly.clustering.cache.group.web.client-mappings",
"org.wildfly.clustering.cache.group.web.default-server",
"org.wildfly.clustering.cache.group.web.passivation",
"org.wildfly.clustering.cache.registry.ejb.passivation",
"org.wildfly.clustering.cache.registry.server.default",
"org.wildfly.clustering.cache.registry.web.passivation",
"org.wildfly.clustering.cache.registry-entry.ejb.passivation",
"org.wildfly.clustering.cache.registry-entry.hibernate.entity",
"org.wildfly.clustering.cache.registry-entry.hibernate.local-query",
"org.wildfly.clustering.cache.registry-entry.hibernate.timestamps",
"org.wildfly.clustering.cache.registry-entry.server.client-mappings",
"org.wildfly.clustering.cache.registry-entry.server.default",
"org.wildfly.clustering.cache.registry-entry.web.client-mappings",
"org.wildfly.clustering.cache.registry-entry.web.default-server",
"org.wildfly.clustering.cache.registry-entry.web.passivation",
"org.wildfly.clustering.cache.registry-factory.ejb.passivation",
"org.wildfly.clustering.cache.registry-factory.hibernate.entity",
"org.wildfly.clustering.cache.registry-factory.hibernate.local-query",
"org.wildfly.clustering.cache.registry-factory.hibernate.timestamps",
"org.wildfly.clustering.cache.registry-factory.server.client-mappings",
"org.wildfly.clustering.cache.registry-factory.server.default",
"org.wildfly.clustering.cache.registry-factory.web.client-mappings",
"org.wildfly.clustering.cache.registry-factory.web.default-server",
"org.wildfly.clustering.cache.registry-factory.web.passivation",
"org.wildfly.clustering.cache.service-provider-registry.ejb.client-mappings",
"org.wildfly.clustering.cache.service-provider-registry.ejb.passivation",
"org.wildfly.clustering.cache.service-provider-registry.hibernate.entity",
"org.wildfly.clustering.cache.service-provider-registry.hibernate.local-query",
"org.wildfly.clustering.cache.service-provider-registry.hibernate.timestamps",
"org.wildfly.clustering.cache.service-provider-registry.server.client-mappings",
"org.wildfly.clustering.cache.service-provider-registry.server.default",
"org.wildfly.clustering.cache.service-provider-registry.web.client-mappings",
"org.wildfly.clustering.cache.service-provider-registry.web.default-server",
"org.wildfly.clustering.cache.service-provider-registry.web.passivation",
"org.wildfly.clustering.command-dispatcher-factory.ejb",
"org.wildfly.clustering.command-dispatcher-factory.hibernate",
"org.wildfly.clustering.command-dispatcher-factory.local",
"org.wildfly.clustering.command-dispatcher-factory.server",
"org.wildfly.clustering.command-dispatcher-factory.web",
"org.wildfly.clustering.default-command-dispatcher-factory",
"org.wildfly.clustering.group.hibernate",
"org.wildfly.clustering.group.server",
"org.wildfly.clustering.group.web",
"org.wildfly.clustering.infinispan.cache.ejb.passivation",
"org.wildfly.clustering.infinispan.cache.hibernate.entity",
"org.wildfly.clustering.infinispan.cache.hibernate.local-query",
"org.wildfly.clustering.infinispan.cache.hibernate.timestamps",
"org.wildfly.clustering.infinispan.cache.server.client-mappings",
"org.wildfly.clustering.infinispan.cache.server.default",
"org.wildfly.clustering.infinispan.cache.store.hibernate.entity",
"org.wildfly.clustering.infinispan.cache.store.hibernate.local-query",
"org.wildfly.clustering.infinispan.cache.store.hibernate.timestamps",
"org.wildfly.clustering.infinispan.cache.store.server.default",
"org.wildfly.clustering.infinispan.cache.store.web.passivation",
"org.wildfly.clustering.infinispan.cache.web.client-mappings",
"org.wildfly.clustering.infinispan.cache.web.default-server",
"org.wildfly.clustering.infinispan.cache.web.passivation",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.entity",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.entity.expiration",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.entity.locking",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.entity.memory",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.entity.transaction",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.local-query",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.local-query.expiration",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.local-query.locking",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.local-query.memory",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.local-query.transaction",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.timestamps",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.timestamps.expiration",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.timestamps.locking",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.timestamps.memory",
"org.wildfly.clustering.infinispan.cache-configuration.hibernate.timestamps.transaction",
"org.wildfly.clustering.infinispan.cache-configuration.server.client-mappings",
"org.wildfly.clustering.infinispan.cache-configuration.server.default",
"org.wildfly.clustering.infinispan.cache-configuration.server.default.expiration",
"org.wildfly.clustering.infinispan.cache-configuration.server.default.locking",
"org.wildfly.clustering.infinispan.cache-configuration.server.default.memory",
"org.wildfly.clustering.infinispan.cache-configuration.server.default.transaction",
"org.wildfly.clustering.infinispan.cache-configuration.web.client-mappings",
"org.wildfly.clustering.infinispan.cache-configuration.web.default-server",
"org.wildfly.clustering.infinispan.cache-configuration.web.passivation",
"org.wildfly.clustering.infinispan.cache-configuration.web.passivation.expiration",
"org.wildfly.clustering.infinispan.cache-configuration.web.passivation.locking",
"org.wildfly.clustering.infinispan.cache-configuration.web.passivation.memory",
"org.wildfly.clustering.infinispan.cache-configuration.web.passivation.transaction",
"org.wildfly.clustering.infinispan.cache-configuration.web.passivation.write",
"org.wildfly.clustering.infinispan.cache-container.hibernate",
"org.wildfly.clustering.infinispan.cache-container.server",
"org.wildfly.clustering.infinispan.cache-container.web",
"org.wildfly.clustering.infinispan.cache-container-configuration.hibernate",
"org.wildfly.clustering.infinispan.cache-container-configuration.hibernate.transport",
"org.wildfly.clustering.infinispan.cache-container-configuration.server",
"org.wildfly.clustering.infinispan.cache-container-configuration.server.transport",
"org.wildfly.clustering.infinispan.cache-container-configuration.web",
"org.wildfly.clustering.infinispan.cache-container-configuration.web.transport",
"org.wildfly.clustering.infinispan.default-cache.ejb",
"org.wildfly.clustering.infinispan.default-cache.web",
"org.wildfly.clustering.infinispan.default-cache-configuration.server",
"org.wildfly.clustering.infinispan.default-cache-configuration.web"
]
}
}
**POM file**
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>CactusServer</artifactId>
<packaging>war</packaging>
<name>Cactus Server Project</name>
<parent>
<groupId>com.xorail</groupId>
<artifactId>com.xorail.talon.pom</artifactId>
<relativePath>../com.xorail.talon.pom</relativePath>
<version>${app-version}</version>
</parent>
<repositories>
<repository>
<id>prime-repo</id>
<name>PrimeFaces Maven Repository</name>
<url>https://repository.primefaces.org</url>
<layout>default</layout>
</repository>
<repository>
<id>Jboss 3rd party</id>
<url>https://repository.jboss.org/nexus/content/repositories/thirdparty-releases/</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.xorail</groupId>
<artifactId>CactusShared</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.xorail</groupId>
<artifactId>com.xorail.util.property</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.xorail</groupId>
<artifactId>com.xorail.qaqc</artifactId>
<version>${project.version}</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.richfaces.framework/richfaces-api
<dependency> <groupId>org.richfaces.framework</groupId> <artifactId>richfaces-api</artifactId>
<version>3.3.3.Final</version> </dependency> -->
<!-- https://mvnrepository.com/artifact/org.richfaces.framework/richfaces-impl
<dependency> <groupId>org.richfaces.framework</groupId> <artifactId>richfaces-impl</artifactId>
<version>3.3.3.Final</version> </dependency> -->
<!-- https://mvnrepository.com/artifact/org.richfaces.framework/richfaces-impl-jsf2
<dependency> <groupId>org.richfaces.framework</groupId> <artifactId>richfaces-impl-jsf2</artifactId>
<version>3.3.3.Final</version> <scope>provided</scope> </dependency> -->
<!-- https://mvnrepository.com/artifact/org.richfaces.ui/richfaces-ui <dependency>
<groupId>org.richfaces.ui</groupId> <artifactId>richfaces-ui</artifactId>
<version>3.3.3.Final</version> </dependency> -->
<!-- https://mvnrepository.com/artifact/org.apache.axis/axis -->
<dependency>
<groupId>org.apache.axis</groupId>
<artifactId>axis</artifactId>
<version>1.4</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.axis2/axis2 -->
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2</artifactId>
<version>1.6.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-httpclient/commons-httpclient -->
<dependency>
<groupId>commons-httpclient</groupId>
<artifactId>commons-httpclient</artifactId>
<version>3.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.quartz-scheduler/quartz -->
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz</artifactId>
<version>1.8.6</version>
</dependency>
<!-- cron-utils -->
<dependency>
<groupId>com.cronutils</groupId>
<artifactId>cron-utils</artifactId>
<version>9.0.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/net.sf.dozer/dozer -->
<dependency>
<groupId>net.sf.dozer</groupId>
<artifactId>dozer</artifactId>
<version>5.4.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-net/commons-net -->
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>3.0.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.jboss.spec.javax.faces/jboss-jsf-api_2.2_spec -->
<dependency>
<groupId>org.jboss.spec.javax.faces</groupId>
<artifactId>jboss-jsf-api_2.2_spec</artifactId>
<version>2.2.12</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.myfaces.tomahawk/tomahawk21 -->
<dependency>
<groupId>org.apache.myfaces.tomahawk</groupId>
<artifactId>tomahawk21</artifactId>
<version>1.1.14</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.commons/commons-compress -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.16.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.sun.facelets/jsf-facelets <dependency>
<groupId>com.sun.facelets</groupId> <artifactId>jsf-facelets</artifactId>
<version>1.1.15</version> </dependency> -->
<dependency>
<groupId>org.jboss.spec.javax.servlet</groupId>
<artifactId>jboss-servlet-api_3.1_spec</artifactId>
<version>1.0.0.Final</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.jboss.spec.javax.servlet.jsp</groupId>
<artifactId>jboss-jsp-api_2.3_spec</artifactId>
<version>1.0.1.Final</version>
<scope>provided</scope>
</dependency>
<!-- Redmine related references -->
<!-- https://mvnrepository.com/artifact/asm/asm -->
<dependency>
<groupId>asm</groupId>
<artifactId>asm</artifactId>
<version>3.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.taskadapter/redmine-java-api -->
<dependency>
<groupId>com.taskadapter</groupId>
<artifactId>redmine-java-api</artifactId>
<version>2.6.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.primefaces/primefaces -->
<dependency>
<groupId>org.primefaces</groupId>
<artifactId>primefaces</artifactId>
<version>8.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.primefaces.themes/casablanca -->
<dependency>
<groupId>org.primefaces.themes</groupId>
<artifactId>bootstrap</artifactId>
<version>1.0.10</version>
</dependency>
<dependency>
<groupId>commons-discovery</groupId>
<artifactId>commons-discovery</artifactId>
<version>0.5</version>
</dependency>
<dependency>
<groupId>wsdl4j</groupId>
<artifactId>wsdl4j</artifactId>
<version>1.6.2</version>
</dependency>
<dependency>
<groupId>com.nimbusds</groupId>
<artifactId>oauth2-oidc-sdk</artifactId>
<version>5.24.1</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20090211</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.0.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-codec/commons-codec -->
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.13</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src</sourceDirectory>
<finalName>${project.artifactId}</finalName>
<resources>
<resource>
<directory>src</directory>
<includes>
<include>**/*.properties</include>
</includes>
</resource>
</resources>
<plugins>
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>2.6</version>
<configuration>
<warSourceDirectory>WebContent</warSourceDirectory>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<source>${java-version}</source>
<target>${java-version}</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
I am able to insert the data in HBase table using the Put method. But, now I want to read data from CSV file and write it to the HBase table.
I wrote this code:
object Read_CSV_Method2 {
case class HotelBooking(hotelName : String, is_cancelled : String, reservation_status : String,
reservation_status_date : String)
def main(args : Array[String]){
Logger.getLogger("org").setLevel(Level.ERROR)
val sparkConf = new SparkConf().setAppName("Spark_Hbase_Connection").setMaster("local[*]")
val sc = new SparkContext(sparkConf)
val conf = HBaseConfiguration.create()
// establish a connection
val connection:Connection = ConnectionFactory.createConnection(conf)
// Table on which different commands have to be run.
val tableName = connection.getTable(TableName.valueOf("hotelTable"))
val sqlContext = new SQLContext(sc)
import sqlContext.implicits._
val textRDD = sc.textFile("/Users/sukritmehta/Documents/hotel_bookings.csv")
var c = 1
textRDD.foreachPartition{
iter =>
val hbaseConf = HBaseConfiguration.create()
val connection:Connection = ConnectionFactory.createConnection(hbaseConf)
val myTable = connection.getTable(TableName.valueOf("hotelTable"))
iter.foreach{
f =>
val col = f.split(",")
val insertData = new Put(Bytes.toBytes(c.toString()))
insertData.addColumn(Bytes.toBytes("hotelFam"), Bytes.toBytes("hotelName"), Bytes.toBytes(col(0).toString()))
insertData.addColumn(Bytes.toBytes("hotelFam"), Bytes.toBytes("is_canceled"), Bytes.toBytes(col(1).toString()))
insertData.addColumn(Bytes.toBytes("hotelFam"), Bytes.toBytes("reservation_status"), Bytes.toBytes(col(30).toString()))
insertData.addColumn(Bytes.toBytes("hotelFam"), Bytes.toBytes("reservation_status_date"), Bytes.toBytes(col(31).toString()))
c = c+1
myTable.put(insertData)
}
}
}
}
But after running this code, I am getting the following error:
Exception in thread "dag-scheduler-event-loop" java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/InputSplitWithLocationInfo
at org.apache.spark.rdd.HadoopRDD.getPreferredLocations(HadoopRDD.scala:356)
at org.apache.spark.rdd.RDD$$anonfun$preferredLocations$2.apply(RDD.scala:275)
at org.apache.spark.rdd.RDD$$anonfun$preferredLocations$2.apply(RDD.scala:275)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.preferredLocations(RDD.scala:274)
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal(DAGScheduler.scala:2003)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2$$anonfun$apply$1.apply$mcVI$sp(DAGScheduler.scala:2014)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2$$anonfun$apply$1.apply(DAGScheduler.scala:2013)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2$$anonfun$apply$1.apply(DAGScheduler.scala:2013)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2.apply(DAGScheduler.scala:2013)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2.apply(DAGScheduler.scala:2011)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal(DAGScheduler.scala:2011)
at org.apache.spark.scheduler.DAGScheduler.getPreferredLocs(DAGScheduler.scala:1977)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$17.apply(DAGScheduler.scala:1112)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$17.apply(DAGScheduler.scala:1110)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at org.apache.spark.scheduler.DAGScheduler.submitMissingTasks(DAGScheduler.scala:1110)
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitStage(DAGScheduler.scala:1069)
at org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:1013)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2065)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2057)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2046)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.InputSplitWithLocationInfo
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 32 more
pom.xml file
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>Spark_Hbase</groupId>
<artifactId>Spark_Hbase</artifactId>
<version>0.0.1-SNAPSHOT</version>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<!-- Scala and Spark dependencies -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.11.11</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.4.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-avro_2.11</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.4.0</version>
<scope>provided</scope>
</dependency>
<!-- <dependency> <groupId>org.codehaus.janino</groupId> <artifactId>commons-compiler</artifactId>
<version>3.0.7</version> </dependency> <dependency> <groupId>org.apache.spark</groupId>
<artifactId>spark-network-common_2.11</artifactId> <version>2.1.1</version>
</dependency> -->
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.9.9</version>
</dependency>
<!-- <dependency> <groupId>org.mongodb.spark</groupId> <artifactId>mongo-spark-connector_2.10</artifactId>
<version>2.1.1</version> </dependency> -->
<!-- <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-mllib_2.10</artifactId>
<version>2.1.1</version> </dependency> -->
<!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase -->
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>2.2.4</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-spark</artifactId>
<version>2.0.0-alpha4</version> <!-- Hortonworks Latest -->
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase-mapreduce -->
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-mapreduce</artifactId>
<version>2.2.4</version>
</dependency>
</dependencies>
</project>
It would be great if anyone could help me out in resolving this issue.
I think you are missing
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>2.6.5</version>
<scope>provided</scope>
</dependency>
I use spring boot starter to run camunda. I can run with h2 database. But when i try to connect to postgres i havethe following error :
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Invocation of init method failed; nested exception is org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1803)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1108)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:868)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at fr.casden.eficas.workflow.Application.main(Application.java:10)
Caused by: org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:275)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
at org.hibernate.id.factory.internal.DefaultIdentifierGeneratorFactory.injectServices(DefaultIdentifierGeneratorFactory.java:152)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.injectDependencies(AbstractServiceRegistryImpl.java:286)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:243)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.<init>(InFlightMetadataCollectorImpl.java:175)
at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:118)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:1214)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:1245)
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:58)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:391)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:378)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1862)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1799)
... 16 common frames omitted
Caused by: org.hibernate.HibernateException: Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.determineDialect(DialectFactoryImpl.java:100)
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.buildDialect(DialectFactoryImpl.java:54)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:137)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:101)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
... 33 common frames omitted
Here are my pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<properties>
<java.version>8</java.version>
</properties>
<groupId>fr.my.work</groupId>
<artifactId>my-workflow</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.camunda.bpm.springboot</groupId>
<artifactId>camunda-bpm-spring-boot-starter-rest</artifactId>
<version>3.4.1</version>
</dependency>
<!-- Driver JDBC Postgresql-->
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.2.10</version>
</dependency>
<dependency>
<groupId>org.camunda.bpm.springboot</groupId>
<artifactId>camunda-bpm-spring-boot-starter-webapp</artifactId>
<version>3.4.1</version>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.4.200</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<version>2.2.1.RELEASE</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
application properties :
camunda.bpm.admin-user.id=demo
camunda.bpm.admin-user.password=demo
camunda.bpm.database.type=postgres
spring.datasource.url=jdbc:postgresql://localhost:5432/workflow
spring.datasource.username=user
spring.datasource.password=workflow
spring.datasource.driver-class-name=org.postgresql.Driver
hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
spring.datasource.driverClassName=org.postgresql.Driver
Spring Boot Application class
#SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class);
}
}
What am i missing please ?
Try to remove the version of postgres in your pom.xml and instead give <scope>runtime</scope>
Try to give this in application.yaml:
# Database
datasource:
driver-class-name: org.postgresql.Driver
url: jdbc:postgresql://localhost:5432/postgres
username: postgres
password: postgres
# JPA properties
jpa:
hibernate:
ddl-auto: none
show-sql: true
database: postgresql
database-platform: org.hibernate.dialect.PostgreSQLDialect
open-in-view: false
generate-ddl: false
Springboot version - 2.1.2.RELEASE works fine with camunda bpm version -7.13.0
Here is the pom.xml entry -
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.2.RELEASE</version>
<relativePath/>
</parent>
<properties>
<java.version>8</java.version>
</properties>
<groupId>com.xxxx.workflow</groupId>
<artifactId>xxxx-camunda-bpm</artifactId>
<version>1.0.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.camunda.bpm.springboot</groupId>
<artifactId>camunda-bpm-spring-boot-starter</artifactId>
<version>7.13.0</version>
</dependency>
<dependency>
<groupId>org.camunda.bpm.springboot</groupId>
<artifactId>camunda-bpm-spring-boot-starter-rest</artifactId>
<version>7.13.0</version>
</dependency>
<dependency>
<groupId>org.camunda.bpm.springboot</groupId>
<artifactId>camunda-bpm-spring-boot-starter-webapp</artifactId>
<version>7.13.0</version>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
and application.yml
camunda:
admin-user:
id: demo
password: demo
bpm:
database:
type: postgres
table-prefix: camunda.
schema-update: true
spring:
main:
allow-bean-definition-overriding: true
jpa:
properties:
hibernate:
dialect: org.hibernate.dialect.PostgreSQLDialect
datasource:
driverClassName: org.postgresql.Driver
#DB connection:
url: jdbc:postgresql://localhost:5432/workflow?stringtype=unspecified
username: xxxxx
password: xxxxx
ArrayOutOfBound Error
enter image description here
I have an xls rule [1 & 2], when I run it as JUnit, it works fine, but when I run it as Maven test, an ArrayOutOfBounds [3] error appears and I couldn't find any explanation. Any hint ?
========================================================
The pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.ibm.cio.cloud.cost</groupId>
<artifactId>bluecost</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>bluecost</name>
<url>http://maven.apache.org</url>
<profiles>
<profile>
<id>dev</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<build.profile.id>dev</build.profile.id>
</properties>
</profile>
<profile>
<id>test</id>
<properties>
<build.profile.id>test</build.profile.id>
</properties>
<build>
<plugins>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<phase>install</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>prod</id>
<properties>
<build.profile.id>prod</build.profile.id>
</properties>
</profile>
</profiles>
<properties>
<jdk.version>1.8</jdk.version>
<spring.version>4.3.9.RELEASE</spring.version>
<spring.batch.version>3.0.8.RELEASE</spring.batch.version>
<mysql.driver.version>5.1.44</mysql.driver.version>
<db2.driver.version>10.1.0</db2.driver.version>
</properties>
<dependencies>
<!-- Spring Core -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${spring.version}</version>
</dependency>
<!-- Spring Core -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<version>${spring.version}</version>
</dependency>
<!-- Spring Test -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>${spring.version}</version>
</dependency>
<!-- Spring Batch dependencies -->
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-core</artifactId>
<version>${spring.batch.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-infrastructure</artifactId>
<version>${spring.batch.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-test</artifactId>
<version>${spring.batch.version}</version>
<scope>test</scope>
</dependency>
<!-- MySQL database driver -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${mysql.driver.version}</version>
</dependency>
<dependency>
<groupId>com.ibm.db2.jcc</groupId>
<artifactId>db2jcc_license_cisuz</artifactId>
<version>${db2.driver.version}</version>
</dependency>
<dependency>
<groupId>com.ibm.db2.jcc</groupId>
<artifactId>db2jcc</artifactId>
<version>${db2.driver.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>commons-dbcp</groupId>
<artifactId>commons-dbcp</artifactId>
<version>1.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>2.4.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>commons-pool</groupId>
<artifactId>commons-pool</artifactId>
<version>1.6</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-spring</artifactId>
<version>5.4.0.Final</version>
</dependency>
<dependency>
<groupId>org.eclipse.jdt</groupId>
<artifactId>org.eclipse.jdt.core</artifactId>
<version>3.7.1</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.14</version>
</dependency>
</dependencies>
<build>
<finalName>bluecost-batch</finalName>
<filters>
<filter>profiles/${build.profile.id}/config.properties</filter>
</filters>
<resources>
<resource>
<filtering>true</filtering>
<directory>src/main/resources</directory>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<version>2.9</version>
<configuration>
<downloadSources>true</downloadSources>
<downloadJavadocs>false</downloadJavadocs>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20.1</version>
<configuration>
<excludes>
<exclude></exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<organization>
<name>IBM Corporation</name>
</organization>`enter code here`
</project>
========================
=====================
And the JUnit I created...
#Before
public void setUp() {
final Logger logger = Logger.getLogger("setup()");
logger.info("-----------------------------");
}
#Test
public void testDefaultValuesbyChannelName(){
BlueReport testBlueReport = new BlueReport();
// Arbitrary value, this should be "SOFTLYER"
testBlueReport.setAccount_id("29302"); // Must exist
testBlueReport.setOrg_co("SOFTLYER");
SSCData testSSCdataOutput = new SSCData();
testSSCdataOutput.setORIG_LOC_CD(null); // values will be filled by the rule
testSSCdataOutput.setSERVICE_CD(null); // this too will be filled by the rule
// values will be filled by the rules
String expectedOrigLocCd = "SLR";
//SSCDataOutput.setORIG_LOC_CD("SLR");
String expectedServiceTypCd = "SLR";
//SSCDataOutput.setSERVICE_TYP_CD("SLR");
String expectedServiceCd = "SLIC";
//SSCDataOutput.setSERVICE_CD("SLIC");
String expectedServiceGroupId = "BASE";
//SSCDataOutput.setSERVICE_GROUP_ID("BASE");
String expectedRateClasCd = "OGS";
//SSCDataOutput.setRATECLAS_CD("OGS");
String expectedLocalField1 = "INVCE ID";
//SSCDataOutput.setLOCAL_FIELD_1("INVCE ID");
//SSCDataOutput.setLOCAL_FIELD_2(null);//Not anymore at the Invoice level. can be Nulled.
String expectedLocalField3 = "ACCT ID";
//SSCDataOutput.setLOCAL_FIELD_3("ACCT ID");
//SSCDataOutput.setLOCAL_FIELD_5(null);
//SSCDataOutput.setLOCAL_FIELD_6(null);
final Logger logger = Logger.getLogger("testDefaultValuesbyChannelName");
dtSession.execute( Arrays.asList(new Object[] { testSSCdataOutput, testBlueReport }) );
logger.info("Rules that activated: " + testSSCdataOutput.getRuleAudit());
dtSession.setGlobal("logger", logger);
logger.info("Expecting ORIG_LOC_CD to be SLR");
assertTrue("I expected SLR", testSSCdataOutput.getORIG_LOC_CD().contains(expectedOrigLocCd));
logger.info("Expecting SERVICE_CD to be SLIC");
assertTrue("I expected SLIC", testSSCdataOutput.getSERVICE_CD().contains(expectedServiceCd));
logger.info("Expecting SERVICE_TYPE_CD to be SLR");
assertTrue("I expected SLR", testSSCdataOutput.getSERVICE_TYP_CD().contains(expectedServiceTypCd));
logger.info("Expecting SERVICE_GROUP_ID to be BASE");
assertTrue("I expected BASE", testSSCdataOutput.getSERVICE_GROUP_ID().contains(expectedServiceGroupId));
logger.info("Expecting RATECLAS_CD to be OGS");
assertTrue("I expected OGS", testSSCdataOutput.getRATECLAS_CD().contains(expectedRateClasCd));
logger.info("Expecting LOCAL_FIELD1 to be INVCE ID");
assertTrue("I expected INVCE ID", testSSCdataOutput.getLOCAL_FIELD_1().contains(expectedLocalField1));
logger.info("Expecting LOCAL_FIELD3 to be ACCT ID");
assertTrue("I expected ACCCT ID", testSSCdataOutput.getLOCAL_FIELD_3().contains(expectedLocalField3));
And this is the error on Run As ... 'Maven test'
======================================================
Running com.ibm.cio.cloud.cost.unit.DroolsDefaultValuesByChannelNameTest
[ERROR] TestContextManager - Caught exception while allowing TestExecutionListener [org.springframework.test.context.support.DependencyInjectionTestExecutionListener#1de6f05] to prepare test instance [com.ibm.cio.cloud.cost.unit.DroolsDefaultValuesByChannelNameTest#114e780] <java.lang.IllegalStateException: Failed to load ApplicationContext>java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124)
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:83)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:117)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:230)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:228)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:287)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:289)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:247)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:369)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:275)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:239)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:160)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dtBase': Invocation of init method failed; nested exception is java.lang.ArrayIndexOutOfBoundsException: 38851
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:742)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:128)
at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:60)
at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.delegateLoading(AbstractDelegatingSmartContextLoader.java:108)
at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:251)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:98)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:116)
... 27 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 38851
at jxl.read.biff.Record.<init>(Record.java:79)
at jxl.read.biff.File.next(File.java:181)
at jxl.read.biff.WorkbookParser.parse(WorkbookParser.java:569)
at jxl.Workbook.getWorkbook(Workbook.java:271)
at org.drools.decisiontable.parser.xls.ExcelParser.parseFile(ExcelParser.java:77)
at org.drools.decisiontable.SpreadsheetCompiler.compile(SpreadsheetCompiler.java:89)
at org.drools.decisiontable.SpreadsheetCompiler.compile(SpreadsheetCompiler.java:68)
at org.drools.decisiontable.DecisionTableProviderImpl.compileStream(DecisionTableProviderImpl.java:37)
at org.drools.decisiontable.DecisionTableProviderImpl.loadFromInputStream(DecisionTableProviderImpl.java:20)
at org.drools.compiler.DecisionTableFactory.loadFromInputStream(DecisionTableFactory.java:15)
at org.drools.compiler.PackageBuilder.decisionTableToPackageDescr(PackageBuilder.java:454)
at org.drools.compiler.PackageBuilder.addPackageFromDecisionTable(PackageBuilder.java:448)
at org.drools.compiler.PackageBuilder.addKnowledgeResource(PackageBuilder.java:690)
at org.drools.builder.impl.KnowledgeBuilderImpl.add(KnowledgeBuilderImpl.java:45)
at org.drools.builder.impl.KnowledgeBuilderImpl.add(KnowledgeBuilderImpl.java:34)
at org.drools.container.spring.beans.KnowledgeBaseBeanFactory.afterPropertiesSet(KnowledgeBaseBeanFactory.java:110)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624)
... 42 more
Environment:
Ubuntu 16.04.1 LTS
Flink 1.1.3
Kakfa 0.10.1.1
I'm trying to connect flink with kafka (Flink 1.1.3 Kakfa 0.10.1.1)
I already try all the fixes that i could find, but none of them work.
pom.xml :
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>ux</groupId>
<artifactId>logs</artifactId>
<version>1.3-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Flink Quickstart Job</name>
<url>http://www.myorganization.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<flink.version>1.3-SNAPSHOT</flink.version>
<slf4j.version>1.7.7</slf4j.version>
<log4j.version>1.2.17</log4j.version>
</properties>
<repositories>
<repository>
<id>apache.snapshots</id>
<name>Apache Development Snapshot Repository</name>
<url>https://repository.apache.org/content/repositories/snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka-0.10_2.10</artifactId>
<version>1.3-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>${log4j.version}</version>
</dependency>
</dependencies>
<profiles>
<profile>
<id>build-jar</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.11</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.11</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>${log4j.version}</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes combine.self="override"></excludes>
</artifactSet>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes>
<exclude>org.apache.flink:flink-annotations</exclude>
<exclude>org.apache.flink:flink-shaded-hadoop2</exclude>
<exclude>org.apache.flink:flink-shaded-curator-recipes</exclude>
<exclude>org.apache.flink:flink-core</exclude>
<exclude>org.apache.flink:flink-scala_2.11</exclude>
<exclude>org.apache.flink:flink-runtime_2.11</exclude>
<exclude>org.apache.flink:flink-optimizer_2.11</exclude>
<exclude>org.apache.flink:flink-clients_2.11</exclude>
<exclude>org.apache.flink:flink-avro_2.11</exclude>
<exclude>org.apache.flink:flink-examples-batch_2.11</exclude>
<exclude>org.apache.flink:flink-examples-streaming_2.11</exclude>
<exclude>org.apache.flink:flink-streaming-scala_2.11</exclude>
<exclude>org.apache.flink:flink-scala-shell_2.11</exclude>
<exclude>org.apache.flink:flink-python</exclude>
<exclude>org.apache.flink:flink-metrics-core</exclude>
<exclude>org.apache.flink:flink-metrics-jmx</exclude>
<exclude>org.apache.flink:flink-statebackend-rocksdb_2.11</exclude>
<exclude>log4j:log4j</exclude>
<exclude>org.scala-lang:scala-library</exclude>
<exclude>org.scala-lang:scala-compiler</exclude>
<exclude>org.scala-lang:scala-reflect</exclude>
<exclude>com.data-artisans:flakka-actor_*</exclude>
<exclude>com.data-artisans:flakka-remote_*</exclude>
<exclude>com.data-artisans:flakka-slf4j_*</exclude>
<exclude>io.netty:netty-all</exclude>
<exclude>io.netty:netty</exclude>
<exclude>commons-fileupload:commons-fileupload</exclude>
<exclude>org.apache.avro:avro</exclude>
<exclude>commons-collections:commons-collections</exclude>
<exclude>org.codehaus.jackson:jackson-core-asl</exclude>
<exclude>org.codehaus.jackson:jackson-mapper-asl</exclude>
<exclude>com.thoughtworks.paranamer:paranamer</exclude>
<exclude>org.xerial.snappy:snappy-java</exclude>
<exclude>org.apache.commons:commons-compress</exclude>
<exclude>org.tukaani:xz</exclude>
<exclude>com.esotericsoftware.kryo:kryo</exclude>
<exclude>com.esotericsoftware.minlog:minlog</exclude>
<exclude>org.objenesis:objenesis</exclude>
<exclude>com.twitter:chill_*</exclude>
<exclude>com.twitter:chill-java</exclude>
<exclude>commons-lang:commons-lang</exclude>
<exclude>junit:junit</exclude>
<exclude>org.apache.commons:commons-lang3</exclude>
<exclude>org.slf4j:slf4j-api</exclude>
<exclude>org.slf4j:slf4j-log4j12</exclude>
<exclude>log4j:log4j</exclude>
<exclude>org.apache.commons:commons-math</exclude>
<exclude>org.apache.sling:org.apache.sling.commons.json</exclude>
<exclude>commons-logging:commons-logging</exclude>
<exclude>commons-codec:commons-codec</exclude>
<exclude>com.fasterxml.jackson.core:jackson-core</exclude>
<exclude>com.fasterxml.jackson.core:jackson-databind</exclude>
<exclude>com.fasterxml.jackson.core:jackson-annotations</exclude>
<exclude>stax:stax-api</exclude>
<exclude>com.typesafe:config</exclude>
<exclude>org.uncommons.maths:uncommons-maths</exclude>
<exclude>com.github.scopt:scopt_*</exclude>
<exclude>commons-io:commons-io</exclude>
<exclude>commons-cli:commons-cli</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<artifact>org.apache.flink:*</artifact>
<excludes>
<exclude>org/apache/flink/shaded/com/**</exclude>
<exclude>web-docs/**</exclude>
</excludes>
</filter>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
my java code :
import java.util.Properties;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010;
import org.apache.flink.streaming.util.serialization.SimpleStringSchema;
public class App
{
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
Properties properties = new Properties();
properties.setProperty("bootstrap.servers", "localhost:9092");
properties.setProperty("zookeeper.connect", "localhost:2181");
properties.setProperty("group.id", "flink_consumer");
DataStream<String> messageStream = env.addSource(new FlinkKafkaConsumer010<>
("ux_logs", new SimpleStringSchema(), properties));
messageStream.rebalance().map(new MapFunction<String, String>() {
private static final long serialVersionUID = -6867736771747690202L;
public String map(String value) throws Exception {
return "Kafka and Flink says: " + value;
}
}).print();
env.execute();
}
}
But I get this error:
java.lang.NoClassDefFoundError: org/apache/flink/streaming/api/checkpoint/CheckpointedFunction
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at ux.App.main(App.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:509)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:403)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:320)
at org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:777)
at org.apache.flink.client.CliFrontend.run(CliFrontend.java:253)
at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1005)
at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1048)
Caused by: java.lang.ClassNotFoundException: org.apache.flink.streaming.api.checkpoint.CheckpointedFunction
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
Do i need to remove my kafka, and run an older version?
Is my flink kafka connector wrong?
i tried to use this plugin but it didn't work. (https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/linking.html)
Thanks.
You have to downgrade your connector:
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka-0.8_2.10</artifactId>
<version>1.1.2</version>
</dependency>
Here is the similar question: https://stackoverflow.com/a/40037895/1252056
If you want to connect to Kafka 0.10~ you will have to move to Flink 1.2, otherwise, as #streetturte mentioned, you will have to downgrade your Kafka connector.
Have a look at the reference table here:
https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/connectors/kafka.html
Kafka 0.9 and newer version don't need to zookeeper.
You need to upgrade your flink or downgrade Kafka version
--topic myTopic --bootstrap.servers 10.123.34.56:9092 --group.id myGroup
FlinkKafkaConsumer09<CarDB> consumerBinden = new FlinkKafkaConsumer09<CarDB>(
kafkaTopic,
new ClassClassSchema(),
parameterTool.getProperties());
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka-0.9_2.10</artifactId>
<version>${flink.version}</version>
</dependency>
Perhaps you are using Flink-Kafka consumer dependency for Scala in your pom file, whereas your code is in Java. Try flink kafka consumer for Java in your pom. Hope it helps.