I use Flink SQL (version 1.11) and would like to process data from Kafka. For this I wrote a job from the scala template and added the dependency to pom.xml.
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-sql-connector-kafka_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
When I want to run the job in the cluster, I get the following error:
Caused by: org.apache.flink.table.api.ValidationException: Could not find any factory for identifier 'kafka' that implements 'org.apache.flink.table.factories.DynamicTableSinkFactory' in the classpath.
Available factory identifiers are:
blackhole
print
If I add the flink-sql-connector-kafka jar to the /lib folder it works but then can't use the SQL client because it then loads once from its own lib folder and this connector and it is already loaded in the cluster. Then comes following error:
java.lang.ClassCastException: cannot assign instance of org.apache.commons.collections.map.LinkedMap to field org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.pendingOffsetsToCommit of type org.apache.commons.collections.map.LinkedMap in instance of org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer
How can I add the flink-sql-connector-kafka to the fat-jar? Or should these SQL connectors rather be added to the /lib folder?
Related
I am configuring spring config server with control bus(rabbitmq) but when i add its dependency the application failed to start with error:
APPLICATION FAILED TO START
Description:
An attempt was made to call a method that does not exist. The attempt was made from the following location:
org.springframework.cloud.stream.binder.rabbit.RabbitMessageChannelBinder.createConsumerEndpoint(RabbitMessageChannelBinder.java:517)
The following method did not exist:
'void org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter.<init>(org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer)'
The method's class, org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter, is available from the following locations:
jar:file:/D:/maven/Repo/org/springframework/integration/spring-integration-amqp/5.5.2/spring-integration-amqp-5.5.2.jar!/org/springframework/integration/amqp/inbound/AmqpInboundChannelAdapter.class
The class hierarchy was loaded from the following locations:
org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter: file:/D:/maven/Repo/org/springframework/integration/spring-integration-amqp/5.5.2/spring-integration-amqp-5.5.2.jar
org.springframework.integration.endpoint.MessageProducerSupport: file:/D:/maven/Repo/org/springframework/integration/spring-integration-core/5.5.2/spring-integration-core-5.5.2.jar
org.springframework.integration.endpoint.AbstractEndpoint: file:/D:/maven/Repo/org/springframework/integration/spring-integration-core/5.5.2/spring-integration-core-5.5.2.jar
org.springframework.integration.context.IntegrationObjectSupport: file:/D:/maven/Repo/org/springframework/integration/spring-integration-core/5.5.2/spring-integration-core-5.5.2.jar
Action:
Correct the classpath of your application so that it contains a single, compatible version of org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter
Using spring cloud version <spring-cloud.version>2020.0.3</spring-cloud.version> and spring boot version 2.5.3 and my pom.xml contains following dependencies (unrelated ones are removed)
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-monitor</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bus-amqp</artifactId>
</dependency>
It's a known issue; fixed in spring-integration-amqp 5.5.3 (Boot 2.5.4).
https://github.com/spring-projects/spring-integration/issues/3606
For a scala project I use the StreamTableEnvironment and when running my code in IntelliJ everything works fine. However when I try to export my project to a jar (I create a fat jar using sbt assembly), no suitable table factory can be found. I've looked inside the jar and the classes it needs are included. Here the complete stacktrace:
Exception in thread "main" org.apache.flink.table.api.TableException: Could not instantiate the executor. Make sure a planner module is on the classpath
at org.apache.flink.table.api.scala.internal.StreamTableEnvironmentImpl$.lookupExecutor(StreamTableEnvironmentImpl.scala:328)
at org.apache.flink.table.api.scala.internal.StreamTableEnvironmentImpl$.create(StreamTableEnvironmentImpl.scala:284)
at org.apache.flink.table.api.scala.StreamTableEnvironment$.create(StreamTableEnvironment.scala:366)
at org.tudelft.plugins.SQLService$.setupEnv(SQLService.scala:40)
at org.tudelft.plugins.SQLStage.main(SQLStage.scala:19)
at org.codefeedr.stages.OutputStage.transform(OutputStage.scala:45)
at org.codefeedr.pipeline.Pipeline.$anonfun$startMock$1(Pipeline.scala:240)
at org.codefeedr.pipeline.Pipeline.$anonfun$startMock$1$adapted(Pipeline.scala:238)
at scala.collection.Iterator.foreach(Iterator.scala:941)
at scala.collection.Iterator.foreach$(Iterator.scala:941)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
at org.codefeedr.pipeline.Pipeline.startMock(Pipeline.scala:238)
at org.tudelft.Main$.main(Main.scala:34)
at org.tudelft.Main.main(Main.scala)
Caused by: org.apache.flink.table.api.NoMatchingTableFactoryException: Could not find a suitable table factory for 'org.apache.flink.table.delegation.ExecutorFactory' in
the classpath.
Reason: No factory implements 'org.apache.flink.table.delegation.ExecutorFactory'.
The following properties are requested:
class-name=org.apache.flink.table.executor.StreamExecutorFactory
streaming-mode=true
The following factories have been considered:
at org.apache.flink.table.factories.TableFactoryService.filterByFactoryClass(TableFactoryService.java:243)
at org.apache.flink.table.factories.TableFactoryService.filter(TableFactoryService.java:186)
at org.apache.flink.table.factories.TableFactoryService.findAllInternal(TableFactoryService.java:172)
at org.apache.flink.table.factories.TableFactoryService.findAll(TableFactoryService.java:126)
at org.apache.flink.table.factories.ComponentFactoryService.find(ComponentFactoryService.java:48)
at org.apache.flink.table.api.scala.internal.StreamTableEnvironmentImpl$.lookupExecutor(StreamTableEnvironmentImpl.scala:312)
... 16 more```
Similar but slightly different to what OP needed: I have unit tests using the Table API that were failing with the same error message, even though the same pipeline worked fine when submitted to a real flink cluster.
org.apache.flink.table.api.TableException: Could not instantiate the executor. Make sure a planner module is on the classpath
The solution was to add:
<dependency>
<!-- this is needed to use the Table API from unit tests -->
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>test</scope>
</dependency>
I believe flink-table-planner-blink is no longer available in recent versions (I'm using 1.15), and has instead replaced flink-table-planner.
Probably a bit late to answer, but I had the same issue and the solution was to add the blink planner dependency (original answer here: https://issues.apache.org/jira/browse/FLINK-14031)
It's useful for me
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner-blink_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
I have created a business process and build it as a jar in Kie Workbench (version 7.16.0.Final).
When trying to execute the process I got the Runtime exception.
2019-01-20 21:35:57.092 [http-nio-9043-exec-8] ERROR o.a.c.c.C.[.[.[.[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.RuntimeException: Unknown resource type: ResourceType = 'jBPM BPMN2 Language'] with root cause
java.lang.RuntimeException: Unknown resource type: ResourceType = 'jBPM BPMN2 Language'
at org.kie.internal.services.KieAssemblersImpl.addResource(KieAssemblersImpl.java:51)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.addPackageForExternalType(KnowledgeBuilderImpl.java:778)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.addKnowledgeResource(KnowledgeBuilderImpl.java:763)
at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl$ResourceBuilder.lambda$static$6(CompositeKnowledgeBuilderImpl.java:307)
at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildResourceType(CompositeKnowledgeBuilderImpl.java:141)
at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildProcesses(CompositeKnowledgeBuilderImpl.java:121)
at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:112)
at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:100)
at org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:251)
at org.drools.compiler.kie.builder.impl.AbstractKieModule.buildKnowledgePackages(AbstractKieModule.java:201)
at org.drools.compiler.kie.builder.impl.AbstractKieModule.createKieBase(AbstractKieModule.java:214)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.createKieBase(KieContainerImpl.java:406)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.getKieBase(KieContainerImpl.java:374)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.getKieBaseFromKieSessionModel(KieContainerImpl.java:575)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:551)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:521)
I faced this same issue (although in KIE/Drools 7.17.0.Final) and the problem was a missing dependency. In my case I had Drools/KIE/JBPM as a JBoss module and didn't have all required JARs declared in module.xml.
What I did to fix it was to add the following two JARs to module.xml (and of course, put the JARs into the module's main folder):
jbpm-bpmn2-7.17.0.Final.jar
jbpm-flow-builder-7.17.0.Final.jar
You can find these JARs under Drools and jBPM tools distribution.
When using as runtime version one of the latest:
<runtime.version>7.42.0.Final</runtime.version>
Then, it's enough to have the following set of libraries (which are automatically added if you create a Drools project with Eclipse+Drools plugin):
<dependency>
<groupId>org.kie</groupId>
<artifactId>kie-api</artifactId>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-core</artifactId>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-decisiontables</artifactId>
</dependency>
<dependency>
<groupId>org.jbpm</groupId>
<artifactId>jbpm-test</artifactId>
</dependency>
There's a sample demo project here: Using Drools Rules with jBPM
I am using JDK 1.7 with Maven 3.0 for my project.
When I have the following in pom.xml
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
<version>6.0</version>
<scope>compile</scope>
</dependency>
I am getting the following while deploying to Weblogic 10.3.6 integrated with Eclipse
WARN (org.hibernate.cfg.beanvalidation.TypeSafeActivator;141) HHH000274:
Unable to apply constraints on DDL for test.entity.Employee
java.lang.NullPointerException at
org.hibernate.cfg.beanvalidation.TypeSafeActivator.applyDDL(TypeSafeActivator.java:172)
If I remove javaee-api dependency, then I do have issues when I compile project using mvn compile with errors like
[ERROR] /domain/view/EmployeeNameConverter.java:[29,76] cannot access
javax.el.ELContext
class file for javax.el.ELContext not found
[ERROR] /EmployeeNameConverter.java:[28,55] cannot access javax.el.ELResolver
class file for javax.el.ELResolver not found
More details here
How can I resolve this issue?
Like we've worked out in the comments Weblogic 11g doesn't support Java EE6. So one solution can be using Java EE5 instead.
I'm using Apache CXF version 2.4.0. I'm trying to create a Restful service.
The below example works in 2.3.4 but does not work in 2.4.0. What should I do different in beans config file?
When I include the below xml line in my bean config file.
<import resource="classpath:META-INF/cxf/cxf-extension-jaxrs-binding.xml" />
I get the following stacktrace error:
org.springframework.beans.factory.parsing.BeanDefinitionParsingException:
Configuration problem: Failed to
import bean definitions from URL
location
[classpath:META-INF/cxf/cxf-extension-jaxrs-binding.xml]
Offending resource: ServletContext
resource [/WEB-INF/beans.xml]; nested
exception is
org.springframework.beans.factory.BeanDefinitionStoreException:
IOException parsing XML document from
class path resource
[META-INF/cxf/cxf-extension-jaxrs-binding.xml];
nested exception is
java.io.FileNotFoundException: class
path resource
[META-INF/cxf/cxf-extension-jaxrs-binding.xml]
cannot be opened because it does not
exist at
org.springframework.beans.factory.parsing.FailFastProblemReporter.error(FailFastProblemReporter.java:68)
My POM dependency is below. This will work for 2.3.4 but not for 2.4.0. Any suggestions? Is that xml extension line deprecated or included in another jar?
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-bundle-jaxrs</artifactId>
<version>2.3.4</version>
<exclusions>
<exclusion>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
</exclusion>
</exclusions>
</dependency>
It's no longer needed. With 2.4.x, most of the META-INF/cxf files are no longer needed and can be removed. Really, the cxf.xml one is the only one needed.
Here is the CXF information about embedding cxf inside spring in version 2.4 and above.
http://cxf.apache.org/docs/embedding-cxf-inside-spring.html