Spring Integration - Apache ActiveMQ to Kafka - apache-kafka

Question

I am using the below configuration for integrating activemq with kafka. I receive message from activemq and forwards it to kafka. However, i am noticing that messages are getting dequeued from JMS Queue but messages are not going to kafka.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jms="http://www.springframework.org/schema/integration/jms"
xmlns:integration="http://www.springframework.org/schema/integration"
xmlns:int-kafka="http://www.springframework.org/schema/integration/kafka"
xmlns:task="http://www.springframework.org/schema/task"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration/jms
http://www.springframework.org/schema/integration/jms/spring-integration-jms.xsd
http://www.springframework.org/schema/integration/kafka
http://www.springframework.org/schema/integration/kafka/spring-integration-kafka.xsd">
<jms:message-driven-channel-adapter
id="helloJMSAdapater" destination="helloJMSQueue" connection-factory="jmsConnectionfactory"
channel="helloChannel" extract-payload="true" />
<integration:channel id="helloChannel" />
<integration:service-activator id="sayHelloServiceActivator"
input-channel="helloChannel" ref="sayHelloService" method="sayHello" />
<int-kafka:outbound-channel-adapter
id="kafkaOutboundChannelAdapter" kafka-template="template"
auto-startup="false" sync="true" channel="helloChannel" topic="test1234"
>
</int-kafka:outbound-channel-adapter>
<bean id="template" class="org.springframework.kafka.core.KafkaTemplate">
<constructor-arg>
<bean class="org.springframework.kafka.core.DefaultKafkaProducerFactory">
<constructor-arg>
<map>
<entry key="bootstrap.servers" value="localhost:9092" />
<!--entry key="retries" value="5" /> <entry key="batch.size" value="16384"
/> <entry key="linger.ms" value="1" /> <entry key="buffer.memory" value="33554432"
/> < entry key="key.serializer" value="org.apache.kafka.common.serialization.StringSerializer"
/> <entry key="value.serializer" value="org.apache.kafka.common.serialization.StringSerializer"
/ -->
</map>
</constructor-arg>
</bean>
</constructor-arg>
</bean>
</beans>
Also, in case there is any issue from Kafka, it is not even reporting any exception stack trace.
Did i miss anything ?

Answer 1

Your messages are consumed by sayHelloServiceActivator.
So change your helloChannel channel type to
<publish-subscribe-channel id="helloChannel"/>
Default is DirectChannel
The DirectChannel has point-to-point semantics but otherwise is more
similar to the PublishSubscribeChannel than any of the queue-based
channel implementations described above. It implements the
SubscribableChannel interface instead of the PollableChannel
interface, so it dispatches Messages directly to a subscriber. As a
point-to-point channel, however, it differs from the
PublishSubscribeChannel in that it will only send each Message to a
single subscribed MessageHandler.

Answer 2

As #Hassen Bennour says, if you want to send a message to two consumers, you need a publish/subscribe channel.
That said, you have auto-startup="false" on the kafka adapter, so it won't even be subscribed to the channel.
If it was started, with your current configuration messages would be sent round-robin alternately to the service activator and adapter.