Is there any example (somewhere) of spring-remoting working on top of (spring-) Kafka ?
There is no Kafka implementation of spring-remoting.
spring-remoting itself is deprecated as of Spring Framework 5.3.
https://github.com/spring-projects/spring-framework/commit/d9ccd618ea9cbf339eb5639d24d5a5fabe8157b5
You can, however, use the RPC support in Spring for Apache Kafka
https://docs.spring.io/spring-kafka/docs/current/reference/html/#replying-template
Related
Has anyone in Fineract-CN replaced ActiveMQ with Kafka?
I want to use Kafka in Fineract-CN modules, but the Spring Boot version is currently 1.4.1 and is not possible.
The Fineract-CN documentation says:
Apache ActiveMQ is used by default, but can be replaced with any Java Messaging Service (JMS) compatible message queue."
However, to my knowledge Kafka does not implement JMS. Therefore, it appears you won't be able to use Fineract-CN with Kafka.
I am creating a kafka streams api application in spring boot and I don't want to use spring-kafka dependency. Is it possible to use only apache kafka with "kafka-client" dependency?
Placing my kafka code in a #Component in spring boot and running it results in StreamsBuilderFactoryBean not found error every time, unless I include spring-kafka dependency.
What am I doing wrong? Is my approach even correct? (new to kafka and spring) :)
Any guidance is welcome!
I want suggestions of alternatives to Spring-Kafka.
I have used Kafka integrated in Spring in my application. I want to explore any alternative libraries available. Would be good if comparative analysis is share between libraries.
In addition to the plain Java API ... Put site:github.com + kafka in your favorite search engine...
wix/greyhound (seems pretty simple, and based on ZIO)
zio-kafka (ZIO is cool)
ExpediaGroup/rhapsody (Worth a look)
dropwizard/dropwizard-kafka (my personal 2nd choice)
Reactor Kafka (I think Spring-Kafka / Spring Cloud Streams kinda took over this project)
Akka Alpakka (Akka is a great framework)
SmallRye Reactive Messaging (with or without Quarkus) (one I would pick for new projects)
Micronaut Kafka (Good for webapps)
Vert.x Kafka client (Reactive, functional programming is awesome)
... probably more
While starting a topology with Kafka Spout with new Kafka version 2.1.0 and Storm version 1.2.2 getting a java.lang.ClassNotFoundException: kafka.api.OffsetRequest. I don't get this when I use Kafka version 0.10.0.1. Can you guys please help as I want to be on latest Kafka version?
I have tried all the latest kafka version starting with 2.*. But not of it works.Caused by: java.lang.ClassNotFoundException: kafka.api.OffsetRequest
kafka.api is the old Scala classes. Many of these were removed in Kafka 2.x
The majority of those classes were moved to org.apache.kafka.common.requests, and there is ListOffsetRequest and OffsetFetchRequest, so not sure which you're trying to use.
If Storm itself is depending on these old APIs, then you are bound to those, your own processor cannot use the new APIs.
Plus, the Kafka server version itself only supports certain API calls of these new request classes.
Adding to this answer, I suspect you're using the storm-kafka library for Kafka integration. You need to migrate to storm-kafka-client, which is based on the new Kafka APIs. The documentation for the new module can be found here.
If you need to migrate committed offsets from storm-kafka, you can use the utility at https://github.com/apache/storm/tree/master/external/storm-kafka-migration. It will let you migrate without having to start over on your Kafka partitions.
I have kafka broker upgraded from 0.8 to 0.11, and now I am trying to upgrade the spark streaming job code to be compatible with the new kafka -I am using spark 1.6.2-.
I searched a lot for steps to follow to do this upgrade I didn't find any article either official or not-official.
The only article I found useful is this one, however it is mentioning spark 2.2 and kafka 0.10, but I got a line saying
However, because the newer integration uses the new Kafka consumer API instead of the simple API, there are notable differences in usage. This version of the integration is marked as experimental, so the API is potentially subject to change
Do anyone have tried to integrate spark streaming 1.6 with kafka 0.11, or is it better to upgrade spark first to 2.X , since there is a lack of info about and support regarding this version mix of spark-streaming and kafka?
After lots of investigations, found no way to do this move, as spark-streaming only supporting kafka version up to 0.10 (which has major differences from kafka 0.11, 1.0.X).
That's why I decided to move from spark-streaming to use the new kafka-streaming api, and simply it was awesome, simple to use, very flexible, and the big advantage is that: IT IS A LIBRARY, you can simply add it to your project, not a framework that is wrapping your code.
Kafka-streaming api almost support all functionality provided by spark (aggregation, windowing, filtering, MR).