Difference between Spring Cloud Kafka Streams Vs Spring Cloud Stream? - apache-kafka

Question

Whats the difference between Spring Cloud Kafka Streams Vs Spring Cloud Stream Vs Spring Cloud Function Vs Spring AMQP and Spring for Apache Kafka?

Answer 1

Spring for Apache Kafka and Spring AMQP are foundational libraries for writing Spring friendly applications for Apache Kafka and AMQP respectively. They provide design patterns such as templates, message listener containers, and a wide array of other mechanisms to interact with the middleware systems at a lower level. These libraries do not require Spring Boot, but Spring Framework is the least common denominator. In other words, you can write a traditional Spring application with only Spring Framework contexts using these libraries.
Spring Cloud Function is a library that is part of the Spring Cloud portfolio projects. This is used as part of Spring Boot applications. It gives a consistent programming model for writing applications that involve various paradigms such as request-response (HTTP), event-driven (pub-sub), stream-processing (pub-sub/streaming), reactive streams, etc. The programming model at the application level is through the Java 8 functional model - for example you can write your business logic as a java.util.function.Function<?, ?>. Spring Cloud Function is not coupled with any middleware or other such technologies.
Spring Cloud Stream is another Spring Cloud project that is specifically built for event-driven and stream-processing usecases. Because this is a Spring Cloud project, it requires to be used as part of a Spring Boot application. The recent versions of Spring Cloud Stream is built on the foundations that Spring Cloud Function provides. This is essentially a destination binding framework that allows you to provide a destination - such as a Kafka topic or a RabbitMQ exchange. Spring Cloud Stream will bind those destinations for the application. The core Spring Cloud Stream does not have any middleware dependencies. That's where the binder implementations come in.
Spring Cloud Stream provides two kinds of Kafka binders - spring-cloud-stream-binder-kafka and spring-cloud-stream-binder-kafka-streams. The first one is a binder implementation where it provides programming model support for writing regular Kafka producers and consumers. For the most part, you can take this same application and provide another binder (such as spring-cloud-stream-binder-rabbit) and it should work (provided that the application makes the right configuration changes). This is because the binders are the ones concerned with lower-level details of communicating to the middleware and not the app itself. Apps can largely focus on the business logic at hand. The Kafka Streams binder in Spring Cloud Stream is a binder implementation specifically built for writing streaming applications using Kafka Streams. Both Kafka binder implementations use Spring for Apache Kafka under the hood.
The rabbit binder in Spring Cloud Stream uses Spring AMQP internally.
To summarize:
Spring for Apache Kafka/Spring AMQP - lower-level foundational libraries, do not require Spring Boot.
Spring Cloud Function - Spring Cloud project providing Java 8 functional programming model, Used with Spring Boot applicaitons.
Spring Cloud Stream - Framework for event-driven applications using Spring Cloud Function. Used with Spring Boot applications.
Spring Cloud Stream Kafka/Kafka Streams - Spring Cloud Stream binder implementation using Spring for Apache Kafka. Used with Spring Boot applications.