We are trying to build a data-torrent application using kafka as a source that will consume messages from web UI using REST.
Can someone provide a good documentation or some steps how to setup kafka produce using java REST API ?
Thanks
Check out kafka-rest.
It will let you produce messages to a Kafka topic with a REST API in JSON or Avro. It is in Java and you may be able to use it out-of-the-box if you don't have special requirements.
Alternatively, if you don't want to use Confluent REST Proxy for Kafka, you can run a web application using a Web Framework which will call into a Kafka Client library during an HTTP request from the client.
Related
I am new to Kafka .I have implemented my consumer as normal Java springboot application.I need to connect to the topic deployed on remote broker using Kafka rest proxy.
I am not able to understand how it will function differently if i use Kafka rest proxy.Where i should do change in my code to include the rest proxy.Do i need to structure my code complete different as i didn't think about rest proxy while creation.
I maybe wrong with the terminologies.
Any help or guidance would be of great help.
REST proxy would be used with any HTTP client, not a Kafka consumer (so create a WebClient bean rather than a ConsumerFactory, etc)
You can refer its documentation for how you can consume records over HTTP, but, simply put, the code will be completely different up until you parse the data
I've made a REST connector to push NodeJS Service response to Kafka but now I want to write or use a REST sink connector to expose my API response stored as message.I'm able to get Kafka message over websocket, what is standard way to expose other than API Gateway using plain Apache kafka(no third party platform)
works with WebSocket but want to expose as API Gateway or something reliable usign kafka
No third part platform just Apache kafka
As of now we are using curl and GET call to get data from outside using their ENDPOINT URL.
i'm planning to setup a new process and is there anyway to leverage kafka here instead of CURL.
unfortunately we dont have kafka confluent version.
Kafka doesn't perform or accept HTTP calls.
You'd have to write some HTTP scraper, then combine this with a Kafka producer.
Note: You don't need Confluent Platform to just setup and run their REST proxy next to your existing brokers
I'm new with kafka and I'm trying to publish data from external application via http but I cannot find the way to do this.
I already created a topic in kafka and test it producing and consuming the message but I don't know how to insert/publish message via http, I tried to invoke the following url to retrieve the topics but it does not retrieve any data http://servername:2181/topics/
I'm using cloudera 5.12.1.
You can access to your topics, if it was already created, using APIs. The easy way...(see client list)
Or see Connects Config to manage connectors by REST (rest.host.name, rest.port parameters). But only connectors...
To consume or produce message in a topic, use a middleware. IT is more feaseble.
Check out the open source Kafka REST Proxy from Confluent. It does exactly what you want.
You can get it standalone, or as part of Confluent Platform.
I have a requirement where messages are coming in json format from ACTIVEMQ and I have to expose end point where I can receive messages. I am using KAFKA but don't know whether can I use KAFKA REST API to receive json messages and store them against a particular topic in Kafka broker?
There is no REST API in the core Apache Kafka distribution but there is a very good open source REST Proxy for Kafka available from Confluent. It enables both publish and subscribe via REST APIs. The code and docs are on github at https://github.com/confluentinc/kafka-rest or you can download the entire Confluent open source platform which includes Apache Kafka, REST Proxy, Schema Registry and a few other useful enhancements at http://www.confluent.io/download/