I am new to kafka and following this tutorial https://www.tutorialspoint.com/apache_kafka/apache_kafka_basic_operations.htm
I have installed and configure everything zookeeper, kafka etc and it's working fine. But when I run any command my output terminal window close within second, so I am not able to see output. Is there property to set output window time?
kafka-topics.sh --list --zookeeper localhost:2181
I used this command to list topic but listed topics window closed not given enough time me to see.
Powershell doesn't run .sh scripts. If CMD or Powershell are closing its because you're getting a fatal exception and should really be using bin\windows\kafka-topics.bat, which ran on its own with no arguments should print help text, and not close any window
Also relevant - Run Kafka on WSL2 instead of direct in windows if you want to avoid LogCleaner errors you'll eventually find with file permissions
I also suggest following the official Kafka website documentation since the Zookeeper flag is deprecated
Related
I started a kafka broker on my computer. I also ran zookeeper. Once i was playing with the commands to see consumer groups or so like, suddenly kafka got time out exception. When i restarted it, all the topics were gone (i mean i couldn't get them or watch them through command line). But when i checked the kafka-logs dir, they were all there. I also created a new topic and it was shown in cmd and interestingly it's log was near other topic logs that faced the problem mentioned. Any idea? I appreciate that.
i want to start my very first kafka, but when i tried to run this on my kafka_2.13-2.8.0 directory bin\windows\zookeeper-server-start.bat .. \ .. \config\zookeeper.properties
why it returns \Kafka\kafka_2.13-2.8.0\bin\windows\../ ../config/log4j.properties was unexpected at this time
idk i already followed this tip to install kafka https://www.youtube.com/watch?v=bYVyRh4C94E&t=303s
It's a known error in the Kafka log4j settings, especially if the install path contains spaces or non alphanumeric characters
If you really want to run Kafka on Windows, you should use WSL2 anyway, or Docker. Otherwise, assuming you did get the bat file working, you'd eventually run into other errors that crash the broker
I am currently using windows machine and able to read whole file through command prompt using Kafka producer and consumer. I need to only get the recent changes in a file and need to use it in as input for Apache flink. I tried using this link but due to kafka client jar mismatch issue, i was not able to use it.
In my current approach when i call my producer each time it loads the whole file and we need to run it every time to see the changes occurred to file. I thought of using threads and some way of comparing difference in file using java code but is there any of doing only by Kafka.
I had similar problem recently (but in Linux) and solved it following way:
tail -f somefile.log | kafka-console-producer.sh ...
In your case you can try some Windows alternatives to Linux's tail: 13 Ways to Tail a Log File on Windows & Linux
As in the title, I'm following the initial Kafka tutorial - have downloaded & unpacked the Kafka package and am now trying to start the Zookeeper server from the command prompt.
I'm getting the behaviour described in the title - no logs. Can anyone help?
I am newbie in Kafka. I want to consume remote kafka message in shell script. Basically I have linux machine where I cannot run any web server (some strange reasons) only thing I can do is use crontab/shell script to listen for kafka message which is hosted remotely. Is it possible to write simple shell script which will consume kafka message, parse it and take corresponding action.
kafka clients are available in multiple languages. you can use any client, you don't need any web server or browser for it.
you may use shell script for consuming message & parsing but that script have to use any kafka client provided here because currently there is no client written in pure shell script.
Kafka has provided kafka client console producer and consumer, you can use that as well.
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
follow the document properly.
You could also use the tool kafkacat which is documented for example here.
This is a very powerful and fast tool to read data out of Kafka from the console and is open source: https://github.com/edenhill/kafkacat.
Many exmples are provided on GitHub and one example is shown below:
kafkacat -C -b mybroker -t mytopic