I am using Kafka Command Line tool to consume data from a Kafka topic, but this is not working as expected.
I am using the following command :
kafka-console-consumer.sh --bootstrap-server totkafka-console-consumer.sh --bootstrap-server localhost1:9092,localhost2:9092, localhost3:9092 --topic topic-name
and I am getting the following error stack:
Exception in thread "main" joptsimple.MultipleArgumentsForOptionException: Found multiple arguments for option bootstrap-server, but you asked for only one
at joptsimple.OptionSet.valueOf(OptionSet.java:179)
at kafka.tools.ConsoleConsumer$ConsumerConfig.<init>(ConsoleConsumer.scala:301)
at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:52)
at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
May you please help me with this?
Best regards
Syntax eror in your command, try below:
kafka-console-consumer.sh --bootstrap-server localhost1:9092,localhost2:9092,localhost3:9092 --topic topic-name
Related
we have kafka linux server with kafka version 2.6
I want the details about the Query of all Topics configuration
I am using bootstrap-server flag
here my approach ( 172.23.248.85 is Kafka IP )
kafka-configs.sh --describe --bootstrap-server 172.23.248.85:6667 --entity-type topics
Dynamic configs for topic ggt_opl are:
Error while executing config command with args '--describe --bootstrap-server 172.23.248.85:6667 --entity-type topics'
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_CONFIGS
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:104)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:272)
at kafka.admin.ConfigCommand$.getResourceConfig(ConfigCommand.scala:543)
at kafka.admin.ConfigCommand$.$anonfun$describeResourceConfig$4(ConfigCommand.scala:504)
at kafka.admin.ConfigCommand$.$anonfun$describeResourceConfig$4$adapted(ConfigCommand.scala:496)
at scala.collection.immutable.List.foreach(List.scala:333)
at kafka.admin.ConfigCommand$.describeResourceConfig(ConfigCommand.scala:496)
at kafka.admin.ConfigCommand$.describeConfig(ConfigCommand.scala:480)
at kafka.admin.ConfigCommand$.processCommand(ConfigCommand.scala:303)
at kafka.admin.ConfigCommand$.main(ConfigCommand.scala:96)
at kafka.admin.ConfigCommand.main(ConfigCommand.scala)
Caused by: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_CONFIGS
where I am wrong ?
other way that is working on my Kafka is:
kafka-topics.sh --zookeeper zoo_server:2181 --describe
but I want to know if we can get the described details with --bootstrap-server
we also try:
kafka-topics.sh --bootstrap-server=172.23.248.85:6667 --describe
Error while executing topic command : The broker does not support DESCRIBE_CONFIGS
[2021-09-02 09:35:34,054] ERROR org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_CONFIGS
(kafka.admin.TopicCommand$)
but on other way the following option is working:
kafka-topics.sh --bootstrap-server=172.23.248.85:6667 --list
so I not understand why ( --list ) works but ( --describe ) isn't ?
--bootstrap-servers + --describe uses the AdminClient class, which your brokers need to be a certain version to support, as the error says.
Related KIP-133 (implemented in Kafka 0.11, according to JIRA)
I am getting an error while reading a topic from a single node cluster
kafka-console-producer.bat -- broker-list localhost:9092 -- topic raja
Error below
Missing required argument "[topic]"
kafka version-kafka_2.13-2.4.0
I suggest you do more research and training.
You use kafka CONSUMER to read from a topic, NOT PRODUCER,
and also, as #Robin Moffatt mentioned while I was typing the answer,
remove spaces between the parameter labels:
kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic raja
This should work.
If you want to read the topic from the beginning then add this as well:
--from-beginning
You've got a space between the parameter labels, so it won't work. Also note consumer not producer if you're reading from the topic.
Try:
kafka-console-consumer.bat --broker-list localhost:9092 --topic raja
Ref: https://kafka.apache.org/documentation/#quickstart_consume
I’m having an issue with using the Kafka command line tools to get consumer lag for a given group/topic. Currently, I'm trying to use kafka-consumer-groups.sh as mentioned in countless online resources. The following command works just fine: kafka-consumer-groups.sh --bootstrap-server $BROKERS --list
However, if I use kafka-consumer-groups.sh --bootstrap-server $BROKERS --describe --group $GROUP, I get the following output and error:
Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).
[2018-07-24 00:19:09,139] ERROR admin-client-network-thread exited (kafka.admin.AdminClient) java.lang.NullPointerException
at org.apache.kafka.common.utils.Utils.join(Utils.java:399)
at org.apache.kafka.common.requests.OffsetFetchRequest$Builder.toString(OffsetFetchRequest.java:74)
at java.lang.String.valueOf(String.java:2994)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at org.apache.kafka.clients.ClientRequest.toString(ClientRequest.java:63)
at org.apache.kafka.clients.NetworkClient.doSend(NetworkClient.java:374)
at org.apache.kafka.clients.NetworkClient.send(NetworkClient.java:332)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.trySend(ConsumerNetworkClient.java:409)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:252)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:208)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:199)
at kafka.admin.AdminClient$$anon$1.run(AdminClient.scala:61)
at java.lang.Thread.run(Thread.java:748) Error: Executing consumer group command failed due to The server experienced an unexpected error when processing the request
I've tried setting GROUP to each and every group available and I've used KafkaTool to confirm that these groups exist and are working properly (consuming messages from various topics). I've tried placing strings in the command directly instead of using an environment variable.
Why am I getting this error and what else can I do to debug?
I am getting "Error: Executing consumer group command failed due to null" when running the following command
./kafka-consumer-groups.bat --zookeeper 192.168.99.101:2181 --describe --group console-consumer-62252
Googled it but couldn't find any solution
Not sure if it's still relevant, but the new Kafka using bootstrap-server instead of zookeeper, try this one:
./kafka-consumer-groups.sh --bootstrap-server 192.168.99.101:9092 --describe --group console-consumer-62252
Try to use list of bootstrap and see if --new-consumer work
sh kafka-consumer-groups --bootstrap-server broker1host:port1,broker2host:port2... --new-consumer --describe --group console-consumer-62252
or without --new-consumer if you are using kafka 2.8.0+
sh kafka-consumer-groups --bootstrap-server broker1host:port1,broker2host:port2... --describe --group console-consumer-62252
FWIW since this was the first result when I googled Error: Executing consumer group command failed due to null, I thought I'd record that plain old PEBCAK can also cause this error. I got it and puzzled over it for a few minutes before realising I was trying to reset offsets for a consumer group on a topic, and had a typo in the topic name...
I am trying to read a file using kafka producer.Zookeeper and Broker server are running. I am able to read inputs from command prompt using Kafka producer and Consumers using below commands -
Kafka Producer
kafka-console-producer --topic incoming --broker localhost:9092
Kafka Consumer
kafka-console-consumer --topic incoming --zookeeper localhost:2181
For reading from file i had tried below command line arguments -
kafka-console-producer -–broker-list localhost:9092 -–topic incoming --new-producer < C:\abc.txt
but it produced below error -
û is not a recognized option
I googled the message and it says about correcting the producer command which looks correct to me.
For kafka-10 you don't need to pass --new-producer flag. Following command is working for me:
kafka-console-producer.sh --broker-list localhost:9092 --topic incoming < C:\abc.txt