I am getting "Error: Executing consumer group command failed due to null" when running the following command
./kafka-consumer-groups.bat --zookeeper 192.168.99.101:2181 --describe --group console-consumer-62252
Googled it but couldn't find any solution
Not sure if it's still relevant, but the new Kafka using bootstrap-server instead of zookeeper, try this one:
./kafka-consumer-groups.sh --bootstrap-server 192.168.99.101:9092 --describe --group console-consumer-62252
Try to use list of bootstrap and see if --new-consumer work
sh kafka-consumer-groups --bootstrap-server broker1host:port1,broker2host:port2... --new-consumer --describe --group console-consumer-62252
or without --new-consumer if you are using kafka 2.8.0+
sh kafka-consumer-groups --bootstrap-server broker1host:port1,broker2host:port2... --describe --group console-consumer-62252
FWIW since this was the first result when I googled Error: Executing consumer group command failed due to null, I thought I'd record that plain old PEBCAK can also cause this error. I got it and puzzled over it for a few minutes before realising I was trying to reset offsets for a consumer group on a topic, and had a typo in the topic name...
Related
we have kafka linux server with kafka version 2.6
I want the details about the Query of all Topics configuration
I am using bootstrap-server flag
here my approach ( 172.23.248.85 is Kafka IP )
kafka-configs.sh --describe --bootstrap-server 172.23.248.85:6667 --entity-type topics
Dynamic configs for topic ggt_opl are:
Error while executing config command with args '--describe --bootstrap-server 172.23.248.85:6667 --entity-type topics'
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_CONFIGS
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:104)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:272)
at kafka.admin.ConfigCommand$.getResourceConfig(ConfigCommand.scala:543)
at kafka.admin.ConfigCommand$.$anonfun$describeResourceConfig$4(ConfigCommand.scala:504)
at kafka.admin.ConfigCommand$.$anonfun$describeResourceConfig$4$adapted(ConfigCommand.scala:496)
at scala.collection.immutable.List.foreach(List.scala:333)
at kafka.admin.ConfigCommand$.describeResourceConfig(ConfigCommand.scala:496)
at kafka.admin.ConfigCommand$.describeConfig(ConfigCommand.scala:480)
at kafka.admin.ConfigCommand$.processCommand(ConfigCommand.scala:303)
at kafka.admin.ConfigCommand$.main(ConfigCommand.scala:96)
at kafka.admin.ConfigCommand.main(ConfigCommand.scala)
Caused by: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_CONFIGS
where I am wrong ?
other way that is working on my Kafka is:
kafka-topics.sh --zookeeper zoo_server:2181 --describe
but I want to know if we can get the described details with --bootstrap-server
we also try:
kafka-topics.sh --bootstrap-server=172.23.248.85:6667 --describe
Error while executing topic command : The broker does not support DESCRIBE_CONFIGS
[2021-09-02 09:35:34,054] ERROR org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_CONFIGS
(kafka.admin.TopicCommand$)
but on other way the following option is working:
kafka-topics.sh --bootstrap-server=172.23.248.85:6667 --list
so I not understand why ( --list ) works but ( --describe ) isn't ?
--bootstrap-servers + --describe uses the AdminClient class, which your brokers need to be a certain version to support, as the error says.
Related KIP-133 (implemented in Kafka 0.11, according to JIRA)
I am using Kafka Command Line tool to consume data from a Kafka topic, but this is not working as expected.
I am using the following command :
kafka-console-consumer.sh --bootstrap-server totkafka-console-consumer.sh --bootstrap-server localhost1:9092,localhost2:9092, localhost3:9092 --topic topic-name
and I am getting the following error stack:
Exception in thread "main" joptsimple.MultipleArgumentsForOptionException: Found multiple arguments for option bootstrap-server, but you asked for only one
at joptsimple.OptionSet.valueOf(OptionSet.java:179)
at kafka.tools.ConsoleConsumer$ConsumerConfig.<init>(ConsoleConsumer.scala:301)
at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:52)
at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
May you please help me with this?
Best regards
Syntax eror in your command, try below:
kafka-console-consumer.sh --bootstrap-server localhost1:9092,localhost2:9092,localhost3:9092 --topic topic-name
When I run the command:
kafka-consumer-groups -bootstrap-server localhost:9092 -list
It produces a list of consumer groups. Here's some lengthy output:
$ kafka-consumer-groups --bootstrap-server localhost:9092 --list
Note: This will not show information about old Zookeeper-based consumers.
simulate_birdfeeder.e67c034c-a96d-11e9-861a-983b8f0e47c8
xfer_server.e65d3732-a96d-11e9-992f-983b8f0e47c8
xfer_server.e654e596-a96d-11e9-992f-983b8f0e47c8
scheduler_node.e6ba65ce-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e695c336-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e681a4c8-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6b936f4-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e6956c2e-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6de0222-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e68ec02c-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6b48f82-a96d-11e9-b125-983b8f0e47c8
scheduler_node.e6b436cc-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e68fed3a-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e658faf0-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e64f28ea-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e691a710-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e6889f30-a96d-11e9-861a-983b8f0e47c8
74b6a2e3-efe6-4a62-ad6b-a4782038db43
simulate_birdfeeder.e67ec7bc-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e67b9178-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e65b4b2a-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6dcad1e-a96d-11e9-b125-983b8f0e47c8
water_connector
simulate_birdfeeder.e6924904-a96d-11e9-861a-983b8f0e47c8
reporter
xfer_server.e652ee8a-a96d-11e9-992f-983b8f0e47c8
scheduler_node.e68d3446-a96d-11e9-b125-983b8f0e47c8
simulate_flexcharger.e560bef8-a96d-11e9-a249-983b8f0e47c8
511bcf37-7e6b-4a33-9a57-9ad498f9a089
simulator.e5d05a10-a96d-11e9-9375-983b8f0e47c8
trajectory.queue
simulate_birdfeeder.e662497a-a96d-11e9-861a-983b8f0e47c8
scheduler
simulate_birdfeeder.e6874ca2-a96d-11e9-861a-983b8f0e47c8
85f138e2-bb87-4885-98b8-a6536595cf7c
scheduler_node.e69bcd9e-a96d-11e9-b125-983b8f0e47c8
scheduler_node.e69b0c42-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e6612c7a-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e691649e-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e665d02c-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e6554ab8-a96d-11e9-861a-983b8f0e47c8
xfer_server.e65b81ee-a96d-11e9-992f-983b8f0e47c8
server.e6159012-a96d-11e9-bbdf-983b8f0e47c8
simulate_birdfeeder.e666c9b4-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e67dffb2-a96d-11e9-861a-983b8f0e47c8
KMOffsetCache-my-computer
simulate_birdfeeder.e697c230-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e683e0bc-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e65fbd40-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e69706c4-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e69cc438-a96d-11e9-b125-983b8f0e47c8
xfer_server.e64a93de-a96d-11e9-992f-983b8f0e47c8
scheduler_node.e6a55e72-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e685be50-a96d-11e9-861a-983b8f0e47c8
product_monitor
simulate_birdfeeder.e688f89a-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e66b29fa-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e6811242-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6a62be0-a96d-11e9-b125-983b8f0e47c8
scheduler_node.e6b9fa58-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e68a2986-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e690480c-a96d-11e9-861a-983b8f0e47c8
server.e60fec02-a96d-11e9-bbdf-983b8f0e47c8
scheduler_node.e6a45856-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e68385ae-a96d-11e9-861a-983b8f0e47c8
server
simulate_birdfeeder.e668c246-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e65f11a6-a96d-11e9-861a-983b8f0e47c8
xfer_server.e64f2bb0-a96d-11e9-992f-983b8f0e47c8
scheduler_node.e6b8e834-a96d-11e9-b125-983b8f0e47c8
scheduler_node.e6ddb13c-a96d-11e9-b125-983b8f0e47c8
Of which I can describe them:
$ kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group server
Note: This will not show information about old Zookeeper-based consumers.
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
reservationist.queue 0 456 456 0 rdkafka-fa9be154-c7ac-471f-a9d1-5bfd3f42da8c /127.0.0.1 rdkafka
However, for some consumer groups, if I describe them, there is no output at all. I'll get something like the following:
$ kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group scheduler_node.e6ddb13c-a96d-11e9-b125-983b8f0e47c8
Note: This will not show information about old Zookeeper-based consumers.
Aside from the warning, there is no output, but the consumer does exist, because it is listed in the former list command. Now, if I use the zookeeper option, the command complains about the group not existing.
Often the consumers that look like widget has a description of offsets, and the ones that look like widget.some-long-random-string have no output. And consumers that look like some-long-random-string also produce a description with offsets.
(I'm not 100% certain, but I'm pretty sure the missing consumer groups are subscribed to, otherwise my application would be bombing, which isn't the case.)
I am using confluent kafka locally. My code and confluent are all running in my dev box:
$ confluent version kafka
This CLI is intended for development only, not for production
https://docs.confluent.io/current/cli/index.html
1.1.1-cp1
Can any give me some insight as to what is going on? Thanks.
kafka-consumer-groups \
--bootstrap-server localhost:9092 \
--describe \
--group your_consumer_group_name
will return no output if the consumers in the consumer group have not committed any offsets.
If the output of
bin/kafka-consumer-groups.sh \
--bootstrap-server localhost:9092 \
--describe \
--group server
and
bin/kafka-consumer-groups.sh \
--bootstrap-server localhost:9092 \
--new-consumer
--describe \
--group server
matches and the consumer group appears in both consumer groups list output, my guess is that your consumers inside the consumer group server have not committed any offsets and this is why no information appears using the describe command.
Make sure that your consumers inside server group commit their offsets succesfully (either manually or automatically).
I'm trying to describe the kafka consumer group via the below command in terminal:
./kafka-consumer-groups.sh --bootstrap-server $IP:$PORT --describe --group $GROUPID
I'm able to get the $GROUPID via this command:
./kafka-consumer-groups.sh --bootstrap-server $IP:$PORT --list
and the response is:
b'TestProcess'
But after trying to get the detail about this group via this command:
./kafka-consumer-groups.sh --bootstrap-server $IP:$PORT --describe --group TestProcess
this result will come up:
Error: Consumer group 'TestProcess' does not exist.
Kafka Version: 2.2.0
It looks like your consumer group is named literally b'TestProcess', so try issuing describe command like this:
./kafka-consumer-groups.sh --bootstrap-server $IP:$PORT --describe --group "b'TestProcess'"
Setting up Kafka first time, Kafka 0.11. Using pretty much default configurations. Produced produced some messages to topic ABC. 2 Consumers are coded to consume messages from the same topic. Each consumer belongs to different group id GROUP.1 and GROUP.2
Want to look into the topic for all the messages and also the offset details.
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group GROUP.1
throws following error,
Error: The consumer group 'GROUP.1' does not exist.
Same error for GROUP.2 also. I got some output without error for one of the group yesterday, but not today. What I'm I missing? Need to configure somewhere to persist consumer group details, or will the command work only when the consumers with given group id is currently running, or?
I tried kafka-consumer-groups --zookeeper localhost:2181 --describe --group GROUP.1 but got the same error.
Also tried Kafka-consumer-offset-checker command.
kafka-consumer-offset-checker --zookeeper localhost:2181 --topic ABC --group GROUP.1
[2017-12-19 19:25:01,654] WARN WARNING: ConsumerOffsetChecker is deprecated and will be dropped in releases following 0.9.0. Use ConsumerGroupCommand instead. (kafka.tools.ConsumerOffsetChecker$)
Exiting due to: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /consumers/GROUP.1/offsets/ABC/2.
As you said you saw the group details yesterday, it's probably worth noting that by default offsets are only stored for 24 hours. So if you group has not committed offsets in 24 hours, Kafka has no more information about it.
If this is indeed the issue, you can increase the time by setting offsets.retention.minutes to a larger value.