I have 3 zookeeper instances and 3 kafka instances.
The configuration is all fine and i can see they are properly connected to each other.
Then I create a new topic using below command :
kafka-topics.sh --create --zookeeper 10.XXX.XXX.XX:2181,10.XXX.XXX.XX:2182,10.XXX.XXX.XX:2183 --replication-factor 3 --partitions 1 --topic <topicName>
But as soon as i create a consumer using :
kafka-console-consumer.sh --bootstrap-server
10.XXX.XXX.XX:9094,10.XXX.XXX.XX:9095,10.XXX.XXX.XX:9096 --topic ankit108 -
consumer-property group.id=test1
I get below errors:
ERROR [ReplicaFetcher replicaId=1, leaderId=2, fetcherId=0] Error for
partition __consumer_offsets-32 to broker
2:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
Related
I'm basically doing the kafka quickstart using kafka_2.11-2.0.0 which comes with zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 00:39 GMT.
I also use Ubuntu:
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.5 LTS
Release: 16.04
Codename: xenial
I do these with the order below:
start zookeeper
start kafka server0 with port 9092
start kafka server1 with port 9093
start kafka server2 with port 9094
create a topic: kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replicated-topic
produce some messages: kafka-console-producer.sh --broker-list localhost:9092 --topic my-replicated-topic
consume from server0: kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic my-replicated-topic
consume from server1: kafka-console-consumer.sh --bootstrap-server localhost:9093 --from-beginning --topic my-replicated-topic
check the leader for my-replicated-topic and find it to be server0 -> here's the tricky part; one should kill server1 and then maybe server2 but never server0 and then restore them just in order to get server0 to be the leader of my-replicated-topic
kill server0
check for the new leader (happens to be server2): kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic
produce some messages to server2: kafka-console-producer.sh --broker-list localhost:9094 --topic my-replicated-topic
consume from server2 (or server1): kafka-console-consumer.sh --bootstrap-server localhost:9094 --from-beginning --topic my-replicated-topic -> this will hang till restarting server0
starting again server0
consumer from server2 outputs all messages including the one sent to server2 when was the leader
What is wrong and how one would solve the problem so not to matter which server becomes the leader?
I think this is due to replication factor of "__consumer_offsets" topic. This is set to one in server.properties file for testing purpose.
set offsets.topic.replication.factor=3 for high availability.
Copied from KAFKA-7526 comment
I am getting below error while trying to pass words in Kafka-consumer,
commands with i entered
console-1:(for producer)
export PATH=$PATH:/usr/hdp/current/kafka-broker/bin
kafka-topics.sh --create --zookeeper ip-172-31-20-58.ec2.internal:2181 --replication-factor 1 --partitions 1 --topic testuday1234
kafka-console-producer.sh --broker-list ip-172-31-20-58.ec2.internal:6667 --topic testuday1234
console-2: (for consumer)
export PATH=$PATH:/usr/hdp/current/kafka-broker/bin
kafka-console-consumer.sh --zookeeper localhost:2181 --topic testuday1234 --from-beginning
Please help me in resolving these error
Error i am getting in Producer console:
[udaychitukula6587#ip-172-31-38-183 ~]$ kafka-console-producer.sh --broker-list ip-172-31-20-58.ec2.internal:6667 --topic testuday1234
hi
[2018-05-28 15:27:36,761] WARN Error while fetching metadata [{TopicMetadata for topic testuday1234 ->
No partition metadata for topic testuday1234 due to kafka.common.LeaderNotAvailableException}] for topic [testuday1234]: class kafka.common.LeaderNotAvailableExcep
tion (kafka.producer.BrokerPartitionInfo)
[2018-05-28 15:27:36,774] WARN Error while fetching metadata [{TopicMetadata for topic testuday1234 ->
No partition metadata for topic testuday1234 due to kafka.common.LeaderNotAvailableException}] for topic [testuday1234]: class kafka.common.LeaderNotAvailableExcep
tion (kafka.producer.BrokerPartitionInfo)
Error i am getting in consumer console:
[udaychitukula6587#ip-172-31-38-183 ~]$ kafka-console-consumer.sh --zookeeper localhost:2181 --topic testuday123 --from-beginning
{metadata.broker.list=ip-172-31-20-58.ec2.internal:6667,ip-172-31-53-48.ec2.internal:6667,ip-172-31-60-179.ec2.internal:6667, request.timeout.ms=30000, client.id=c
onsole-consumer-63526, security.protocol=PLAINTEXT}
{metadata.broker.list=ip-172-31-20-58.ec2.internal:6667,ip-172-31-53-48.ec2.internal:6667,ip-172-31-60-179.ec2.internal:6667, request.timeout.ms=30000, client.id=c
onsole-consumer-63526, security.protocol=PLAINTEXT}
There is a couple of things that I've noted here.
First, in the newer versions (I think from 0.10.1) of Kafka for the console consumer, we need to use --bootstrap-server option as opposed to --zookeeper. Could you please confirm the version you're using? and also try to run the consumer command with --bootstrap-server option?
Second, for the producer in such a scenario, I would recommend checking 3 things to confirm where the issue might be:
The leader of a Kafka cluster is elected by the zookeeper, so it might be worth checking by running a zookeeper-client shell to see if there is an active controller in the Kafka cluster (in the znode path - /brokers/ids/[brokerId]).
Try running a Kafka-topics --describe --topic command to see if the topic has an active leader partition i.e. the Leader column in the output of the command should NOT have None. I've run into this myself before.
The last one is about the port number of the broker, could you please check and confirm if the broker is actually listening on that port. You'll find this information (listeners and advertised.listeners) in server.properties file on the broker. I found this post that you might find useful where the user had a problem with the port 6667.
I hope this helps!
Kafka Version : kafka_2.11-0.11.0.1
when I try to send messages from producer to consumer in a Multi-node Kafka Ecosystem, the message is getting sent, and broadcasted from Kafka server! These messages can be read by using Kafka Consumer via Zookeeper [which is going to be deprecated on following version!], and also via kafka-simple-consumer-shell.sh
./bin/kafka-simple-consumer-shell.sh --broker-list <broker-ip>:9092 --topic myTopic
But cannot be read from Consumer via consumer-bootstrap-server!
Call to Consumer via Zookeeper-server :
./bin/kafka-console-consumer.sh --topic myTopic --zookeeper <broker-ip>:2181 --from-beginning
Call to Consumer via Bootstrap-server :
./kafka-console-consumer.sh --topic myTopic --bootstrap-server <broker-ip>:9092 --consumer.config ../config/consumer.properties
My config/producer.properties contains:
bootstrap.servers= {broker-ip}:9092
compression.type=none
My config/consumer.properties contains(nothing):
# zookeeper.connect={private-ip-1}:2181,{private-ip-2}:2181
# timeout in ms for connecting to zookeeper
# zookeeper.connection.timeout.ms=6000
#consumer group id
group.id=test-consumer-group
#consumer timeout
# consumer.timeout.ms=7000
# consumer.timeout.ms=-1
My config/server.properties contains:
listeners=PLAINTEXT://{private-ip-1}:9092
advertised.host.name={private-ip-1}
advertised.listeners=PLAINTEXT://{private-ip-1}:9092
advertised.port=9092
host.name={private-ip-1}
zookeeper.connect={private-ip-1}:2181
When i try to consume messages from the kafka server which is hosted in ec2 with kafka console tool (V 0.9.0.1 , i think this uses old consumer APIs)
I get following exception.
How can i overcome this?
kafka-console-consumer.sh --zookeeper zookeeper.xx.com:2181 --topic my-replicated-topic
[2016-04-06 15:52:40,247] WARN [console-consumer-12572_Rathas-MacBook-Pro.local-1459921957380-6ebc238f-leader-finder-thread], Failed to add leader for partitions [my-replicated-topic,1],[my-replicated-topic,2],[my-replicated-topic,0]; will retry (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:110)
at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:98)
at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:83)
at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:149)
at kafka.consumer.SimpleConsumer.earliestOrLatestOffset(SimpleConsumer.scala:188)
at kafka.consumer.ConsumerFetcherThread.handleOffsetOutOfRange(ConsumerFetcherThread.scala:84)
at kafka.server.AbstractFetcherThread$$anonfun$addPartitions$2.apply(AbstractFetcherThread.scala:187)
at kafka.server.AbstractFetcherThread$$anonfun$addPartitions$2.apply(AbstractFetcherThread.scala:182)
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:778)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:116)
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:777)
at kafka.server.AbstractFetcherThread.addPartitions(AbstractFetcherThread.scala:182)
at kafka.server.AbstractFetcherManager$$anonfun$addFetcherForPartitions$2.apply(AbstractFetcherManager.scala:88)
at kafka.server.AbstractFetcherManager$$anonfun$addFetcherForPartitions$2.apply(AbstractFetcherManager.scala:78)
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:778)
at scala.collection.immutable.Map$Map3.foreach(Map.scala:161)
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:777)
at kafka.server.AbstractFetcherManager.addFetcherForPartitions(AbstractFetcherManager.scala:78)
at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:95)
at
I created topic like
/kafka-topics.sh --create --zookeeper zookeeper.xx.com:2181 --replication-factor 3 --partitions 3 --topic my-replicated-topic
I am running Kafka 0.8.1.1 with one Zookeeper and one broker. I created a partition 'test2' as below:
/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic test2
I noticed this exception in the state-change.log. Why is this occuring ? Should I be running more brokers if I have more than one partition ?
Exception:
kafka.common.NoReplicaOnlineException: No replica for partition [test2,2] is alive. Live brokers are: [Set()], Assigned replicas are: [List(0)]
at kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:61)
at kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:336)
at kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:185)
at kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:99)
at kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:96)
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:743)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:95)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:95)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:157)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:190)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:45)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:95)
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:742)
at kafka.controller.PartitionStateMachine.triggerOnlinePartitionStateChange(PartitionStateMachine.scala:96)
at kafka.controller.PartitionStateMachine.startup(PartitionStateMachine.scala:68)
at kafka.controller.KafkaController.onControllerFailover(KafkaController.scala:312)
at kafka.controller.KafkaController$$anonfun$1.apply$mcV$sp(KafkaController.scala:162)
at kafka.server.ZookeeperLeaderElector.elect(ZookeeperLeaderElector.scala:63)
at kafka.server.ZookeeperLeaderElector$$anonfun$startup$1.apply$mcZ$sp(ZookeeperLeaderElector.scala:49)
at kafka.server.ZookeeperLeaderElector$$anonfun$startup$1.apply(ZookeeperLeaderElector.scala:47)
at kafka.server.ZookeeperLeaderElector$$anonfun$startup$1.apply(ZookeeperLeaderElector.scala:47)
at kafka.utils.Utils$.inLock(Utils.scala:538)
at kafka.server.ZookeeperLeaderElector.startup(ZookeeperLeaderElector.scala:47)
at kafka.controller.KafkaController$$anonfun$startup$1.apply$mcV$sp(KafkaController.scala:637)
at kafka.controller.KafkaController$$anonfun$startup$1.apply(KafkaController.scala:633)
at kafka.controller.KafkaController$$anonfun$startup$1.apply(KafkaController.scala:633)
at kafka.utils.Utils$.inLock(Utils.scala:538)
at kafka.controller.KafkaController.startup(KafkaController.scala:633)
at kafka.server.KafkaServer.startup(KafkaServer.scala:96)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)