I ran into an issue while trying to set up kafaka (version 2.12-2.4.0) on my local machine by following:
https://kafka.apache.org/quickstart
I created a very simple spring-boot app that had a producer and consumer by following some online tutorials. When I would start up my app it would spin for 30 seconds and then start throwing errors in the logs trying to create a topic with connection errors.
I thought maybe my spring-boot app was misconfigured so I tried creating a topic from the command line but I got a similar error:
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic testing
Error while executing topic command : org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
[2020-02-11 21:57:06,545] ERROR java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at kafka.admin.TopicCommand$AdminClientTopicService.createTopic(TopicCommand.scala:225)
at kafka.admin.TopicCommand$TopicService.createTopic(TopicCommand.scala:194)
at kafka.admin.TopicCommand$TopicService.createTopic$(TopicCommand.scala:189)
at kafka.admin.TopicCommand$AdminClientTopicService.createTopic(TopicCommand.scala:217)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:61)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
Kafka and zookeeper were up and running but nothing could connect to kafka.
I noticed that the kafka logs said it was listening on 0.0.0.0:9092:
INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
I went into the server.properties and changed all the localhost values to:
# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
# FORMAT:
# listeners = listener_name://host_name:port
# EXAMPLE:
# listeners = PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://127.0.0.1:9092
This solved the problem. I don't know if this is an issue with my laptop but I wanted to save other people time. I didn't have to change my spring-boot connection configuration, connecting to localhost still worked.
Related
I have installed confluent-oss-5.0.0 on Azure VM and exposed all necessary ports to access using public IP Address.
I tried to change the etc/kafka/server.properties below things to achieve but no luck
Approach - 1
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://<publicIP>:9092
--------------------------------------
Approach - 2
advertised.listeners=PLAINTEXT://<publicIP>:9092
--------------------------------------
Approach - 3
listeners=PLAINTEXT://<publicIP>:9092
I experienced below error
pj#pj-HP-EliteBook-840-G1:~/confluent-kafka/confluent-oss-5.0.0/bin$ kafka-console-producer --broker-list <publicIp>:9092 --topic pj_test123>dfsds
[2019-03-25 19:13:38,784] WARN [Producer clientId=console-producer] Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
pj#pj-HP-EliteBook-840-G1:~/confluent-kafka/confluent-oss-5.0.0/bin$ kafka-console-producer --broker-list <publicIp>:9092 --topic pj_test123
>message1
>message2
>[2019-03-25 19:20:13,216] ERROR Error when sending message to topic pj_test123 with key: null, value: 3 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s) for pj_test123-0: 1503 ms has passed since batch creation plus linger time
[2019-03-25 19:20:13,218] ERROR Error when sending message to topic pj_test123 with key: null, value: 3 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
pj#pj-HP-EliteBook-840-G1:~/confluent-kafka/confluent-oss-5.0.0/bin$ kafka-console-consumer --bootstrap-server <publicIp>:9092 --topic pj_test123 --from-beginning
[2019-03-25 19:29:27,742] WARN [Consumer clientId=consumer-1, groupId=console-consumer-42352] Error while fetching metadata with correlation id 2 : {pj_test123=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
pj#pj-HP-EliteBook-840-G1:~/confluent-kafka/confluent-oss-5.0.0/bin$ kafka-console-consumer --bootstrap-server <publicIp>:9092 --topic pj_test123 --from-beginning
[2019-03-25 19:27:06,589] WARN [Consumer clientId=consumer-1, groupId=console-consumer-33252] Connection to node 0 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
All other service like zookeeper, kafka-connect and restAPI are working fine using the <PublicIP>:<port>
kafka-topics --zookeeper 13.71.115.20:2181 --list --- This is working
Ref:
Not able to access messages from confluent kafka on EC2
https://kafka.apache.org/documentation/#brokerconfigs
Why I cannot connect to Kafka from outside?
Solutions
Thanks, #Robin Moffatt, It works for me. I do below changes along with allowing all Kafka related ports on Azure networking
kafka#kafka:~/confluent-oss-5.0.0$ sudo vi etc/kafka/server.properties
listeners=INTERNAL://0.0.0.0:9092,EXTERNAL://0.0.0.0:19092
listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
advertised.listeners=INTERNAL://<privateIp>:9092,EXTERNAL://<publicIp>:19092
inter.broker.listener.name=INTERNAL
You need to configure both internal and external listeners for your broker. This article details how: https://rmoff.net/2018/08/02/kafka-listeners-explained/.
You will also have to give public access to port 9092 (your broker). TO do that,
Go to your Virtual machine in Azure portal
Select Networking under settings in the left menu
Add inbound port rule
Add port 9092 to be accessbile from anywhere
I have an environment problem.
I want using zookeeper and Kafka cluster to solve my problem.
My zookeeper version is 3.4.12 and Kafka is 2.12-2.1.0
I also change the zoo.cfg in zookeeper.
dataDir=D:/WEBSOCKET/zookeeper-3.4.12/data
and server.properties in kafka.
log.dirs=D:/WEBSOCKET/kafka_2.12-2.1.0/logs
I see all the tutorial and do it the exact same way.
And also usgin kafka open zookeeper.
this is my command:
1) open zookeeper (zkServer.cmd)
2) in kafka
.\bin\windows\kafka-server-start.bat .\config\server.properties
3) create topic
.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic hello
4) create a producer
.\bin\windows\kafka-console-producer.bat --bootstrap-server localhost:2181 --topic hello
5) create a consumer
.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:2181 --topic hello
when I get in step 5, I always fail.
zookeeper will give me a lot of console like :
WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn#383] - Exception causing close of session 0x0: null
INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory#215] - Accepted socket connection from /127.0.0.1:55192
and
2019-01-08 17:05:24,822 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn#1040] - Closed socket connection for client /127.0.0.1:50874 (no session established for client)
2019-01-08 17:05:25,783 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory#215] - Accepted socket connection from /127.0.0.1:56089
I don't know how to fix it. I google for two days...
when I open my Kafka with step 2, my zookeeper some times doesn't hvae any response or shows me this:
[ProcessThread(sid:0 cport:2181)::PrepRequestProcessor#596] - Got user-level KeeperException when processing sessionid:0x1000058f8960000 type:multi cxid:0x36 zxid:0x69 txntype:-1 reqpath:n/a aborting remaining multi ops. Error Path:/admin/preferred_replica_election Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election
I also google this, but not helpful.
I set this in kfaka before:
advertised.host.name = localhost
listeners=PLAINTEXT://127.0.0.1:9092
my host have set
127.0.0.1 localhost
please help me create local server I want to coding my project..
thank you read all.
Producers and consumers need to use port 9092 (Kafka)
You are seeing Zookeeper logs and errors because you are trying to use bootstrap-server or broker-list and port 2181 (Zookeeper)
Check again the quickstart guide
I am new to Kafka and seem to be having several issues with the 'Quickstart' guide for Apache Kafka found here:
https://kafka.apache.org/quickstart#quickstart_kafkaconnect
Ultimately I am trying to learn how to load a kafka queue with many kafka messages and so the Step 7 part of this Quickstart guide seemed relevant.
I installed the binary download (Scala 2.11 - kafka_2.11-1.1.0.tgz ) found here:
https://kafka.apache.org/downloads
I had initially tried to jump straight to step 7 but realised after finding this question (Kafka Connect implementation errors) I had to do the few steps prior to that
Therefore I followed the first step successfully:
tar -xzf kafka_2.11-1.1.0.tgz
cd kafka_2.11-1.1.0
Then I followed step 2:
bin/zookeeper-server-start.sh config/zookeeper.properties
But I get the error
ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain)
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:90)
at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:117)
at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87)
at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
But when I run the next command in that same step:
bin/kafka-server-start.sh config/server.properties
The Kafka server seems to run successfully?
So then I tried to continue to step 3 to create a topic:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
But this produces the error:
Error while executing topic command : Replication factor: 1 larger than available brokers: 0.
[2018-04-09 14:13:26,908] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 1 larger than available brokers: 0.
(kafka.admin.TopicCommand$)
Then trying step 4:
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
This seems to work and I can write a message but then I get a connection error (which is probably due to the fact previous steps haven't worked successfully)
kafka_2.11-1.1.0 user$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
>This is a message
[2018-04-09 14:17:52,631] WARN [Producer clientId=console-producer] Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-04-09 14:17:52,687] WARN [Producer clientId=console-producer] Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
Does anyone know why these issues are occurring and how I can fix them? I can't find anymore inforomation in that tutorial about these problems
As the error suggests, you have something running on the default port for ZK. Either close it or change the zookeeper properties file to use another port.
Address localhost:2181 is already in use. Since Zookeeper cannot start, then Kafka brokers won't start too. replication-factor must be less or equal to the number of available brokers, and since no broker is available then the following error will be reported (even if you are using --replication-factor 1).
Error while executing topic command : Replication factor: 1 larger than available brokers: 0.
[2018-04-09 14:13:26,908] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 1 larger than available brokers: 0.
(kafka.admin.TopicCommand$)
You either need stop the process which is running in 2181 or change the ZK default port to a port which is not currently in use.
To see what is running (PID) in port 2181, run
lsof -i -n -P | grep 2181
If you want to kill that process, then run
kill -9 PID
where PID is the process ID which you can get from lsof command.
Otherwise, you need to change the port in the zookeeper.properties file by modifying the parameter clientPort=2181. And finally, you need to change zookeeper.connect=localhost:2181 parameter in the server.properties file accordingly.
I am trying to send messages using the Kafka console producer to a Kafka broker that is running on the same machine. When I run
echo "Hello world" | ./kafka-console-producer.sh --broker-list localhost:9092 --topic test
I receive the following error message:
[2016-06-29 15:00:44,069] ERROR Error when sending message to topic test with key: null, value: 11 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Batch containing 1 record(s) expired due to timeout while requesting metadata from brokers for test-0
When I check if the Kafka broker is listening on port 9092, it is there and running. How do I debug why cannot the command line producer connect to the broker? Thank you!
I experienced the same problem and it seems like Kafka wrote wrong metadata to zookeeper.
The easiest way to get pass this is to remove the znode in which brokers data is being registered. I.e if you use the defaults, the name should be '/brokers' and the command:
$ zkCli.sh
...
[zk: localhost:2181(CONNECTED) 1] rmr /brokers
Kafka 0.8 works great. I am able to use CLI as well as write my own producers/consumers!
Checking Zookeeper... and I see all the topics and partitions created successfully for 0.8.
Kafka 0.7 does not work!
Why Kafka 0.7? I am using Kafka Spout from Storm which is made for Kafka 0.7.
First I just want to run CLI based producer/consumer for Kafka 0.7, which I am unable to. I carry out the following steps:
I delete all the topics/partitions etc. in Zookeeper that were created from my Kafka 0.8
I change the dataDir in zoo.cfg to point to different location.
Now I start the kafka server 0.7. It starts successfully. However I don’t know why it again registers the broker topics I deleted?
Now I start the Kafka Producer :
bin/kafka-console-producer.sh --zookeeper localhost:2181 --topic topicime
& it starts successfully:
[2013-06-28 14:06:05,521] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2013-06-28 14:06:05,606] INFO Creating async producer for broker id = 0 at 0:0 (kafka.producer.ProducerPool)
Time to send some messages & oops I get this error:
[2013-06-28 14:07:19,650] INFO Disconnecting from 0:0 (kafka.producer.SyncProducer)
[2013-06-28 14:07:19,653] ERROR Connection attempt to 0:0 failed, next attempt in 1 ms (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:364)
at sun.nio.ch.Net.connect(Net.java:356)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:623)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:173)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:196)
at kafka.producer.SyncProducer.send(SyncProducer.scala:92)
at kafka.producer.SyncProducer.multiSend(SyncProducer.scala:135)
at kafka.producer.async.DefaultEventHandler.send(DefaultEventHandler.scala:58)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:44)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:116)
at scala.collection.immutable.Stream.foreach(Stream.scala:254)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:70)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:41)
Note that Zookeeper is already running.
Any help would really be appreciated.
EDIT:
I don't even see the topic being created in zookeeper. I am running the following command:
bin/kafka-console-producer.sh --zookeeper localhost:2181 --topic topicime
After the command everything is fine & I get the following message:
[2013-06-28 14:30:17,614] INFO Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x13f805c6673004b, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2013-06-28 14:30:17,615] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2013-06-28 14:30:17,700] INFO Creating async producer for broker id = 0 at 0:0 (kafka.producer.ProducerPool)
However now when i type a string to send I get the above error (Connection refused!)
INFO Disconnecting from 0:0 (kafka.producer.SyncProducer)
The above line has the error hidden in it. 0:0 is not a valid host and port. The solution is to explicitly set the host ip to be registered in Zookeeper by setting the "hostname" property in server.properties.
Consider checking out the storm-kafka fork, available at https://github.com/wurstmeister/storm-kafka-0.8-plus
I'm installing it right now for our servers =).