sorry but i am racking my brains to figure this out.
My kafka broker says waiting to connect to zookeper and then shuts down (time out).I checked broker config / port did netstat etc also confirmed zookeper settings. But then if i restart my ubuntu kernel (where kafka broker is running) all is well. Zookeeper is on Linux mint. Please can someone advice where what log files can give me more detail - i did the usual netstat and lsof no luck.
its windows 10 for some reason it is not recognizing my partitioned ip.
Fixed it was a config item on my end.
Related
I installed kafka on windows subsystem linux and started using the command confluent local services start and all services are up. Now, when I'm trying to run my kafka-spring application from windows, it is showing the following error:- connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
My server.properties are:-
listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://localhost:9092
Where I'm going wrong???
You need to run the Spring app within WSL2 session such that it's on the same network bridge as the hypervisor
I am facing below error message when i was trying to connect and see the topic/consumer details of one of my kafka clusters we have.
we have 3 brokers in the cluster which I able to see but the topic and its partitions.
Note : I have kafka 1.0 and kafka tool version is 2.0.1
I had the same issue on my MacBook Pro. The tool was using "tshepo-mbp" as the hostname which it could not resolve. To get it to work I added 127.0.0.1 tshepo-mbp to the /etc/hosts file.
kafka tool is most likely using the hostname to connect to the broker and cannot reach it. You maybe connecting to the zookeeper host by IP address but make sure you can connect/ping the host name of the broker from the machine running the kafka tool.
If you cannot ping the broker either fix the network issues or as a workaround edit the host file on your client to let it know how to reach the broker by its name
This issue occurs if you have not set listeners and advertised.listeners property in server.properties file.
For Ex:
config/server.properties
...
listeners=PLAINTEXT://:9092
...
advertised.listeners=PLAINTEXT://<public-ip/host-name>:9092
...
To fix this issue, we need to change the server.properties file.
$ vim /usr/local/etc/kafka/server.properties
Here update the listeners value from
listeners=PLAINTEXT://:9092
to
listeners=PLAINTEXT://localhost:9092
source:https://medium.com/#Ankitthakur/apache-kafka-installation-on-mac-using-homebrew-a367cdefd273
For better visibility (even already commented the same in early days thread)
In my case, I got to know when I used Kafkatool from my local machine, tool tris to find out Kafka broker port which was blocked from my cluster admins for my local machine, that is the reason I was not able to connect.
Resolution:
Either ask the admin to open the port for intranet if they can, if they can not you can use tunnelling for your testing purpose or time being for your port.
Hope this would help a few.
I have a problem with Apache kafka
I have 4 clusters where I want to install kafka instances. On 3 clusters its works, they can product, and consume messages between each other, zookepers work fine. But on 4th cluster I can't run zookeeper connected with others zookeepers. If I set in zoo.cfg only local server (0.0.0.0:2888:3888) zookeeper runs in mode standalone, but if I add others servers I get error
./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /etc/zookeeper/conf/zoo.cfg
Error contacting service. It is probably not running.
How I can fix this error? I will add, that I can ping servers, so they can see each other.
I have Hortonworks DataFlow (HDF3.0.2) running in VmWare on my mac. Kafka broker is running on port 6667 and the IP address of sandbox is 172.17.0.2
In java program, running locally on my mac, I have bootstrap server configured as below:
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "172.17.0.2:6667");
Java program just hangs cannot produce message in kafka topic. I have tried disabling the firewall, added entry in /etc/hosts as:
172.17.0.2 sandbox-hdf.hortonworks.com
and changed the bootstrap servers config entry to use sandbox-hdf.hortonworks.com, but no luck.
telnet command to 172.17.0.2 6667 hangs as well and it gives connection timeout error.
Any help to produce message in Kafka running in HDF 3.0.2 from outside of Vmware is highly appreciated. Please let me know if I am missing anything.
Thanks for your time and help.
I am trying to setup Kafka referring to the quick start guide on the apache kafka page and while starting the zookeeper server it gets stuck at this step...Would appreciate if someone can help guide me through on how to start the zookeeper server
[2015-05-26 15:41:39,216] INFO tickTime set to 3000 (org.apache.zookeeper.server.ZooKeeperServer)
[2015-05-26 15:41:39,216] INFO minSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2015-05-26 15:41:39,216] INFO maxSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2015-05-26 15:41:39,235] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
The terminal getting stuck at that step actually means that the server is running. So, while you may have thought you were getting an error, it actually means that the server has started.
I'm running through the same setup. Since you've opened the port in that terminal, you can proceed with the rest of the instructions from another terminal. You'll have to open multiple terminals throughout the setup, and eventually will be entering messages in one terminal (the producer), and seeing them appear in another terminal (the consumer).
My logs were stuck at this line, meaning zookeper could not start properly. I could not start kafka too.
0.0.0.0 might mean something is wrong, sometimes. I renewed my DHCP lease on my mac and (system preferences -> Advanced -> TCP/IP -> Renew DHCP lease), restarted zookeeper and it started working.
based on the screenshot posted, below line tells that zookeeper is running on the port 2181
[2015-05-26 15:41:39,235] INFO binding to port 0.0.0.0/0.0.0.0:2181
it seems like you have started the zookeeper server successfully but without running the Kafka server you are trying to run Kafka commands and because of you are getting an error of broker is not available.
in mac run the following two commands in two tabs:
start-zookeeper:
zookeeper-start-server /usr/local/kafka/zookeeper.properties
start kafka server:
kafka-server-start /usr/local/etc/kafka/server.properties
open another terminal and try out the Kafka commands, commands will run without any issues