I have docker in a linux machine, there I download kafka version 1.1.0, also I have run a zookeeper container and expose the port 2181, I change the zookeeper.connect of my server.properties to the one that the zookeeper container, I create an image where I include kafka, this is my dockerfile:
FROM openjdk:8
WORKDIR /app
ADD kafka_2.11-1.1.0 /app
ENTRYPOINT bin/kafka-server-start.sh config/server.properties
After the image is created I run my kafka container and everything is works great, my problem is when I try to send messages from an external producer, the messages never arrive, I search for this error and I found this link where I need to configure the advertised.host.name and advertised.port, but after change those properties and try to run my kafka container I get the next error:
kafka.common.KafkaException: Socket server failed to bind to ip:9092: Cannot assign requested address.
Also I try with this property:
listeners=PLAINTEXT://ip:9092
But I get the same error, if I don't make any change in kafka and I try to send a message from my spring boot application I get this error:
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for prueba-0: 30041 ms has passed since batch creation plus linger time
Can someone help me with this?,
Thanks in advance.
After searching a lot I found this link, I only need to specify the ip address of the host not the ip of docker container, this ip need to be on the advertised.listeners property:
advertised.listeners=PLAINTEXT://myip:9092
Hope this helps someone else.
Related
Able to connect successfully to local kafka broker/cluster running locally (dockerized) using Conduktor, but when trying to connect to Kafka cluster running on Unix VM, getting below error.
Error:
"The broker [...] is reachable but Kafka can't connect. Ensure you have access to the advertised listeners of the the brokers and the proper authorization"
Appreciate any assistance.
running locally (dockerized)
When running in docker, you need to ensure that the ports are accessible from outside of your container. To verify this, try doing a telnet <ip> <port> and check if you are able to connect.
Since the error message says, the broker is reachable, I suppose you would be able to successfully telnet to the broker.
Next, check your broker config called advertised.listeners. Here you need to mention your IP:Port combination where IP is what you will be giving in your client program i.e. Conduktor.
An example for that would be
advertised.listeners=PLAINTEXT://1.2.3.4:9092
and then restart your broker and reconnect. If you are using ssl then you need to provide some extra configuration. See Configuring Kafka brokers for more.
Try to add in /etc/hosts (Unix-like) or C:\Windows\System32\drivers\etc\hosts (windows-like) the Kafka server in such manner kafka_server_ip kafka_server_name_in_dns (e.g. 10.10.0.1 kafka).
We have a kafka broker setup on internal cloud. We find actual url using zookeeper and provide in bootstrap.server config.
Now the problem is when the broker restarts the internal cloud restarts it on dynamically allotted machine with new host port. Now the host port which i have initially given in producer config is not valid.
Question is how can i reload this config without restart.
Note: i know this is a bad design to host broker where it can restart in a different machine, but this is how it is right now.
I think you can use the domain name instead of ip in bootstrap.server config.
I've started learning some big data tools for a new project, and right now I'm on Kafka and Zookeeper.
I have them both install on my local machine, and I can start them up and start producing and consuming messages just fine. Now, I want to try it having two machines, one with a kafka broker, zookeepr and a producer, and the other with a consumer. Lets call them Machine A and Machine B.
Machine A has runs the Zookeeper server, the broker and a producer. Machine B runs a consumer. From what I think I understand, I should be able to setup the consumer to listen to a topic from the producer on Machine A, using Zookeeper. Since both machines are on the same network (i.e. my local home network), I thought I could change the kafka broker server.properties to use my static ip address for Machine A, and then have the consumer on Machine B connect to it.
My problem, is that zookeeper keeps spinning up on localhost, and connecting to 0.0.0.0/0.0.0.0:2181 so when my broker tries to connect to it using my static ip address (i.e 192.168.x.x), it times out. I have looked all over for a solution, but I cannot find anything that tells me how to configure the Zookeeper sever to start on a different ip address.
Maybe my understanding of these technologies is simply wrong, but I thought this would be a fairly simple thing to do. Does anyone know any way to resolve this? Or else if I'm doing it completely wrong, what is the correct approach
zookeeper keeps spinning up on localhost, and connecting to 0.0.0.0/0.0.0.0:2181
Well, that is the bind address.
You need to also (preferably) have a static IP for Zookeeper, then set zookeeper.connect within the server.properties file of Kafka to reach to that other machine's external address.
From the Zookeeper configuration file, you would make sure you have the myid file and have a line in the property file that looks like this (without the double brackets)
server.{{ myid }}={{ ip_address }}:2888:3888
You wouldn't find this in the Kafka documentation, but it is in the Zookeeper documentation
However, if Kafka and Zookeeper are on the same machine, this isn't necessary.
Your external consumer should be setting bootstrap.servers property and the Kafka IP address(es) w/ port 9092.
Your problem might me related instead to the advertised.listeners setting within Kafka.
For example, start with listeners=PLAINTEXT://:9092
As of Zookeeper 3.3.0 (see Advanced Configuration):
clientPortAddress : New in 3.3.0: the address (ipv4, ipv6 or hostname)
to listen for client connections; that is, the address that clients
attempt to connect to. This is optional, by default we bind in such a
way that any connection to the clientPort for any
address/interface/nic on the server will be accepted
So you could use:
clientPortAddress=127.0.0.1
I am facing below error message when i was trying to connect and see the topic/consumer details of one of my kafka clusters we have.
we have 3 brokers in the cluster which I able to see but the topic and its partitions.
Note : I have kafka 1.0 and kafka tool version is 2.0.1
I had the same issue on my MacBook Pro. The tool was using "tshepo-mbp" as the hostname which it could not resolve. To get it to work I added 127.0.0.1 tshepo-mbp to the /etc/hosts file.
kafka tool is most likely using the hostname to connect to the broker and cannot reach it. You maybe connecting to the zookeeper host by IP address but make sure you can connect/ping the host name of the broker from the machine running the kafka tool.
If you cannot ping the broker either fix the network issues or as a workaround edit the host file on your client to let it know how to reach the broker by its name
This issue occurs if you have not set listeners and advertised.listeners property in server.properties file.
For Ex:
config/server.properties
...
listeners=PLAINTEXT://:9092
...
advertised.listeners=PLAINTEXT://<public-ip/host-name>:9092
...
To fix this issue, we need to change the server.properties file.
$ vim /usr/local/etc/kafka/server.properties
Here update the listeners value from
listeners=PLAINTEXT://:9092
to
listeners=PLAINTEXT://localhost:9092
source:https://medium.com/#Ankitthakur/apache-kafka-installation-on-mac-using-homebrew-a367cdefd273
For better visibility (even already commented the same in early days thread)
In my case, I got to know when I used Kafkatool from my local machine, tool tris to find out Kafka broker port which was blocked from my cluster admins for my local machine, that is the reason I was not able to connect.
Resolution:
Either ask the admin to open the port for intranet if they can, if they can not you can use tunnelling for your testing purpose or time being for your port.
Hope this would help a few.
I have installed zookeeper and kafka,
first step :
running zookeeper by the following commands :
bin/zkServer.sh start
bin/zkCli.sh
second step :
running kafka server
bin/kafka-server-start.sh config/server.properties
kafka should run at localhost:9092
but I am getting the following error :
WARN Unexpected error from /0:0:0:0:0:0:0:1; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1195725856 larger than 104857600)
I am following the following link :
Link1
Link2
I am new to kafka ,please help me to set it up.
1195725856 is GET[space] encoded as a big-endian, four-byte integer (see here for more information on how that works). This indicates that HTTP traffic is being sent to Kafka port 9092, but Kafka doesn't accept HTTP traffic, it only accepts its own protocol (which takes the first four bytes as the receive size, hence the error).
Since the error is received on startup, it is likely benign and may indicate a scanning service or similar on your network scanning ports with protocols that Kafka doesn't understand.
In order to find the cause, you can find where the HTTP traffic is coming from using tcpdump:
tcpdump -i any -w trap.pcap dst port 9092
# ...wait for logs to appear again, then ^C...
tcpdump -qX -r trap.pcap | less +/HEAD
Overall though, this is probably annoying but harmless. At least Kafka isn't actually allocating/dirtying the memory. :-)
Try to reset socket.request.max.bytes value in $KAFKA_HOME/config/server.properties file to more than your packet size and restart kafka server.
My initial guess would be that you might be trying to receive a request that is too large. The maximum size is the default size for socket.request.max.bytes, which is 100MB. So if you have a message which is bigger than 100MB try to increase the value of this variable under server.properties and make sure to restart the cluster before trying again.
If the above doesn't work, then most probably you are trying to connect to a non-SSL-listener.
If you are using the default broker of the port, you need to verify that :9092 is the SSL listener port on that broker.
For example,
listeners=SSL://:9092
advertised.listeners=SSL://:9092
inter.broker.listener.name=SSL
should do the trick for you (Make sure you restart Kafka after re-configuring these properties).
This is how I resolved this issue after installing a Kafka, ELK and Kafdrop set up:
First stop every application one by one that interfaces with Kakfa
to track down the offending service.
Resolve the issue with that application.
In my set up it was Metricbeats.
It was resolved by editing the Metricbeats kafka.yml settings file located in modules.d sub folder:
Ensuring the Kafka advertised.listener in server.properties was
referenced in the hosts property.
Uncomment the metricsets and client_id properties.
The resulting kafka.yml looks like:
# Module: kafka
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.6/metricbeat-module-kafka.html
# Kafka metrics collected using the Kafka protocol
- module: kafka
metricsets:
- partition
- consumergroup
period: 10s
hosts: ["[your advertised.listener]:9092"]
client_id: metricbeat
The answer is most likely in one of the 2 areas
a. socket.request.max.bytes
b. you are using a non SSL end point to connect the producer and the consumer too.
Note: the port you run it really does not matter. Make sure if you have an ELB the ELB is returning all the healthchecks to be successful.
In my case i had an AWS ELB fronting KAFKA. I had specified the Listernet Protocol as TCP instead of Secure TCP. This caused the issue.
#listeners=PLAINTEXT://:9092
inter.broker.listener.name=INTERNAL
listeners=INTERNAL://:9093,EXTERNAL://:9092
advertised.listeners=EXTERNAL://<AWS-ELB>:9092,INTERNAL://<EC2-PRIVATE-DNS>:9093
listener.security.protocol.map=INTERNAL:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
Here is a snippet of my producer.properties and consumer.properties for testing externally
bootstrap.servers=<AWS-ELB>:9092
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
In my case, some other application was already sending data to port 9092, hence the starting of server failed. Closing the application resolved this issue.
Please make sure that you use .security.protocol=plaintext or you have mismatch server security compared to the clients trying to connect.