Apache Kafka : Producer -Consume API not running on GCP - apache-kafka

Question

I have been trying to run my producer and consumer api on Kafka cluster but it is not working.
1) Zookeeper is running on one GCP VM instance
2) Kafka is running on another GCP VM instance
Steps Followed:
Step 1) Running Zookeeper using the following command:
bin/zookeeper-server-start.sh config/zookeeper.properties
Zookeeper.properties:
#the directory where the snapshot is stored.
dataDir=/tmp/zookeeper
#the port at which the clients will connect
clientPort=2181
#disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
# In milliseconds
tickTime=2000
# In ticks
initLimit=10
syncLimit=5
maxClientCnxns=30
# All Zookeeper servers need to be aware of other Zookeepers part of the cluster
server.1=zook-1:2888:3888
Note : zook-1 is the name of the Zookeeper instance
Step 2)
Running Kafka on another GCP VM instance as :
bin/kafka-console-producer.sh --broker-li de-kafka-1:9092 --topic test
server.properties:
############################# Server Basics ############################# # The id of the broker. This must be set to a unique integer for each broker.
broker.id=0
############################ Zookeeper ############################# #
# root directory for all kafka znodes.
zookeeper.connect=zook-1:2181
# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000
host.name=de-kafka-1
Step 3)
Running producer api on the Kafka instance from Step 2 (using another SSH tunnel) as follows :
bin/kafka-console-producer.sh --broker-list de-kafka-1:9 092 --topic test
Note : Here de-kafka-1 is the name of the GCP Kafka VM instance I have tried and replaced de-kafka-1 with External IP of de-Kafka-1 instance, localhost, zook-1 (Name of Zookeeper VM instance), external IP of zook-1 vm instance
Error: OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N Sending msg1 sending msg2
Error: error when sending message to topic test single node with key: null, value: 5 bytes with error (org.apache.kafka.clients.producer.internals.errorloggingcallback)org.apache.kafka.common.errors.timeoutexception: failed to update metadata after 60000 ms.
Step 4:
Running Consumer API on Kafka instance (using third SSH tunnel):
bin/kafka-console-consumer.sh --bootstrap-server de-kafka-1:2181 --topic test --from-beginning
Would appreciate if anyone could help on this. Thanks

Answer 1

Try to create topic before producing bin/kafka-topics.sh --create --zookeeper zook-1:2181 --replication-factor 1 --partitions 1 --topic test.
If its not working read about zkCli.sh script here - with this script you can check if the broker is registered in the zookeeper.
If you still have a problems, and you trying to produce from outside the kafka cluster, try to read about advertised.listeners property here.