Logstash: Kafka Output Plugin - Issues with Bootstrap_Server - apache-kafka

Question

I am trying to use the logstash-output-kafka in logstash:
Logstash Configuration File
input {
stdin {}
}
output {
kafka {
topic_id => "mytopic"
bootstrap_server => "[Kafka Hostname]:9092"
}
}
However, when executing this configuration, I am getting this error:
[ERROR][logstash.agent ] Failed to execute action
{:action=>LogStash::PipelineAction::Create/pipeline_id:main,
:exception=>"LogStash::ConfigurationError", :message=>"Something is wrong
with your configuration."
I tried to change "[Kafka Hostname]:9092" to "localhost:9092", but that also fails to connect to kafka. Only when I remove the bootstrap_server configuration (which then defaults to localhost:9092) then the kafka connection seems to be established.
Is there something wrong with the bootstrap_server configuration of the kafka output plugin? i am using logstash v6.4.1, logstash-output-kafka v7.1.3

Answer 1

I think there is a typo in your configuration. Instead of bootstrap_server you need to define bootstrap_servers.
input {
stdin {}
}
output {
kafka {
topic_id => "mytopic"
bootstrap_servers => "your_Kafka_host:9092"
}
}
According to Logstash Docs,
bootstrap_servers
Value type is string
Default value is "localhost:9092"
This is for bootstrapping and the producer will only
use it for getting metadata (topics, partitions and replicas). The
socket connections for sending the actual data will be established
based on the broker information returned in the metadata. The format
is host1:port1,host2:port2, and the list can be a subset of brokers or
a VIP pointing to a subset of brokers.