Kafka not receiving message from external producer - apache-kafka

Question

Update #2 - I believe I found the solution and I am answering this for completeness. It seems that I need to have the following configurations set with my instances public ip address and port 9092.
advertised.host.name
advertised.port
I am running a two node Kafka server running version 0.9.0.1. I have gone through the quick-start and tested that I can send and receive messages internally, as in producing a message on the first Kafka server node and consuming it on the second Kafka server node. Both nodes are on the same network.
I then tried to produce some messages from an external box outside of the network that the Kafka nodes were on. I have ensured the appropriate ports are open and tested that I can hit both boxes from my producer machine using telnet on port 9092.
Everything seems to work fine; my small Java app will send the messages without error, but the consumer never receives anything. I have checked the Kafka logs and nothing is there. Is there any additional configurations that need to be set in the server.properties file in order for Kafka servers to consume from producers outside of its internal network?
Updated ** the issue seems to be isolated on my producer. When Kafka is running locally on the same machine that is running the Java producer code it works as expected. When I run the Java code and produce the same messages to a external machine, the message is not included in the producer.send call.
I have verified this by running the following command.
sudo tcpdump -n -i en0 -xX port 9092
This monitors the traffic from my producer machine being sent out on port 9092. I can make out the topic, but the message is clearly not there.
Here is the code I am using for the producer:
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.storm.shade.org.yaml.snakeyaml.Yaml;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.util.Map;
import java.util.Properties;
import static constants.Constants.*;
public class TestProducer {
// variable declarations
private KafkaProducer<String, String> _producer;
private Properties _props;
private String _topic;
public TestProducer(String configPath, String topic) {
this._props = initialize(configPath);
this._producer = new KafkaProducer<>(_props);
this._topic = topic;
}
private Properties initialize(String configPath) {
Yaml yaml = new Yaml();
Properties props = new Properties();
try {
InputStream is = new FileInputStream(new File(configPath));
// Parse the YAML file and return the output as a series of Maps and Lists
Map<String, Object> config = (Map<String, Object>) yaml.load(is);
props.put("bootstrap.servers", config.get(PROD_BOOTSTRAP_SERVERS));
props.put("acks", config.get(PROD_ACKS));
props.put("retries", config.get(PROD_RETRIES));
props.put("batch.size", config.get(PROD_BATCH_SIZE));
props.put("linger.ms", config.get(PROD_LINGER_MS));
props.put("buffer.memory", config.get(PROD_BUFFER_MEMORY));
props.put("key.serializer", config.get(PROD_KEY_SERIALIZER));
props.put("value.serializer", config.get(PROD_VALUE_SERIALIZER));
} catch( FileNotFoundException e ){
LOG.error("FileNotFoundException: could not initialize Properties");
}
return props;
}
public void produce(String message) {
try {
//verify that we have supplied a topic
if( _topic != null && !_topic.isEmpty() ) {
System.out.println("Producing message:" + message);
_producer.send(new ProducerRecord<String, String>(_topic, message));
}
} catch (Throwable throwable) {
System.out.printf("%s", throwable.getStackTrace());
}
}
public void close() {
_producer.close();
}
}