read output of kafka coming from MQTT - apache-kafka

Question

I am using Kafka connect for MQTT-KAFKA connection. I am publishing sample data from MQTTLENSES and have written consumer code for Kafka in java:
package test;
import java.io.UnsupportedEncodingException;
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Base64;
import java.util.Iterator;
import java.util.Properties;
import javax.crypto.Cipher;
import javax.crypto.spec.SecretKeySpec;
import org.json.*;
import org.apache.kafka.common.serialization.StringDeserializer;
import com.fasterxml.jackson.databind.util.JSONWrappedObject;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
#SuppressWarnings("unused")
public class ConsumerTest {
public static void main(String[] args) throws UnsupportedEncodingException {
System.out.println("consumer123");
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "group-1");
props.put("enable.auto.commit", "true");
props.put("auto.commit.interval.ms", "1000");
props.put("auto.offset.reset", "earliest");
props.put("session.timeout.ms", "30000");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
#SuppressWarnings("resource")
KafkaConsumer<String, String> kafkaConsumer = new KafkaConsumer<>(props);
kafkaConsumer.subscribe(Arrays.asList(MQTT TOPIC));
while (true) {
ConsumerRecords<String, String> records = kafkaConsumer.poll(100);
for (ConsumerRecord<String, String> record : records) {
System.out.println(records);
try
{
String record_data = record.value().toString();
JSONObject obj = new JSONObject(record_data);
String payload = obj.getString("payload");
String s = new String(payload.getBytes(), StandardCharsets.UTF_8);
System.out.println(s);
System.out.println(record_data);
System.out.println(record.key());
//System.out.println(decryptData(payload));
}
catch(Exception je)
{
System.out.println(je.toString());
}
}
}
}
for the input "Hello", it prints output in consumer => {"schema":{"type":"bytes","optional":false},"payload":"SGVsbG8K"}
How to decode the payload coming in kafka consumer?

Answer 1

The payload that you are receiving is in "base64" format. you need only to decode this base64 string:
byte[] plainText = Base64.getDecoder().decode(payload);
System.out.println(new String(plainTexts));