Read Avro messages from Kafka in terminal - kafka-avro-console-consumer alternative - apache-kafka

Question

I'm trying to find easiest way how to read Avro messages from the Kafka topics in readable format. There is option to use Confluent kafka-avro-console-consumer in following way
./kafka-avro-console-consumer \
--topic topic \
--from-beginning \
--bootstrap-server bootstrap_server_url \
--max-messages 10 \
--property schema.registry.url=schema_registry_url
but for this I need to download whole Confluent platform (1.7 GB) that I see as an overkill in my scenario.
Is there any alternative how I could get Avro messages from the Kafka topics in the terminal easily?

Answer 1

I was able to get the last Avro messages in the readable form with kcat
kcat -C -b bootstrap_server \
-t topic \
-r schema_registry \
-p 0 -o -1 -s value=avro -e

Answer 2

You will need to download additional Kafka tools that supports the Schema Registry and Avro format, such as ksqlDB or Conduktor or AKHQ or similar GUI tools
kcat might support Avro now, I cannot recall
You could write your own consumer script. The Python library from Confluent doesn't require much code to consume Avro records
You could also clone the Schema Registry project from Github and build it on its own, then use the CLI scripts there

Answer 3

Another way to read messages in Avro is to use Kafdrop. Test it by adding such section to your docker-compose.yml along with broker, server-regestry and other containers:
kafdrop:
image: obsidiandynamics/kafdrop
restart: "no"
ports:
- "9001:9000"
environment:
KAFKA_BROKERCONNECT: "broker:9092"
JVM_OPTS: "-Xms16M -Xmx48M -Xss180K -XX:-TieredCompilation -XX:+UseStringDeduplication -noverify"
CMD_ARGS: "--message.format=AVRO --schemaregistry.connect=http://schema-registry:8081"
depends_on:
- "broker"
After that, open Kafdrop at http://localhost:9001, click at topic where avro-messages will be put, and choose AVRO message format in drop down.