Micronaut kafka project - mutiple consumers each with different bootstrap server and ssl certs - tags

Question

I am trying to set up a micronaut project with multiple consumers each with different bootstrap server and ssl certs. I am not setting global bootstrap server and certs. This does not work. Any suggestion is appreciated.
One another option is to combine the certs into one jks file and set global bootstrap and ssl configuration.
kafka:
consumers:
group1:
bootstrap:
servers: $someserver1
ssl:
keystore:
location: /keystore.jks
password: password
truststore:
location: /truststore.jks
password: password
type: PKCS12
security:
protocol: ssl
group2:
bootstrap:
servers: $someserver2
ssl:
keystore:
location: /keystore-1.jks
password: password
truststore:
location: /truststore-1.jks
password: password
type: PKCS12
security:
protocol: ssl

Answer 1

The above config does work fine but u need to disable the kafka health check or provide a combined ssl certs otherwise micronaut kafka health check fails. This is true with 1.2.7 version of micronaut.
kafka:
health.enabled: false
consumers:
group1:
bootstrap:
servers: $someserver1
ssl:
keystore:
location: /keystore.jks
password: password
truststore:
location: /truststore.jks
password: password
type: PKCS12
security:
protocol: ssl
group2:
bootstrap:
servers: $someserver2
ssl:
keystore:
location: /keystore-1.jks
password: password
truststore:
location: /truststore-1.jks
password: password
type: PKCS12
security:
protocol: ssl

Answer 2

The issue is that consumer classes are not annotated quite correctly. Here is what worked for me:
In application.yml (note that both servers use the same SSL certs):
kafka:
consumers:
group1:
topic: some-topic-1
bootstrap:
servers: server-1:9092
group2:
topic: some-topic-2
bootstrap:
servers: server-2:9092
ssl:
keystore:
location: /keystore.jks
password: "password1"
truststore:
location: /truststore.jks
password: "password2"
security:
protocol: ssl
Then, annotate your classes to set consumer specific properties. Example in Kotlin:
#KafkaListener(
properties = [Property(name = ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, value = "\${kafka.consumers.group1.bootstrap.servers}")],
)
class TestConsumer1() {
#Topic("\${kafka.consumers.group1.topic}")
fun receiveMessage(record: ConsumerRecord<String, String>, acknowledgement: Acknowledgement) {
TODO("Handle events from server-1:9092 and some-topic-1 :)")
}
}
More information on the #KafkaListener and properties can be found here: https://micronaut-projects.github.io/micronaut-kafka/latest/guide/#kafkaListenerConfiguration