Unable to trigger Custom Producer Interceptor via JDBC Source Connector - apache-kafka

Question

I have created a custom Producer Interceptor (AuditProducerInterceptor) which accepts some custom configs(application_id, type etc.). I have generated a jar from the AuditProducerInterceptor project and placed the jar inside kafka-connect at /usr/share/java/monitoring-interceptors. When i try to post JDBC-Source connector with below configurations, my audit interceptor is not triggered.
{
"name": "jdbc-source-xx-xxxx-xxx-xxx",
"config": {
"connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
"connection.url": "jdbc:sqlserver://{{ip}}:1433;databaseName=XX;useNTLMv2=true",
"connection.user": "SA",
"connection.password": "Admin1234",
"producer.interceptor.classes": "com.optum.payer.common.kafka.audit.interceptor.AuditProducerInterceptor",
"topic.prefix": "MyTestTopic",
"query": "SELECT ID, chart_id, request_id, UpdatedDate FROM xxx.xxx WITH (NOLOCK)",
"mode": "timestamp",
"timestamp.column.name": "UpdatedDate",
"producer.audit.application.id": "HelloApplication",
"producer.audit.type": "test type",
"poll.interval.ms": "10",
"tasks.max": "1",
"batch.max.rows": "100",
"validate.non.null": "false",
"numeric.mapping":"best_fit",
"key.converter": "io.confluent.connect.avro.AvroConverter",
"key.converter.schema.registry.url": "http://{{ip}}:8081",
"value.converter": "io.confluent.connect.avro.AvroConverter",
"value.converter.schema.registry.url": "http://{{ip}}:8081"
}}
As u can see in the configuration, i have added below props in connector config to trigger the custom interceptor. But I dont see any logs in Kafka-Connect related to AuditProducerInterceptor.
"producer.interceptor.classes": "com.optum.payer.common.kafka.audit.interceptor.AuditProducerInterceptor"
"producer.audit.application.id": "HelloApplication",
"producer.audit.type": "test type"
I tried adding these three config in kafka-connect config and I am able to trigger the interceptor. But I want to trigger the interceptor via JDBC source connector so that i can pass the custom props(application_id,type etc) via connector.
Please help me solve this issue

Answer 1

If you have allowed client overrides (enabled by default) in the Connect worker, you'll want to use producer.override prefix
From docs
Starting with 2.3.0, client configuration overrides can be configured individually per connector by using the prefixes producer.override. and consumer.override. for Kafka sources or Kafka sinks respectively.