Troubleshooting high time taken to publish event to Kafka - apache-kafka

Question

Publishing to kafka takes 3 seconds in one of the environment, however we see that, in other environments it takes only 20 milli seconds .
We have done following to troubleshoot this issue:
We have run route trace and we dont see an network latency.
Analyzed the logs and found following :
2023-01-05T07:26:24.627Z DEBUG 40 --- [ad | producer-5] o.a.k.c.NetworkClient : [Producer clientId=producer-5] Using older server API v7 to send PRODUCE {acks=-1,timeout=30000,partitionSizes=[esd.mob.vocc.datacollector.sampio.plab01.raw-9=464]} with correlation id 1156 to node 46
2023-01-05T07:26:24.627Z TRACE 40 --- [ad | producer-5] o.a.k.c.p.i.Sender : [Producer clientId=producer-5] Sent produce request to 46: (type=ProduceRequest, acks=-1, timeout=30000, partitionRecords=({esd.mob.vocc.datacollector.sampio.plab01.raw-9=MemoryRecords(size=464, buffer=java.nio.HeapByteBuffer[pos=0 lim=464 cap=464])}), transactionalId=''
2023-01-05T07:26:27.700Z TRACE 40 --- [ad | producer-5] o.a.k.c.NetworkClient : [Producer clientId=producer-5] Completed receive from node 46 for PRODUCE with correlation id 1156, received {responses=[{topic=esd.mob.vocc.datacollector.sampio.plab01.raw,partition_responses=[{partition=9,error_code=0,base_offset=31376,log_append_time=-1,log_start_offset=31001}]}],throttle_time_ms=0}
2023-01-05T07:26:27.700Z TRACE 40 --- [ad | producer-5] o.a.k.c.p.i.Sender : [Producer clientId=producer-5] Received produce response from node 46 with correlation id 1156
the log clearly states that it takes 3 seconds to publish the message .
We are using below mentioned code .
CompletableFuture<SendResult<String, String>> sendResultCompletableFuture = kafkaTemplate.send(payloadMsg);
sendResultCompletableFuture.whenComplete((data, ex) -> {
if (Objects.nonNull(ex)) {
LOGGER.error("Exception while publishing message to kafka, {}", ExceptionUtils.getStackTrace(ex));
throw new KafkaConnectorException(ExceptionUtils.getStackTrace(ex), topic, message);
} else {
LOGGER.info("Message handle published to kafka successfully for key ::");
}
});
Please help in troubleshooting this issue further and share thoughts.