I am trying to deveopemet synchronous poc. My objective is i want to send message to request topic and get back updated message. I have request-topic for producer, and reply-topic with consumer.
i am able send the message to consumer and able to update the message as well. but i am not getting response on consumer side. getting an error " No pending reply-timeout, perhaps using shared reply topic.
Related
I have an application in which there is an https service that receives a message and there are a few microservices that process this message and at the end the message is put into a Kafka queue.
I want to know the performance of the microservices end to end.
I want to run tests and want to calculate the time difference between message sent to the http service and time message received from http Kafka queue.
Can we do this with Gatling?
In a spring boot app, i use JMS with QPID to receive messages from an Azure ServiceBus Queue.
I create my own connection factory with properties:
SessionsAcknownlegdeMode: CLIENT_ACKNOWlEDGE
RedeliveryPolicy Outcome: REJECTED
MaxRedelivery: 5
I use the annotation #JmsListener
Problem: When I consume the message, we try to send a mail with JavaMail, this normally works but it happened that the smtp server we use was having problem so the org.springframework.mail.MailSendException was thrown.
The message is correctly retried and put in DLQ after max retries but after a few messages in error, my #JmsListener method is not invoked for the following messages, and they are put directly in DLQ. That is not what I want.
I tried to replicate this behavior locally by manually throwing exceptions in the listener for given messages, but the consumer correctly sends bad messages to DLQ and consumes good messages.
Does anyone know what is happening ?
Suppose that kafka message producer send an event message to a topic. And then a consumer process this event message. However, this consumer process it throw exception because business error, so he want to let the message producer know it and resent again.
Are there any solution?
You should, generally, not create such coupling between producers and consumers.
If a consumer fails to read a message then it ideally shouldn't commit that offset. If the record didn't expire in Kafka, then there's no need for the producer to send anything as data is still in the broker, so having the consumer application be restarted will attempt to read the failed offsets again.
If you want it to never miss a message but might duplicate the occasional message, you can use delivery semantic "at-least-once".
I'm trying very hard to understand the flow of a web request to a server which has a queue or message broker in the middle, but I can't find information about when and where the reply is given.
Imagine this use case:
Client A:
sends a invoice order request
the invoice is enqueued
the request is processed and dequeued.
at which time the client will receive a response?
right after the message is received by the queue?
right after the message is processed and dequeued? Other?
I'm asking because if the reply only comes after the message being processed the client might wait a long time. Imagine the message takes 3 minutes to process, would the client need to keep requesting the server to see if it is processed? or a connection is maintained using something like long polling?
I'm interested in scenarios using RabbitMq and kafka.
Advantage of having a messaging system is to ensure the frontend webserver and backend processing is decoupled. Best practice is Web server should publish the message and just wait for the messaging system to acknowledge receiving the message.
I have a message a want to dequeue then right after its dequeued I want to queue another message to a different queue. I want to do all this in the same transaction. is this possible with rabbitmq or any other queueing service?
The closest you can get to what you want with RabbitMQ is:
Use acks and publisher confirms
You receive a message and do not ack it.
Send your reply message.
Wait for confirm from the broker.
Once confirm had arrived, ack initial message.
But then, consider this failure situation:
Initial message received
Reply message sent
Your service failed before ACKing initial message
When your service is back, it will receive the initial message again
So you will need to use some deduplication mechanism etc.