I am using org.hornetq.api.core.client
how can I guarantee the message that I am sending actually reached the queue (not the client, just the queue) ?
producer.send("validQueue",clientMessage)
please note that the queue is a valid queue .
this similar question is referring to invalid queue. other ones such as this one is relevant to the delivery to the client .
It really depends on how you are sending.
First question of yours was about
First of all, on JMS you have to way to send to an invalid queue since the producer will validate the queue's existence. On HornetQ core api you send to an address (not to a queue), and you may have an unbound queue. So you have to query if the address has queues or not.
Now, for confirmation the message was received:
Scenario I, persistent messages, non transactionally
Every message is sent blocked. The client will unblock as soon as the server acknowledged receiving the message. This is done automatically.. you don't have to do anything.
Scenario II, non persistent messages, non transactionally
There are no confirmations by default. The message is sent asynchronously. We assume the message is transient and it's not a big deal if you lost it. you can change that by setting block-on-non-persistent-send on the ServerLocator.
Scenario III, transactionally (either persistent or not).
As soon as you call commit the message is on the queues.
Scenario IV, Confirmation send
You set a callback and you get a method call as soon as the server acked it on the queues. Look on the manual for confirmation callback. There's also the same feature on JMS2.
Related
I have an [Azure Storage] queue where I put e-mail messages to be sent. Then, there is a separate service which monitors that queue and send e-mails using some service. In this particular case I'm using SendGrid.
So, theoretically, if the sender crashes right after a successful call to SendGrid Mail Send API (https://sendgrid.com/docs/API_Reference/Web_API_v3/Mail/index.html), the message will be returned to the queue and retried later. This may result in the same e-mail being delivered more than once, which could be really annoying for some type of e-mail.
The normal way to avoid this situation would be to provide some sort of idempotency key to Send API. Then the side being called can make sure the operation is performed at most once.
After careful reading of SendGrid documentation and googling, I could not find any way to achieve what I'm looking for here (at most once semantic). Any ideas?
Without support for an idempotency key in the API itself your options are limited I think.
You could modify your email sending service to dequeue and commit before calling the Send API. That way if the service fails to send the message will not be retried as it has already been removed from the queue, it will be sent at most once.
Additionally, you could implement some limited retries on particular http responses (e.g. 429 & 5xx) from SendGrid where you are certain the message was not sent and retrying might be useful - this would maintain "at most once" whilst lowering the failure rate. Probably this should include some backoff time between each attempt.
I'm trying very hard to understand the flow of a web request to a server which has a queue or message broker in the middle, but I can't find information about when and where the reply is given.
Imagine this use case:
Client A:
sends a invoice order request
the invoice is enqueued
the request is processed and dequeued.
at which time the client will receive a response?
right after the message is received by the queue?
right after the message is processed and dequeued? Other?
I'm asking because if the reply only comes after the message being processed the client might wait a long time. Imagine the message takes 3 minutes to process, would the client need to keep requesting the server to see if it is processed? or a connection is maintained using something like long polling?
I'm interested in scenarios using RabbitMq and kafka.
Advantage of having a messaging system is to ensure the frontend webserver and backend processing is decoupled. Best practice is Web server should publish the message and just wait for the messaging system to acknowledge receiving the message.
I have a message a want to dequeue then right after its dequeued I want to queue another message to a different queue. I want to do all this in the same transaction. is this possible with rabbitmq or any other queueing service?
The closest you can get to what you want with RabbitMQ is:
Use acks and publisher confirms
You receive a message and do not ack it.
Send your reply message.
Wait for confirm from the broker.
Once confirm had arrived, ack initial message.
But then, consider this failure situation:
Initial message received
Reply message sent
Your service failed before ACKing initial message
When your service is back, it will receive the initial message again
So you will need to use some deduplication mechanism etc.
Here, at the end of this page. last paragraph ,
They mentioned some problems that occurs in This protocol.
i am unable to understand what are these problems. ?
for example. He told. "If a request processing long time"
I am unable to understand this statement. Where is the request which processing taking long time, on client ? or on server ?
Or i am unable to understand where is the Clock(time) ? is it on Client side or Server Side? because here mentioned in the end of 2 point. "if the reply is not received within the time period , the kernel of the client machine re-transmits the request message."
Consider this:
The client sends a message. If it doesn't get a reply from the server within - say - 1 minute it will transmit the message again.
When the server receives a message, it only sends a reply after having generated a full response to the message that the client sent.
No suppose you, as client, send a message to the server. The server receives your message, and starts processing it. At this time, you, the client, have no idea of whether the server got the message or not. Assume you send a complicated task to the server, which takes it 1 minute and 5 seconds to complete. After 1 minute (ignoring transmission times), the server is still busy doing your work, but you as the client don't know of any of this and send your message again.
Now, depending on the actual protocol implementation, there are a few potential issues:
It's possible that by sending the message again, you increase some sequence count and are therefore unable to receive the reply to the original message afterwards.
It's possible that the server isn't able to determine whether a message that arrives is the first message or a message that had to be send again. So it could be doing work that it already did, leading either to needless processing or in the worst case to (business) logic errors.
Additionally, by sending both the message and the reply possibly needless more than once, you increase the amount of total data transmitted, without gaining anything from it.
To "solve" this, you could increase the waiting time before the client sends its message again. This will "fix" the issue with long running tasks on the server, but will also hurt in case the message actually got lost on the way, because you're waiting longer to even send a new message.
The "real" solution here is to have the server acknowledge as soon as it receives a message from the client, just as saying "i got your message, i'll send the reply soon!" before even starting to actually process the message.
If a quick fix session is created by server(acceptor) at say 9AM, but the StartTime is at 11AM. This means the session exists but not active.
If the server receives an unsolicited message from an exchange that it needs to send on this session, will it persist this if I have configuration PersistMessages=Y and sends it to the client(initiator) when it connects after 11AM?
No, it would not persist messages received before start time and would send you a reject message. The message will be rejected at the interface itself, message isn't handled. You would have to resend it to get a response.
QuickFIX does persist (but not send) messages before a session is connected. The sequence numbers are updated and when the session is connected and the first message is sent, the counterparty FIX engine will see the gap in the sequence numbers and request a resend. QuickFIX will then resend the persisted messages. However, depending on your QuickFIX configuration, the outgoing messages might be considered to be too old and rejected locally.
As I understand, these are kept to take into account timings under which corresponding exchange would accept the orders.
Application or its sub-modules do not need to maintain timings and take some action on closing the fix session. Rather, QuickFix shall automatically deactivate the session.
Persistence of the message or re-sesnding when the session becomes active does not look desirable to me.
You can rather maintain some kind of queue to buffer such messages in sending application, and send them only when the time matches with active session timings.
That's my thoughts, hope that helps.