So, I am new to MSMQ and NServiceBus. I played around with demos and got a working scenario going with NServiceBus. (Getting my own up and running was even easier than following the demo thanks to the new Modeling tools!)
I then went and presented my plan (based off my work with the demo and my own model) to my co-workers. Two of them were versed in using MSMQ and started asking me questions about how I will handle "Dead Letters".
I had never heard of "Dead Letters". They explained that it is queue used for messages that things cannot be sent (either because the other end refuses them or if the other end is not there).
The concern of my co-workers is that if we don't have Dead Letter queues then how will we stop a message from blocking the queue? (If the queue is FIFO and the top message can't be sent, then it blocks the other messages behind it right?)
On the other hand, if we have "Dead Letter" queues how are they managed? (Do I get an event from NServiceBus that tells me a new message is in the Dead Letter queue? How do I configure when a message will go to the Dead Letter queue? How can I try to re-send a Dead Letter message?)
So basically, how does NServiceBus deal with undeliverable messages?
Typically, you specify an error queue as well, which is where messages that couldn't be sent will go. Look at their MSMQ example:
http://docs.particular.net/nservicebus/msmq/transportconfig &
http://docs.particular.net/nservicebus/msmq/connection-strings
<MsmqTransportConfig InputQueue="MyClient" ErrorQueue="error"
NumberOfWorkerThreads="1" MaxRetries="5"/>
In this scenario, the error queue will reside on the same machine as the input queue. This may also be a remote queue, if preferred. MaxRetries refers to the number of attempts to send before it gets put into the error queue. How you choose to handle the error queue is up to you, however.
Related
I am using Spring Cloud Stream over RabbitMQ for my project. I have a processor that reads from a source, process the message and publish it to the sink.
Is my understanding correct that if my application picks up an event from the stream and fails (e.g. app sudden death):
unless I ack the message or
I save the message after reading it from the queue
then my event would be lost? What other option would I have to make sure not to lose the event in such case?
DIgging through the Rabbit-MQ documentation I found this very useful example page for the different types of queues and message deliveries for RabbitMQ, and most of them can be used with AMPQ.
In particular looking at the work queue example for java, I found exactly the answer that I was looking for:
Message acknowledgment
Doing a task can take a few seconds. You may wonder what happens if
one of the consumers starts a long task and dies with it only partly
done. With our current code, once RabbitMQ delivers a message to the
consumer it immediately marks it for deletion. In this case, if you
kill a worker we will lose the message it was just processing. We'll
also lose all the messages that were dispatched to this particular
worker but were not yet handled. But we don't want to lose any tasks.
If a worker dies, we'd like the task to be delivered to another
worker.
In order to make sure a message is never lost, RabbitMQ supports
message acknowledgments. An ack(nowledgement) is sent back by the
consumer to tell RabbitMQ that a particular message has been received,
processed and that RabbitMQ is free to delete it.
If a consumer dies (its channel is closed, connection is closed, or
TCP connection is lost) without sending an ack, RabbitMQ will
understand that a message wasn't processed fully and will re-queue it.
If there are other consumers online at the same time, it will then
quickly redeliver it to another consumer. That way you can be sure
that no message is lost, even if the workers occasionally die.
There aren't any message timeouts; RabbitMQ will redeliver the message
when the consumer dies. It's fine even if processing a message takes a
very, very long time.
Manual message acknowledgments are turned on by default. In previous
examples we explicitly turned them off via the autoAck=true flag. It's
time to set this flag to false and send a proper acknowledgment from
the worker, once we're done with a task.
Thinking about it, using the ACK seems to be the logic thing to do. The reason why I didn't think about it before, is because I thought of a ACK just under the perspective of the publisher and not of the broker. The piece of documentation above was very useful to me.
I use NServiceBus with MSMQ on Windows Server 2008.
In one of our scenario we send a command to another server queue but sometimes the message never arrives in the queue and MSQM doesn't throw an exception.
I have written a small application that sends a message directly to the queue without NServiceBus and I have encountered the same problem.
I think it isn't a nservicebus problem but a MSMQ problem : windows security, network,...
How can I force MSMQ to return an error when the message can't be saved in queue ?
What are the possible reasons of this behavior ?
Where are the messages that have disappeared ?
In the MSMQ Management snap-in take a look at the system queues and look at transactional dead letter and non-transactional dead letter queues.
Also be sure to have setup queue ACL correctly.
Windows Server 2012
MSMQ 6 Workgroup Mode
We've had issues trying to recover MSMQ messages that were sent to the transaction dead letter queue. We've tried moving them to the outbound queue, the message seems to send fine (even the Event Log says so) however it never gets to the destination queue.
After trial and error we've figured out how to get them to another queue on the same server but not to the destination queue on a remote server. We don't want to lose anymore messages. Does anyone have any suggestion on how we can deliver these messages?
Thank you,
David
As I understood your question, it's a one time problem with some number of messages you already have in MSMQ, and not general connectivity issue between machines? If so, you should be able to solve it with some MSMQ management tool. Disclaimer: I'm the author of one such tool - QueueExplorer. I don't know what other tools can do, but with QueueExplorer you can copy/paste or drag/drop messages to another machine opened in separate tab/window. In order to do that QueueExplorer has to perform MSMQ Send operation, so messages will have to pass through MSMQ between these two machines.
So if there's still that issue that prevented original delivery you'll still be stuck. In that case you can save all messages to a file, transfer it to another machine through file system and load it there to whichever queue they should go. This is obviusly just a manual workaround for one time situation. Btw. this could be done in QueueExplore's trial mode.
If however problem is with connectivity and messages always end up in dead letter queue, it's better to check them from Computer Management. It's one area where it's better than our tool - you can turn on "Class" column and see reason why messages couldn't be delivered. For instance if you see "The time-to-be-received has elapsed" you'll know what's the problem.
I have a transactional private message queue (among other message queues on which I have not seen this problem) on a Windows Server 2008 R2 server.
This particular queue has a recurring problem happening every few weeks where the console shows a nonzero count of messages in the queue, but it does not have any messages in the queue itself or any subqueue. Queue Explorer shows the same thing. Performance counters indicate there are messages like the count in the built-in msmq console and queue explorer.
I cannot find any messages. I understand that I could see a situation like this for outgoing queues with dead letter tracking such that it may have been delivered to a remote machine but not yet processed. This is not an outgoing queue, though. Messages are sourced from remote machines and have landed here on this machine.
Also, I am certain that the count I'm seeing are not journal messages or subqueues.
Does this make any sense? Is there a logical explanation for this and under some circumstance this is expected? If so, what is it?
EDIT: Removed info about purging queue removing the count - that was incorrect. Purging actually does nothing and leaves me in the same state as before with a count reflected, but no messages showing.
As you noted, you can see a message count on an outgoing queue if source journaling is in use. The invisible messages are there in case they need to be moved to the DLQ.I would expect your problem to be similar - there should be a visible message in the outgoing queue and an invisible message in the destination queue because delivery hasn't completed. I assume a handshaking or storage acknowledgement has been lost along the way. Or maybe the message has been processed and removed from the queue but MSMQ couldn't update the sender of the fact. Check the outgoing queues on the remote machines sending TO this queue.
I'm working on a BizTalk application that sends messages to a Transactional MSMQ location. If it is unable to write to the Queue, the message gets sent to the Dead Letter Queue on the BizTalk server. Is there a better way to handle this? I would prefer the message to suspend so we would get notification from BizTalk360. If it is not possible to suspend the message, and the message must go to the Dead Letter Queue, how can I recover the message once the issue that prevented writing to the Queue is corrected?
Ok, so the best way that I have found to accomplish this is a bit of a hack in my opinion, but it works. I am setting up a receive port/location listening to the Dead Letter Queue at DIRECT=.\SYSTEM$;DEADXACT. I also added an orchestration that listens for messages coming from that receive port and immediately throws an exception suspending the message allowing BizTalk360 to send me notification that there is an issue.