I have one MSMQ queue which is listened by five windows services. I used BeginPeek and PeekCompleted event for this purpose. My problem is among five services, only one service is the right recipient of the message. All four just read message, but no action is performed. This can only be identified when we read MQ message.
Now, I added a code in my services to check, if the criteria matches and the message is being processed by the right service, then I am using Receive to dequeue the message from MSMQ. Is that a good idea?
Secondly, If the message doesnot satisfy condition and all five services just peeked it, but not received, the message still lies in queue. I understand. But the same message is being processed infinite times, as the message was never removed.
private void queue_PeekCompleted(object sender, PeekCompletedEventArgs e)
{
MessageQueue queue = (MessageQueue)sender;
//Message msg = queue.EndPeek(e.AsyncResult);
Message msg = e.Message;
//Read message and check if the criteria matches
if(CriteriaMatches)
{
queue.ReceiveById(e.Message.Id);
}
queue.EndPeek(e.AsyncResult);
queue.BeginPeek();
}
Appreciate your help.
Thanks,
Fayaz
Set the messages to expire after a set (short) period. They will then move to the dead letter queue where you can have another service waiting for arrivals. This service could then raise an alert, for example, as soon as a message arrives.
Related
I am using BizTalk Server SB-Messaging adapter to retreive messages from Azure Service Bus Queue. I have successfully managed to send message to queue myself (using same adapter), and retreive message from queue and do further processing.
Problem arises when a 3rd party software supplier is sending messages to the queue, and for BizTalk Server to retreive and process message. I then receive the following additional "header"-information and control characters in the beginning of the message:
In text: #ACKstringBShttp://schemas.microsoft.com/2003/10/Serialization/?$SOH
Seems like there is some sort of enveloped message, including headers to handle ACKnowledgement of the message to the queue.
SB-Messaging adapter gave following initial error message:
"The WCF service host at address has faulted and as a result no
more messages can be received on the corresponding receive location.
To fix the issue, BizTalk Server will automatically attempt to restart
the service host."
And, another error message:
"No Disassemble stage components can recognize the data."
Did anyone hit this problem before, and, what can be the cause of the problem? Can character encoding be a possible cause of this problem?
Here comes the feedback!
Turned out 3rd party software supplier had a setting to send message as stream, instead of string. Turns out it is a .Net application using BrokeredMessage object. Using string makes message serialized, and meta-data is added to the message. Using stream, no such serialization takes place, and message is kept untouched.
So, problem was using string and automatic serialization when sending to Service Bus Queue.
I have legacy Microsoft.ServiceBus.Messaging clients sending BrokeredMessage Xml content as <string> and I want to receive using the latest Microsoft.Azure.ServiceBus library and Message type.
Using Encoding.UTF8.GetString(message.Body) I get a unusable string prefaced with
#\u0006string\b3http://schemas.microsoft.com/2003/10/Serialization/��#
My approach is to explicitly use XmlDictionaryReader binary deserialization to undo the hidden serialization magic from the legacy library
private static string GetMessageBodyHandlingLegacyBrokeredMessage(Message message)
{
string str = null;
if (message.ContentType == null)
{
using (var reader = XmlDictionaryReader.CreateBinaryReader(
new MemoryStream(message.Body),
null,
XmlDictionaryReaderQuotas.Max))
{
var doc = new XmlDocument();
doc.Load(reader);
str = doc.InnerText;
}
}
else
throw new NotImplementedException("Unhandled Service Bus ContentType " + message.ContentType);
return str;
}
References
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messages-payloads#payload-serialization
https://carlos.mendible.com/2016/07/17/step-by-step-net-core-azure-service-bus-and-amqp/
I am new NATS. Not sure how NATS request reply works.
As per my understanding, this pattern can be use for bi-directional communication but questions is, Does it works between same message id/thread ? If not, can't we use two different queue for the same purpose? How it is different from pub-sub or queue pattern of NATS?
Can someone provide more use case on this?
Thanks.
You added nats-streaming-server tag, so I would first want to clarify that there is no request/reply API in NATS Streaming, because it does not really make sense.
In NATS, you would use request/reply when your publishing application wants to know that the subscribing application did receive and process the message. It is an end-to-end confirmation that the published message was received and processed.
It can also be simply because the subscribing application processes a job and send the result of that job back to the requestor.
A simple example would be:
// Request will create an internal subscription on
// a private inbox and set it to the message's Reply
// field.
msg, err := nc.Request("job", payload, time.Second)
if err != nil {
...
} else {
// msg is the reply sent by the subscribing application.
}
In the other side, you would have registered a subscription to handle the job requests.
nc.Subscribe("job", func(req *nats.Msg) {
// req is the request received by the publisher above.
// Send back a reply to the request reply subject.
nc.Publish(req.Reply, []byte(reply))
})
Not sure what language you use, but here is a link to the Go client
I'm using Kafka 0.11.0.0. I have a test program that publishes to a Kafka topic; if the zookeeper and Kafka servers are down (which is normal in my development environment; I bring them up as needed) then the call to KafkaProducer<>.send() hangs indefinitely.
I either need to have send() return, preferably indicating the error; or I need a way to check whether the servers are up or down. Basically, I want my test tool to be able tell me, "Hey, dummy, start up Kafka!" instead of hanging.
Is there a way for my producer task to determine whether the servers are up or down?
I'm calling the send() like this:
kafkaProducer.send(new ProducerRecord<>(KAFKA_TOPIC, KAFKA_KEY,
message), (rm, ex) -> {
System.out.println("**** " + rm + "\n**** " +ex);
});
I have linger.ms = 1; I've tried retries=0, 1, and 2, and send() still blocks. I've never seen the callback called.
Older messages suggest setting metadata.fetch.timeout.ms to a small value, but that's gone in 0.11. Others suggest calling command line utilities to see if the servers are OK...but the referenced utilities also seem to be gone.
What's the graceful way to get this done?
We can send messages to broker in three ways :
Fire-and-forget :
We send a message to the server and don’t really care if it arrives successfully or not. Most of the time, it will arrive successfully, since Kafka is highly available and the producer will retry sending messages automatically. However, some messages will get lost using this method.
Asynchronous send
We call the send() method with a callback function, which gets triggered when it receives a response from the Kafka broker.
Synchronous send
We send a message, the send() method returns a Future object, and we use get() to wait on the future and see if the send() was successful or not.
The simplest way to send a message synchronously is as follows:
ProducerRecord<String, String> record =
new ProducerRecord<>(KAFKA_TOPIC, KEY, message);
try {
producer.send(record).get();
} catch (Exception e) {
e.printStackTrace();
}
Here, we are using Future.get() to wait for a reply from Kafka. This method will throw an exception if the record is not sent successfully to Kafka. If there were no errors, we will get a RecordMetadata object that we can use to retrieve the offset the message was written to.
hope this helps.
That is strange. It should return with an error saying either "Failed to update metadata" or "Expiring x number of records".
Check request.timeout.ms and max.block.ms setting for your producer. By default request.timeout.ms is 60 seconds long
Let's say I try to send to an authenticated transactional queue,
by calling msg.send(object,MessageQueueTransactionType.Single), message does not receive in transactional queue, no exception thrown.
What I want to accomplish is after sending, if message fail to send, perform some function and abort transaction, yet it doesn't throw exception, so I am unable to process it.
I am sending object from Web Application in local to local message queue.
My code is as follows in my web application:
MessageQueueTransaction mqTran=new MessageQueueTransaction();
try
{
using(System.Messaging.Message msg=new System.Messaging.Message(){
mqTran.Begin();
MessageQueue adminQ = new MessageQueue(AdminQueuePath);
MessageQueue msgQ = new MessageQueue(queuePath);
msgQ.DefaultPropertiesToSend.Recoverable = true;
msg.body = object;
msg.Recoverable=true;
msg.Label="Object";
msg.TimeToReachQueue=new TimeSpan(0,0,30);
msg.AcknowledgeType=AcknowledgeTypes.FullReachQueue;
msg.ResponseQueue=adminQ;
msg.AdministrationQueue=adminQ;
msgQ1.Send(msg,MessageQueueTransactionType.Single);
mqTran.Commit();
}
catch(Exception e)
{
mqTran.Abort();
//Do some processing if fail to send
}
It's not going to throw an exception for failure to deliver, only for failure to place on the queue. One of the points of message queueing is that the messages are durable so that you can take appropriate measures if delivery fails. This means you need to program another process to read the dead letter queue. The image below is taken from MSDN.
Because the entire process is asynchronous, your code flow is not going to be exception-driven the way your code block would like. Your transaction is simply the "sending transaction" in this workflow.
Recommendation: Check your message queue to find the messages, either in the outgoing queue or the transactional dead-letter queue.
I have program which reads MSMQ using GetAllMessages but it does not remove messages from Queue so I have following code; which keep getting same messages. I do not want to process same message again and again. How can I make sure that MSMQ deletes those already received messages or atleast I don't receive it ?
while()
{
Messages[] receivedMessage = queue.GetAllMessages()
foreach(Message msg in receivedMessage)
{
... Processing
}
}
GetAllMessages() gives you a copy of the messages in the queue, but doesn't delete them.
Use any of the Receive methods to receive and remove the messages from a queue