My system has one server and many clients of 2 types. First type of client sends events to server. Second type receives notifications from server on these events. I'm currently testing RabbitMQ and NServiceBus to build message queue with the following requirements:
First type of client should have incoming queue for events (physically running on it) to prevent data loss on server disconnection.
Server should have outgoing queue for notifications (physically running on it) to prevent data loss on second type client disconnection.
[Client Type 1 + queue] -> [Server + queue] -> [Client Type 2]
Can this be achieved with one of specified components (or both of them)? If yes how?
I am not very familiar with rabbit, so I'll answer the question with nservicebus (NSB) in mind.
My system has one server and many clients of 2 types
OK, first thing, NSB does not have equivalent concepts of client and server. In NSB, all participating applications are called endpoints or services. Some endpoints are publishers, some are subscribers, some are senders, some are receivers. Some are any combination of the above.
First type of client sends events to server.
By convention, there are two types of message in NSB, commands and events. Commands are sent, events are published. So in this scenario, the type 1 clients would send commands to the server. In this scenario, a type 1 client would be a sender endpoint. The server is therefore a receiver endpoint.
Second type receives notifications from server on these events
So in this scenario, the server is a publisher endpoint, and a type 2 client is a subscriber endpoint. The server will publish an event which all subscribers would receive.
First type of client should have incoming queue for events (physically running on it) to prevent data loss on server disconnection
I am assuming that you mean the type 1 client needs to receive the data which it needs to send to the server from somewhere.
Well, in NSB, every endpoint has a queue, called the input queue. This is the means by which the endpoint receives messages.
In NSB the queue transport is abstracted, but out of the box the default is MSMQ (NSB also has support for Rabbit as the queue transport).
This provides a store-and-forward messaging pattern, which guarantees reliability. What this means is that if the server is unavailable, the queuing transport would wait until it was available again before transmitting the message.
So you could send a message onto the type 1 client input queue, which would then be converted into a command and sent to the server.
Server should have outgoing queue for notifications (physically
running on it) to prevent data loss on second type client
disconnection.
Similarly, when the server publishes the event (on receipt of the command from the type 1 client), the queuing transport would guarantee delivery of the event to all the subscribing type 2 clients.
A point of note: this would not be based on the server having an "outgoing queue", but rather the queuing transport would deliver the messages to the input queues of all subscribing endpoints.
So all your scenarios would be satisfied by using NServiceBus as part of your approach.
Related
I'm trying very hard to understand the flow of a web request to a server which has a queue or message broker in the middle, but I can't find information about when and where the reply is given.
Imagine this use case:
Client A:
sends a invoice order request
the invoice is enqueued
the request is processed and dequeued.
at which time the client will receive a response?
right after the message is received by the queue?
right after the message is processed and dequeued? Other?
I'm asking because if the reply only comes after the message being processed the client might wait a long time. Imagine the message takes 3 minutes to process, would the client need to keep requesting the server to see if it is processed? or a connection is maintained using something like long polling?
I'm interested in scenarios using RabbitMq and kafka.
Advantage of having a messaging system is to ensure the frontend webserver and backend processing is decoupled. Best practice is Web server should publish the message and just wait for the messaging system to acknowledge receiving the message.
I am using org.hornetq.api.core.client
how can I guarantee the message that I am sending actually reached the queue (not the client, just the queue) ?
producer.send("validQueue",clientMessage)
please note that the queue is a valid queue .
this similar question is referring to invalid queue. other ones such as this one is relevant to the delivery to the client .
It really depends on how you are sending.
First question of yours was about
First of all, on JMS you have to way to send to an invalid queue since the producer will validate the queue's existence. On HornetQ core api you send to an address (not to a queue), and you may have an unbound queue. So you have to query if the address has queues or not.
Now, for confirmation the message was received:
Scenario I, persistent messages, non transactionally
Every message is sent blocked. The client will unblock as soon as the server acknowledged receiving the message. This is done automatically.. you don't have to do anything.
Scenario II, non persistent messages, non transactionally
There are no confirmations by default. The message is sent asynchronously. We assume the message is transient and it's not a big deal if you lost it. you can change that by setting block-on-non-persistent-send on the ServerLocator.
Scenario III, transactionally (either persistent or not).
As soon as you call commit the message is on the queues.
Scenario IV, Confirmation send
You set a callback and you get a method call as soon as the server acked it on the queues. Look on the manual for confirmation callback. There's also the same feature on JMS2.
I'm transferring our web application to new infrastructure and I'm stuck at the MSMQ part.
1st screenshot: Server A sends messages to server B. I see the outgoing messages appear on server A.
2nd screenshot: Server B shows incoming traffic, but the messages don't appear in the queue.
The service picking up the messages at server B is not running!
Any ideas how to debug this situation?
The status of the outgoing queue is connected but the messages aren't moving. Likely to be that the acknowledgement messages are not being sent back successfully from server B. As server A never sees the acknowledgements, it is stuck in a permanent state of retrying to send awaiting a response. There should be an outgoing queue on server B pointing back to server A. Check its status. It is very likely that the IP address of the outgoing queue is incorrect.
If the messages are queuing in your outgoing queue on server A that means that they are definitely not being sent to the destination queue on server B.
If you have messages arriving on server B but not being delivered then this is probably due to queue permissions. However, based on your assertion that messages queue up on the outbound queue I can't see how server B can be receiving any messages.
I have a strange issue setting up an existing application on our new internal Cloud.
I have a simple messaging system that pushes a message from one server (Server1) onto a MSMQ on another server (Server2). The messages disappear off the outbound but never appear in the inbound queue.
When I take Server2 msmq off line the messages build up on Server1. Restarting Msmq on Server2 causes the messages in the outbound queue on Server1 to disappear - but the message still never arrives at Server2.
The details:
MSMQ is set up in Workgroup mode, as that's the virtual networks requirement.
Queues are private.
Permissions are set to allow certain users access.
Has anybody any ideas on why this is happening or how I could track down the issue.
It could be that the remote private queue is a transactional queue and you send the message as non-transactional or vice versa. If the transaction setting on the queue and the message does not match, the message will disappear!
I have seen this in the past with the direct format name where it was set to something like
DIRECT=OS:192.16.8.0.1\PRIVATE$\MyQueue
where I should have specified DIRECT=TCP:192.168.0.1\PRIVATE$\MyQueue
see:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms700996(v=vs.85).aspx
#John Breakwell had noted here http://blogs.msdn.com/b/johnbreakwell/archive/2010/01/22/why-does-msmq-keep-losing-my-messages.aspx:
Server name used to address message doesn't match destination machine
When MSMQ receives a message from over the wire, it always validates that this machine is the correct recipient. This is to ensure that something like a DNS misconfiguration does not result in messages being delivered to the wrong place. The messages are, instead, discarded unless the IgnoreOSNameValidation registry value is set appropriately. You may want to do this with an Internet-facing MSMQ server, for example, where the domain and server names visible to MSMQ clients on the Internet often bear no resemblance to the real ones (for good security reasons).
It sounds like a permissions or addressing issue.
Try to enable the event log under Applications and Services Logs -> Microsoft -> Windows -> MSMQ called End2End.
This log should tell you exactly what is going wrong with the delivery of messages to the expected destination queue.
Nope: For every successful delivery there should be three events raised in this log:
Message with ID blah came over the network (ie, message has arrived from a remote sender)
Message with ID blah was sent to queue blah (ie, message forwarded to local queue)
Message with ID blah was put into queue blah (ie, message arrives in local queue)
Assumes you are using Server 2008 and above.
You can add Negative Source Journaling to the sending application code to find out exactly what the root cause is. Most likely one of the two answers you have already received.
Are the messages arriving in the dead-letter queue on Server 2?
I am basically writing a XMPP client to automatically reply to "specific" chat messages.
My setup is like this:
I have pidgin running on my machine configured to run with an account x#xyz.com.
I have my own jabber client configured to run with the same account x#xyz.com.
There could be other XMPP clients .
Here is my requirement:
I am trying to automate certain kind of messages that I receive on gtalk. So whenever I receive a specific message eg: "How are you" , my own XMPP client should reply automatically with say "fine". How are you". All messages sent (before and after my client replies) to x#xyz.com but should be received by all clients (my own client does not have a UI and can only respond to specific messages.).
Now I have already coded my client to reply automatically. This works fine. But the problem I am facing is that as soon as I reply (I use the smack library), all subsequent messages that are sent to x#xyz.com are received only by my XMPP client. This is obviously a problem as my own client is quite dump and does not have a UI, so I don't get to see the rest of the messages sent to me, thereby making me "lose" messages.
I have observed the same behavior with other XMPP clients as well. Now the question is, is this is a requirement of XMPP (I am sorry but I haven't read XMPP protocol too well). Is it possible to code an XMPP client to send a reply to a user and still be able to receive all subsequent messages in all clients currently listening for messages? Making my client a full fledged XMPP client is a solution, but I don't want to go that route.
I hope my question is clear.
You may have to set a negative presence priority for your bot..
First thing to know is that in XMPP protocol every client is supposed to have a full JID. This is a bare JID - in your case x#xyz.com with a resource in the end e.g. x#xyz.com/pidgin or x#xyz.com/home (where /pidgin and /home are the resource). This is a part of how routing messages to different clients is supposed to be achieved.
Then there are the presence stanzas. When going online a client usually sends a presence stanza to the server. This informs about e.g. if the client is available for chat or away for lunch. Along with this information can be sent a priority. When there are more than one clients connected the one with the highest priority will receive the messages sent to the bare JID (e.g. ClientA(prio=50) and ClientB(prio=60) -> ClientB receives the messages sent to x#xyz.com). But there are also negative priorities. A priority less than 0 states that this client should never be sent any messages. Such a stanza might look like this
<presence from="x#xyz.com/bot">
<priority>-1</priority>
</presence>
This may fit your case. Please keep in mind it also depends on the XMPP server where your account is located, which may or may have not fully implemented this part of the protocol.
So to summarize: I recommend you to look through the Smack API how to set a presence and set the priority to <0 for your bot client right after it connected.