I'm trying very hard to understand the flow of a web request to a server which has a queue or message broker in the middle, but I can't find information about when and where the reply is given.
Imagine this use case:
Client A:
sends a invoice order request
the invoice is enqueued
the request is processed and dequeued.
at which time the client will receive a response?
right after the message is received by the queue?
right after the message is processed and dequeued? Other?
I'm asking because if the reply only comes after the message being processed the client might wait a long time. Imagine the message takes 3 minutes to process, would the client need to keep requesting the server to see if it is processed? or a connection is maintained using something like long polling?
I'm interested in scenarios using RabbitMq and kafka.
Advantage of having a messaging system is to ensure the frontend webserver and backend processing is decoupled. Best practice is Web server should publish the message and just wait for the messaging system to acknowledge receiving the message.
At the server, CometD provides a MaxQueueListener hook to drop messages but if the Bayeux Client wants to stop receiving messages from a server without disconnecting , can it achieve that?
A BayeuxClient receives message from the server only for the channels it is subscribed to.
For a BayeuxClient to stop receiving messages from the server is enough to unsubscribe from all channels it subscribed to.
The BayeuxClient will still receive meta messages on meta channels that are part of the Bayeux protocol, but no application message will be delivered by the server.
There could be scenarios where client get disconnected from server just after receiving a message. Server is not sure if the client received the message or not and re-sends the message. Now, how does client identifies whether this message is new or already processed by it? Does XMPP associates any unique identifier with every message to eliminate duplicates?
Servers do not re-deliver messages by default, just because the server is "not sure" if the client received it. Sent to client == "delivered".
However for greater reliability, it is possible to use the XEP-0198 extension. This avoids duplicate messages because when the client reconnects to the server, it tells the server how many messages (stanzas, rather) it received. The server compares this to the number of stanzas it sent to the client. If the client did not receive them all, it re-sends only the ones that the client missed.
This mechanism avoids having to assign a unique identifier to every message.
I am using ejabberd server and ios xmppframework.
there are two clients, A and B.
When A and B are online, A can send message to B successfully.
If B is offline, B can receive the message when B is online again.
But when B is suddenly/unexpectedly lost connection, such as manually close wi-fi, the message sent by A is lost. B will never
receive this message.
I guess the reason is that B lost connection suddenly and the server still think B is online. Thus the offline message does work under this condition.
So my question is how to ensure the message that sent by A will be received by B? To ensure there is no messages lost.
I've spent the last week trying to track down missing messages in my XMPPFramework and eJabberd messaging app. Here are the full steps I went through to guarantee message delivery and what the effects of each step are.
Mod_offline
In the ejabberd.yml config file ensure that you have this in the access rules:
max_user_offline_messages:
admin: 5000
all: 100
and this in the modules section:
mod_offline:
access_max_user_messages: max_user_offline_messages
When the server knows the recipient of a message is offline they will store it and deliver it when they re-connect.
Ping (XEP-199)
xmppPing = XMPPPing()
xmppPing.respondsToQueries = true
xmppPing.activate(xmppStream)
xmppAutoPing = XMPPAutoPing()
xmppAutoPing.pingInterval = 2 * 60
xmppAutoPing.pingTimeout = 10.0
xmppAutoPing.activate(xmppStream)
Ping acts like a heartbeat so the server knows when the user is offline but didn't disconnect normally. It's a good idea to not rely on this by disconnecting on applicationDidEnterBackground but when the client looses connectivity or the stream disconnects for unknown reasons there is a window of time where a client is offline but the server doesn't know it yet because the ping wasn't expected until sometime in the future. In this scenario the message isn't delivered and isn't stored for offline delivery.
Stream Management (XEP-198)
xmppStreamManagement = XMPPStreamManagement(storage: XMPPStreamManagementMemoryStorage(), dispatchQueue: dispatch_get_main_queue())
xmppStreamManagement.autoResume = true
xmppStreamManagement.addDelegate(self, delegateQueue: dispatch_get_main_queue())
xmppStreamManagement.activate(xmppStream)
and then in xmppStreamDidAuthenticate
xmppStreamManagement.enableStreamManagementWithResumption(true, maxTimeout: 100)
Nearly there. The final step is to go back to the ejabberd.yml and add this line to the listening ports section underneath access: c2s:
resend_on_timeout: true
Stream Management adds req/akn handshakes after each message delivery. On it's own it won't have any effect on the server side unless that resend_on_timeout is set (which it isn't by default on eJabberd).
There is a final edge case which needs to be considered when the acknowledgement of a received message doesn't get to the server and it decides to hold it for offline delivery. The next time the client logs in they are likely to get a duplicate message. To handle this we set that delegate for the XMPPStreamManager. Implement the xmppStreamManagement getIsHandled: and if the message has a chat body set the isHandledPtr to false. When you construct an outbound message add an xmppElement with a unique id:
let xmppMessage = XMPPMessage(type: "chat", to: partnerJID)
let xmppElement = DDXMLElement(name: "message")
xmppElement.addAttributeWithName("id", stringValue: xmppStream.generateUUID())
xmppElement.addAttributeWithName("type", stringValue: "chat")
xmppElement.addAttributeWithName("to", stringValue: partnerJID.bare())
xmppMessage.addBody(message)
xmppMessage.addChild(xmppElement)
xmppMessage.addReceiptRequest()
xmppStream.sendElement(xmppMessage)
Then when you receive a message, inform the stream manager that the message has been handled with xmppStreamManager.markHandledStanzaId(message.from().resource)
The purpose of this final step is to establish a unique identifier that you can add to the XMPPMessageArchivingCoreDataStorage and check for duplicates before displaying.
I guess the reason is that B lost connection suddenly and the server
still think B is online. Thus the offline message does work under this
condition
Yes you are absolutely correct,this is well known limitation of TCP connections.
There are two approaches to your problem
1 Server side
As I can see you are using ejabbed as XMPP server you can implement
mod_ping , Enabling this module will enables server side
heartbeat[ping] ,in case of broken connection to server[ejabbed] will
try to send heartbeat to connection and will detect connection is lost
between server and client. Use of this approach has one
drawback,module mod_ping has property called ping_interval which
states how often to send heartbeat to connected clients, here lower
limit is 32 seconds any value below 32 is ignored by ejabbed,means
you have 32 seconds black window in which messages can be lost if user
is sowing as online
2 Client side
From client side you can implement Message Delivery Receipts
mechanism .With each Chat message send a receipt to receiver user of
as soon as receiver user receives message send back this receipt
id. This way you can detect that your message is actually delivered to
receiver. If you don't receive such acknowledgement between certain
time interval you can show user as offline locally(on mobile
phone),store any further messages to this user as offline message
locally[in SQLLight database ],and wait for offline presence stanza for that user
,as soon as you receive offline presence stanza it means that server
has finally detected connection to that user is lost and makes user
status as offline ,now you can send all messages to that user ,which
will be again stored as offline messages on server.This is best
approach to avoid black-window.
Conclusion
You can either use Approach 2 and design you client such way ,you can also use Approach 1 along with approach 2 to minimize server broken connection detraction time.
If B goes offline suddenly then user A have to check if B is online/offline while sending message to user B. If user B is offline then user A have to upload that message on Server using Web service. And user B have to call web service on below function.
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
So user B will get that all offline message which was lost due to connection Lost.
At last, I use Ping together with Stream Management:
http://xmpp.org/extensions/xep-0198.html
This problem is solved.
I'm using an Openfire server to support Jabber chat between Spark clients. I want to use GitHub's Hubot to monitor chatrooms and listen for users to send messages to Hubot, then log the messages. I'm having a hard time figuring out how I can get the timestamp of the message from the point of view of the Openfire server... is this even possible or should I just use the system time of the system running Hubot?
Use the system time of the receiving process. Only messages that were stored before you join a room will have timestamps with the server time.