xmpp messages are lost when client connection lost suddently - xmpp

I am using ejabberd server and ios xmppframework.
there are two clients, A and B.
When A and B are online, A can send message to B successfully.
If B is offline, B can receive the message when B is online again.
But when B is suddenly/unexpectedly lost connection, such as manually close wi-fi, the message sent by A is lost. B will never
receive this message.
I guess the reason is that B lost connection suddenly and the server still think B is online. Thus the offline message does work under this condition.
So my question is how to ensure the message that sent by A will be received by B? To ensure there is no messages lost.

I've spent the last week trying to track down missing messages in my XMPPFramework and eJabberd messaging app. Here are the full steps I went through to guarantee message delivery and what the effects of each step are.
Mod_offline
In the ejabberd.yml config file ensure that you have this in the access rules:
max_user_offline_messages:
admin: 5000
all: 100
and this in the modules section:
mod_offline:
access_max_user_messages: max_user_offline_messages
When the server knows the recipient of a message is offline they will store it and deliver it when they re-connect.
Ping (XEP-199)
xmppPing = XMPPPing()
xmppPing.respondsToQueries = true
xmppPing.activate(xmppStream)
xmppAutoPing = XMPPAutoPing()
xmppAutoPing.pingInterval = 2 * 60
xmppAutoPing.pingTimeout = 10.0
xmppAutoPing.activate(xmppStream)
Ping acts like a heartbeat so the server knows when the user is offline but didn't disconnect normally. It's a good idea to not rely on this by disconnecting on applicationDidEnterBackground but when the client looses connectivity or the stream disconnects for unknown reasons there is a window of time where a client is offline but the server doesn't know it yet because the ping wasn't expected until sometime in the future. In this scenario the message isn't delivered and isn't stored for offline delivery.
Stream Management (XEP-198)
xmppStreamManagement = XMPPStreamManagement(storage: XMPPStreamManagementMemoryStorage(), dispatchQueue: dispatch_get_main_queue())
xmppStreamManagement.autoResume = true
xmppStreamManagement.addDelegate(self, delegateQueue: dispatch_get_main_queue())
xmppStreamManagement.activate(xmppStream)
and then in xmppStreamDidAuthenticate
xmppStreamManagement.enableStreamManagementWithResumption(true, maxTimeout: 100)
Nearly there. The final step is to go back to the ejabberd.yml and add this line to the listening ports section underneath access: c2s:
resend_on_timeout: true
Stream Management adds req/akn handshakes after each message delivery. On it's own it won't have any effect on the server side unless that resend_on_timeout is set (which it isn't by default on eJabberd).
There is a final edge case which needs to be considered when the acknowledgement of a received message doesn't get to the server and it decides to hold it for offline delivery. The next time the client logs in they are likely to get a duplicate message. To handle this we set that delegate for the XMPPStreamManager. Implement the xmppStreamManagement getIsHandled: and if the message has a chat body set the isHandledPtr to false. When you construct an outbound message add an xmppElement with a unique id:
let xmppMessage = XMPPMessage(type: "chat", to: partnerJID)
let xmppElement = DDXMLElement(name: "message")
xmppElement.addAttributeWithName("id", stringValue: xmppStream.generateUUID())
xmppElement.addAttributeWithName("type", stringValue: "chat")
xmppElement.addAttributeWithName("to", stringValue: partnerJID.bare())
xmppMessage.addBody(message)
xmppMessage.addChild(xmppElement)
xmppMessage.addReceiptRequest()
xmppStream.sendElement(xmppMessage)
Then when you receive a message, inform the stream manager that the message has been handled with xmppStreamManager.markHandledStanzaId(message.from().resource)
The purpose of this final step is to establish a unique identifier that you can add to the XMPPMessageArchivingCoreDataStorage and check for duplicates before displaying.

I guess the reason is that B lost connection suddenly and the server
still think B is online. Thus the offline message does work under this
condition
Yes you are absolutely correct,this is well known limitation of TCP connections.
There are two approaches to your problem
1 Server side
As I can see you are using ejabbed as XMPP server you can implement
mod_ping , Enabling this module will enables server side
heartbeat[ping] ,in case of broken connection to server[ejabbed] will
try to send heartbeat to connection and will detect connection is lost
between server and client. Use of this approach has one
drawback,module mod_ping has property called ping_interval which
states how often to send heartbeat to connected clients, here lower
limit is 32 seconds any value below 32 is ignored by ejabbed,means
you have 32 seconds black window in which messages can be lost if user
is sowing as online
2 Client side
From client side you can implement Message Delivery Receipts
mechanism .With each Chat message send a receipt to receiver user of
as soon as receiver user receives message send back this receipt
id. This way you can detect that your message is actually delivered to
receiver. If you don't receive such acknowledgement between certain
time interval you can show user as offline locally(on mobile
phone),store any further messages to this user as offline message
locally[in SQLLight database ],and wait for offline presence stanza for that user
,as soon as you receive offline presence stanza it means that server
has finally detected connection to that user is lost and makes user
status as offline ,now you can send all messages to that user ,which
will be again stored as offline messages on server.This is best
approach to avoid black-window.
Conclusion
You can either use Approach 2 and design you client such way ,you can also use Approach 1 along with approach 2 to minimize server broken connection detraction time.

If B goes offline suddenly then user A have to check if B is online/offline while sending message to user B. If user B is offline then user A have to upload that message on Server using Web service. And user B have to call web service on below function.
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
So user B will get that all offline message which was lost due to connection Lost.

At last, I use Ping together with Stream Management:
http://xmpp.org/extensions/xep-0198.html
This problem is solved.

Related

How to check if Siebel has successfully delivered an email?

We send a lot of email messages from our Siebel 7.8 application, and we'd like to determine whether they have been successfully delivered or not.
According to the Bookshelf, if the SMTP server is down, the Communications Outbound Manager retries to send the message later, so that's not a problem. However, there are still plenty of issues which could cause an email to not be delivered, such as a typo in the address, the receiver having reached its storage quota, etc.
We send our messages this way:
var ps = TheApplication().NewPropertySet();
ps.SetProperty("ActivityId", outboundEmailActivityId);
ps.SetProperty("CommProfile", commProfile);
ps.SetProperty("ProcessMode", "Local");
var bs = TheApplication().GetService("Outbound Communications Manager");
bs.InvokeMethod("SendMessage", ps, psOut);
Using ProcessMode = Local allows us to detect a few errors. For example, if we try to send a message to a non-existant account in the same domain of our SMTP server, it returns 550 Unknown user and then 503 Must have sender and recipient first. The Outbound Communications Manager raises an exception, and we capture and handle it.
However, if we send a message to a non-existant account in a different domain, our SMTP server can't know that it will fail, and therefore it returns 250 Queued, and our code completes successfully. Later (it can range from seconds to a few hours later), we will receive a "Message undeliverable" error message, but at this point we only know that an outbound message failed, we don't know which one.
Is there any way in which Siebel can handle these 'Message undeliverable' notifications automatically?
We are thinking of writing our own process for that, but it seems like a huge task: we'd have to parse the delivery failure notification, identify the failing recipient, search for all the recent messages sent to that address, and somehow, guess which one failed (based on the Message-Id if we are lucky and can read it within Siebel, or on the Subject otherwise).
The problem is that SMTP is by its nature neither a synchronous nor reliable protocol (i.e. in the sense of "engineered for guaranteed delivery"). Your Siebel app server will connect to its assigned SMTP server and ask it to accept a message for delivery and at that time there are a few high level validations that can be perform (some of which you've mentioned but which can also include policy enforcement such as checking whether your (possibly anonymous) identity is authorized for relaying messages to external domains). Once that conversation ends, there is not much else you can reliably do because again, everything from that point is asynchronous and not guaranteed for delivery (any number of intermediate relay agents can be involved, each with their own potential for outages with or without retry, each with the ability to honor or ignore requests for delivery or read receipts or to report invalid recipients, throwing your message in a junk folder or not, etc.). Certainly you can attempt to work with any bounce notifications you do happen to get to try to correlate them back to the sender but that would be outside the context of your sending code.

Strange errors with stream management in ejabberd

I’m building an instant messenger app on iOS that uses ejabberd. I’m currently testing the stream management feature and in particular the resumption that seems to work in most cases. However there is a case I don’t understand, that I can replicate through the following steps, taking in account the settings: resume_timeout: 30, resend_on_timeout: if_offline
at the beginning client A and client B are connected, no other resources are connected
client B crashes or disconnects in a not clean way
client A starts to send a bunch of messages (10+) very quickly
ejabberd sends an ack to A for each message sent to confirm that the messages reached the server
around 20 seconds since the crash, B reconnects. At this instant A receives an error for each message sent before
<message xmlns="jabber:client" from="clientB#mydomain" to="clientA#mydomain/resourceID" type="error" id="CFBF4583-209A-4453-2567-CCCC7894827E">
<body>test</body>
<active xmlns="http://jabber.org/protocol/chatstates" />
<request xmlns="urn:xmpp:receipts" />
<error code="503" type="cancel">
<service-unavailable xmlns="urn:ietf:params:xml:ns:xmpp-stanzas" />
</error>
</message>
I tried with ejabberd 16.01.
This happens 80% of the time; sometimes messages sent by A are correctly delivered to B on reconnection within the 30 seconds.
My questions are:
is this behavior correct? I would expect that no error is bounced to client A if an ack has already been received for a message.
since resend_on_timeout is set to if_offline and no other resource is connected, I would expect no errors at all. Am I correct?
Stream Management acks only indicate that the message has been received by your server. It doesn't imply that the message has been processed or delivered to the specified address. Even if it were delivered to the address, then that device can still return an error for the stanza.
This is really just a stab in the dark, but after having a glance over the ejabberd code, this could be what happens:
clientB#mydomain/ResourceB drops their connection, there is now a session awaiting resumption using ResourceB.
Client B reconnects, doesn't resume (because it crashed and lost its state).
Client B binds resource ResourceB again.
Now the server has to terminate the sleeping session that was waiting for resumption because client B requested the same resource.
The server checks whether there are other sessions because it is set to if_offline.
The server sees there is a session (the new session) and therefore chooses to bounce instead of resend.
So my theory is that if_offline only checks if there are other sessions when queue of unacknowledged messages needs to be handled, not at the time the message was originally received.
#xnyhps' response is correct, and I fixed this particular corner case for the next ejabberd release. However, #xnyhps is also correct that there are other corner cases, so if you want reliable message delivery, you should be using XEP-0313. The main feature of XEP-0198 is session resumption.

Using FIX protocol to connect to the broker

I am currently trying to build a program using FIX protocol to communicate with a broker (currenex). I sent my (self-generated) logon message to the server and got something back.
This is what I sent
8=FIX.4.49=8835=A49=xxxxxxxx56=CNX34=152=20140718-11:40:18.24498=0108=30141=Y554=xxxxxx10=128
(the SenderCompID and the password were replaced)
and I got
8=FIX.4.49=7635=A34=149=CNX52=20140718-11:40:33.22456=xxxxxxxx141=Y98=0108=3010=1458=FIX.4.49=7035=h34=249=CNX52=20140718-11:40:33.22656=xxxxxxxx336=0340=210=128
back from the server.
I think I built the logon message correct (or did I?). But when I sent a second request MarketDataRequest
8=FIX.4.49=13735=V49=xxxxxxxx56=CNX34=252=20140718-11:42:53.504262=363263=1264=0265=1266=N267=2269=1269=0146=155=GBP/USD554=xxxx10=013
I had no response at all. I asked the broker and they said the connection dropped right away every time after I logged in.
I thought it was some connection problem and I tried using RESTClient (Postman) to send the message but the result was the same.
Could any one take a look at my messages and point out if there is something stupid please?
All I need is the real-time exchange rate so a simple FIX message example will be very helpful. Thanks a lot!
Regards,
Bo
Your logon response message says that your trading session is open (340=2) so it's not broker-side problem. I think your program disconnects TCP/IP connection from server after login message. FIX protocol insists that TCP/IP connection must be kept alive during the whole FIX session - otherwise the session will be closed. So you need rewrite your program to keep connection alive and just send there yor requests and listen for responses. Don't close the connection.
Try using Minifix tool which will maintain the heartbeats and the session connection.
Ideally for a 35=V request you should get
35=W = Market Data-Snapshot/Full Refresh
35=X = Market Data-Incremental Refresh
35=Y = Market Data Request Reject
you get a 35=3 (reject) or 35=4 (or a seq reset) in response to a 35=A request.

Quickfix engine - does it persist messages before the start time on the server side

If a quick fix session is created by server(acceptor) at say 9AM, but the StartTime is at 11AM. This means the session exists but not active.
If the server receives an unsolicited message from an exchange that it needs to send on this session, will it persist this if I have configuration PersistMessages=Y and sends it to the client(initiator) when it connects after 11AM?
No, it would not persist messages received before start time and would send you a reject message. The message will be rejected at the interface itself, message isn't handled. You would have to resend it to get a response.
QuickFIX does persist (but not send) messages before a session is connected. The sequence numbers are updated and when the session is connected and the first message is sent, the counterparty FIX engine will see the gap in the sequence numbers and request a resend. QuickFIX will then resend the persisted messages. However, depending on your QuickFIX configuration, the outgoing messages might be considered to be too old and rejected locally.
As I understand, these are kept to take into account timings under which corresponding exchange would accept the orders.
Application or its sub-modules do not need to maintain timings and take some action on closing the fix session. Rather, QuickFix shall automatically deactivate the session.
Persistence of the message or re-sesnding when the session becomes active does not look desirable to me.
You can rather maintain some kind of queue to buffer such messages in sending application, and send them only when the time matches with active session timings.
That's my thoughts, hope that helps.

Send XMPP message without starting a chat

I am basically writing a XMPP client to automatically reply to "specific" chat messages.
My setup is like this:
I have pidgin running on my machine configured to run with an account x#xyz.com.
I have my own jabber client configured to run with the same account x#xyz.com.
There could be other XMPP clients .
Here is my requirement:
I am trying to automate certain kind of messages that I receive on gtalk. So whenever I receive a specific message eg: "How are you" , my own XMPP client should reply automatically with say "fine". How are you". All messages sent (before and after my client replies) to x#xyz.com but should be received by all clients (my own client does not have a UI and can only respond to specific messages.).
Now I have already coded my client to reply automatically. This works fine. But the problem I am facing is that as soon as I reply (I use the smack library), all subsequent messages that are sent to x#xyz.com are received only by my XMPP client. This is obviously a problem as my own client is quite dump and does not have a UI, so I don't get to see the rest of the messages sent to me, thereby making me "lose" messages.
I have observed the same behavior with other XMPP clients as well. Now the question is, is this is a requirement of XMPP (I am sorry but I haven't read XMPP protocol too well). Is it possible to code an XMPP client to send a reply to a user and still be able to receive all subsequent messages in all clients currently listening for messages? Making my client a full fledged XMPP client is a solution, but I don't want to go that route.
I hope my question is clear.
You may have to set a negative presence priority for your bot..
First thing to know is that in XMPP protocol every client is supposed to have a full JID. This is a bare JID - in your case x#xyz.com with a resource in the end e.g. x#xyz.com/pidgin or x#xyz.com/home (where /pidgin and /home are the resource). This is a part of how routing messages to different clients is supposed to be achieved.
Then there are the presence stanzas. When going online a client usually sends a presence stanza to the server. This informs about e.g. if the client is available for chat or away for lunch. Along with this information can be sent a priority. When there are more than one clients connected the one with the highest priority will receive the messages sent to the bare JID (e.g. ClientA(prio=50) and ClientB(prio=60) -> ClientB receives the messages sent to x#xyz.com). But there are also negative priorities. A priority less than 0 states that this client should never be sent any messages. Such a stanza might look like this
<presence from="x#xyz.com/bot">
<priority>-1</priority>
</presence>
This may fit your case. Please keep in mind it also depends on the XMPP server where your account is located, which may or may have not fully implemented this part of the protocol.
So to summarize: I recommend you to look through the Smack API how to set a presence and set the priority to <0 for your bot client right after it connected.