I am trying to write an implementation of XMPP client (no BOSH, direct TCP connection).
trying to figure out how to handle incoming messages in response to my own requests and to know which response relates to which request, most protocol tag the request with some unique id and the server sends that id with the response.
I have seen an ID parameters for IQ stanzas but they are reported to identify a session and not a unique message...
what is the best way to handle this issue ?
The ID attribute in IQ stanzas bind a request (get or set) to a response (result or error). For each request, a new ID should be used. That's the way to track responses for each request.
The new RFC 6120 has a good description of how IQs work. Make sure that you're matching both the sender and the id of the received stanza to ensure that someone else isn't guessing your id numbers. The typical pattern is to use a IQtracker, which you call like this (pseudo-code):
iq = create_xml_dom_that_is(<iq id='[counter++]' type='get' to='receiver'/>)
track(iq, callback)
where callback gets called when the we receive an iq stanza from the receiver with the expected id, or when some timespan has been reached without getting a response.
Related
At the moment I have a single AWS EC2 instance which handles all incoming http client requests. It analyses each request and then decides which back end worker server should handle the request and then makes a http call to the chosen server. The back end server then responds when it has processed the request. The front end server will then respond to the client. The front end server is effectively a load balancer.
I now want to go to a Pub-Sub architecture instead of the front end server pushing the requests to the back end instances. The front end server will do some basic processing and then simply put the request into an SNS queue and the logic of which back end server should handle the request is left to the back end servers themselves.
My question is with this model what is the best way to have the back end servers notify the front end server that they have processed the request? Previously they just replied to the http request the front end server sent but now there is no direct request, just an item of work being published to a queue and a back end instance picking it off the queue.
Pubsub architectures are not well suited to responses/acknowledgements. Their fire-and-forget broadcasting pattern decouples publishers and the subscribers: a publisher does not know if or how many subscribers there are, and the subscribers do no know which publisher generated a message. Also, it can be difficult to guarantee sequence of responses, they won't necessarily match the sequence of messages due to the nature of network comms and handling of messages can take different amounts of time etc. So each message that needs to be acknowledge needs a unique ID that the subscriber can include in its response so the publisher can match a response with the message sent. For example:
publisher sends message "new event" and provides a UUID for the
event
many subscribers get the message; some may be the handlers for
the request, but others might be observers, loggers, analytics, etc
if only one subscriber handles the message (e.g. the first
subscriber to get a key from somewhere), that subscriber generates a
message "new event handled" and provides a UUID
the original
publisher, as well as any number of other subscribers, may get that
message;
the original publisher sees the ID is
in its cache as an unconfirmed message, and now marks it as
confirmed
if a certain amount of time passes without receiving a
confirmation with given ID, the original publisher republishes the
original message, with a new ID, and removes the old ID from cache.
In step 3, if many subscribers handled the message instead of just one, then it
less obvious how the original publisher should handle "responses": how does it
know how many subscribers handle the message, some could be down or
too busy to respond, or some may be in the process of responding by the time
the original publisher determines that "not enough handlers have
responded".
Publish-subscribe architectures should be designed to not request any response, but instead to check for some condition that should have happened as a result of the command being handled, such as a thumbnail having gotten generated (it can assume as a result of a handler of the message).
I heard that websocket messages are received in order, because websocket runs over TCP.
Then what is the purpose of 'sequence number'?
This is the explanation of sequence number in websocket.
But I'm wondering why does that sequence number is needed, if we have a 'in-order' received message.
The sequence number allows you to map your requests to responses even if the responses don't come in the order you make them.
HTTP and other relevant protocols support pipelining. Also there is no need for the request responses to be sent back to you in any specific order. Each one may be processed according to its individual cost or dispatched across a server farm and reassembled in an order that is not predetermined. Either way, if they are out of order you will need a key to map the response back to your request.
This question is regarding the ACK request in SIP (Session Initiation Protocol). Quoting RFC-3261 - 18.1.1 Sending Requests
If an element sends a request over TCP because of these message size
constraints, and that request would have otherwise been sent over
UDP, if the attempt to establish the connection generates either an
ICMP Protocol Not Supported, or results in a TCP reset, the element
SHOULD retry the request, using UDP.
This looks OK for the INVITE and other non-invite request except ACK. Below are the points why i think the above statement may not be applicable for ACK.
ACK is just a request and not transaction. So the SIP transaction State Machine must not be applicable for ACK request. Since State Machine is the only place which talks about the retransmission, So the 18.1.1 is not applicable for ACK request. Quoting 3261 - "17.1 Client Transaction" - in favour for above argument.
There are two types of client transaction state machines, depending
on the method of the request passed by the TU. One handles client
transactions for INVITE requests. This type of machine is referred
to as an INVITE client transaction. Another type handles client
transactions for all requests except INVITE and ACK. This is
referred to as a non-INVITE client transaction. There is no client
transaction for ACK. If the TU wishes to send an ACK, it passes one
directly to the transport layer for transmission.
ACK can be re-transmissted only if next B-party retransmits the final response.
Q:- Is my assumption correct that SIP ACK request can not be retransmitted on receiving the transport error (such as connection error, ICMP error)?
Quoting 3261 - "17.1 Client Transaction" -
If the TU wishes to send an ACK, it passes one
directly to the transport layer for transmission.
Q:- What does TU wishes mean? Does it mean that TU can send any time it wish OR it means whenever final response come then only TU sends?
Regards,
Sudhansu
1) Let's go back to basic stuff
"If an element sends a request over TCP because of these message size
constraints, and that request would have otherwise been sent over
UDP, if the attempt to establish the connection generates either an
ICMP Protocol Not Supported, or results in a TCP reset, the element
SHOULD retry the request, using UDP."
My comments :-
"SHOULD retry the request" --> It says SHOULD retry not MUST retry request.
So this is not mandatory at all to send ACK request.
2)
Q:- Is my assumption correct that SIP ACK request can not be retransmitted on receiving the transport error (such as connection error, ICMP error)?
Quoting 3261 - "17.1 Client Transaction" -
If the TU wishes to send an ACK, it passes one directly to the transport layer for transmission.
comments :- This clearly states if TU ( Transaction user ) wishes to {not must} pass ACK request to transport layer for tranmission.
So as per statement, it's application wish that don't do anything once you pass that to ACK to transport layer. OR if on another side if application wants if there any TCP error (ICMP) then it can retransmit same ACK request over UDP.
See RFC will never exclude ACK from this, it is application responsibility & how it is implemented.
Also please can also ask question at Sip-implementors mailing list. You will get more like minded people.
First some clarifications. From [RFC3261 Section 6][1]:
SIP Transaction: A SIP transaction occurs between a client and a
server and comprises all messages from the first request sent
from the client to the server up to a final (non-1xx) response
sent from the server to the client. If the request is INVITE
and the final response is a non-2xx, the transaction also
includes an ACK to the response. The ACK for a 2xx response to
an INVITE request is a separate transaction
So an ACK to a non-2xx is part of the same transaction as the INVITE, any 1xx responses and the final response. In particular it has the same CSeq and the same branch tag. ACKs to non-2xx responses are hop-by-hop.
An ACK to a 2xx is a new transaction (although this is a special ACK transaction, and there are no responses). It has the same CSeq, but a different branch tag. ACKs to 2xx responses go all the way from the client to the server.
On your specific point, section 17 is about the transaction Layer. Here ACKs are only ever re-transmitted in response to a re-transmitted final response. When the Transaction Layer decides it needs to send an ACK, it sends it to the Transport Layer, and it is the Transport Layers job to deliver it.
Section 18 is about the Transport Layer. An ICMP error or a TCP reset are Transport Layer errors, they are not covered by the SIP transaction model. If the Transport Layer needs to promote a message from UDP to TCP, or to fall-back from TCP to UDP, that is separate from the Transaction Layers model. In fact, if the TCP socket cannot be established, the ACK has not been transmitted yet. In this case the Transport Layer isn't retransmitting it, it is attempting to transmit it for the first time, by a different protocol.
In practise, this case is going to be rare. For the size of the ACK to be a problem for UDP, it has to be larger then both the INVITE and the final response - these have already succeeded via UDP. In the usual case the INVITE and final response contain an SDP body; in every success case the 2xx contains a body. The only case where the ACK is going to be larger is if there was no SDP in the INVITE and the ACK adds to the headers or body.
If you hit this case - TCP not supported and ACKs too big for UDP, you may be heading for failure anyway.
[1]: https://www.rfc-editor.org/rfc/rfc3261#section-6
I am trying to implement Fix.4.2 protocol, but It is difficult to understand the message log I attached below. Here Logon(35=A) request was sent with MsgSeqNum(34=1) from client. Then for testing ResendRequest and SequenceReset session level messages I sent a NewOrderSingle request with MsgSeqNum=7 (instead of MsgSeqNum=2, as subsequent messages should have incremeted msgseqnum after logon request). As expected MsgSeqNum is too high than recieved one Fiximulator responded with a ResendRequest(35=2) to send from 2 to 0 (i.e., from 2 to 7). Here why the Fiximulator is not waiting for client's reply ? instead it is sending an heartbeat message. Why the client is sending ResendRequest in response to ResendRequest of Fiximulator instead of sending SequenceReset message ?.
Also explain remaining cases if possible.
Thanks in advance.
What is your status of ResetOnLogon in your config file for the acceptor ? Default value is N so it isn't being reset. Always check your config file or try debugging to figure out issues.
ResetOnLogon Determines if sequence numbers should be reset when recieving a logon request. Acceptors only
I'm reading the XEP-0124 / BOSH specification and do not understand the following sentence in chapter 9.1 Request Acknowledgements:
The only exception is that, after its
session creation response, the
connection manager SHOULD NOT include
an 'ack' attribute in any response if
the value would be the 'rid' of the
request being responded to.
In my words: I should not send an ACK if the respond is dedicated for the last and only request (in connection manager's queue).
But: There is a client with it's own state machine. Maybe the client already send a second request -- where the first one is not replied -- and expect to get two answers. In this case the client except a ACK with RID of the "older" request and the connection manager have to set ACK.
Conclusion: The connection mananager MUST set ACK as long multiple requests are allowed.
I'm not sure, but is this text paragraph dedicated only for the use case where no further request is send by the client but the session creation phase is finished successfully and the connection manager have to send "ping" messages to the client due to "wait" timeouts ?
So, as I read it:
If the highest RID (in sequence) that you have received is 11 (you might have received 14 after that, but it is out of sequence since 12 & 13 are missing), and you are responding on:
The same request, then you should not (it is recommended that you do not, but if you have a good reason to, then you may) send an 'ack' attribute.
An earlier held request (say RID 10) then you should set 'ack' to 11 since that is the highest in-sequence RID that you have received so far.
It's okay if the client sent multiple requests and the server doesn't yet know about them. This is because there is a chance that when the client sent 11, the server has no held connections and it will respond back on the same connection. In that case, there are 2 requests sent out (11 & 12), but the response for each one acks that same request since the server always has something to send back immediately.