In Spray, for chunked response we can send chunked message with an ack like
peer ! MessageChunk(buffer).withAck(MyAck(k))
and we can get ack message back which confirms the message delivery to the OS network layer. Do we have something like timeouts for these acks ?
I don't think there is an Ack Timeout feature built in to spray.
It should be possible, for the Chunking Actor to keep track of sent messages & timeouts if your Ack contains details about the sent chunk. See ChunkingActor[T] used to marshall Stream[T] in the spray source code to get an idea of how this could be done. You may choose to have this actor keep track of the last chunk sent along with the time sent. If an Ack does not come back for that chunk during the set timeout, you can handle the case where chunk delivery has "timed out".
Related
The question is when to send the ACK or NACK frames to the broker. The STOMP specification says:
ACK is used to acknowledge consumption of a message from a
subscription using client or client-individual acknowledgment. Any
messages received from such a subscription will not be considered to
have been consumed until the message has been acknowledged via an ACK.
"Consumption" would mean for me "after received and processed". So I can imagine two scenarios.
Scenario A after receiving the message:
function on_message(message)
message.ack()
heavy_processing(message)
or Scenario B after the message has been processed:
function on_message(message)
heavy_processing(message)
message.ack()
As I understood the NACK is to tell the broker that this listener for example has been temporary marked as inactive. And as I understood as well, the NACK is not to to mark a message as unprocessed because of an exception.
So the following pseudo code would handle ACK, NACK, and exceptions correctly from my understanding:
function on_message(message)
if online(): # checks resources etc
message.ack()
else:
message.nack()
return
try:
heavy_processing(message) # processing takes 5-10 minutes
catch Exception: # could be problem with this Listener or malformed message
message.put_to_dlq() # putting to dlq is a "manual" process
return
FYI the system I'm talking about is build in Python 3.7.x with Stomp.py module and ActiveMQ.
As you said, 'Consumption' would mean 'after received and processed', you need to acknowledge the message when you have successfully processed your message without any exceptions. So scenario B will be the apt one.
From the documentation
NACK is the opposite of ACK. It is used to tell the server that the client did not consume the message.
So NACK in this context would mean that you have received the message and did not process it successfully.
Note: If you are maintaining a separate queue for failed messages(ones that cause Exceptions), then you can publish those message to another(dlq queue in your case) and positively acknowledge(ACK) to the original queue for those failed messages.
The STOMP specification says this about the NACK frame:
NACK is the opposite of ACK. It is used to tell the server that the client did not consume the message. The server can then either send the message to a different client, discard it, or put it in a dead letter queue. The exact behavior is server specific.
So a NACK frame tells the server that the client didn't consume the message, but it doesn't indicate why.
A NACK frame does not indicate that the listener is somehow temporarily marked as inactive. The broker will continue to send messages to the listener (assuming it's active).
A NACK could indicate the message wasn't processed due to an exception. Again, the specification doesn't make a distinction between cases here.
The resulting behavior from a NACK is left up to the server.
This question is regarding the ACK request in SIP (Session Initiation Protocol). Quoting RFC-3261 - 18.1.1 Sending Requests
If an element sends a request over TCP because of these message size
constraints, and that request would have otherwise been sent over
UDP, if the attempt to establish the connection generates either an
ICMP Protocol Not Supported, or results in a TCP reset, the element
SHOULD retry the request, using UDP.
This looks OK for the INVITE and other non-invite request except ACK. Below are the points why i think the above statement may not be applicable for ACK.
ACK is just a request and not transaction. So the SIP transaction State Machine must not be applicable for ACK request. Since State Machine is the only place which talks about the retransmission, So the 18.1.1 is not applicable for ACK request. Quoting 3261 - "17.1 Client Transaction" - in favour for above argument.
There are two types of client transaction state machines, depending
on the method of the request passed by the TU. One handles client
transactions for INVITE requests. This type of machine is referred
to as an INVITE client transaction. Another type handles client
transactions for all requests except INVITE and ACK. This is
referred to as a non-INVITE client transaction. There is no client
transaction for ACK. If the TU wishes to send an ACK, it passes one
directly to the transport layer for transmission.
ACK can be re-transmissted only if next B-party retransmits the final response.
Q:- Is my assumption correct that SIP ACK request can not be retransmitted on receiving the transport error (such as connection error, ICMP error)?
Quoting 3261 - "17.1 Client Transaction" -
If the TU wishes to send an ACK, it passes one
directly to the transport layer for transmission.
Q:- What does TU wishes mean? Does it mean that TU can send any time it wish OR it means whenever final response come then only TU sends?
Regards,
Sudhansu
1) Let's go back to basic stuff
"If an element sends a request over TCP because of these message size
constraints, and that request would have otherwise been sent over
UDP, if the attempt to establish the connection generates either an
ICMP Protocol Not Supported, or results in a TCP reset, the element
SHOULD retry the request, using UDP."
My comments :-
"SHOULD retry the request" --> It says SHOULD retry not MUST retry request.
So this is not mandatory at all to send ACK request.
2)
Q:- Is my assumption correct that SIP ACK request can not be retransmitted on receiving the transport error (such as connection error, ICMP error)?
Quoting 3261 - "17.1 Client Transaction" -
If the TU wishes to send an ACK, it passes one directly to the transport layer for transmission.
comments :- This clearly states if TU ( Transaction user ) wishes to {not must} pass ACK request to transport layer for tranmission.
So as per statement, it's application wish that don't do anything once you pass that to ACK to transport layer. OR if on another side if application wants if there any TCP error (ICMP) then it can retransmit same ACK request over UDP.
See RFC will never exclude ACK from this, it is application responsibility & how it is implemented.
Also please can also ask question at Sip-implementors mailing list. You will get more like minded people.
First some clarifications. From [RFC3261 Section 6][1]:
SIP Transaction: A SIP transaction occurs between a client and a
server and comprises all messages from the first request sent
from the client to the server up to a final (non-1xx) response
sent from the server to the client. If the request is INVITE
and the final response is a non-2xx, the transaction also
includes an ACK to the response. The ACK for a 2xx response to
an INVITE request is a separate transaction
So an ACK to a non-2xx is part of the same transaction as the INVITE, any 1xx responses and the final response. In particular it has the same CSeq and the same branch tag. ACKs to non-2xx responses are hop-by-hop.
An ACK to a 2xx is a new transaction (although this is a special ACK transaction, and there are no responses). It has the same CSeq, but a different branch tag. ACKs to 2xx responses go all the way from the client to the server.
On your specific point, section 17 is about the transaction Layer. Here ACKs are only ever re-transmitted in response to a re-transmitted final response. When the Transaction Layer decides it needs to send an ACK, it sends it to the Transport Layer, and it is the Transport Layers job to deliver it.
Section 18 is about the Transport Layer. An ICMP error or a TCP reset are Transport Layer errors, they are not covered by the SIP transaction model. If the Transport Layer needs to promote a message from UDP to TCP, or to fall-back from TCP to UDP, that is separate from the Transaction Layers model. In fact, if the TCP socket cannot be established, the ACK has not been transmitted yet. In this case the Transport Layer isn't retransmitting it, it is attempting to transmit it for the first time, by a different protocol.
In practise, this case is going to be rare. For the size of the ACK to be a problem for UDP, it has to be larger then both the INVITE and the final response - these have already succeeded via UDP. In the usual case the INVITE and final response contain an SDP body; in every success case the 2xx contains a body. The only case where the ACK is going to be larger is if there was no SDP in the INVITE and the ACK adds to the headers or body.
If you hit this case - TCP not supported and ACKs too big for UDP, you may be heading for failure anyway.
[1]: https://www.rfc-editor.org/rfc/rfc3261#section-6
I'm trying to figure out if it is somehow possible to setup Mirth to send 2 ACK back to the caller Application:
A) 1 ACK sent from Mirth to the caller when the transmission has been received from Mirth;
B) 1 ACK sent from Mirth to the caller after the channel is finished processing the message.
I know that Mirth can either be configured to send ACK before processing (case A above) or after processing (Case B above), but I could not find any way to send both.
Has anyone had experience in doing this?
Thank you all for your help.
Mirth uses a single responseMap to store acknowledgement which is processed after all scripts. So, if you put anything there when a message is received, this Ack will be overridden with a new Ack placed into the same map at the end. And only the latter will be sent, which you've already experienced I guess.
If I'm correct, what you are trying to achieve is, first, to confirm that the message is received by a remote location (let's call it System B) and, second, is to confirm that the message successfully processed. If your client (System A) is capable to send a message to two endpoints at System B then you may create two receiving channels on the System B side, one of these channels sends ACK immediately after receiving the message and does nothing. The other channel processes the message and sends ACK in postprocessor.
There are other options, say, on System B side redirect an incoming message to another channel which forms Ack and sends it back to System A, but then System A should have a listener on its side.
Or, System B may have a receiving channel that sends Ack immediately, routes the message to another channel that is connected to its destinations, and remove that destination to prevent incoming message to propagate to that channel. The second channel processes the message and sends Ack back to the first channel. First channel resends that Ack back to System A. (I have not tested such configuration, so this is just an idea to overcome a single responseMap. It may not work.)
I have send Udp packet to the Server. If the server is OK then I can receive the response packet nicely but when the server is down then I did not get any response packet. Anybody can help me that how can I send my packet to server multiple time when fail to receive the response packet. Moreover, want to keep alive the connection with server. Thanks in advance.
Well,
After you've sent the packet, you wait for the ACK (response) package from the server. You could use the DatagramSocket.setSoTimeout() to an appropriate time, if you get the Timeout Exception increment a counter, if that counter is less than 2/3 send the packet again and repeat these steps. If the counter is bigger than 2/3 the server is down, just quit.
According to Java documentation, receive will block until a package is received or a timeout has expired.
To keep the connection alive you need to implement a ping-pong. In another thread of your program, you send a Keep-Alive packet (any small packet will do) and wait for a response. I suggest using a different port number for this purpose so that these packets won't mess up with the normal data packets. These packets can be send every 2 seconds o 2 minutes depends on your particular needs. When the thread receives the ACK packet it will update a private time variable with the current time, for example:
lastTimeSeen = System.currentTimeMillis();
put a method in your thread class to access the value of that variable.
I already read this question about socket synchronization but I still dont get it yet.
Recently I was working on a relatively simple client/server app where the communication happens over a tcp socket. The client is written in PHP using the C-like functions (especially fsockopen and fgetc) PHP provides to interact with sockets, the server is written in node.js using a Stream for outputting data.
The protocol is quite simple, the message is just a string which ends with a 0-byte character.
Basically it works like this:
SERVER: Message 1
CLIENT: Ack 1
SERVER: Message 2
CLIENT: Ack 2
....
Which really worked fine as my client processed one message at a time by reading char by char from the socket until a 0-byte was encountered which designates the end of the message. Then the client writes back to the server that it has successfully received the message (thats the Ack <message id> part).
Now this happened:
SERVER: Message 1
CLIENT: Ack 1
SERVER: Message 2
CLIENT: Ack 2
SERVER: Message 3
Message 4
Message 5
Message 6
CLIENT: <DOH!>
....
Meaning the server unexpectedly sent multiple messages in one "batch" to the client, although every message is a single stream.write(...) operation on the server. It seemed like the messages were buffered somewhere and then sent to the client at once. My client code couldnt cope with multiple messages in the socket WITHOUT an Ack response in between, so it cut off the remaining messages after id 3.
So my question is:
How synchronized are sockets in their read and writes? From the question above I understand that a socket is basically two uni-directional pipes, which means they are not synchronized at all?
How can it happen that some messages were sent to my client in a simple "one message-one ack" manner and then suddendly multiple messages are written to the stream?
Does it actually change the picture if the socket is opened in a blocking/non-blocking manner?
I tested this on a Ubuntu VM (so no load or anything that could provoke strange behaviour) using PHP 5.4 and node 0.6.x.
TCP is an abstraction of a bi-directional stream, and as such has no concept of messages and cannot preserve message boundaries. There is no guarantee how multiple send() or recv() calls will map to TCP packets. You should treat send() as if calling it multiple times is equivalent to calling it once with the concatenation of all the data. More importantly, when receiving, you should make sure that your code interprets the incoming data exactly the same way, no matter how it was split over indvidual recv() calls.
To receive properly, you can use a buffer where you store incomplete messages. But be careful that when you have an incomplete message in a buffer, the next recv() call may complete the current message, as well as provide zero or more complete messages, and possibly part of another incomplete message.
The blocking or non-blocking mode doesn't change anything here - it's only about the way your application interfaces with the OS.
There are two synchronization concepts to deal with:
The (generally) synchronous operation of send() or recv().
The asynchronous way that one process sends a message and the way the other process handles the message.
If you can, try to avoid a design that keeps a client and server in process-synchronized "lock step" with each other. That's asking for trouble. What if the one of the processes closes unexpectedly? The other process/thread might hang on a recv() that will never come. It's one thing for your design to expect each message to be acknowledged eventually, but it's quite another for your design to expect that only one message can be sent, then it must be acknowledged, before you may send another.
Consider this:
Server: send 1
Client: ack 1
Server: send 2
Server: send 3
Client: ack 2
Server: send 4
Client: ack 3
Client: ack 4
A design that can accommodate this situation is better than one that expects:
Server: send 1
Client: ack 1
Server: send 2
Client: ack 2
Server: send 3
Client: ack 3
Server: send 4
Client: ack 4