The question is when to send the ACK or NACK frames to the broker. The STOMP specification says:
ACK is used to acknowledge consumption of a message from a
subscription using client or client-individual acknowledgment. Any
messages received from such a subscription will not be considered to
have been consumed until the message has been acknowledged via an ACK.
"Consumption" would mean for me "after received and processed". So I can imagine two scenarios.
Scenario A after receiving the message:
function on_message(message)
message.ack()
heavy_processing(message)
or Scenario B after the message has been processed:
function on_message(message)
heavy_processing(message)
message.ack()
As I understood the NACK is to tell the broker that this listener for example has been temporary marked as inactive. And as I understood as well, the NACK is not to to mark a message as unprocessed because of an exception.
So the following pseudo code would handle ACK, NACK, and exceptions correctly from my understanding:
function on_message(message)
if online(): # checks resources etc
message.ack()
else:
message.nack()
return
try:
heavy_processing(message) # processing takes 5-10 minutes
catch Exception: # could be problem with this Listener or malformed message
message.put_to_dlq() # putting to dlq is a "manual" process
return
FYI the system I'm talking about is build in Python 3.7.x with Stomp.py module and ActiveMQ.
As you said, 'Consumption' would mean 'after received and processed', you need to acknowledge the message when you have successfully processed your message without any exceptions. So scenario B will be the apt one.
From the documentation
NACK is the opposite of ACK. It is used to tell the server that the client did not consume the message.
So NACK in this context would mean that you have received the message and did not process it successfully.
Note: If you are maintaining a separate queue for failed messages(ones that cause Exceptions), then you can publish those message to another(dlq queue in your case) and positively acknowledge(ACK) to the original queue for those failed messages.
The STOMP specification says this about the NACK frame:
NACK is the opposite of ACK. It is used to tell the server that the client did not consume the message. The server can then either send the message to a different client, discard it, or put it in a dead letter queue. The exact behavior is server specific.
So a NACK frame tells the server that the client didn't consume the message, but it doesn't indicate why.
A NACK frame does not indicate that the listener is somehow temporarily marked as inactive. The broker will continue to send messages to the listener (assuming it's active).
A NACK could indicate the message wasn't processed due to an exception. Again, the specification doesn't make a distinction between cases here.
The resulting behavior from a NACK is left up to the server.
Related
I'm working on an gameServer that communicate with game client, but wonder whether the packet server send to client remain sequence when client received it ?
like server sends packets A,B,C
but the client received B,A,C ?
I have read the great blog http://packetlife.net/blog/2010/jun/7/understanding-tcp-sequence-acknowledgment-numbers/
It seems that every packet send by the server has an ack corresponding by client, but it does not say why the packet received by client has the same sequence with server
It's worth reading TCP's RFC, particularly section 1.5 (Operation), which explains the process. In part, it says:
The TCP must recover from data that is damaged, lost, duplicated, or delivered out of order by the internet communication system. This is achieved by assigning a sequence number to each octet transmitted, and requiring a positive acknowledgment (ACK) from the receiving TCP. If the ACK is not received within a timeout interval, the data is retransmitted. At the receiver, the sequence numbers are used to correctly order segments that may be received out of order and to eliminate duplicates. Damage is handled by adding a checksum to each segment transmitted, checking it at the receiver, and discarding damaged segments.
I don't see where it's ever made explicit, but since the acknowledgement (as described in section 2.6) describes the next expected packet, the receiving TCP implementation is only ever acknowledging consecutive sequences of packets from the beginning. That is, if you never receive the first packet, you never send an acknowledgement, even if you've received all other packets in the message; if you've received 1, 2, 3, 5, and 6, you only acknowledge 1-3.
For completeness, I'd also direct your attention to section 2.6, again, after it describes the above-quoted section in more detail:
An acknowledgment by TCP does not guarantee that the data has been delivered to the end user, but only that the receiving TCP has taken the responsibility to do so.
So, TCP ensures the order of packets, unless the application doesn't receive them. That exception probably wouldn't be common, except for cases where the application is unavailable, but it does mean that an application shouldn't assume that a successful send is equivalent to a successful reception. It probably is, for a variety of reasons, but it's explicitly outside of the protocol's scope.
TCP guarantees sequence and integrity of the byte stream. You will not receive data out of sequence. From RFC 793:
Reliable Communication: A stream of data sent on a TCP connection is delivered reliably and in
order at the destination.
My recent project requires the use of i2c communication using a single master with multiple slaves. I know that with each data byte (actual data) sent by master,the slave responds with Nack\Ack(1,0).
I am confused that how this Nack and ACK are interpreted. I searched the web but i didn't got clear picture about this. My understanding is something like this.
ACK- I have successfully received the data. Send me more data.
NACK- I haven't received the data.Send again.
Is this something like this or I am wrong.
Please clarify and suggest the right answer.
Thanks
Amit kumar
You really should read the I2C specification here, but briefly, there are two different cases to consider for ACK/NACK:
After sending the slave address: when the I2C master sends the address of the slave to talk to (including the read/write bit), a slave which recognizes its address sends an ACK. This tells the master that the slave it is trying to reach is actually on the bus. If no slave devices recognize the address, the result is a NACK. In this case, the master must abort the request as there is no one to talk to. This is not generally something that can be fixed by retrying.
Within a transfer: after the side reading a byte (master on a receive or slave on a send) receives a byte, it must send an ACK. The major exception is if the receiver is controlling the number of bytes sent, it must send a NACK after the last byte to be sent. For example, on a slave-to-master transfer, the master must send a NACK just before sending a STOP condition to end the transfer. (This is required by the spec.)
It may also be that the receiver can send a NACK if there is an error; I don't remember if this is allowed by the spec.
But the bottom line is that a NACK either indicates a fatal condition which cannot be retried or is simply an indication of the end of a transfer.
BTW, the case where a receiving device needs more time to process is never indicated by a NACK. Instead, a slave device either does "clock stretching" (or the master simply delays generating the clock) or it uses a higher-layer protocol to request retrying.
Edit 6/8/19: As pointed out by #DavidLedger, there are I2C flash devices that use NACK to indicate that the flash is internally busy (e.g. completing a write operation). I went back to the I2C standard (see above) and found the following:
There are five conditions that lead to the generation of a NACK:
No receiver is present on the bus with the transmitted address so there is no device to respond with an acknowledge.
The receiver is unable to receive or transmit because it is performing some real-time function and is not ready to start
communication with the master.
During the transfer, the receiver gets data or commands that it does not understand.
During the transfer, the receiver cannot receive any more data bytes.
A master-receiver must signal the end of the transfer to the slave transmitter.
Therefore, these NACK conditions are valid per the standard.
Short delays, particularly within a single operation will normally use clock stretching but longer delays, particularly between operations, as well as invalid operations, my well produce a NACK.
I2C Protocol starts with a start bit followed by the slave address (7 bit address + 1 bit for Read/Write).
After sending the slave address Master releases the data bus(SDA line), put the line in high impedance state leaving it for slave to drive the line.
If address matches the slave address, slave pull the line low for the ACK.
If the line is not pull low by any of the slave, then Master consider it as NACK and sends the Stop bit or repeated start bit in next clock pulse to terminate the or restart the communication.
Besides this the NACK is also sent whenever receiver is not able to communicate or understand the data.
NACK is also used by master (receiver) to terminate the read flow once it has all the data followed by stop bit.
I'm trying to figure out if it is somehow possible to setup Mirth to send 2 ACK back to the caller Application:
A) 1 ACK sent from Mirth to the caller when the transmission has been received from Mirth;
B) 1 ACK sent from Mirth to the caller after the channel is finished processing the message.
I know that Mirth can either be configured to send ACK before processing (case A above) or after processing (Case B above), but I could not find any way to send both.
Has anyone had experience in doing this?
Thank you all for your help.
Mirth uses a single responseMap to store acknowledgement which is processed after all scripts. So, if you put anything there when a message is received, this Ack will be overridden with a new Ack placed into the same map at the end. And only the latter will be sent, which you've already experienced I guess.
If I'm correct, what you are trying to achieve is, first, to confirm that the message is received by a remote location (let's call it System B) and, second, is to confirm that the message successfully processed. If your client (System A) is capable to send a message to two endpoints at System B then you may create two receiving channels on the System B side, one of these channels sends ACK immediately after receiving the message and does nothing. The other channel processes the message and sends ACK in postprocessor.
There are other options, say, on System B side redirect an incoming message to another channel which forms Ack and sends it back to System A, but then System A should have a listener on its side.
Or, System B may have a receiving channel that sends Ack immediately, routes the message to another channel that is connected to its destinations, and remove that destination to prevent incoming message to propagate to that channel. The second channel processes the message and sends Ack back to the first channel. First channel resends that Ack back to System A. (I have not tested such configuration, so this is just an idea to overcome a single responseMap. It may not work.)
In Spray, for chunked response we can send chunked message with an ack like
peer ! MessageChunk(buffer).withAck(MyAck(k))
and we can get ack message back which confirms the message delivery to the OS network layer. Do we have something like timeouts for these acks ?
I don't think there is an Ack Timeout feature built in to spray.
It should be possible, for the Chunking Actor to keep track of sent messages & timeouts if your Ack contains details about the sent chunk. See ChunkingActor[T] used to marshall Stream[T] in the spray source code to get an idea of how this could be done. You may choose to have this actor keep track of the last chunk sent along with the time sent. If an Ack does not come back for that chunk during the set timeout, you can handle the case where chunk delivery has "timed out".
I already read this question about socket synchronization but I still dont get it yet.
Recently I was working on a relatively simple client/server app where the communication happens over a tcp socket. The client is written in PHP using the C-like functions (especially fsockopen and fgetc) PHP provides to interact with sockets, the server is written in node.js using a Stream for outputting data.
The protocol is quite simple, the message is just a string which ends with a 0-byte character.
Basically it works like this:
SERVER: Message 1
CLIENT: Ack 1
SERVER: Message 2
CLIENT: Ack 2
....
Which really worked fine as my client processed one message at a time by reading char by char from the socket until a 0-byte was encountered which designates the end of the message. Then the client writes back to the server that it has successfully received the message (thats the Ack <message id> part).
Now this happened:
SERVER: Message 1
CLIENT: Ack 1
SERVER: Message 2
CLIENT: Ack 2
SERVER: Message 3
Message 4
Message 5
Message 6
CLIENT: <DOH!>
....
Meaning the server unexpectedly sent multiple messages in one "batch" to the client, although every message is a single stream.write(...) operation on the server. It seemed like the messages were buffered somewhere and then sent to the client at once. My client code couldnt cope with multiple messages in the socket WITHOUT an Ack response in between, so it cut off the remaining messages after id 3.
So my question is:
How synchronized are sockets in their read and writes? From the question above I understand that a socket is basically two uni-directional pipes, which means they are not synchronized at all?
How can it happen that some messages were sent to my client in a simple "one message-one ack" manner and then suddendly multiple messages are written to the stream?
Does it actually change the picture if the socket is opened in a blocking/non-blocking manner?
I tested this on a Ubuntu VM (so no load or anything that could provoke strange behaviour) using PHP 5.4 and node 0.6.x.
TCP is an abstraction of a bi-directional stream, and as such has no concept of messages and cannot preserve message boundaries. There is no guarantee how multiple send() or recv() calls will map to TCP packets. You should treat send() as if calling it multiple times is equivalent to calling it once with the concatenation of all the data. More importantly, when receiving, you should make sure that your code interprets the incoming data exactly the same way, no matter how it was split over indvidual recv() calls.
To receive properly, you can use a buffer where you store incomplete messages. But be careful that when you have an incomplete message in a buffer, the next recv() call may complete the current message, as well as provide zero or more complete messages, and possibly part of another incomplete message.
The blocking or non-blocking mode doesn't change anything here - it's only about the way your application interfaces with the OS.
There are two synchronization concepts to deal with:
The (generally) synchronous operation of send() or recv().
The asynchronous way that one process sends a message and the way the other process handles the message.
If you can, try to avoid a design that keeps a client and server in process-synchronized "lock step" with each other. That's asking for trouble. What if the one of the processes closes unexpectedly? The other process/thread might hang on a recv() that will never come. It's one thing for your design to expect each message to be acknowledged eventually, but it's quite another for your design to expect that only one message can be sent, then it must be acknowledged, before you may send another.
Consider this:
Server: send 1
Client: ack 1
Server: send 2
Server: send 3
Client: ack 2
Server: send 4
Client: ack 3
Client: ack 4
A design that can accommodate this situation is better than one that expects:
Server: send 1
Client: ack 1
Server: send 2
Client: ack 2
Server: send 3
Client: ack 3
Server: send 4
Client: ack 4