In a Modbus server implementation, what response should the server send if it receives a request from the client that contains too few (or no) data bytes to interpret correctly?
For example, a modbus RTU server (with address 0x01) receives the ADU datagram: 0x01, 0x01, 0xE0, 0xC1. In this case no physical transport layer errors are detected, the address is correct, the CRC is correct and the function (Read Coils) is correct and implented on the server, but the PDU does not contain the Starting Address or Quantity of Inputs fields required to process the request.
Should the server assume that a (very rare) bit error has occurred and not respond at all?
Should the server interpret this as 'a value in the query data field' being not allowed for the server and respond with an ILLEGAL DATA VALUE exception?
Should the server do something completely different?
In my experience, at least with Modbus TCP, devices tend to just ignore malformed requests.
From the specification MODBUS APPLICATION PROTOCOL SPECIFICATION V1.1b3 the exception (code 3) is correct. Figure 9 MODBUS Transaction state diagram clearly indicates the exception response to an incorrectly formed message.
I suspect the common response that rejects the message is indistinguishable from a transmission error, and the implementor of the faulty client will then be induced to correct their implementation.
Your suggestion that a communication error triggers this is possible, but only if the underlying link does not detect missing bytes. Any byte, other than 0xFF will introduce a start bit into a serial channel, and a missing byte in the TCP/UDP implementations is even less likely.
Related
Suppose a packet is sent by UDP. I'm wondering in which layer the big/little endian conversion of payload is done.
Why do you think this is done? The transport protocol has no notion of your data, it transmits bytes. You, as an application protocol designer, will have to decide and ensure to send your data in a certain endianness.
UDP doesn't know that four successive bytes somewhere in a packet form a 32-bit integer, for example. They might as well form four 1-byte values, for example UTF-8 code points. Do you want UDP to randomly invert your strings?
See also Sending UDP packets in the correct Endianness.
'Endian conversion' of a protocol is usually done in the layer that handles the protocol.
For example: NTP program/library itself creates a NTP-message in network byte order before passing the message to UDP-layer (=sending). UDP layer will create UDP header in network byte order, but does not change the payload (=NTP message).
Lower layer protocols doesn't know and care what kind of upper layer protocols are carried in the payload. There are few exceptions or borderline cases, like checksum offloading in NIC.
I'm working with special-purpose hardware that is connected on a 10G Ethernet link. I've got some questions on the handling of incoming datagrams, as follows:
What happens if the NIC discovers an incorrect link-level Ethernet CRC? Some searching shows that errors may not be reliably reported (here, for instance). Can I expect to get better stats from more recent kernels (2.6 - 3.10?)
What does the kernel actually check before deciding whether to return a packet to a recv? I'm guessing that for IPv4, the IPv4 header checksum must be correct, but what about the optional UDP header checksum?
Can recv ever return 0 for a UDP/SOCK_DGRAM?
For a non-blocking SOCK_DGRAM socket, does recv always return the entire packet when data is available? I guess it has to, but it's not obvious from the docs.
Thanks.
My knowledge may be out of date here, but historically, packets with FCS errors were not delivered at all and were not counted toward the interface statistics. The Ethernet layer error counts are usually reported by ethtool -S <interface>. The problem has always been that the interface statistics were maintained above the driver level and there was no standard API internally for network drivers to report those statistics. (Also, of course, in the very old days of 10Mb half duplex, collisions happened pretty frequently and Ethernet layer statistics weren't terribly informative as to your own adapter's behavior.)
You should not receive a packet if its IP header checksum is wrong, or if the UDP checksum is wrong when a checksum is provided (i.e. non-zero).
Yes. If you provide a zero length buffer, you will receive the next incoming datagram but then the entire content will be truncated, resulting in a return value of zero. Also, UDP permits zero-length datagrams: so if you receive a datagram with no content, the return value would also be zero. Aside from those two cases, I don't believe you'll get a return value of zero.
Yes, you should get the entire datagram provided there is space in your buffer. Otherwise, no. If you don't provide enough space to hold the entire datagram, the part that doesn't fit is discarded (i.e. your next recv will get a subsequent packet, not the end of the truncated one).
I'm working on an gameServer that communicate with game client, but wonder whether the packet server send to client remain sequence when client received it ?
like server sends packets A,B,C
but the client received B,A,C ?
I have read the great blog http://packetlife.net/blog/2010/jun/7/understanding-tcp-sequence-acknowledgment-numbers/
It seems that every packet send by the server has an ack corresponding by client, but it does not say why the packet received by client has the same sequence with server
It's worth reading TCP's RFC, particularly section 1.5 (Operation), which explains the process. In part, it says:
The TCP must recover from data that is damaged, lost, duplicated, or delivered out of order by the internet communication system. This is achieved by assigning a sequence number to each octet transmitted, and requiring a positive acknowledgment (ACK) from the receiving TCP. If the ACK is not received within a timeout interval, the data is retransmitted. At the receiver, the sequence numbers are used to correctly order segments that may be received out of order and to eliminate duplicates. Damage is handled by adding a checksum to each segment transmitted, checking it at the receiver, and discarding damaged segments.
I don't see where it's ever made explicit, but since the acknowledgement (as described in section 2.6) describes the next expected packet, the receiving TCP implementation is only ever acknowledging consecutive sequences of packets from the beginning. That is, if you never receive the first packet, you never send an acknowledgement, even if you've received all other packets in the message; if you've received 1, 2, 3, 5, and 6, you only acknowledge 1-3.
For completeness, I'd also direct your attention to section 2.6, again, after it describes the above-quoted section in more detail:
An acknowledgment by TCP does not guarantee that the data has been delivered to the end user, but only that the receiving TCP has taken the responsibility to do so.
So, TCP ensures the order of packets, unless the application doesn't receive them. That exception probably wouldn't be common, except for cases where the application is unavailable, but it does mean that an application shouldn't assume that a successful send is equivalent to a successful reception. It probably is, for a variety of reasons, but it's explicitly outside of the protocol's scope.
TCP guarantees sequence and integrity of the byte stream. You will not receive data out of sequence. From RFC 793:
Reliable Communication: A stream of data sent on a TCP connection is delivered reliably and in
order at the destination.
Assume I have defined my own application layer protocol on top of TCP for Instant Messaging. I have used a packet structure for the messages. As I am using symmetric (AES) and asymmetric (RSA) encryption, I obtain a different
packet size for different message types. Now to my questions.
How to read from a socket that I receive a single application layer packet?
What size should I specify?
Thanks in advance.
I have two approaches in mind.
Read from the TCP stream a fixed amount of bytes that represents the
actual packet size, and finally re-read from the stream the former gathered size of bytes.
Read the maximal packet size from the stream. Verify the actual size of
obtained bytes and decide so which message type it was.
Now, a more general question. Should I provide metadata like the packet size, encryption method, receiver, sender, etc.? If yes, should I encrypt these meta data as well?
Remember that with TCP, when reading from the network, there is no guarantee about the number of bytes received at that point in time. That is, a client might send a full packet in its write(), but that does not mean that your read() will receive the same number of bytes. Thus your code will always need to read some number of bytes from the network, then verify (based on the accumulated data) that you have received the necessary number of bytes, and then you can verify the packet (type, contents, etc) from there.
Some applications use state machine encoders/decoders and fixed size buffers for reading/writing their network data; other applications dynamically allocate buffers large enough for the "full packet", then continue reading bytes from the network until the "full packet" buffer is full. Which approach you take depends on your application. Thus the size you use for reading is not as important as how your code ensures that it has received a full packet.
As for whether you should encrypt additional metadata, that depends very much on your threat model (i.e. what threats your protocol wants to guard against, what assurances your protocol needs to provide to its clients/users). There's no easy way to answer that question without more context/details.
Hope this helps!
I am a newbie to networks and in particular TCP (I have been fooling a bit with UDP, but that's it).
I am developing a simple protocol based on exchanging messages between two endpoints. Those messages need to be certified, so I implemented a cryptographic layer that takes care of that. However, while UDP has a sound definition of a packet that constitutes the minimum unit that can get transferred at a time, the TCP protocol (as far as my understanding goes) is completely stream oriented.
Now, this puzzles me a bit. When exchanging messages, how can I tell where one starts and the other one ends? In principle, I can obviously communicate fixed length messages or first communicate the size of each message in some header. However, this can be subject to attacks: while of course it is going to be impossible to distort or determine the content of the communication, the above technique would make it easy to completely disrupt my communication just by adding a single byte in the middle.
Say that I need to transfer a message 1234567 bytes long. First of all, I communicate 4 bytes with an integer representing the size of the message. Okay. Then I start sending out the actual message. That message gets split in several packets, which get separately received. Now, an attacker just sends in an additional packet, faking it as if it was part of the conversation. It can just be one byte long: this completely destroys any synchronization mechanism I have implemented! The message has a spurious byte in the middle, and it doesn't successfully get decoded. Not only that, the last byte of the first message disrupts the alignment of the second message and so on: the connection is destroyed, and with a simple, simple attack! How likely and feasible is this attack anyway?
So I am wondering: what is the maximum data unit that can be transferred at once? I understand that to a call to send doesn't correspond a call to receive: the message can be split in different chunks. How can I group the packets together in some way so that I know that they get packed together? Is there a way to define an higher level message that gets reconstructed and aligned all together and triggers a single call to a receive-like function? If not, what other solutions can I find to keep my communication re-alignable even in presence of an attacker?
Basically it is difficult to control the way the OS divides the stream into TCP packets (The RFC defining TCP protocol states that TCP stack should allow the clients to force it to send buffered data by using push function, but it does not define how many packets this should generate. After all the attacker can modify any of them).
And these TCP packets can get divided even more into IP fragments during their way through the network (which can be opted-out by a 'Do not fragment' IP flag -- but this flag can cause that your packets are not delivered at all).
I think that your problem is not about introducing packets into a stream protocol, but about securing it.
IPSec could be very beneficial in your scenario, as it operates on the network layer.
It provides integrity for every packet sent, so any modification on-the-wire gets detected and the invalid packets are dropped. In case of TCP the dropped packets get re-transmitted automatically.
(Almost) everything is done automatically by the OS -- so yo do not need to worry about it (and make mistakes doing so).
The confidentiality can be assured as well (with the same advantage of not re-inventing the wheel).
IPSec should provide you a reliable transport protocol ontop of which you can use whatever framing format you like.
Another alternative is using SSL/TLS on top of TCP session which is less robust (as it does close the whole connection on integrity error).
Now, an attacker just sends in an additional packet, faking it as if it was part of the conversation. It can just be one byte long: this completely destroys any synchronization mechanism I have implemented!
Thwarting such an injection problem is dealt with by securing the stream. Create an encrypted stream and send your packets through that.
Of course the encrypted stream itself then has this problem; its messages can be corrupted. But those messages have secure integrity checks. The problem is detected, and the connection can be torn down and re-established to resynchronize it.
Also, some fixed-length synchronizing/framing bit sequence can be used between messages: some specific bit pattern. It doesn't matter if that pattern occurs inside messages by accident, because we only ever specifically look for that pattern when things go wrong (a corrupt message is received), otherwise we skip that sequence. If a corrupt message is received, we then receive bytes until we see the synchronizing pattern, and assume that whatever follows it is the start of a message (length followed by payload). If that fails, we repeat the process. When we receive a correct message, we reply to the peer, which will re-transmit anything we didn't get.
How likely and feasible is this attack anyway?
TCP connections are identified by four items: the source and destination IP, and source and destination port number. The attacker has to fake a packet which matches your stream in these four identifiers, and sneak that packet past all the routers and firewalls between that attacker and the receiving machine. The attacker also has to be in the right ballpark with regard to the TCP sequence number.
Basically, this is next to impossible for an attacker C to perpetrate against endpoints A and B which are both distant from C on the network. The fake source IP will be rejected long before C is able to reach its destination. It's more plausible as an inside job (which includes malware): C is close to A and B.