Some protocols like HTTP can specify a message length, then send a (possibly very long) message. No other messages can be received while this message is being sent (something HTTP/2.0 tried to solve) so if you decide to ignore the message, you can't just continue waiting for messages and not pull its data.
Normally I read() up to the length of the message repeatedly into a junk buffer and just ignore the bytes. But that involves copying possibly millions of bytes from kernelspace into userspace (at 1 copy per memory page, so not millions of copies). Isn't there some way to just tell the kernel to discard the bytes instead of providing them?
It seemed like an obvious question, but the only answer I've been able to come up with is oddly resource heavy, either using splice() to dump the bytes into a pipe and seeking the pipe back to 0, or opening "/dev/null" and using sendfile() to send the bytes there. I could do that, and reserve a (single) file descriptor for flushing data out of clogged connections, without reading, but isn't there just a... ignore(descriptor, length) function?
Related
I want to understand how Indy socket timeouts works, because I want to use them in the following way.
I have an application (TCP server/client) that transfers a file over the Internet. When I start the transfer, I want to be able to stop it fast enough (let's say, 1500 ms) if I decide that. If some socket is reading data, and something happens on the wire that makes it late, I won't be able to stop the transfer, because the socket is hung reading data. So I need to set some short timeouts, that in normal operation will not be triggered. But if something happens and data is running late, the control will be passed to the main proc and I'll be able to check for the abort request.
Now, I don't know what to do next... If a socket read times out, what do that mean? The socket did not receive any data for that period of time... Or, the socket received some data in the buffer but doesn't have time to finish? I have a feeling that those timeouts are the waiting periods for something to happen (start a read or a write operation). But (let's say a read), once started, what happens if the socket receives half of the data (which he was asked to read) and then nothing comes? Will that call block the program execution forever? Because if that happens, then again I will not be able to check for abort request.
Anyway... when the timeout occurs, it will raise an exception? I can catch it and try again, in the same connection, like nothing happened? Will the in/out buffer be modified after a timeout?
I am using this to set the Read and Write timeouts:
Socket.ReadTimeout:= WorkingRTimeOut;
Socket.Binding.SetSockOpt(SOL_SOCKET, SO_SNDTIMEO, WorkingWTimeOut);
Socket timeouts are applied on a per-byte basis.
If you ask a socket to read N number of bytes, it will return as many bytes as it can, up to N bytes max, from the socket's receive buffer. It can (and frequently does) return fewer bytes, requiring another read to receive the remaining bytes. If a timeout error occurs, it means no bytes at all arrived in time for the current read. There is no way to know why, or whether they ever will arrive.
If you ask a socket to send N number of bytes, it will accept as many bytes as it can, up to N bytes max, into the socket's write buffer. It can (and sometimes does) buffer fewer bytes, requiring another send to buffer the remaining bytes. If a timeout occurs, it means the socket's write buffer has filled up, the receiver is not reading fast enough (or at all) to clear space in the sender's write buffer in time.
If you ask Indy to read/send N number of bytes, it may perform multiple socket reads/sends internally, waiting for all of the expected bytes to be received/sent. So it may have read/sent X number of bytes, where X < N, before the timeout occured. Sure, you could try another read/send again, asking for only the remaining bytes you haven't received/sent yet (N - X), but don't ask for the bytes you already received/sent (X). You might receive/send more bytes, or you might get another timeout, there is no way to know until you try. However, depending on context, it may not be easy/possible to know how many bytes were received/sent before the timeout, so you might not know how many remaining bytes to ask for again. In which case, about all you can sensibly do is just close the TCP connection, reconnect, and resume/start over.
As for your ability to abort a connection quickly, you could move your read/send code to a worker thread, and then Disconnect() the socket from your main proc when needed. That will generally abort any blocking read/send in progress.
If Two processes communicate via sockets and Process A sends Process B 100 bytes.
Process B tries to read 150 bytes. Later Process A sends 50 bytes.
What is the result of Process B's read?
Will the process B read wait until it receives 150 bytes?
That is dependent on many factors, especially related to the type of socket, but also to the timing.
Generally, however, the receive buffer size is considered a maximum. So, if a process executes a recv with a buffer size of 150, but the operating system has only received 100 bytes so far from the peer socket, usually the available 100 are delivered to the receiving process (and the return value of the system call will reflect that). It is the responsibility of the receiving application to go back and execute recv again if it is expecting more data.
Another related factor (which will not generally be the case with a short transfer like 150 bytes but definitely will if you're sending a megabyte, say) is that the sender's apparently "atomic" send of 1000000 bytes will not all be delivered in one packet to the receiving peer, so if the receiver has a corresponding recv with a 1000000 byte buffer, it's very unlikely that all the data will be received in one call. Again, it's the receiver's responsibility to continue calling recv until it has received all the data sent.
And it's generally the responsibility of the sender and receiver to somehow coordinate what the expected size is. One common way to do so is by including a fixed-length header at the beginning of each logical transmission telling the receiver how many bytes are to be expected.
Depends on what kind of socket it is. For a STREAM socket, the read will return either the amount of data currently available or the amount requested (whichever is less) and will only ever block (wait) if there is no data available.
So in this example, assuming the 100 bytes have (all) been transmitted and received into the receive buffer when B reads from the socket and the additional 50 bytes have not yet been transmitted, the read will return those 100 bytes and will not wait.
Note also, the dependency of all the data being transmitted and received -- when process A writes data to a socket it will not necessarily be sent immediately or all at once. Depending on the underlying transport, there's an MTU size and any write larger than that will be broken up. Smaller writes may also be delayed and combined with later writes to make up the MTU. So in your case the send of 100 bytes might be too large (and broken up), or might be too small and not be transmitted immediately.
I am currently writing code to transfer data to a remote vendor. The transfer will take place over a TCP socket. The problem I have is the data is variable length and there are no framing or size markers. Sending the data is no problem, but I am unsure of the best way to handle the returned data.
The data is comprised of distinct "messages" but they do not have a fixed size. Each message has an 8 or 16 byte bitmap that indicates what components are included in this message. Some components are fixed length and some are variable. Each variable length component has a size prefix for that portion of the overall message.
When I first open the socket I will send over messages and each one should receive a response. When I begin reading data I should be at the start of a message. I will need to interpret the bitmap to know what message fields are included. As the data arrives I will have to validate that each field indicated by the bitmap is present and of the correct size.
Once I have read all of the first message, the next one starts. My concern is if the transmission gets cut partway through a message, how can I recover and correctly find the next message start?
I will have to simulate a connection failure and my code needs to automatically retry a set number of times before canceling that message.
I have no control over the code on the remote end and cannot get framing bytes or size prefixes added to the messages.
Best practices, design patterns, or ideas on the best way to handle this are all welcomed.
From a user's point of view, TCP is a stream of data, just like you might receive over a serial port. There are no packets and no markers.
A non-blocking read/recv call will return you what has currently arrived at which point you can parse that. If, while parsing, you run out of data before reaching the end of the message, read/recv more data and continue parsing. Rinse. Repeat. Note that you could get more bytes than needed for a specific message if another has followed on its heels.
A TCP stream will not lose or re-order bytes. A message will not get truncated unless the connection gets broken or the sender has a bug (e.g. was only able to write/send part and then never tried to write/send the rest). You cannot continue a TCP stream that is broken. You can only open a new one and start fresh.
A TCP stream cannot be "cut" mid-message and then resumed.
If there is a short enough break in transmission then the O/S at each end will cope, and packets retransmitted as necessary, but that is invisible to the end user application - as far as it's concerned the stream is contiguous.
If the TCP connection does drop completely, both ends will have to re-open the connection. At that point, the transmitting system ought to start over at a new message boundary.
For something like this you would probably have a lot easier of a time using a networking framework (like netty), or a different IO mechansim entirely, like Iteratee IO with Play 2.0.
I know that it is possible that multiple packets would be stacked to the buffer to be read from and that a long packet might require a loop of multiple send attempts to be fully sent. But I have a question about packaging in these cases:
If I call recv (or any alternative (low-level) function) when there are multiple packets awaiting to be read, would it return them all stacked into my buffer or only one of them (or part of the first one if my buffer is insufficient)?
If I send a long packet which requires multiple iterations to be sent fully, does it count as a single packet or multiple packets? It's basically a question whether it marks that the package sent is not full?
These questions came to my mind when I thought about web sockets packaging. Special characters are used to mark the beginning and end of a packet which sorta leads to a conclusion that it's not possible to separate multiple packages.
P.S. All the questions are about TCP/IP but you are welcomed to share information (answers) about UDP as well.
TCP sockets are stream based. The order is guaranteed but the number of bytes you receive with each recv/read could be any chunk of the pending bytes from the sender. You can layer a message based transport on top of TCP by adding framing information to indicate the way that the payload should be chunked into messages. This is what WebSockets does. Each WebSocket message/frame starts with at least 2 bytes of header information which contains the length of the payload to follow. This allows the receiver to wait for and re-assemble complete messages.
For example, libraries/interfaces that implement the standard Websocket API or a similar API (such as a browser), the onmessage event will fire once for each message received and the data attribute of the event will contain the entire message.
Note that in the older Hixie version of WebSockets, each frame was started with '\x00' and terminated with '\xff'. The current standardized IETF 6455 (HyBi) version of the protocol uses the header information that contains the length which allows much easier processing of the frames (but note that both the old and new are still message based and have basically the same API).
TCP connection provides for stream of bytes, so treat it as such. No application message boundaries are preserved - one send can correspond to multiple receives and the other way around. You need loops on both sides.
UDP, on the other hand, is datagram (i.e. message) based. Here one read will always dequeue single datagram (unless you mess with low-level flags on the socket). Event if your application buffer is smaller then the pending datagram and you read only a part of it, the rest of it is lost. The way around it is to limit the size of datagrams you send to something bellow the normal MTU of 1500 (less IP and UDP headers, so actually 1472).
I've seen several uses of sockets where programmers send a command or some information over a TCP/IP socket, and expect it to be received in one call on the receiving side.
For eg, transmitting
mySocket.Send("SomeSpecificCommand")
They assume the receive side will receive all the data in one call. For eg:
Dim data(255) As Byte
Dim nReceived As Long = s.Receive(data, 0, data.Count, SocketFlags.None)
Dim str As String = Encoding.ASCII.GetString(data, 0, n)
If str = "SomeSpecificCommand" Then
DoStuff()
...
The example above doesn't use any terminator, so the programmer is relying on the fact that the sockets implementation is not allowed, for example, to return "SomeSpecif" in a first call to Receive(), and "cCommand" in a later call to Receive(). (NOTE - In the example, the buffer is sized to be larger than the expected string).
I've never before given this much thought and had just assumed that this type of coding is unsafe and have always used delimiters. Have I been wasting my time (and processor cycles)?
There is no guarantee that it will all arrive at the same time. The code (the app's protocol) needs to deal with the possibility that data from one send may arrive in multiple pieces or the possibility that data from more than one send could arrive in one receive.
Short snippets of data sent in one short call to send() will usually arrive in one call to recv(), which is why code like that will work most of the time. However, it's not guaranteed and therefore bad practice to rely on it.
TCP buffers the data and may split it up as it sees fit. TCP tries to send as few packets as possible to conserve bandwidth, so it won't split up the data for no good reason. However, if it's been queueing up some data and the data from one call to send() happens to straddle a packet boundary, that data will be split up.
Alternately, TCP could try to send it in one packet, but then a router anywhere along the path to the destination could come back and say "this packet is too big!". Then TCP will split it into smaller packets.
When sending data across a network, you should expect your data to be fragmented across multiple packets and structure your code and data to deal with this. In the example case where you are sending a handful of bytes, everything will work fine.. until you start sending larger packets.
If you are expecting to receive one message at a time then you can just loop reading bytes for an interval after the first bytes arrive. This is simple but inefficient.
A delimiter could be used as suggested but then you have to guard against accidentally including the delimiter within the regular data. If you are only sending text then you can use null or some non-printable character. If you are sending binary data then this becomes more difficult as any occurrence of the delimiter within the data needs to be escaped by the sender and un-escaped by the receiver.
An alternative to delimiters is to add a field to the front of the data containing a message length. This is better than using a delimiter as it removes the need for escaping data and better than simply looping until a timer expires as it will be more responsive.
No, its not a good idea to assume that the server (assuming your the client) is gonna only send you one socket response. The server could be running though a list of procedures that returns multiple results. I would continue to read from the socket until there is nothing left to pick up, then wait a few miliseconds and test again. If nothing shows up, chances are good that the server has finished sending responses.
There are several types of sockets. TCP uses SOCK_STREAM, which don't preserve message boundaries. SOCK_SEQPACKET sockets do preserve message boundaries.
EDIT: SCTP supports both SOCK_STREAM and SOCK_SEQPACKET.