I have a very annoying problem that I found several times on other forums but I can't find a proper solution.
The problem is recv() returns 0 on the last few bytes of a connection. Here are some background information.
Both (client / server) applications run on the same machine.
Both (client / server) sockets are non-blocking
The transfered data size is 53 bytes.
Both (client / server) call shutdown and closesocket when the last send()/recv() was executed.
I also tried with SO_LINGER and 10 seconds, no success either
I call send() several times (small chunks) and from the client side 53 bytes are transfered.
The server calls recv() several times (4 byte requests) and read 49 bytes and then returns 0 (54 Bytes - 49 Bytes, so 4 bytes are missing).
MSDN and some forums write for non-blocking sockets:
recv() definitely returns < 0 on error and errno / WSAGetLastError is set
recv() definitely returns = 0 when the other side closed the connection
recv() definitely returns > 0 when data was read
MSDN also says:
Using the closesocket or shutdown functions with SD_SEND or SD_BOTH
results in a RELEASE signal being sent out on the control channel. Due
to ATM's use of separate signal and data channels, it is possible that
a RELEASE signal could reach the remote end before the last of the
data reaches its destination, resulting in a loss of that data. One
possible solutions is programming a sufficient delay between the last
data sent and the closesocket or shutdown function calls for an ATM
socket.
This is regarded in the example of recv() and send(): http://msdn.microsoft.com/en-us/library/windows/desktop/ms740121(v=vs.85).aspx
But still no success, I still get some interrupts in 10% of all connections after the 49 Byte is received, 90% of the connections succeed. Any ideas? Thx.
recv() returns 0 only when you request a 0-byte buffer or the other peer has gracefully disconnected. If you are not receiving all of the data you are expecting, then you are not reading the data correctly to begin with. Please update your question with your actual code.
My guess is that you're not really sending all the data you think your are sending. Check out:
The Ultimate SO_LINGER page
recv() definitely returns = 0 when the other side closed the connection
This is not completely true, in the following code using non-blocking winsock2 tcp, when no data is available, select returns 1 and recv returns 0, as does WSAGetLastError().
fd_set test = {1, socket};
const timeval timeout = {0, 0};
if (!::select(0, &test, nullptr, nullptr, &timeout)) return 0;
int done = ::recv(socket, buffer, 1, 0);
This continues even after the other end has called:
::shutdown(socket, SD_BOTH);
::closesocket(socket);
and then ended. Communication works as expected, it is just ::recv that seems to be "broken".
Related
The recv() library function man page mention that:
It returns the number of bytes received. It normally returns any data available, up to the requested amount, rather than waiting for receipt of the full amount requested.
If we are using blocking recv() call and requested for 100 bytes:
recv(sockDesc, buffer, size, 0); /* Where size is 100. */
and only 50 bytes are send by the server then this recv() is blocked until 100 bytes are available or it will return receiving 50 bytes.
The scenario could be that:
server crashes after sendign only 50 bytes
bad protocol design where server is only sending 50 bytes while client is expecting 100 and server is also waiting for client's reply (i.e. socket close connection has not been initiated by server in which recv will return)
I am interested on Linux / Solaris platform. I don't have the development environment to check it out myself.
recv will return when there is data in the internal buffers to return. It will not wait until there is 100 bytes if you request 100 bytes.
If you're sending 100 byte "messages", remember that TCP does not provide messages, it is just a stream. If you're dealing with application messages, you need to handle that at the application layer as TCP will not do it.
There are many, many conditions where a send() call of 100 bytes might not be read fully on the other end with only one recv call when calling recv(..., 100); here's just a few examples:
The sending TCP stack decided to bundle together 15 write calls, and the MTU happened to be 1460, which - depending on timing of the arrived data might cause the clients first 14 calls to fetch 100 bytes and the 15. call to fetch 60 bytes - the last 40 bytes will come the next time you call recv() . (But if you call recv with a buffer of 100 , you might get the last 40 bytes of the prior application "message" and the first 60 bytes of the next message)
The sender buffers are full, maybe the reader is slow, or the network is congested. At some point, data might get through and while emptying the buffers the last chunk of data wasn't a multiple of 100.
The receiver buffers are full, while your app recv() that data, the last chunk it pulls up is just partial since the whole 100 bytes of that message didn't fit the buffers.
Many of these scenarios are rather hard to test, especially on a lan where you might not have a lot of congestion or packet loss - things might differ as you ramp up and down the speed at which messages are sent/produced.
Anyway. If you want to read 100 bytes from a socket, use something like
int
readn(int f, void *av, int n)
{
char *a;
int m, t;
a = av;
t = 0;
while(t < n){
m = read(f, a+t, n-t);
if(m <= 0){
if(t == 0)
return m;
break;
}
t += m;
}
return t;
}
...
if(readn(mysocket,buffer,BUFFER_SZ) != BUFFER_SZ) {
//something really bad is going on.
}
The behavior is determined by two things. The recv low water mark and whether or not you pass the MSG_WAITALL flag. If you pass this flag the call will block until the requested number of bytes are received, even if the server crashes. Other wise it returns as soon as at least SO_RCVLOWAT bytes are available in the socket's receive buffer.
SO_RCVLOWAT
Sets the minimum number of bytes to
process for socket input operations.
The default value for SO_RCVLOWAT is
1. If SO_RCVLOWAT is set to a larger value, blocking receive calls normally
wait until they have received the
smaller of the low water mark value or
the requested amount. (They may return
less than the low water mark if an
error occurs, a signal is caught, or
the type of data next in the receive
queue is different than that returned,
e.g. out of band data). This option
takes an int value. Note that not all
implementations allow this option to
be set.
If you read the quote precisely, the most common scenario is:
the socket is receiving data. That 100 bytes will take some time.
the recv() call is made.
If there are more than 0 bytes in the buffer, recv() returns what is available and does not wait.
While there are 0 bytes available it blocks and the granularity of the threading system determines how long that is.
I'm currently implementing IOCP server for a game and I'm trying zero bytes recv technic.
I have 3 questions.
If you know disconnection from the client by checking if bytestransferred is 0, then how do you distinguish between receive completion and disconnection?
I'm not performing non-block mode recv() when I process actual receive process because clients send the bytes of actual data first, so I know how many bytes I'm receiving. Do I need to use non-block mode recv() still?
I'm doing it like so.
InputMemoryBitStream incomming;
std::string data;
uint32_t strLen = 0;
recv(socket, reinterpret_cast<char*>(&strLen), sizeof(uint32_t), 0);
incomming.resize(strLen);
recv(socket, reinterpret_cast<char*>(incomming.getBufferPtr()), strLen, 0);
incomming.read(data, strLen);
(InputMemoryBitStream is for reading compressed data.)
I'm dynamically allocating per io data every time I do WSARecv() and WSASend() and I free them as soon as I finish processing completed I/Os. Is it inefficient to do that? or is it acceptable? Should I reuse per io data maybe?
Thank you in advance.
If the server sends 4 bytes
send(sock, buffer1, 4, 0);
And client waits for exactly 4 bytes
recv(sock, buffer2, 4, 0);
Is there a possibility that less than 4 bytes will be written to buffer2?
No other send or recv was made before.
If there is no possibility, what's the maximum size of buffer that send can do, so that recv could get the same buffer size in one call.
There's no such thing as a "message", except what you delimit yourself.
REPEAT: THERE IS NO SUCH THING AS A MESSAGE.
TCP does not send messages, it sends an octet stream.
You need to send in a loop, in case there is a backlog of unacknowledged data and send does not use the whole buffer you passed in. You need to recv in a loop, in case the sending stack chunked it in an unexpected way. You need to delimit your messages (eg by prepending a network-endian length), so you can recover them properly.
From the man page for recv:
"These calls return the number of bytes received, or -1 if an error occurred.
For TCP sockets, the return value 0 means the peer has closed its half side of the connection."
As such recv is always allowed return all of the bytes sent, fewer than the bytes sent, or none of the bytes sent. You cannot assume anything simply because you happen to know what send is doing at the other end of the connection.
There is no way to guarantee that you can always get complete message.
Yes, there's this possibility. In fact, recv gives you as return value the amount of bytes received.
I have read that the combination of three things causes something like a 200ms delay with TCP: Nagle's algorithm, delayed acknowledgement, and the "write-write-read" combination. However, I cannot reproduce this delay with Java sockets and I am therefore not sure if I have understood correctly.
I am running a test on Windows 7 with Java 7 with two threads using sockets over the loopback address. I have not touched the tcpNoDelay option on any socket (false by default) nor played around with any TCP settings on the OS. The main piece of the code in the client is as below. The server is responding with a byte after each two bytes it receives from the client.
for (int i = 0; i < 100; i++) {
client.getOutputStream().write(1);
client.getOutputStream().write(2);
System.out.println(client.getInputStream().read());
}
I do not see any delay. Why not?
I believe you see delay acknowledgment.
You write 4 and 4 bytes to the socket. The server's TCP stack receives a segment (that probably contains at least 4 bytes from an int number) and wakes up the server application thread. This thread writes a byte back to the stream and this byte is sent to the client within ACK segment. I.e. TCP stack gives a chance to an application to send a reply immediately. So you see no delay.
You can write a dump of traffic and also make an experiment between two computers to see what really happens.
I am writing a simple client which reads binary data through a TCP stream.
Since the file is sent part-by-part I have a loop similar to this one:
while (1 /* Maybe a check for EOF or if the socket is closed here ? */) {
ssize_t bytes_red = recv(
if (bytes_read <= 0) // not working properly ?
break;
...
}
The reading is done via recv() method, which returns the number of bytes received.
The idea is obviously to read data as long the TCP stream is opened, but it seems the above approach does not work for me.
Any other ideas? What is a clean way to read from socket until the stream is closed or EOF is encountered?
EDIT: The above approach works fine, I had a bug elsewhere.
recv() returns 0 if the socket was properly closed, -1 if there was an error. So, you read until recv() returns <=0. Your posted code snippet looks correct - why do you think it's not working for you? What happens, what did you expect?
If your socket is non-blocking, you will also have to handle the case where recv() returns -1 and errno is set to EAGAIN or EWOULDBLOCK. This means that the socket is still alive and well, but there's no more data right now.