Under what circumstances, if any, can ReadFile hang after WSAEventSelect said it was ready? - sockets

Question

This is rather ugly. We're getting weird hangups caused (read: triggered) by some security scanning software. I suspect it has a nonstandard TCP/IP stack, and the customer is asking why hangups.
Static analysis suggests only two possible locations for the hangup. The hangup either has to be in ReadFile() or WriteFile() on a socket; and WriteFile() cannot hang here unless the scanner is designed to make WriteFile() hang by setting the window size to zero. If WriteFile() were to return at all even if it didn't make progress I'd be able to knock the thing out of its wedged state. I also don't think the log state is consistent with WriteFile() returning.
So onto ReadFile(): this is the calling sequence:
SOCKET conn;
HANDLE unwedgeEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
HANDLE listenEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
if (listenEvent == NULL) return;
//...
conn = accept(lstn);
for (;;) {
HANDLE wakeup[2];
wakeup[0] = unwedgeEvent;
wakeup[1] = listenEvent;
if (WSAEventSelect(conn, socket, FD_READ | FD_CLOSE) == 0) {
// Log error
break;
}
which = WaitForMultipleObjects(2, wakeup, FALSE, INFINITE);
if (which < 0) {
// Log error
break;
}
if (which == 1) {
DWORD read;
r = ReadFile(conn, chunk, 4096, &read, NULL);
if (r == 0) {
// Handle error -- not stuck here
} else {
// Handle data -- not stuck here either
}
}
if (which == 0) break;
}
Where signalling unwedgeEvent doesn't manage to accomplish anything and the thread remains stuck forever.
So the real question is have I gone nuts or is this really a thing that can happen?
So this has gone somewhat off the deep end; I don't need non-blocking sockets at all. I need a select() that takes handle arguments to things that are sockets and things that are not sockets.
The following API sequences do not hang in ReadFile:
---------------------------------------------------------------
Sender B Receiver
WSAEventSelect
* WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 1)
WSAEventSelect
* WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 1)
................................................................
WSAEventSelect
* WaitForMultipeObjects
send(buffer size = 2)
ReadFile(size = 1)
WaitForMultipeObjects
ReadFile(size = 1)
................................................................
WSAEventSelect
* WaitForMultipeObjects
send(buffer size = 1)
send(buffer size = 1)
ReadFile(size = 1)
WaitForMultipeObjects
ReadFile(size = 1)
................................................................
WSAEventSelect
send(buffer size = 1)
WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 1)
WaitForMultipeObjects
ReadFile(size = 1)
................................................................
WSAEventSelect
send(buffer size = 1)
WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 2)
* WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 1)
................................................................
WSAEventSelect
send(buffer size = 1)
WaitForMultipeObjects
ReadFile(size = 1)
send(buffer size = 1)
WaitForMultipeObjects
ReadFile(size = 1)

Answer 1

I see a number of issues with your code:
You should use WSACreateEvent() and WSAWaitForMultipleEvents() instead of CreateEvent() and WaitForMultipleObjects(). Although the current implementation is that the former APIs simply map to the latter APIs, Microsoft is free to change that implementation at any time without breaking code that uses the former APIs properly.
In your call to WSAEventSelect(), socket should be listenEvent instead.
WSAEventSelect() returns SOCKET_ERROR (-1) on failure, not 0 like you have coded.
You are not calling WSAEnumNetworkEvents() at all, which you need to do in order to determine if FD_READ was the actual type of event triggered, and to clear the socket's event state and reset the event object. So, you may be acting on a stale read state, which could explain why you end up calling ReadFile() when there is actually nothing available to read.
WSAEventSelect() puts the socket into non-blocking mode (per its documentation), so it is actually not possible for ReadFile() (or any other reading function) to block on the socket. However, it could fail immediately with a WSAEWOULDBLOCK error, so make sure you are not treating that condition as a fatal error.
The WSAEventSelect() documentation does not list ReadFile() as a supported function for re-enabling events when calling WSAEnumNetworkEvents(). Although the Socket Handles documentation does say that a Winsock SOCKET can be used with non-Winsock I/O functions like ReadFile(), it recommends that a SOCKET should only be used with Winsock functions. So, you should use send()/recv() or WSASend()/WSARecv() instead of WriteFile()/ReadFile().
With that said, try something more like the following:
HANDLE unwedgeEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
if (unwedgeEvent == NULL) {
// Log error
return;
}
...
SOCKET conn = accept(lstn, NULL, NULL);
if (conn == INVALID_SOCKET) {
// Log error
return;
}
...
WSAEVENT listenEvent = WSACreateEvent();
if (listenEvent == NULL) {
// Log error
}
else if (WSAEventSelect(conn, listenEvent, FD_READ | FD_CLOSE) == SOCKET_ERROR) {
// Log error
}
else {
WSAEVENT wakeup[2];
wakeup[0] = (WSAEVENT) unwedgeEvent;
wakeup[1] = listenEvent;
char chunk[4096];
int read;
do {
DWORD which = WSAWaitForMultipleEvents(2, wakeup, FALSE, WSA_INFINITE, FALSE);
if (which == WSA_WAIT_FAILED) {
// Log error
break;
}
if (which == WSA_WAIT_EVENT_0) {
break;
}
if (which == (WSA_WAIT_EVENT_0+1)) {
WSANETWORKEVENTS events = {};
if (WSAEnumNetworkEvents(conn, listenEvent, &events) == SOCKET_ERROR) {
// Log error
break;
}
if (events.lNetworkEvents & FD_READ) {
read = recv(conn, chunk, sizeof(chunk), 0);
if (read == SOCKET_ERROR) {
if (WSAGetLastError() != WSAEWOULDBLOCK) {
// Log error
break;
}
}
else if (read == 0) {
break;
}
else {
// Handle chunk up to read number of bytes
}
}
if (events.lNetworkEvents & FD_CLOSE) {
break;
}
}
}
while (true);
WSACloseEvent(listenEvent);
}
closesocket(conn);