Abort socket operation Windows Phone - sockets

I am using pseudo-synchronous sockets in a Windows Phone 7 application. My socket code is based on the sample from http://msdn.microsoft.com/en-us/library/hh202858(v=vs.92).aspx.
The server's sending pattern is somewhat unpredictable. It starts with a fixed-size header that contains the length of the rest of the message. I first read in this header, and then I read the specified number of bytes from the socket.
Since I need to send messages to the server as well, and my attempts at duplexing the socket with a thread for receiving and another thread for sending caused lots of problems, I have a loop like this in my code:
while (KeepConnectionGoing)
{
byte[] Rcvd;
Rcvd = Socket.Receive();//Returns null if no message received in 50 ms
if (Rcvd != null)
{
ParseMessage(Rcvd);
}
if (HasMessageThatNeedsToBeSent())
{
byte[] Message = GetMessageToSend();
Socket.Send(Message);
}
}
This works fine for the majority of the time, but strange things happen when the message is null.
Because the timeout in the Receive method (see the linked sample) uses a ManualResetEvent, the receive request on the socket is never actually cancels. Even though the method returns, that request waits around somewhere, and when data is available on the socket, chomps up the header. The event handler has nothing to do with the data it received (since the method has returned and the variables in the method will never be used again), the data basically disappears. The read request I expect to return the header skips reads the bytes after the header, and I have no idea how long the message is.
I'd like to be able to cancel all outstanding requests if the socket times out. I am using anonymous methods like in the sample since it simplifies everything and prevents me from having to write all the state transfer code myself. Thus, I cannot unhook the event handler. I think though, that even if I were using a method as the event handler, but unhooking before the asynchronous operation is done, the callback method would still be called. (I haven't tested this, it's just my understanding)
Right now, the only solution I can see is hacking together some static byte arrays (ie. having a static byte[] Header and if it is null, I read the header, otherwise I read the message), but that seems like a really inelegant solution and very prone to race conditions.
Is there a better way?
Thanks

It appears there really is no good way to do this. A poll method would be nice, but Silverlight doesn't have it. I hacked together a solution using static flags to tell me what state I am in (Has the header been requested, has the message been requested), a static int for the length and a static buffer.
At the beginning of the method, either the header or the body can be requested. If the header has already been requested, the thread waits until a valid body length is available. If this wait times out, that means that the header receive operation is still pending, but there really is no message available. Otherwise, it reads in that length of a message.
If the header has not been requested, receive the header. In the event handler, after completion, check to see if the control flow has already continued (i.e. the receive operation took too long, so the function returned already, but is now actually done). Update the length, then request the body unless it timed out.

Related

How can I know if bytes have not been written by the ChannelWriter?

After I call the ChannelWriter.wite() method to send a message, it returns me a ChannelFuture. If the receiving end on the other side has disabled read in epoll_wait, I think the socket write is supposed to be partially complete since zero bytes are supposed to be written. I thought of using the returned ChannelFuture, but isSuccess() returns true.
How do I know if the ChannelWriter.write() was only partially complete or zero bytes written?
It will only notify the future once the whole bytes are even written or if the write failed.

Why does `libpq` use polling rather than notification for data fetch?

I am reading libpq reference. It has both of sync and async methods. Bu I discovered something strange.
When I see PQsendQuery function, it seems to send a query and return immediately. And I expected a callback function to get notified, but there was no such thing and the manual says to poll for data availability.
I don't understand why async method is written in polling way. Anyway, as libp is the official client implementation, I believe there should be a good reason for this design. What is that? Or am I missing correct callback stuffs mentioned somewhere else?
In the execution model of a mono-threaded program, the execution flow can't be interrupted by data coming back from an asynchronous query, or more generally a network socket. Only signals (SIGTERM and friends) may interrupt the flow, but signals can't be hooked to data coming in.
That's why having a callback to get notified of incoming data is not possible. The piece of code in libpq that would be necessary to emit the callback would never run if your code doesn't call it. And if you have to call it, that defeats the whole point of a callback.
There are libraries like Qt that provide callbacks, but they're architectured from the ground up with a main loop that acts as an event processor. The user code is organized in callbacks and event-based processing of incoming data is possible. But in this case the library takes ownership of the execution flow, meaning its mainloop polls the data sources. That just shifts the responsibility to another piece of code outside of libpq.
This page is describing how I can get be notified for async result fetch.
http://www.postgresql.org/docs/9.3/static/libpq-events.html#LIBPQ-EVENTS-PROC
PGEVT_RESULTCREATE
The result creation event is fired in response to any query execution
function that generates a result, including PQgetResult. This event
will only be fired after the result has been created successfully.
typedef struct {
PGconn *conn;
PGresult *result; } PGEventResultCreate; When a PGEVT_RESULTCREATE event is received, the evtInfo pointer should be cast to a
PGEventResultCreate *. The conn is the connection used to generate the
result. This is the ideal place to initialize any instanceData that
needs to be associated with the result. If the event procedure fails,
the result will be cleared and the failure will be propagated. The
event procedure must not try to PQclear the result object for itself.
When returning a failure code, all cleanup must be performed as no
PGEVT_RESULTDESTROY event will be sent.

How to request a replay of an already received fix message

I have an application that could potentitally throw an error on receiving a ExecutionReport (35=8) message.
This error is thrown at the application level and not at the fix engine level.
The fix engine records the message as seen and therefore will not send a ResendRequest (35=2). However the application has not processed it and I would like to manually trigger a re-processing of the missed ExecutionReport.
Forcing a ResendRequest (35=2) does not work as it requires modifying the expected next sequence number.
I was wonderin if FIX supports replaying of messages but without requiring a sequence number reset?
When processing an execution report, you should not throw any exceptions and expect FIX library to handle it. You either process the report or you have a system failure (i.e. call abort()). Therefore, if your code that handles execution report throws an exception and you know how to handle it, then catch it in that very same function, eliminate the cause of the problem and try processing again. For example (pseudo-code):
// This function is called by FIX library. No exceptions must be thrown because
// FIX library has no idea what to do with them.
void on_exec_report(const fix::msg &msg)
{
for (;;) {
try {
// Handle the execution report however you want.
handle_exec_report(msg);
} catch(const try_again_exception &) {
// Oh, some resource was temporarily unavailable? Try again!
continue;
} catch(const std::exception &) {
// This should never happen, but it did. Call 911.
abort();
}
}
}
Of course, it is possible to make FIX library do a re-transmission request and pass you that message again if exception was thrown. However, it does not make any sense at all because what is the point of asking the sender (over the network, using TCP/IP) to re-send a message that you already have (up your stack :)) and just need to process. Even if it did, what's the guarantee it won't happen again? Re-transmission in this case is not only doesn't sound right logically, the other side (i.e. exchange) may call you up and ask to stop doing this crap because you put too much load on their server with unnecessary re-transmit (because IRL TCP/IP does not lose messages and FIX sequence sync process happens only when connecting, unless of course some non-reliable transport is used, which is theoretically possible but doesn't happen in practice).
When aborting, however, it is FIX library`s responsibility not to increment RX sequence unless it knows for sure that user has processed the message. So that next time application starts, it actually performs synchronization and receives missing messages. If QuickFIX is not doing it, then you need to either fix this, take care of this manually (i.e. go screw with the file where it stores RX/TX sequence numbers), or use some other library that handles this correctly.
This is the wrong thing to do.
A ResendRequest tells the other side that there was some transmission error. In your case, there wasn't, so you shouldn't do that. You're misusing the protocol to cover your app's mistakes. This is wrong. (Also, as Vlad Lazarenko points out in his answer, if they did resend it, what's to say you won't have the error again?)
If an error occurs in your message handler, then that's your problem, and you need to find it and fix it, or alternately you need to catch your own exception and handle it accordingly.
(Based on past questions, I bet you are storing ExecutionReports to a DB store or something, and you want to use Resends to compensate for DB storage exceptions. Bad idea. You need to come up with your own redundancy solution.)

GetQueuedCompletionStatus returns ERROR_NETNAME_DELETED on remote socket closure

I am writing a small server-client-stuff using an I/O-Completion Port.
I get the server and client connected successfully via AcceptEx over my completion port.
After the client has connected the client socket is associated with the completion port and an overlapped call to WSARecv on that socket is invoked.
Everything works just fine, until I close the client test program.
GetQueuedCompletionStatus() returns FALSE and GetLastError returns
ERROR_NETNAME_DELETED
, which makes sense to me (after I read the description on the MSDN).
But my problem is, that I thought the call to GetQueuedCompletionStatus would return me a packet indicating the failure due to closure of the socket, because WSARecv would return the apropriate return value.
Since this is not the case I don´t know which clients´ socket caused the error and cant act the way i need to (freeing structures , cleanup for this particular connection, etc)...
Any suggestion on how to solve this, Or hints?
Thanks:)
EDIT: http://codepad.org/WeYINasO <- the code responsible... the "error" occures at the first functions beginning of the while-loop (the call to GetCompletionStatus() which is only a wrapper for GetQueuedCompletionStatus() working fine in other cases) [Did post it there, because it looks shitty & messy in here]
Here are the scenarios that you need to watch for when calling GetQueuedCompletionStatus:
GetQueuedCompletionStatus returns TRUE: A successful completion packet has been received, all the out parameters have been populated.
GetQueuedCompletionStatus returns FALSE, lpOverlapped == NULL: No packet was dequeued. The other out parameters contain indeterminate values.
GetQueuedCompletionStatus returns FALSE, lpOverlapped != NULL: The function has dequeued a failed completion packet. The error code is available via GetLastError.
To answer your question, when GetQueuedCompletionStatus returns FALSE and lpOverlapped != NULL, there was a failed I/O completion. It's the value of lpOverlapped that you need to be concerned about.
I know this is an old question, but I found this page while fruitlessly googling for details about ERROR_NETNAME_DELETED. It is an error which I get while doing an overlapped Readfile().
After some debugging it turned out that the problem was caused by a program which was writing to a socket but forgetting to call closesocket() before using ExitProcess() (due to garbage collection issues). Calling CloseHandle() did not prevent the error, nor did adding WSACleanup() before ExitProcess(). However, adding a short sleep before the client exited did prevent the error. Maybe avoiding ExitProcess() would have prevented the problem also.
So I suspect your problem is caused by the program exiting without closing down the socket properly.
I don't think this would be an issue on Unix where sockets are just ordinary file descriptors.

MSG_READALL is to recv() as ?? is to send()

From the recv(2) man page:
MSG_WAITALL
This flag requests that the operation block until the full request is satisfied. However, the call may still return less data than requested if a signal is caught, an error or disconnect occurs, or the next data to be received is of a different type than that returned.
It doesn't look like there's an equivalent flag for send(2), which strikes me as strange. Maybe send()s are guaranteed to always accept the whole buffer, but I don't see that written anywhere (and anyway, that seems unlikely to me).
Is there a way to tell send() to wait until it's sent the whole buffer before returning, equivalently to recv()'s MSG_WAITALL?
Edit: I understand that send() just copies data into a buffer in the operating system and that I can't force send() to put data onto the wire. My question is: Can I force send() to block until all the data I gave it has been copied into the OS's buffer?
You can't. send just offloads the buffer to the kernel, then returns.
To quote from the Unix standard:
The send() function shall initiate transmission of a message from the specified socket to its peer (...)
Successful completion of a call to send() does not guarantee delivery of the message.
Note the word "initiate". It doesn't actually send anything, rather tells the OS to send the message when it's ready for it (when its buffers are full or after some time has passed).
send(2) for TCP does not actually "send" anything on the wire, but places your bytes into the socket send buffer. It tells you how many bytes it was able to copy there in the return value.
Make the send buffer bigger (see setsockopt(2) and tcp(7)), pay attention to the syscall return value. In any case, TCP is a stream. You need to manage application-level protocol yourself.