A coworker and I are having a disagreement on what constitutes "completion" of a WSASend overlapped IO request. He asserts using fWait as TRUE in the WSAGetOverlappedResult call only waits until the message is queued for sending. He believes waiting for the write/send operation to "complete" only means the message was successfully initiated. In my opinion that is far from a "completed" message to the other side of the socket...that would simple be a beginning of a send and not a completion. If the fWait of TRUE does not block until the bytes have been sent and been ACKed (or error returned), then this is far from synchronous...it would in fact be acting the same as asynchronous IO because it's just fire and forget.
I have been maintaining our company's communication library with my understanding of how to do, and what is, "synchronous" IO for decades so I'll be shocked if I'm indeed wrong in my understanding. But my coworker is a brilliant developer with TONS of TCP/IP experience and is adamant he's right. Says he even posed this question here on stackoverflow and was told he was right. I can't imagine how I could be misunderstanding "completion" of a send to mean anything other than the sending of the bytes requested were indeed sent and ACKed. But I've been wrong before LOL
So...who is right? What EXACTLY does it mean to wait for a WSASend request to be "complete"? Simply waiting until the message is queued for sending in the TCP/IP stack...or waiting for all the packets that constitute the message to be both sent and ACKed??? Or is the truth somewhere in-between?
You are wrong.
A send request is completed when the request is no longer being processed. Once the stack has taken over the operation, the request is no longer being processed and its resources can be used for some other purpose.
If you were right, overlapped I/O would be nearly useless. Who would want to have to wait until the other side ACKed a previous send in its entirety before they could queue any more data to the TCP connection?
How would overlapped I/O be useful if there was no way to determine when the queing process was complete? That's what the application needs to know so it can send another chunk of data.
You would always have dead time as the send queue would always have to get completely empty before more data could be queued on the sending side. Yuck!
And knowing when the ACK is received is useless anyway. It doesn't mean the application on the other end got the data. So it has no meaning at application layer. Knowing that the send has been queued does have meaning at application layer -- it means you can now queue more data to be sent and be guaranteed that all of the data from the previous send will be passed to the receiver's application layer before any data from the next send. That's what the application needs to know.
A synchronous call to WASSend also completes when the send is queued. An asynchronous operation just means you don't have to wait for whatever you'd wait for in a synchronous operation. So that's another reason your understanding would be quite strange.
Lastly, there is no standard or common way to wait for a remote ACK with synchronous operations. So you would have to think that asynchronous operations default to providing a semantic that is not even really available with synchronous ones. That's also pretty strange.
David is correct.
WSASend will complete when that data is handed off to the TCPIP stack (buffered at their layer) to be sent whenever the transport will allow. If it's a blocking call, it will wait until it's ready to pend; just like if it's async, the OVERLAPPED I/O will complete once it pends.
Some may ask why is this even necessary? This is behavior is crucial for keeping as much data in-flight over a TCP connection. In order to keep as much data in-flight, a sender should call WSASend until it pends (recall if it's a synchronous WSASend call then that thread will just block until WSASend can complete; if it's asynchronous, the async completion will occur once that WSASend call can pend).
Why would WSASend ever pend, why not just complete immediately? WSASend will only complete once the transport (in kernel) is ready to buffer more data. So looping until WSASend pends will keep that connection flush with enough data to keep the maximum data in-flight.
Hope this makes sense.
My testing seems to show I am indeed wrong. Synchronous behavior does not exist for sending...just receiving. I see how this is a performance help, but I object to all documentation saying sending fWait of TRUE in WSAGetOverlappedResult will wait until the IO request is "complete". This word implies much more than just it being queued up. Well..I'm flabbergasted that my understanding was off...but TCP handles things well enough that it hasn't caused me issues before.
Thanks for the excellent replies and patience with my frustrated replies. I'm incredibly disappointed at how all the documentation is written using words that would imply my understanding was right...when all along waiting for IO to "end", "complete", "finish" absolutely does NOT wait.
Related
I am using InfiniBand/RDMA for this. My client machine is sending a block to the server (via RDMA Send), and I want the client to get back a special 8 byte acknowledgement. I know the server could send it along separately, but that would duplicate work since my understanding for RDMA Send, the NIC is already sending some form of ACK via the completion queue. However, I was looking at InfiniBand docs, the completion queue doesn't seem to have an immediate along with it, just an id (which is itself less than 8 bytes, so there aren't any funny tricks I could play). I was wondering if I could get suggestions on how to do this?
Thank you!
I'm not sure I fully understand the question, but on the client side, a completion queue entry will be generated when the send work request is fully executed. "Fully executed" means that the RDMA-level ACK has been received from the server in your case. The work request ID in that completion entry is indeed 64 bits, and it is common to use it for a pointer to an auxiliary structure holding whatever info you want for the work request.
If you want the server to actively send back some data that it chooses after it receives the send from the client, then I don't think there's any way around having the server post a work request to do that.
I am trying to write a HTTP API server which does basic CRUD operation on a specific resource. It talks to an external db server to do the operations.
Future support in scala is pretty good, and for all non-blocking computation, future is used. I have used future in many places where we wrap an operation with future and move on, when the value is eventually available and the call back is triggered.
Coming to an HTTP API server's context, it is possible to implement non-blocking asynchronous calls, but when a GET or a POST call still blocks the main thread right?
When a GET request is made, a success 200 means the data is written to the db successfully and not lost. Until the data is written to the server, the thread that was created is still blocking until the final acknowledgement has been received from the database that the insert is successful right?
The main thread(created when http request was received) could delegate and get a Future back, but is it still blocked until the onSuccess is trigged which gets triggered when the value is available, which means the db call was successful.
I am failing to understand how efficiently a HTTP server could be designed to maximize efficiency, what happens when few hundred requests hit a specific endpoint and how it is dealt with. I've been told that slick takes the best approach.
If someone could explain a successful http request lifecycle with future and without future, assuming there are 100 db connection threads.
When a GET request is made, a success 200 means the data is written to
the db successfully and not lost. Until the data is written to the
server, the thread that was created is still blocking until the final
acknowledgement has been received from the database that the insert is
successful right?
The thread that was created for the specific request need not be blocked at all. When you start an HTTP server, you always have the "main" thread ongoing and waiting for requests to come in. Once a request starts, it is usually offloaded to a thread which is taken from the thread pool (or ExecutionContext). The thread serving the request doesn't need to block anything, it only needs to register a callback which says "once this future completes, please complete this request with a success or failure indication". In the meanwhile, the client socket is still pending a response from your server, nothing returns. If, for example, we're on Linux and using epoll, then we pass the kernel a list of file descriptors to monitor for incoming data and wait for that data to become available, in which we will get back a notification for.
We get this for free when running on top of the JVM due to how java.NIO is implemented for Linux.
The main thread (created when http request was received) could delegate
and get a Future back, but is it still blocked until the onSuccess is
trigged which gets triggered when the value is available, which means
the db call was successful.
The main thread usually won't be blocked, as it is whats in charge of accepting new incoming connections. If you think about it logically, if the main thread blocked until your request completed, that means that we could only serve one concurrent request, and who wants a server which can only handle a single request at a time?
In order for it to be able to accept multiple request, it will never handle the processing of the route on the thread in which it accepts the connection, it will always delegate it to a background thread to do that work.
In general, there are many ways of doing efficient IO in both Linux and Windows. The former has epoll while the latter has IO completion ports. For more on how epoll works internally, see https://eklitzke.org/blocking-io-nonblocking-io-and-epoll
First off, there has to be something blocking the final main thread for it to keep running. But it's no different than having a threadpool and joining to it. I'm not exactly sure what you're asking here, since I think we both agree that using threads/concurrency is better than a single threaded operation.
Future is easy and efficient because it abstracts all the thread handling from you. By default, all new futures run in the global implicit ExecutionContext, which is just a default threadpool. Once you kick of a Future request, that thread will spawn and run, and your program execution will continue. There are also convenient constructs to directly manipulate the results of a future. For example, you can map, and flatMap on futures, and once that future(thread) returns, it will run your transformation.
It's not like single threaded languages where a single future will actually block the entire execution if you have a blocking call.
When you're comparing efficiency, what are you comparing it to?
In general "non-blocking" may mean different things in different contexts: non-blocking = asynchronous (your second question) and non-blocking = non-blocking IO (your first question). The second question is a bit simpler (addresses more traditional or well-known aspect let's say), so let's start from it.
The main thread(created when http request was received) could delegate and get a Future back, but is it still blocked until the onSuccess is trigged which gets triggered when the value is available, which means the db call was successful.
It is not blocked, because Future runs on different thread, so your main thread and thread where you execute your db call logic run concurrently (main thread still able to handle other requests while db call code of previous request is executing).
When a GET request is made, a success 200 means the data is written to the db successfully and not lost. Until the data is written to the server, the thread that was created is still blocking until the final acknowledgement has been received from the database that the insert is successful right?
This aspect is about IO. Thread making DB call (Network IO) is not necessary blocked. It is the case for old "thread per request" model, when thread is really blocked and you need create another thread for another DB request. However, nowadays non-blocking IO became popular. You can google for more details about it, but in general it allows you to use one thread for several IO operations.
What is the difference between asynchronous and synchronous HTTP request?
Synchronous:
A synchronous request blocks the client until operation completes. In such case, javascript engine of the browser is blocked.
Asynchronous
An asynchronous request doesn’t block the client i.e. browser is responsive. At that time, user can perform another operations also. In such case, javascript engine of the browser is not blocked.
Check out Determining synchronous vs. asynchronous in web applications for previous discussion. In short:
Asynchronous APIs do not block. Every synchronous call waits and blocks for your results to > come back. This is just a sleeping thread and wasted computation.
Asynchronous APIs do not block. Every synchronous call waits and blocks for your results to come back. This is just a sleeping thread and wasted computation.
If you need something to happen, send an asynchronous request and do further computation when the request returns. This means your thread sits idle and can pick up other work.
Asynchronous requests is the way to scale to thousands of concurrent users.
Sachin Gandhwani's answer is very well explained in simple words. In case you are still not convinced with the difference of asynchronous HTTP request and synchronous HTTP request, you can read this - https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Synchronous_and_Asynchronous_Requests
A synchronous client constructs an HTTP structure, sends a request, and waits for a response. An asynchronous client constructs an HTTP structure, sends a request, and moves on. In this case, the client is notified when the response arrives. The original thread, or another thread, can then process the response. Although asynchronous behavior can result in faster overall execution, synchronous behavior might be preferred in certain cases where more simplified code is necessary.
We are using recv(2) in order to listen to incoming ICMP packets in a network management application in which there is a status poll feature to ping all managed hosts.
In a productive environment, the number of managed hosts can become pretty large (1,000+), and at the moment, when said status poll is performed, we send out all the ping requests sequentially with no delay inbetween.
Obviously, this will lead to many ping replies being sent back almost simultaneously, and it appears that our dispatching routine cannot catch up. This seems to cause that packets are dropped and the ping replies are never actually received by our application. We believe so because many hosts are falsely detected as being "unavailable".
The dispatcher doesn't do anything more than adding incoming replies to an "inbox" of sorts, which is processed later by another thread, ie it does not take much time and can probably not be optimized any further.
Is it possible that the internal packet buffer used by recv (in the kernel? in the network hardware?) fills up and starts dropping packets? If so, is there a good way to determine a reasonable maximum amount of simultaneous pings that can be performed safely (ie by getting that buffer size from the OS)?
Is there any way of checking if data sent using winsock's send() or WSASend() are really delivered to destination?
I'm writing an application talking with third party server, which sometimes goes down after working for some time, and need to be sure if messages sent to that server are delivered or not. The problem is sometimes calling send() finishes without error, even if server is already down, and only next send() finishes with error - so I have no idea if previous message was delivered or not.
I suppose on TCP layer there is information if certain (or all) packets sent were acked or not, but it is not available using socket interface (or I cannot find a way).
Worst of all, I cannot change the code of the server, so I can't get any delivery confirmation messages.
I'm sorry, but given what you're trying to achieve, you should realise that even if the TCP stack COULD give you an indication that a particular set of bytes has been ACK'd by the remote TCP stack it wouldn't actually mean anything different to what you know at the moment.
The problem is that unless you have an application level ACK from the remote application which is only sent once the remote application has actioned the data that you have sent to it then you will never know for sure if the data has been received by the remote application.
'but I can assume its close enough'
is just delusional. You may as well make that assumption if your send completes as it's about as valid.
The issue is that even if the TCP stack could tell you that the remote stack had ACK'd the data (1) that's not the same thing as the remote application receiving the data (2) and that is not the same thing as the remote application actually USING the data (3).
Given that the remote application COULD crash at any point, 1, 2 OR 3 the only worthwhile indication that the data has arrived is one that is sent by the remote application after it has used the data for the intended purpose.
Everything else is just wishful thinking.
Not from the return to send(). All send() says is that the data was pushed into the send buffer. Connected socket streams are not guarenteed to send all the data, just that the data will be in order. So you can't assume that your send() will be done in a single packet or if it will ever occur due to network delay or interruption.
If you need a full acknowledgement, you'll have to look at higher application level acks (server sending back a formatted ack message, not just packet ACKs).