I am receiving some data over socket (with some start and end character). I can use a byte receiving mechanism that should receive one byte at a time, add it to some queue kind of thing and receive next until ending character found. Or i can make a chunk receiver and find an ending character to terminate my message...
My question is, what is cost of increasing / decreasing buffer size?? in my perception, decreasing buffer size should increase memory io but does increasing buffer verify that I'll be increasing IO performance as well?
Never re-size a buffer in a socket application. It might not matter for a socket application where there aren't that many simultaneous operations. But it's a bad habit that's easy to get used to.
Handling a buffer larger than the actual data isn't that hard to work with. Just check all Stream methods. They have a offset and count property which tells where you should start processing and how many bytes you can process. Same thing here.
And to answer your question: The cost is that .NET need to allocate a new memory "slot" and that the memory gets more fragmented for each request.
Simply allocate a 15kb buffer directly when the socket is connected. Create a buffer pool if you can handle multiple (asynchronous) receives per connection.
Related
I want to understand how Indy socket timeouts works, because I want to use them in the following way.
I have an application (TCP server/client) that transfers a file over the Internet. When I start the transfer, I want to be able to stop it fast enough (let's say, 1500 ms) if I decide that. If some socket is reading data, and something happens on the wire that makes it late, I won't be able to stop the transfer, because the socket is hung reading data. So I need to set some short timeouts, that in normal operation will not be triggered. But if something happens and data is running late, the control will be passed to the main proc and I'll be able to check for the abort request.
Now, I don't know what to do next... If a socket read times out, what do that mean? The socket did not receive any data for that period of time... Or, the socket received some data in the buffer but doesn't have time to finish? I have a feeling that those timeouts are the waiting periods for something to happen (start a read or a write operation). But (let's say a read), once started, what happens if the socket receives half of the data (which he was asked to read) and then nothing comes? Will that call block the program execution forever? Because if that happens, then again I will not be able to check for abort request.
Anyway... when the timeout occurs, it will raise an exception? I can catch it and try again, in the same connection, like nothing happened? Will the in/out buffer be modified after a timeout?
I am using this to set the Read and Write timeouts:
Socket.ReadTimeout:= WorkingRTimeOut;
Socket.Binding.SetSockOpt(SOL_SOCKET, SO_SNDTIMEO, WorkingWTimeOut);
Socket timeouts are applied on a per-byte basis.
If you ask a socket to read N number of bytes, it will return as many bytes as it can, up to N bytes max, from the socket's receive buffer. It can (and frequently does) return fewer bytes, requiring another read to receive the remaining bytes. If a timeout error occurs, it means no bytes at all arrived in time for the current read. There is no way to know why, or whether they ever will arrive.
If you ask a socket to send N number of bytes, it will accept as many bytes as it can, up to N bytes max, into the socket's write buffer. It can (and sometimes does) buffer fewer bytes, requiring another send to buffer the remaining bytes. If a timeout occurs, it means the socket's write buffer has filled up, the receiver is not reading fast enough (or at all) to clear space in the sender's write buffer in time.
If you ask Indy to read/send N number of bytes, it may perform multiple socket reads/sends internally, waiting for all of the expected bytes to be received/sent. So it may have read/sent X number of bytes, where X < N, before the timeout occured. Sure, you could try another read/send again, asking for only the remaining bytes you haven't received/sent yet (N - X), but don't ask for the bytes you already received/sent (X). You might receive/send more bytes, or you might get another timeout, there is no way to know until you try. However, depending on context, it may not be easy/possible to know how many bytes were received/sent before the timeout, so you might not know how many remaining bytes to ask for again. In which case, about all you can sensibly do is just close the TCP connection, reconnect, and resume/start over.
As for your ability to abort a connection quickly, you could move your read/send code to a worker thread, and then Disconnect() the socket from your main proc when needed. That will generally abort any blocking read/send in progress.
I am doing an application that will imply reading a lot of data sent to my socket.
The problem I have is whether if should I rely on the socket setReceiveBufferSize, put a big value there to hope that it will gather all the data that I have until I am able to process it, or use a BlockingQueue to put everything there and then process it from another thread that keeps pooling and processing data?
Also is it a bad design if I let the queue with the max number of elements? ( so I'm just telling it, "yeah receive as many element as you'd like"), I'm referring to the memory consumption if I will receive a really big number of elements?
Regards,
Aurelian
A large socket receive buffer is always a good idea, but for TCP windowing/throughput reasons, not because you may be slow reading. You should be aiming to read the input as fast as possible, certainly as fast as it arrives. The proposed BlockingQueue is a complete waste of time and space. If the socket receive buffer fills up, the sender will stall. That's all you need.
suppose I write data really fast [I have all the data in memory] to a blocking socket.
further suppose the other side will read data very slow [like sleep 1 second between each read].
what is the expected behavior on the writing side in this case?
would the write operation block until the other side reads enough data, or will the write return an error like connection reset?
For a blocking socket, the send() call will block until all the data has been copied into the networking stack's buffer for that connection. It does not have to be received by the other side. The size of this buffer is implementation dependent.
Data is cleared from the buffer when the remote side acknowledges it. This is an OS thing and is not dependent upon the remote application actually reading the data. The size of this buffer is also implementation dependent.
When the remote buffer is full, it tells your local stack to stop sending. When data is cleared from the remote buffer (by being read by the remote application) then the remote system will inform the local system to send more data.
In both cases, small systems (like embedded systems) may have buffers of a few KB or smaller and modern servers may have buffers of a few MB or larger.
Once space is available in the local buffer, more data from your send() call will be copied. Once all of that data has been copied, your call will return.
You won't get a "connection reset" error (from the OS -- libraries may do anything) unless the connection actually does get reset.
So... It really doesn't matter how quickly the remote application is reading data until you've sent as much data as both local & remote buffer sizes combined. After that, you'll only be able to send() as quickly as the remote side will recv().
Output (send) buffer gets filled until it gets full and send() block until the buffer get freed enough to enqueue the packet.
As send manual page says:
When the message does not fit into the send buffer of the socket,
send() normally blocks, unless the socket has been placed in non-
blocking I/O mode.
Look at this: http://manpages.ubuntu.com/manpages/lucid/man2/send.2.html
I am currently writing code to transfer data to a remote vendor. The transfer will take place over a TCP socket. The problem I have is the data is variable length and there are no framing or size markers. Sending the data is no problem, but I am unsure of the best way to handle the returned data.
The data is comprised of distinct "messages" but they do not have a fixed size. Each message has an 8 or 16 byte bitmap that indicates what components are included in this message. Some components are fixed length and some are variable. Each variable length component has a size prefix for that portion of the overall message.
When I first open the socket I will send over messages and each one should receive a response. When I begin reading data I should be at the start of a message. I will need to interpret the bitmap to know what message fields are included. As the data arrives I will have to validate that each field indicated by the bitmap is present and of the correct size.
Once I have read all of the first message, the next one starts. My concern is if the transmission gets cut partway through a message, how can I recover and correctly find the next message start?
I will have to simulate a connection failure and my code needs to automatically retry a set number of times before canceling that message.
I have no control over the code on the remote end and cannot get framing bytes or size prefixes added to the messages.
Best practices, design patterns, or ideas on the best way to handle this are all welcomed.
From a user's point of view, TCP is a stream of data, just like you might receive over a serial port. There are no packets and no markers.
A non-blocking read/recv call will return you what has currently arrived at which point you can parse that. If, while parsing, you run out of data before reaching the end of the message, read/recv more data and continue parsing. Rinse. Repeat. Note that you could get more bytes than needed for a specific message if another has followed on its heels.
A TCP stream will not lose or re-order bytes. A message will not get truncated unless the connection gets broken or the sender has a bug (e.g. was only able to write/send part and then never tried to write/send the rest). You cannot continue a TCP stream that is broken. You can only open a new one and start fresh.
A TCP stream cannot be "cut" mid-message and then resumed.
If there is a short enough break in transmission then the O/S at each end will cope, and packets retransmitted as necessary, but that is invisible to the end user application - as far as it's concerned the stream is contiguous.
If the TCP connection does drop completely, both ends will have to re-open the connection. At that point, the transmitting system ought to start over at a new message boundary.
For something like this you would probably have a lot easier of a time using a networking framework (like netty), or a different IO mechansim entirely, like Iteratee IO with Play 2.0.
In order not to flood the remote endpoint my server app will have to implement a "to-send" queue of packets I wish to send.
I use Windows Winsock, I/O Completion Ports.
So, I know that when my code calls "socket->send(.....)" my custom "send()" function will check to see if a data is already "on the wire" (towards that socket).
If a data is indeed on the wire it will simply queue the data to be sent later.
If no data is on the wire it will call WSASend() to really send the data.
So far everything is nice.
Now, the size of the data I'm going to send is unpredictable, so I break it into smaller chunks (say 64 bytes) in order not to waste memory for small packets, and queue/send these small chunks.
When a "write-done" completion status is given by IOCP regarding the packet I've sent, I send the next packet in the queue.
That's the problem; The speed is awfully low.
I'm actually getting, and it's on a local connection (127.0.0.1) speeds like 200kb/s.
So, I know I'll have to call WSASend() with seveal chunks (array of WSABUF objects), and that will give much better performance, but, how much will I send at once?
Is there a recommended size of bytes? I'm sure the answer is specific to my needs, yet I'm also sure there is some "general" point to start with.
Is there any other, better, way to do this?
Of course you only need to resort to providing your own queue if you are trying to send data faster than the peer can process it (either due to link speed or the speed that the peer can read and process the data). Then you only need to resort to your own data queue if you want to control the amount of system resources being used. If you only have a few connections then it is likely that this is all unnecessary, if you have 1000s then it's something that you need to be concerned about. The main thing to realise here is that if you use ANY of the asynchronous network send APIs on Windows, managed or unmanaged, then you are handing control over the lifetime of your send buffers to the receiving application and the network. See here for more details.
And once you have decided that you DO need to bother with this you then don't always need to bother, if the peer can process the data faster than you can produce it then there's no need to slow things down by queuing on the sender. You'll see that you need to queue data because your write completions will begin to take longer as the overlapped writes that you issue cannot complete due to the TCP stack being unable to send any more data due to flow control issues (see http://www.tcpipguide.com/free/t_TCPWindowSizeAdjustmentandFlowControl.htm). At this point you are potentially using an unconstrained amount of limited system resources (both non-paged pool memory and the number of memory pages that can be locked are limited and (as far as I know) both are used by pending socket writes)...
Anyway, enough of that... I assume you already have achieved good throughput before you added your send queue? To achieve maximum performance you probably need to set the TCP window size to something larger than the default (see http://msdn.microsoft.com/en-us/library/ms819736.aspx) and post multiple overlapped writes on the connection.
Assuming you already HAVE good throughput then you need to allow a number of pending overlapped writes before you start queuing, this maximises the amount of data that is ready to be sent. Once you have your magic number of pending writes outstanding you can start to queue the data and then send it based on subsequent completions. Of course, as soon as you have ANY data queued all further data must be queued. Make the number configurable and profile to see what works best as a trade off between speed and resources used (i.e. number of concurrent connections that you can maintain).
I tend to queue the whole data buffer that is due to be sent as a single entry in a queue of data buffers, since you're using IOCP it's likely that these data buffers are already reference counted to make it easy to release then when the completions occur and not before and so the queuing process is made simpler as you simply hold a reference to the send buffer whilst the data is in the queue and release it once you've issued a send.
Personally I wouldn't optimise by using scatter/gather writes with multiple WSABUFs until you have the base working and you know that doing so actually improves performance, I doubt that it will if you have enough data already pending; but as always, measure and you will know.
64 bytes is too small.
You may have already seen this but I wrote about the subject here: http://www.lenholgate.com/blog/2008/03/bug-in-timer-queue-code.html though it's possibly too vague for you.