A lot of examples can be found about non blocking TCP sockets, however I find it difficult to find good explanation about how UDP sockets should be handled with poll/select/epoll system calls.
Blocking or non blocking ?
When we have to deal with tcp sockets then it makes sense to set them to non blocking, since it takes only one slow client connection to prevent the server from serving other clients. However, there are no ACK messages in UDP, so my assumption is that writing to UDP should be fast enough for both cases. Does that mean that we can safely use blocking UDP socket with the family of poll system calls if each time we are going to send small amount of data (10Kb for example)? From this discussion I assume that ARP request is the only point that can substantially block the sendto function, however isn't it a one time thing?
Return value of sendto
Let's say the socket is non-blocking. Can there be a scenario that I try to send 1000 bytes of data, and the sendto function sends only some part of it (say 300 bytes)? Does that mean that it has just sent a UDP packet with 300 bytes, and next time I use sendto I have to consider that it will send in a new UDP packet again? Is this situation still possible for blocking sockets?
Return value of recvfrom
The same question applies for recvfrom. Can there be a situation that I will need to call recvfrom more than once to obtain the full UDP packet. Is
that behaviour different for blocking/non-blocking sockets.
Related
I have an application whose main purpose is to transform a RTP stream into an HTTP stream. One thread is receiving RTP packets and write them into a circular buffer and another thread acts as a mini webserver and answers HTTP request by reading from that buffer (only one GET request can happen at a time).
This HTTP thread, once the GET has been received is a simple loop that call send() whenever there is something in the circular buffer. But sometimes, the send() blocks for an insane amount of time (like >1s), creating audio dropout.
To be clear, RTP packets arrive in due real time, no over or underflow here. The HTTP socket is, on purpose, blocking as it is expected that the receiver regulates its flow using TCP when it does not need audio (enough on its own buffers). But the HTTP client is not overwhelmed by audio as the RTP source is, again, just doing realtime.
But obviously, something else happens and I've observed that on Linux, MacOS and Windows (the code works on all these) and on two different network topologies.
I'm wondering if the send() long blocks are not due to something else than the TCP flow control, like something I'm missing with what happens when a thread blocks in a send()
Get a wireshark trace so you can see where the TCP stall is happening. I suspect what is happening is any of the following:
You're actually sending faster than client is consuming. I think you've already ruled that out...
The more likely case is that an IP packet is getting lost and TCP is stuck waiting for the ACK, times out, and then retransmits. Meanwhile your sending thread is trying to stuff more data into the socket and it's getting backed up and eventually blocks.
One simple things you can do is to try increasing the send buffer (SO_SNDBUF) on the socket you send with. This value specifies how many untransmitted bytes that the app can write to the socket before blocking. And if possible, increase the receive buffer (SO_RCVBUF) on the client side. That way, if the network takes a burp for a couple of seconds, your socket will take longer to fill up before blocking.
int size = 512*1024;
setsockopt(sock, SOL_SOCKET, SO_SNDBUF, &size, sizeof(size));
I know, I know. This question has been asked many times before. But I've spent an hour googling now without finding what I am looking for so I will ask it again and mention my context along with what makes the decision hard for me:
I am writing the server for a game where the response time is very important and a packet loss every now and then isn't a problem.
Judging by this and the fact that I as a server mostly have to send the same data to many different clients, the obvious answer would be UDP.
I had already started writing the code when I came across this:
In some applications TCP is faster (better throughput) than UDP.
This is the case when doing lots of small writes relative to the MTU size. For example, I read an experiment in which a stream of 300 byte packets was being sent over Ethernet (1500 byte MTU) and TCP was 50% faster than UDP.
In my case the information units I'm sending are <100 bytes, which means each one fits into a single UDP packet (which is quite pleasant for me because I don't have to deal with the fragmentation) and UDP seems much easier to implement for my purpose because I don't have to deal with a huge amount of single connections, but my top priority is to minimize the time between
client sends something to server
and
client receives response from server
So I am willing to pick TCP if that's the faster way.
Unfortunately I couldn't find more information about the above quoted case, which is why I am asking: Which protocol will be faster in my case?
UDP is still going to be better for your use case.
The main problem with TCP and games is what happens when a packet is dropped. In UDP, that's the end of the story; the packet is dropped and life continues exactly as before with the next packet. With TCP, data transfer across the TCP stream will stop until the dropped packet is successfully retransmitted, which means that not only will the receiver not receive the dropped packet on time, but subsequent packets will be delayed also -- most likely they will all be received in a burst immediately after the resend of the dropped packet is completed.
Another feature of TCP that might work against you is its automatic bandwidth control -- i.e. TCP will interpret dropped packets as an indication of network congestion, and will dial back its transmission rate in response; potentially to the point of dialing it down to near zero, in cases where lots of packets are being lost. That might be useful if the cause really was network congestion, but dropped packets can also happen due to transient network errors (e.g. user pulled out his Ethernet cable for a couple of seconds), and you might not want to handle those problems that way; but with TCP you have no choice.
One downside of UDP is that it often takes special handling to get incoming UDP packets through the user's firewall, as firewalls are often configured to block incoming UDP packets by default. For an action game it's probably worth dealing with that issue, though.
Note that it's not a strict either/or option; you can always write your game to work over both TCP and UDP, and either use them simultaneously, or let the program and/or the user decide which one to use. That way if one method isn't working well, you can simply use the other one, and it only takes twice as much effort to implement. :)
In some applications TCP is faster (better throughput) than UDP. This
is the case when doing lots of small writes relative to the MTU size.
For example, I read an experiment in which a stream of 300 byte
packets was being sent over Ethernet (1500 byte MTU) and TCP was 50%
faster than UDP.
If this turns out to be an issue for you, you can obtain the same efficiency gain in your UDP protocol by placing multiple messages together into a single larger UDP packet. i.e. instead of sending 3 100-byte packets, you'd place those 3 100-byte messages together in 1 300-byte packet. (You'd need to make sure the receiving program is able to correctly intepret this larger packet, of course). That's really all that the TCP layer is doing here, anyway; placing as much data into the outgoing packets as it has available and can fit, before sending them out.
I'm trying to write a light-weight non-blocking performance statistic agent program, unix domain socket with datagram for IPC seems a reasonable choice. I want to know the system behavior when the agent gets unexpectedly slow.
Eventually UDP datagrams will be dropped somewhere or other.
There is a rmem_max for udp server. looks like the same applies to unix domain socket with datagram. The system will drop the requests when buffer gets full.
I have a question about socket programming. When I use socket to send the data, we can use the API such as sendto() to send using TCP or UDP.
For sendto(), we give a array pointer and the byte number we want to send.
In this case, if I gave a large byte number (e.g.: 20000 bytes), based on my understanding, MTU of the network will not be that big so socket actually send mutiple packets instead of one big packet. Since these 20000 bytes are split into several UDP/TCP packets, but will these 20000 bytes be seen as one packet at beginning? Is this process UDP/TCP fragmentation ?
My another question is if I put the data size smaller than MTU into sendto(), then I can gurantee call sendto() once, socket only sends one TCP/UDP packet?
Thanks in advance.
will these 20000 bytes be seen as one packet at beginning? Is this process UDP/TCP fragmentation?
UDP will send it as one datagram if your socket send buffer is large enough to hold it. Otherwise you will get EMSGSIZE. It may subsequently get fragmented at the IP layer, and if a fragment gets lost so does the whole datagram, but if all the fragments arrive the entire datagram will be received intact.
TCP wil send it all, segmenting and fragmenting it however it sees fit. It will all arrive, intact and in order, unless there is a long enough network outage.
My another question is if I put the data size smaller than MTU into sendto(), then I can guarantee call sendto() once, socket only sends one TCP/UDP packet?
UDP: yes.
TCP: no.
Say Server S have a successful TCP connection with Client C.
C is keep sending 256-byte-long packets to S.
Is it possible that one of packets only receive part of it, but the connection does not break (Can continue receive new packets correctly)?
I thought TCP protocol itself will guarantee not lose any bytes while connecting. But seems not?
P.S. I'm using Python's socketserver library.
The TCP protocol does guarantee delivery. Thus (assuming there are no bugs in your code and in the TCP stack), the scenario you describe is impossible.
Do bear in mind that TCP is stream- rather than packet-oriented. This means that you may need to call recv() multiple times to read the entire 256-byte packet.
As #NPE said, TCP is a stream oriented protocol, that means that there is no guarantee on how much data bytes are sent in each TCP packet nor how many bytes are available for reading in the receiving socket. What TCP ensures is that the receiving socket will be provided with the data bytes in the same order that they were sent.
Consider a communication through a TCP connection socket between two hosts A and B.
When the application in A requests to send 256 bytes, for example, the A's TCP stack can send them in one, or several individual packets, or even wait before sending them. So, B may receive one or several packets with all or part of the bytes requested to be sent by A, and so, when the application in B is notified of the availability of received bytes, it's not sure that it could read at once the 256 bytes.
The only guaranteed thing is that the bytes B reads are in the same order that A sent them.