Suppose I have a UDP socket and send a msg to the server. Now how to figure out if the msg has been received by the server?
Coded acknowledgment from server is a trivial option, is there any other way to find out? I guess no..
You can never be sure that a message arrives, even with TCP not. That is a general limitation of communication networks (also applies to snail mail, for example). You can start with the Bizantine Generals Problem if you want to know more.
The thing you can do, is to increase the likelihood of detecting a message loss. Usually that is done be sending an acknowledgement to the sender. But that may get lost too, so for 100% reliability, you would need to send an acknowledgement for the acknowledgement. And then an acknowledgement for the acknowledgement's acknowledgement. And so on.
My advice: Use TCP, if reliability is your main concern. It has been around for some time, and probably won't have some of the flaws a custom solution would have. If you don't need the reliability of TCP, but need low latencies or something else UDP is good at, use UDP. In that case better make sure that it is not a problem if some packets get lost.
Related
I'm writing a game server for a turn-based game. One criteria is that the game needs to be as fair for all players as possible.
So far it works like this:
Each client has a TCP connection. (If relevant, the connection is opened via WebSockets)
While running, continually check for incoming socket messages via epoll.
Iterate through clients with sockets ready to read:
Read all messages from the client.
Update the internal game state for each message.
Queue outgoing messages to affected clients.
At the end of each "window" (turn):
Iterate through clients and write all queued outgoing messages to their sockets
My concern for fairness raises the following questions:
Does it matter in which order I send messages to the clients?
Calling write() on all the sockets takes only a fraction of a second for my program, but somewhere in the underlying OS or networking would it make a difference if I sorted the client list?
Perhaps I should be sending to the highest-latency clients first?
Does it matter how I write the outgoing messages to the sockets?
Currently I'm writing them as one large chunk. The size can exceed a single packet.
Would it be faster for the client to begin its processing if I sent messages in smaller chunks than 1 packet?
Would it be better to write 1 packet worth to each client at a time, and iterate over the clients multiple times?
Are there any linux/networking configurations that would bear impact here?
Thanks in advance for your feedback and tips.
Does it matter in which order I send messages to the clients?
Yes, by fractions of milliseconds. If the network interface is available for sending the OS will immediately start sending. Why would it wait?
Perhaps I should be sending to the highest-latency clients first?
I think you should be sending in random order. Shuffle the list prior to sending. This makes it fair. I think your question is valid and this should be addressed.
Currently I'm writing them as one large chunk. [...]
First, realize that TCP is stream-based and that there are no packets/messages at the protocol level. On a physical level data is indeed packetized.
It is not necessary to manually split off packets because clients will read data as it arrives anyway. If a client issues a read, that read will complete immediately once the first packet has arrived. There is no artificial waiting in the OS.
Are there any linux/networking configurations that would bear impact here?
I don't know. Be sure to disable nagling.
IS there a way I can in user-space get notification about a packet being dropped at Layer-2 in 802.11.
According to my understanding what happens is, when a packet is sent out on the medium, there are Layer-2 ACKs which are received if it is delivered correctly (if not,it does the retransmission and ultimately drops the packet if not delivered after several retries..)
I want to be able to access this notification (in user-space)and change the behavior of packet transmission.
I want to be able to send the packet to another host from the FIB rather than dropping the packet.
I have read about libpcap libraries and netfilter hooks which allows me to capture packet and inject them back on the networking stack..
But I'm not able to find hooks (if any, for the wireless stack) to help me capture the packet notification in Layer-2.
Please correct me if I'm not understanding something correctly. Also, any heads-up or links to read would be great.
No, you cannot, at least not using the standardised sockets interfaces. 802.11 is a link layer, and the sockets API is strictly link-layer agnostic: unless it's going to work on all link layers, it's not in sockets. There are good reasons for that: the kind of cross-layer interaction that you envision has been tried many times, and it's always turned out more trouble than it's worth.
You didn't give us any details about the application — but the best solution is most probably to change your application-layer protocol to send explicit acknowledgments, and send your data over the fallback route when you fail to receive an ACK.
I have several apps which communicate through serial comm (RS-232 and RS-422), and i would like them to communicate through TCP or UDP without changing them. Another point is that some of the apps must run on linux.
I would like to know if there are exsiting tools for that purpose..
Thanks a lot!
Tal
If all you do with your serial port is read and write bytes, and if precise timing is not a concern, then you may be able to replace your serial port object with a TCP socket and send the exact same data over the socket as you would have sent over the port. The biggest complications are that the timing on a TCP socket is much looser than on a serial port, and TCP sockets' mechanisms for sending "out-of-band" data are different from those of a serial port.
I am unaware of any standards for sending serial data via UDP. Conceptually, it would seem like a useful thing to have since there are many serial-port protocols-based in which it would be more useful to drop data that can't be delivered within a certain time frame than to deliver it late. For example, if the intended recipient of serial-port data is an embedded controller that will sometimes so busy that it drops some incoming data, but which will respond within a few milliseconds to everything it does receive, a one-second hiccup on a TCP connection [not unusual] may cause software which expects to be talking directly to the controller to retransmit a command a dozen times. Even if the device would be capable of detecting and rejecting retransmissions it receives, it would be better for the earlier transmission requests to be abandoned than to have them be delivered late. Note that to be useful, a UDP-based scheme would have to include enough wrapper logic to guarantee that packets would never be delivered out of order; once the data for a packet has been sent to the serial port, any UDP packets which are received later despite having been sent earlier must be discarded; the recipient should probably include logic so that if many out-of-sequence packets are received, it will wait a little while after receiving any packet whose sequence number does not immediately follow the last one, to see if missing packets show up before it commits itself to discarding them.
There is no standard tool to do that. I am thinking to develop one. UDP is ideal in this case since it is 100% guaranteed that there is no out of order packet delivery on a short LAN as is in your case.
In several projects I have used free tool Hercules (https://www.hw-group.com/software/hercules-setup-utility) for protoyping and testing phases. No advertising intended, just a recommendation.
I have an application that communicates in real time with other clients over LAN. The application requires packets to be in order and all to arrive. It also requires as fast transfer as possible and I seem to have some problems with TCP in this matter.
So I was thinking about this, as a non-experienced network programmer, what if I first send a UDP protocoled message and then the same data with TCP. If the UDP-message arrive I will have it as fast as possible if not I still have the TCP message that will make sure I'll atleast get the packet. Obviously I'll make sure that I don't read the same data twice by giving each message an ID or similar.
Is this any good approach? I was thinking that maybe sending the tcpmessage simultaneously will just slow the udp message down so It wont make a difference anyways.
No, this is not a good approach.
You are doubling your network bandwidth and significantly increasing the complexity of your networking code for very little gain.
TCP and UDP have very different characeristics. If you care about data arriving in a timely manner, where if data is late it is no use, then TCP is not useful and as such you should use and only use UDP. If you do not care about data arriving in a timely manner, then UDP is not useful, as it is not reliable.
UDP has very specific use cases. i.e. say an online game which sends a players co-ordinates. You state the order and acknowledgment is needed, therefore TCP seems like the most sensible approach.
Although just to put a twist in the mix, TCP can sometimes surprise you and be better performance wise under specific circumstances.
TCP will try and buffer the data before it sends it across the network (more efficient use of bandwidth). UDP on the other hand puts a packet across the network immediately.
Now imagine writing lots of small packets across a network, UDP may cause congestion whereas TCP is better controlled.
No it is not a good approach at all. You will have now twice the data being sent.
For real time communication UDP is the best approach. You should design your reciever algorithm to manage out of data arrival and sort it and also non arrival of some data.
Also the kind of data being sent can be a deciding factor. If its transactions of a financial kind, udp is not a good idea. But then you should be on a different network.
If it is video data, real time is very important, losses can be tolerated.
So see if you can use the properties of your data to manage udp connection well.
While sending packets across a network, how can one determine where one packet ends and where another starts?
Is sending/receiving acknowledgment one of the ways of doing so?
TCP is a stream-based protocol. That is, it provides a stream vs. packet or message-based interface to the application. If using TCP, an application must implement its own method of determining packets or messages. For example, (a) all message are a fixed size, or (b) each message is prefixed with its subsequent size, or (c) there is a special "end-of-record" sequence in the data stream to indicate a message boundary. Search google for lots of information on how one can implement message boundaries in TCP.
I assume here that you mean application-level 'packets'.
If you use UDP, you don't need to since it's a message protocol. TCP is a byte streaming protocol, so it cannot send packets, just bytes. If you need to send anything more complex than a byte-stream across TCP, you have to add another protocol on top - HTTP is one such protocol. Text is fairly easy since lines have terminating characters, usually CR/LF/CRLF. Sending non-text messages will require a different protocol.
One approach that is often used with TCP is to connect, stream a protocol-unit, disconnect. This works OK, but slowly because of the huge latency of continually opening and closing TCP connections. HTTP usually works like this in order to serve up web pages to large numbers of users who, if left permanently connected while they viewed pages, would needlessly use up all the server sockets.
Waiting for an application-level ACK from the peer is sometimes necessary if it absolutely essential that peer receipt is known before the next message is sent, but again, this is slow because of the connection latency. TCP was not designed with this approach in mind.
If the commonly available IP protocols cannot directly provide what you need, you will have to resort to implementing your own.
What sort of 'packet' are you sending?
Rgds,
Martin
With TCP sockets, you just see the datastream where you can receive and send bytes. You have no way of knowing where a packet ends and another begins.
This is a feature (and a problem) of TCP. Most people just read data into a buffer until a linefeed (\n) is seen. Then process the data and wait for the next line. If transferring chunks of binary data, one can first inform the receiver of how many bytes of data are coming.
If packet boundaries are important, you could use UDP but then the packet order might change or some packets might be lost on the way without you knowing.
The newer SCTP protocol behaves much like TCP (lost packets are resend, packet ordering is retained) but with SCTP sockets you can send packets so that receiver gets exactly the same packet.