If I broadcast using a UDP socket using sendto() and call recvfrom() immediately. I am receiving the broadcast message that I have sent from the same PC a little while ago? Can this loop back kind of thing be avoided some how?? any explanation why am I experiencing such loop back kind of thing? I mean how long does broadcast message stays in the local n/w??
This is normal behavior. You need to read up on how UDP broadcasting actually works. Your local PC is on the same network that you are broadcasting to, so it receives the broadcast it is sending. You cannot avoid the broadcast being sent back to the local PC, but what you can do is look at the sending IP/Port that recvfrom() reports and ignore the message if it came from the same process that owns the socket that broadcasted it.
Otherwise, switch to Multicasting instead of UDP broadcasting (especially since UDP broacasting is not supported in IPv6 anyway). With Multicasting, receivers have to subscribe to the Multicast in order to receive messages, so simply do not have your sending PC subscribe to its own Multicast.
Related
I'm having issues understanding what socket types are negatively impacted in the event that TCP must try retransmmitting messages.
We have a distributed system that uses a combination of inprocess and TCP connections for internal processes and external devices and applications. My concern is that in the event there is a significant traffic that causes latency and dropped packets, that a TCP retransmit will cause delay in the system.
What I'd like to avoid is an application that has messages compile in a queue waiting to be sent (via a single ZeroMQ TCP socket) because TCP is forcing the socket to repeatedly retransmit messages that never sent an acknowledge.
Is this an issue that can happen using ZeroMQ? Currently I am using PUSH/PULL on a Linux OS.
Or is this not a concern, and if not, why?
It is crucial that messages from the external devices/applications do not feed stale data.
First, the only transport where retransmits are possible is TCP over an actual physical network. And then likely not on a LAN, as it's unlikely that Ethernet packets will go missing on a LAN.
TCP internal to a computer, and especially IPC, INPROC, etc will all have guaranteed delivery of data first time, every time. There is no retransmit mechanism.
If one of the transports being used by socket does experience delays due to transmission errors, that will slow things up. ZMQ cannot consider a message "sent" until it's been propagated via all the transports used by the socket. The external visibility of "sent" is that the outbound message queue has moved away from the high water mark by 1.
It's possible that any one single message will arrive sooner over IPC than TCP, and possible that message 2 will arrive over IPC before message 1 has arrived via TCP. But if you're relying on message timing / relative order, you shouldn't be using ZMQ in the first place; it's Actor model, not CSP.
EDIT For Frank
The difference between Actor and CSP is that the former is asynchronous, the latter is synchronous. Thus for Actor model, the sender has zero information as to when the receiver actually gets a message. For CSP, the sending / receiving is an execution rendevous - the send completes only when the receive is complete.
This can be remarkably useful. If in your system it makes no sense for A to instruct C to do something before (in time, not just in A's code flow) instructing B, then you can do that with CSP (but not Actor model). That's because when A sends to B, B receives the message before A's send completes, freeing A to then send to C.
Unsurprisingly it's real time systems that benefit from CSP.
So consider ZMQ's Actor model with a mix of TCP, IPC and INPROC transports in ZMQ. There's a good chance that messages send via TCP will arrive a good deal later than messages sent through INPROC, even if they were sent first.
In our application we have UDP socket(s) that are used only to send packets (these sockets are never read, and never bound to a port). The sockets are "connected" to the destination address. Are there messages like ICMP etc that could conceivably be directed back at these ports and delivered to the receive buffers of these sockets? If so for what types of messages would this occur?
(these sockets are never read, and never bound to a port).
They are bound to a port when you call connect() if not already bound.
Are there messages like ICMP etc that could conceivably be directed back at these ports
Yes. ICMP UNREACHABLE for a start.
and delivered to the receive buffers of these sockets?
No. ICMP UNREACHABLE will cause an exception next time you use the socket.
If so for what types of messages would this occur?
None. If you are getting data in the socket send buffer, someone is sending you UDP datagrams. In fact the connect target is sending you UDP a datagrams.
So I see that there are methods provided to specify timeout value while sending data over UDP packets (at least saw that in C#). I was wondering what's the use of that as I thought UDP is a connectionless protocol and all the sender socket has to do is just send the packet without worrying about the packet being actually received. It makes sense to have a timeout on receive, but I am not sure why would you need a timeout on send. Can someone please explain?
That brings up another question, is UDP send blocking? (I would say not). If yes, what things does it get "blocked" on?
Thanks in advance.
Both questions have the same answer. send() just copies the datagram to the socket send buffer, which is then sent to the network by the OS in the background. If the buffer is full, it blocks. For UDP the buffer can fill up if you are sending datagrams faster than the OS can write them to the network.
Sockets can be either blocking (default) or non-blocking, whether they are TCP or UDP.
I'm trying to find a way for client to know socket server ip:port, without explicitly defining it. Generally I have a socket server running on portable device that's connect to network over DHCP (via WiFi), and ideally clients should be able to find it automaticaly.
So I guess a question is whether socket server can somehow broadcast it's address over local network? I think UPnP can do this, but I'd rather not get into it.
I'm quite sure that this question was asked on Stack lot's of times, but I could find proper keywords to search for it.
One method of doing this is via UDP broadcast packets. See beej's guide if you're using BSD sockets. And here is Microsoft's version of the same.
Assuming all the clients of the application are on the same side of a router then a broadcast address of 255.255.255.255 (or ff02::1 for IPv6) should be more than adequate.
Multicast is another option, but if this is a LAN-only thing I don't think that's necessary.
Suggestion
Pick a UDP port number (say for the sake of an example we pick 1667). The client should listen to UDP messages on 255.255.255.255:1667 (or whatever the equivalent is. e.g.: IPEndPoint(IPAddress.Any, 1667)). The server should broadcast messages on the same address.
Format Suggestion
UDP Packet: First four bytes as a magic number, next four bytes an IPv4 address (and you might want to add other things like a server name).
The magic number is just in case there is a collision with another application using the same port. Check both the length of the packet and the magic number.
Server would broadcast the packet at something like 30 second time intervals. (Alternatively you could have the server send a response only when a client sends a request via broadcast.)
Some options are:
DNS-SD (which seems to translate to "Apple Bonjour"): it has libraries on macOS, but it needs to install the Bonjour service on Windows. I don't know the Linux situation for this. So, it's multi-platform but you need external libraries.
UDP broadcast or multicast
Some other fancy things like Ethernet broadcast, raw sockets, ...
For your case (clients on a WiFi network), a UDP broadcast packet would suffice, it's multi-platform, and not too difficult to implement from the ground up.
Choosing this option, the two main algorithms are:
The server(s) send an "announce" broadcast packet, with clients listening to the broadcast address. Once clients receive the "announce" packet, they know about the server address. Now they can send UDP packets to the server (which will discover their addresses for sending a reply), or connect using TCP.
The client(s) send a "discover" broadcast packet, with the server(s) listening to the broadcast address. Once the server(s) receive the "discover" packet, it can reply directly to it with an "announce" UDP packet.
One or the other could be better for your application, it depends.
Please consider these arguments:
Servers usually listen to requests and send replies
A server that sends regular "announce" broadcast packets over a WiFi network, for a client that may arrive or not, wastes the network bandwidth, while a client knows exactly when it needs to poll for available servers, and stop once it's done.
As a mix of the two options, a server could send a "gratuitous announce" broadcast packet once it comes up, and then it can listen for "discover" broadcast requests from clients, replying directly to one of them using a regular UDP packet.
From here, the client can proceed as needed: send direct requests with UDP to the server, connect to a TCP address:port provided in the "announce" packet, ...
(this is the scheme I used in an application I am working on)
UDP doesnot sends any ack back, but will it send any response?
I have set up client server UDP program. If I give client to send data to non existent server then will client receive any response?
My assumption is as;
Client -->Broadcast server address (ARP)
Server --> Reply to client with its mac address(ARP)
Client sends data to server (UDP)
In any case Client will only receive ARP response. If server exists or not it will not get any UDP response?
Client is using sendto function to send data. We can get error information after sendto call.
So my question is how this info is available when client doesn't get any response.
Error code can be get from WSAGetLastError.
I tried to send data to non existent host and sendto call succeeded . As per documentation it should fail with return value SOCKET_ERROR.
Any thoughts??
You can never receive an error, or notice for a UDP packet that did not reach destination.
The sendto call didn't fail. The datagram was sent to the destination.
The recipient of the datagram or some router on the way to it might return an error response (host unreachable, port unreachable, TTL exceeded). But the sendto call will be history by the time your system receives it. Some operating systems do provide a way to find out this occurred, often with a getsockopt call. But since you can't rely on getting an error reply anyway since it depends on network conditions you have no control over, it's generally best to ignore it.
Sensible protocols layered on top of UDP use replies. If you don't get a reply, then either the other end didn't get your datagram or the reply didn't make it back to you.
"UDP is a simpler message-based connectionless protocol. In connectionless protocols, there is no effort made to set up a dedicated end-to-end connection. Communication is achieved by transmitting information in one direction, from source to destination without checking to see if the destination is still there, or if it is prepared to receive the information."
The machine to which you're sending packets may reply with an ICMP UDP port unreachable message.
The UDP protocol is implemented on top of IP. You send UDP packets to hosts identified by IP addresses, not MAC addresses.
And as pointed out, UDP itself will not send a reply, you will have to add code to do that yourself. Then you will have to add code to expect the reply, and take the proper action if the response is lost (typically resend on a timer, until you decide the other end is "dead"), and so on.
If you need reliable UDP as in ordering or verification such that TCP/IP will give you take a look at RUDP or Reliable UDP. Sometimes you do need verification but a mixture of UDP and TCP can be held up on the TCP reliability causing a bottleneck.
For most large scale MMO's for isntance UDP and Reliablity UDP are the means of communication and reliability. All RUDP does is add a smaller portion of TCP/IP to validate and order certain messages but not all.
A common game development networking library is Raknet which has this built in.
RUDP
http://www.javvin.com/protocolRUDP.html
An example of RUDP using Raknet and Python
http://pyraknet.slowchop.com/