My understanding was that UDP doesn't form connections; it just blindly sends packets. However, when I run
nc -u -l 10002
and
nc -u 127.0.0.1 10002
simultaneously (and so can send messages back and forth between terminals), lsof reports two open UDP connections:
nc ... UDP localhost:10002->localhost:35311
nc ... UDP localhost:35311->localhost:10002
If I open a third terminal and do nc -u 127.0.0.1 10002 again, to send another message to the original listener, the listener does not receive (or acknowledge, at least) the message, suggesting it is indeed tied to a specific connection.
If I implement a UDP echo server in Java like this and do sorta the same thing (on 10001), I get
java ... UDP *:10001
nc ... UDP localhost:52295->localhost:10001
aka, Java is just listening on 10001, but nc has formed a connection.
Based on my understanding of UDP, I'd expect both sides to behave like the Java version. What's going on? Can I make the Java version do whatever nc is doing? Is there a benefit to doing so?
I'm on Ubuntu 20.04.3 LTS.
UDP sockets can be connected (after a call to connect) or they can be unconnected. In the first case the socket can only exchange data with the connected peer, while in the second case it can exchange data with arbitrary peers. What you see in lsof is if the socket is connected or not.
My understanding was that UDP doesn't form connections; it just blindly sends packets.
That's a different meaning of the term connection here. TCP has always "real" connections, i.e. an association between two endpoints which has a clear start (SYN based handshake) and end (FIN based teardown). TCP sockets used for data exchange are therefor always connected.
UDP can have associations between two endpoints too, i.e. it can have connected sockets. There is no explicit setup and teardown of such a connection though. And UDP sockets don't need to be connected. From looking at the traffic it can therefore not be determined if connected UDP sockets are in use or unconnected.
Can I make the Java version do whatever nc is doing?
Yes, see What does Java's UDP DatagramSocket.connect() do?
.
Is there a benefit to doing so?
An unconnected UDP socket will receive data from any peer and the application has to check for each received datagram where they came from and if they should be accepted. A connected UDP socket will only receive data from the connected peer, i.e. no checks in the application are needed to check this.
Apart from that it might scale better if different sockets are used for communication with different peers. But if only few packets are exchanged with each peer and/or if one need to communicate with lots of peers at the same time, then using multiple connected sockets instead of a single unconnected one might mean too much overhead.
Below my infrastructure:
[ Packet Sender Machine ] -------> [ S W I T CH ] ----------> [ Client Machine ]
Now let's say I have 10 different applications running independently in my Client Machine and each one of them has joined the same multicast group to receive the packet being sent to the multicast group.
Question: Does the SWITCHsend 10 copies of the packet to the Client Machine? Or does it send only one and the network card in the machine distributes to all 10 applications? Does it matter? Should I code a PROXY on the Client Machine so that only one packet in transmitted to the PROXY and the 10 applications connect locally to the PROXY? Is there a standard network solution/configuration to this problem?
No. A packet sent to a multicast group is like a broadcast, it goes everywhere. The difference is that hosts not subscribed to a multicast group simply ignore the multicast packet.
In fact multicast, by default, is restricted to a single network, so it never goes through a router. Routers route packets between networks, not from a network back to the same network. To route multicast to a different network, you must enable multicast routing, which is very different than unicast routing.
Can it be done in XE7+ to Broadcast the UDP packets to specific machines/IPs?
For example, I have 10 machine in LAN. Server Broadcasted to all 10 machines. Two machines accepted the packets (clients) and Server then tool its details and keep it in DB. Now these machines are down and up again. Server again Broadcasted the UDP packets but this time to only 8 machines/IPs; and two active machines who were already hand-shaked are not sent these packets. In this process we keep on adding the machine/IPs in our DB till the time all the machines/IPs are added. Once all are added then no UDP packets is sent by the Server. It would become responsibility of the machine to send data packets once it is up.
Update:
I need only suggestion and guidance. Rest would be taken care by me like coding, threading etc.
By definition, a UDP broadcast cannot target a specific peer IP, only a network subnet. If you target a specific IP, it is not a broadcast anymore, you are just doing direct peer-to-peer communication normally.
For what you are attempting, you will have to broadcast across the subnet each time so you can discover new machines, and just ignore responses from any machines you already know about.
I am using multicast UDP between hosts that have multiple network interfaces.
I am using boost::asio, and am confused by the 2 operations receivers have to make: bind, then join-group.
Why do you need to specify the local address of an interface, during bind, when you do that with every multicast group that you join?
The sister-question regards the multicast port: Since during sending, you send to a multicast address & port, why, during subscription to a multicast group, you only specify the address, not the port - the port being specified in the confusing call to bind.
Note: the "join-group" is a wrapper over setsockopt(IP_ADD_MEMBERSHIP), which as documented, may be called multiple times on the same socket to subscribe to different groups (over different networks?). It would therefore make perfect sense to ditch the bind call and specify the port every time I subscribe to a group.
From what I see, always binding to "0.0.0.0" and specifying the interface address when joining the group, works very well. Confused.
To bind a UDP socket when receiving multicast means to specify an address and port from which to receive data (NOT a local interface, as is the case for TCP acceptor bind). The address specified in this case has a filtering role, i.e. the socket will only receive datagrams sent to that multicast address & port, no matter what groups are subsequently joined by the socket. This explains why when binding to INADDR_ANY (0.0.0.0) I received datagrams sent to my multicast group, whereas when binding to any of the local interfaces I did not receive anything, even though the datagrams were being sent on the network to which that interface corresponded.
Quoting from UNIX® Network Programming Volume 1, Third Edition: The Sockets Networking API by W.R Stevens.
21.10. Sending and Receiving
[...] We want the receiving socket to bind the multicast group and
port, say 239.255.1.2 port 8888. (Recall that we could just bind the
wildcard IP address and port 8888, but binding the multicast address
prevents the socket from receiving any other datagrams that might
arrive destined for port 8888.) We then want the receiving socket to
join the multicast group. The sending socket will send datagrams to
this same multicast address and port, say 239.255.1.2 port 8888.
The "bind" operation is basically saying, "use this local UDP port for sending and receiving data. In other words, it allocates that UDP port for exclusive use for your application. (Same holds true for TCP sockets).
When you bind to "0.0.0.0" (INADDR_ANY), you are basically telling the TCP/IP layer to use all available adapters for listening and to choose the best adapter for sending. This is standard practice for most socket code. The only time you wouldn't specify 0 for the IP address is when you want to send/receive on a specific network adapter.
Similarly if you specify a port value of 0 during bind, the OS will assign a randomly available port number for that socket. So I would expect for UDP multicast, you bind to INADDR_ANY on a specific port number where multicast traffic is expected to be sent to.
The "join multicast group" operation (IP_ADD_MEMBERSHIP) is needed because it basically tells your network adapter to listen not only for ethernet frames where the destination MAC address is your own, it also tells the ethernet adapter (NIC) to listen for IP multicast traffic as well for the corresponding multicast ethernet address. Each multicast IP maps to a multicast ethernet address. When you use a socket to send to a specific multicast IP, the destination MAC address on the ethernet frame is set to the corresponding multicast MAC address for the multicast IP. When you join a multicast group, you are configuring the NIC to listen for traffic sent to that same MAC address (in addition to its own).
Without the hardware support, multicast wouldn't be any more efficient than plain broadcast IP messages. The join operation also tells your router/gateway to forward multicast traffic from other networks. (Anyone remember MBONE?)
If you join a multicast group, all the multicast traffic for all ports on that IP address will be received by the NIC. Only the traffic destined for your binded listening port will get passed up the TCP/IP stack to your app. In regards to why ports are specified during a multicast subscription - it's because multicast IP is just that - IP only. "ports" are a property of the upper protocols (UDP and TCP).
You can read more about how multicast IP addresses map to multicast ethernet addresses at various sites. The Wikipedia article is about as good as it gets:
The IANA owns the OUI MAC address 01:00:5e, therefore multicast
packets are delivered by using the Ethernet MAC address range
01:00:5e:00:00:00 - 01:00:5e:7f:ff:ff. This is 23 bits of available
address space. The first octet (01) includes the broadcast/multicast
bit. The lower 23 bits of the 28-bit multicast IP address are mapped
into the 23 bits of available Ethernet address space.
Correction for What does it mean to bind a multicast (udp) socket? as long as it partially true at the following quote:
The "bind" operation is basically saying, "use this local UDP port for sending and receiving data. In other words, it allocates that UDP port for exclusive use for your application
There is one exception. Multiple applications can share the same port for listening (usually it has practical value for multicast datagrams), if the SO_REUSEADDR option applied. For example
int sock = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP); // create UDP socket somehow
...
int set_option_on = 1;
// it is important to do "reuse address" before bind, not after
int res = setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, (char*) &set_option_on,
sizeof(set_option_on));
res = bind(sock, src_addr, len);
If several processes did such "reuse binding", then every UDP datagram received on that shared port will be delivered to each of the processes (providing natural joint with multicasts traffic).
Here are further details regarding what happens in a few cases:
attempt of any bind ("exclusive" or "reuse") to free port will be successful
attempt to "exclusive binding" will fail if the port is already "reuse-binded"
attempt to "reuse binding" will fail if some process keeps "exclusive binding"
It is also very important to distinguish a SENDING multicast socket from a RECEIVING multicast socket.
I agree with all the answers above regarding RECEIVING multicast sockets.
The OP noted that binding a RECEIVING socket to an interface did not help.
However, it is necessary to bind a multicast SENDING socket to an interface.
For a SENDING multicast socket on a multi-homed server, it is very important to create a separate socket for each interface you want to send to. A bound SENDING socket should be created for each interface.
// This is a fix for that bug that causes Servers to pop offline/online.
// Servers will intermittently pop offline/online for 10 seconds or so.
// The bug only happens if the machine had a DHCP gateway, and the gateway is no longer accessible.
// After several minutes, the route to the DHCP gateway may timeout, at which
// point the pingponging stops.
// You need 3 machines, Client machine, server A, and server B
// Client has both ethernets connected, and both ethernets receiving CITP pings (machine A pinging to en0, machine B pinging to en1)
// Now turn off the ping from machine B (en1), but leave the network connected.
// You will notice that the machine transmitting on the interface with
// the DHCP gateway will fail sendto() with errno 'No route to host'
if ( theErr == 0 )
{
// inspired by 'ping -b' option in man page:
// -b boundif
// Bind the socket to interface boundif for sending.
struct sockaddr_in bindInterfaceAddr;
bzero(&bindInterfaceAddr, sizeof(bindInterfaceAddr));
bindInterfaceAddr.sin_len = sizeof(bindInterfaceAddr);
bindInterfaceAddr.sin_family = AF_INET;
bindInterfaceAddr.sin_addr.s_addr = htonl(interfaceipaddr);
bindInterfaceAddr.sin_port = 0; // Allow the kernel to choose a random port number by passing in 0 for the port.
theErr = bind(mSendSocketID, (struct sockaddr *)&bindInterfaceAddr, sizeof(bindInterfaceAddr));
struct sockaddr_in serverAddress;
int namelen = sizeof(serverAddress);
if (getsockname(mSendSocketID, (struct sockaddr *)&serverAddress, (socklen_t *)&namelen) < 0) {
DLogErr(#"ERROR Publishing service... getsockname err");
}
else
{
DLog( #"socket %d bind, %# port %d", mSendSocketID, [NSString stringFromIPAddress:htonl(serverAddress.sin_addr.s_addr)], htons(serverAddress.sin_port) );
}
Without this fix, multicast sending will intermittently get sendto() errno 'No route to host'.
If anyone can shed light on why unplugging a DHCP gateway causes Mac OS X multicast SENDING sockets to get confused, I would love to hear it.
I have an FPGA device with which my code needs to talk. The protocol is as follows:
I send a single non-zero byte (UDP) to turn on a feature. The FPGA board then begins spewing data on the port from which I sent.
Do you see my dilemma? I know which port I sent the message to, but I do not know from which port I sent (is this port not typically chosen automatically by the OS?).
My best guess for what I'm supposed to do is create a socket with the destination IP and port number and then reuse the socket for receiving. If I do so, will it already be set up to listen on the port from which I sent the original message?
Also, for your information, variations of this code will be written in Python and C#. I can look up specific API's as both follow the BSD socket model.
This is exactly what connect(2) and getsockname(2) are for. As a bonus for connecting the UDP socket you will not have to specify the destination address/port on each send, you will be able to discover unavailable destination port (the ICMP reply from the target will manifest as error on the next send instead of being dropped), and your OS will not have to implicitly connect and disconnect the UDP socket on each send saving some cycles.
You can bind a socket to a specific port, check man bind
you can bind the socket to get the desired port.
The only problem with doing that is that you won't be able to run more then one instance of your program at a time on a computer.
You're using UDP to send/receive data. Simply create a new UDP socket and bind to your desired interface / port. Then instruct your FPGA program to send UDP packets back to the port you bound to. UDP does not require you to listen/set up connections. (only required with TCP)