For my UDP application I want to spread the handling per session (4 tuple) over to different threads. But looks like if there are multiple server sockets (bind calls) for unicast packets kernel will deliver the packet only to one of the threads.
Is this the behavior always or are there any config knobs to enable per session routing of the incoming packets to different sockets ?
Pointer to the relevant functions in the kernel code that delivers the incoming packet to the socket will also be helpful.
Thanks.
But looks like if there are multiple server sockets (bind calls) for unicast packets kernel will deliver the packet only to one of the threads.
This is correct, the packet will only be delivered to one socket.
... enable per session routing of the incoming packets to different sockets ?
There are no sessions in UDP and thus no per sessions routing.
It is possible though to connect UDP sockets, i.e. call both bind and connect. In this case the packet will be delivered to the connected socket. If there are multiple connected sockets for the same source it will again be delivered to only one. And note that a connected socket will only receive packets for this connection, i.e. not from arbitrary sources like an unconnected socket.
Pointer to the relevant functions in the kernel code that delivers the incoming packet to the socket will also be helpful.
This behavior is not specific to Linux.
Related
I am building an application that will deploy in effect on multiple "clients" with a common "server". Clearly I could communicate between each client and the server using a single read-write socket for each client-server link, or a read socket and a write socket per link if I really wanted to.
But what if there are (hopefully good) reasons that the server wants to read from any client, and broadcast back to all? If you have a connectionless protocol like UDP, can the server use only a single read-write socket, or must it use one for reading and one for writing? What about the clients? And does this change if you use a connection-based protocol like TCP?
If you have a connectionless protocol like UDP, can the server use only a single read-write socket, or must it use one for reading and one for writing? What about the clients? And does this change if you use a connection-based protocol like TCP?
A socket as an endpoint which has at least a local address and port in case of UDP and TCP. Only data received for this ip and port are delivered to the socket and all data send from this socket contain the local ip and port as the source. A socket can be connected, in which case also the destination IP and port is known. With TCP a socket needs to be connected, with UDP not.
This means:
You can use the same unconnected UDP socket to send data to multiple peers (destination is an argument for the sendto function). You cannot do this with TCP, i.e. you need a connected socket for each single peer.
You can receive data from multiple peers on an unconnected UDP socket. You cannot do this with TCP.
The special broadcast address can be used with UDP but not with TCP, since with TCP you need to have a connection between only two clients which is not the case with broadcast.
See also a related question with answer for more information: Bidirectional UDP Multicast
But what if there are (hopefully good) reasons that the server wants
to read from any client, and broadcast back to all?
Well, then you'd probably want to use a UDP socket (either instead of, or in addition to, some TCP sockets) :)
If you have a connectionless protocol like UDP, can the server use
only a single read-write socket, or must it use one for reading and
one for writing?
A single UDP socket is sufficient for both reading and writing (although some multithreaded designs might find it easier to use two separate sockets instead; either way will work).
What about the clients?
Clients can also use a single socket for both sending and receive UDP packets, if that's what you're asking.
And does this change if you use a connection-based protocol like TCP?
With TCP sockets you can also use a single socket for both sending and receiving. However you will need one TCP socket for each destination that you want to send or receive to/from. (Contrast this with UDP where a single UDP socket can be used in conjunction with sendto() or recvfrom() to communicate with multiple peers)
As per your requirement, you have two ways :
By using TCP connection only : Server reads message from client and for the broadcasting to all clients,server writes message to all client's TCP sockets(connected to clients) and clients read that message from TCP socket(connected to server).This method requires that client and server knows the IP addresses of each other
By using TCP connection for the client-server direct communication and UDP for broadcasting : In this method,client and server communicates (directly one to one) using TCP connection. For broadcasting the message, server can broadcast the message over the network using UDP socket and clients have UDP broadcast receiver for receiving the broadcast message.
In our application we have UDP socket(s) that are used only to send packets (these sockets are never read, and never bound to a port). The sockets are "connected" to the destination address. Are there messages like ICMP etc that could conceivably be directed back at these ports and delivered to the receive buffers of these sockets? If so for what types of messages would this occur?
(these sockets are never read, and never bound to a port).
They are bound to a port when you call connect() if not already bound.
Are there messages like ICMP etc that could conceivably be directed back at these ports
Yes. ICMP UNREACHABLE for a start.
and delivered to the receive buffers of these sockets?
No. ICMP UNREACHABLE will cause an exception next time you use the socket.
If so for what types of messages would this occur?
None. If you are getting data in the socket send buffer, someone is sending you UDP datagrams. In fact the connect target is sending you UDP a datagrams.
Can one application handle incoming UDP packets coming from two different ip's? If so, can those two connections use the same port number?
Yes, that's the point of a server in general: multiple clients can connect to the server on the given UDP port and they can all broadcast data on the same channel. The server doesn't need to have a separate socket connection for each client, instead it just broadcasts data via its socket connection to the same channel that the clients broadcast.
UDP is a little like sitting in a room where everybody yells at each other, while TCP is like talking on the phone with multiple people at the same time.
I'm trying to find a way for client to know socket server ip:port, without explicitly defining it. Generally I have a socket server running on portable device that's connect to network over DHCP (via WiFi), and ideally clients should be able to find it automaticaly.
So I guess a question is whether socket server can somehow broadcast it's address over local network? I think UPnP can do this, but I'd rather not get into it.
I'm quite sure that this question was asked on Stack lot's of times, but I could find proper keywords to search for it.
One method of doing this is via UDP broadcast packets. See beej's guide if you're using BSD sockets. And here is Microsoft's version of the same.
Assuming all the clients of the application are on the same side of a router then a broadcast address of 255.255.255.255 (or ff02::1 for IPv6) should be more than adequate.
Multicast is another option, but if this is a LAN-only thing I don't think that's necessary.
Suggestion
Pick a UDP port number (say for the sake of an example we pick 1667). The client should listen to UDP messages on 255.255.255.255:1667 (or whatever the equivalent is. e.g.: IPEndPoint(IPAddress.Any, 1667)). The server should broadcast messages on the same address.
Format Suggestion
UDP Packet: First four bytes as a magic number, next four bytes an IPv4 address (and you might want to add other things like a server name).
The magic number is just in case there is a collision with another application using the same port. Check both the length of the packet and the magic number.
Server would broadcast the packet at something like 30 second time intervals. (Alternatively you could have the server send a response only when a client sends a request via broadcast.)
Some options are:
DNS-SD (which seems to translate to "Apple Bonjour"): it has libraries on macOS, but it needs to install the Bonjour service on Windows. I don't know the Linux situation for this. So, it's multi-platform but you need external libraries.
UDP broadcast or multicast
Some other fancy things like Ethernet broadcast, raw sockets, ...
For your case (clients on a WiFi network), a UDP broadcast packet would suffice, it's multi-platform, and not too difficult to implement from the ground up.
Choosing this option, the two main algorithms are:
The server(s) send an "announce" broadcast packet, with clients listening to the broadcast address. Once clients receive the "announce" packet, they know about the server address. Now they can send UDP packets to the server (which will discover their addresses for sending a reply), or connect using TCP.
The client(s) send a "discover" broadcast packet, with the server(s) listening to the broadcast address. Once the server(s) receive the "discover" packet, it can reply directly to it with an "announce" UDP packet.
One or the other could be better for your application, it depends.
Please consider these arguments:
Servers usually listen to requests and send replies
A server that sends regular "announce" broadcast packets over a WiFi network, for a client that may arrive or not, wastes the network bandwidth, while a client knows exactly when it needs to poll for available servers, and stop once it's done.
As a mix of the two options, a server could send a "gratuitous announce" broadcast packet once it comes up, and then it can listen for "discover" broadcast requests from clients, replying directly to one of them using a regular UDP packet.
From here, the client can proceed as needed: send direct requests with UDP to the server, connect to a TCP address:port provided in the "announce" packet, ...
(this is the scheme I used in an application I am working on)
For an UDP server architecture that will have long-lived connections, one architecture is to have one socket that listens to all incoming UDP traffic, and then create separate sockets for each connection using connect() to set the remote address. My question is whether it is possible to do this atomically similar to what accept() does for TCP.
The reason for creating a separate socket and using connect() is that this makes it easy to spread the packet-processing across multiple threads, and also make it easier to have the socket directly associated with the data structures that are needed for processing.
The demultiplexing logic in the networking stack will route the incoming packets to the most specific socket.
Now my question is basically what happens when one wants to emulate accept() for UDP like this:
Use select() with a fd-set that includes the UDP server-socket.
Then read a packet from the UDP server-socket.
Then create a new UDP socket which is then connect()ed to the remote address
I call select() with a fd-set that includes both sockets.
What is returned?
given that a packet arrives to the OS somewhere between 1 and 3.
Will the packet be demultiplexed to the UDP server-socket, or will it be demultiplexed to the more specific socket created in 3. That is, at what point does demultiplexing take place? When the packet arrives, or must it happen "as if" it arrived at point 4?
Follow-up question in case the above does not work: What's the best way to do this?
I see that this discussion is from 2009, but since it keeps popping up when I search, I thought I should share my approach. Both to get some feedback and because I am curios about how the author of the question solved the problem.
The way I chose emulate UDP-accept was a combination of number one and two in nik's answer. I have a root thread which listens on a given socket. I have chosen to use TCP for simplicity, but changing this socket to UDP is not very hard. When a client wants to "connect" to my server using UDP, it first connects to the TCP socket and requests a new connection.
The root thread then proceeds by creating a UDP socket, binds it to a local interface, does connect and sets up data structures. This file descriptor is then passed to the thread that will be responsible for the connection. The IP/port information of the new UDP socket is passed back to the client, which creates a new UDP socket and sends data to the provided IP/port.
This approach works well for my use, but the additional steps for setting up a flow introduces an overhead. In some cases, this overhead might not be acceptable.
I found this question after asking it myself here...
UDP server and connected sockets
Since connect() is available for UDP to specify the peer address, I wonder why accept() wasn't made available to effectively complete the connected UDP session from the server side. It could even move the datagram (and any others from the same client) that triggered the accept() over to the new descriptor.
This would enable better server scalability (see the rationale behind SO_REUSEPORT for more background), as well as reliable DTLS authentication.
This will not work.
You have two simple options.
Create a multi-threaded program that has a 'root' thread listening on the UDP socket and 'dispatching' received packets to the correct thread based on the source. This is because you want to segregate processing by source.
Extend your protocol so the the sources accept an incoming connection on some fixed port and then continue with the protocol communication. In this case you would let the source request on the standard UDP port (of your choice), then your end will respond from a new UDP socket to the sources' UDP port. This way you have initiated a new UDP path from your end backwards to the known UDP port of each source. That way you have different UDP sockets at your end.