I'm working on simple traffic tunneling solution (Linux).
Client side creates tun interface, routes all traffic on it, packages all arrived packets and sends to the server side via udp or tcp connection.
Server side expected to work like NAT. Change source ip address, source port (for tcp/udp) put packet on external network interface via sock_raw, listen for response via sock_raw, keep map of original-source-port <-> replaced-source-port and send responses back to the client.
The question is: how should I choose replaced-source-port ? OS chooses them from ephemeral ports. I can't choose it by myself, it would cause conflicts. OS kernel chooses port after I send packet via sock_raw and I have no chance to build original-source-port <-> replaced-source-port map. Even if I choose port by myself – OS kernel will reply with tcp rst to all incoming tcp packets with dst port not associated with particular app.
P.S. I'm not sure on the overall solution for tunneling too. Your suggestions would be highly appreciated.
I have a TCP server which runs in localhost (127.0.0.1), I am trying to connect to the server by injecting SYN packets to the loopback interface, but the server doesn't answer them. These packets have the source IP of the Ethernet interface of my internet adapter (and not localhost IP).
I watch the SYN packet that goes to my loopback server in Wireshark, but the server does not answer it with a SYN/ACK. I think it is because the IP source is not 127.0.0.1, which for example is 192.168.1.24.
If I go to the browser and I connect to my localhost server it works fine, but the source IP that I am using is 127.0.0.1 and the destination IP is 127.0.0.1 too; the only difference between the packets is the source IP.
I want to establish a TCP connection with my loopback server (localhost) by using different IP source addresses than 127.0.0.1. Is that possible?
For example, a Loopback TCP SYN packet which comes from 192.168.1.24 to 127.0.0.1 should be answered by the loopbackserver?
Thanks and regards!
You can send packets to localhost via Npcap Loopback Adapter and get response from the counterpart (e.g. a process on the same machine). An example is Nmap, Nmap uses Npcap Loopback Adapter to scan the ports of localhost. The command is: nmap -v -A 127.0.0.1. Nmap is open-sourced here, so you can see its code about the implementation. If you think Nmap is too complicated, you can see the source code of Nping here, a ping tool shipped by Nmap. Nping also uses Npcap Loopback Adapter when pinging localhost, which works differently with the original ping shipped by Windows.
Using IP of one of local adapters or using 127.0.0.1 should be the same. You can run Nmap to test it. Whatever, using 127.0.0.1 is the best and recommended by Npcap when talking to localhost.
So I think the issue still relates to your own implementation.
Does the server bind() using INADDR_LOOPBACK? If so, you could try changing it to INADDR_ANY to see if that helps. See also man 7 ip.
(These links are obviously Linux-specific; if your platform is something else, then refer to the documentation applicable to your system. For example, if you're on Windows, then maybe refer to https://msdn.microsoft.com/en-us/library/windows/desktop/ms737550(v=vs.85).aspx.)
I solved the problem, thank you very much for your answers.
The problem was a bit stupid, I was trying to establish a TCP connection with the loopback server (localhost) with IP source addresses that were not in the range of the loopback, loopback gateway: 127.0.0.1, loopback netmask: 255.255.0.0; It cant accept packets from IP source addresses that are not in the range of 127.0.X.X ; if I do NAT and I translate the packet from for example 192.168.1.154 to 127.0.1.154 the packet is received by the server and I can establish the server connection, I do not know how I did not realize it before.
Thank you for the time, regards!.
I think too that maybe it is better to bind the server to other virtual network adapter and not to the loopback, I am studing this: https://github.com/Microsoft/Windows-driver-samples/tree/master/network/ndis/netvmini/6x
It would be fine to create a miniport driver and bind the server there, we would have the advantage of having our own gateway and netmask and the layer would be ethernet and not BSD loopback. Your opinions will be interesting for me.
For a client socket, I can use bind() to bind it to a specific source Ip address to select a specific interface. Or I can use connect() directly then it will pick the source ip based on routing table.
I wonder can I bind a client socket to an ip not belongs to any interfaces ? E.g.: I have two interfaces:
eth0 : ip0
eth1 : ip1
(1) If I bind the client socket to ip2. Is this feasible ?
(2) If (1) is feasible, assuming client socket sent packets thru eth0. Then I configure the iptables in this client host, to forward all incoming packets to ip0 (eth0). In this case, if there are packets sent back from server side with destination ip address is ip2 (assuming this packet will reach my client host). Will my client socket receive the packet ?
Thanks in advance.
I don't really understand your question, but here goes:
For client sockets, you typically want the the OS and its routing table to pick the best interface for you using any available port. In which case, you bind to INADDR_ANY (0) and port 0. Or don't explicitly call bind at at all. Just call connect() and it will do the right thing.
If you need the client connection to occur through a specific interface, then bind the socket to a specific IP address. And then the OS will attempt to use that interface for the subsequent connect call and all traffic after that.
Attempting to bind the socket to an IP that doesn't belong to a local interface is surely going to result in an error.
Not sure what you mean about the iptables stuff. Sounds dicey.
Please have a look at:
https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html#ipfreebind
MAN:
https://man7.org/linux/man-pages/man7/ip.7.html
IP_FREEBIND (since Linux 2.4)
If enabled, this boolean option allows binding to an IP
address that is nonlocal or does not (yet) exist. This
permits listening on a socket, without requiring the
underlying network interface or the specified dynamic IP
address to be up at the time that the application is
trying to bind to it. This option is the per-socket
equivalent of the ip_nonlocal_bind /proc interface
described below.
This might be a very basic question but it confuses me.
Can two different connected sockets share a port? I'm writing an application server that should be able to handle more than 100k concurrent connections, and we know that the number of ports available on a system is around 60k (16bit). A connected socket is assigned to a new (dedicated) port, so it means that the number of concurrent connections is limited by the number of ports, unless multiple sockets can share the same port. So the question.
TCP / HTTP Listening On Ports: How Can Many Users Share the Same Port
So, what happens when a server listen for incoming connections on a TCP port? For example, let's say you have a web-server on port 80. Let's assume that your computer has the public IP address of 24.14.181.229 and the person that tries to connect to you has IP address 10.1.2.3. This person can connect to you by opening a TCP socket to 24.14.181.229:80. Simple enough.
Intuitively (and wrongly), most people assume that it looks something like this:
Local Computer | Remote Computer
--------------------------------
<local_ip>:80 | <foreign_ip>:80
^^ not actually what happens, but this is the conceptual model a lot of people have in mind.
This is intuitive, because from the standpoint of the client, he has an IP address, and connects to a server at IP:PORT. Since the client connects to port 80, then his port must be 80 too? This is a sensible thing to think, but actually not what happens. If that were to be correct, we could only serve one user per foreign IP address. Once a remote computer connects, then he would hog the port 80 to port 80 connection, and no one else could connect.
Three things must be understood:
1.) On a server, a process is listening on a port. Once it gets a connection, it hands it off to another thread. The communication never hogs the listening port.
2.) Connections are uniquely identified by the OS by the following 5-tuple: (local-IP, local-port, remote-IP, remote-port, protocol). If any element in the tuple is different, then this is a completely independent connection.
3.) When a client connects to a server, it picks a random, unused high-order source port. This way, a single client can have up to ~64k connections to the server for the same destination port.
So, this is really what gets created when a client connects to a server:
Local Computer | Remote Computer | Role
-----------------------------------------------------------
0.0.0.0:80 | <none> | LISTENING
127.0.0.1:80 | 10.1.2.3:<random_port> | ESTABLISHED
Looking at What Actually Happens
First, let's use netstat to see what is happening on this computer. We will use port 500 instead of 80 (because a whole bunch of stuff is happening on port 80 as it is a common port, but functionally it does not make a difference).
netstat -atnp | grep -i ":500 "
As expected, the output is blank. Now let's start a web server:
sudo python3 -m http.server 500
Now, here is the output of running netstat again:
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:500 0.0.0.0:* LISTEN -
So now there is one process that is actively listening (State: LISTEN) on port 500. The local address is 0.0.0.0, which is code for "listening for all ip addresses". An easy mistake to make is to only listen on port 127.0.0.1, which will only accept connections from the current computer. So this is not a connection, this just means that a process requested to bind() to port IP, and that process is responsible for handling all connections to that port. This hints to the limitation that there can only be one process per computer listening on a port (there are ways to get around that using multiplexing, but this is a much more complicated topic). If a web-server is listening on port 80, it cannot share that port with other web-servers.
So now, let's connect a user to our machine:
quicknet -m tcp -t localhost:500 -p Test payload.
This is a simple script (https://github.com/grokit/quickweb) that opens a TCP socket, sends the payload ("Test payload." in this case), waits a few seconds and disconnects. Doing netstat again while this is happening displays the following:
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:500 0.0.0.0:* LISTEN -
tcp 0 0 192.168.1.10:500 192.168.1.13:54240 ESTABLISHED -
If you connect with another client and do netstat again, you will see the following:
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:500 0.0.0.0:* LISTEN -
tcp 0 0 192.168.1.10:500 192.168.1.13:26813 ESTABLISHED -
... that is, the client used another random port for the connection. So there is never confusion between the IP addresses.
A server socket listens on a single port. All established client connections on that server are associated with that same listening port on the server side of the connection. An established connection is uniquely identified by the combination of client-side and server-side IP/Port pairs. Multiple connections on the same server can share the same server-side IP/Port pair as long as they are associated with different client-side IP/Port pairs, and the server would be able to handle as many clients as available system resources allow it to.
On the client-side, it is common practice for new outbound connections to use a random client-side port, in which case it is possible to run out of available ports if you make a lot of connections in a short amount of time.
A connected socket is assigned to a new (dedicated) port
That's a common intuition, but it's incorrect. A connected socket is not assigned to a new/dedicated port. The only actual constraint that the TCP stack must satisfy is that the tuple of (local_address, local_port, remote_address, remote_port) must be unique for each socket connection. Thus the server can have many TCP sockets using the same local port, as long as each of the sockets on the port is connected to a different remote location.
See the "Socket Pair" paragraph in the book "UNIX Network Programming: The sockets networking API" by
W. Richard Stevens, Bill Fenner, Andrew M. Rudoff at: http://books.google.com/books?id=ptSC4LpwGA0C&lpg=PA52&dq=socket%20pair%20tuple&pg=PA52#v=onepage&q=socket%20pair%20tuple&f=false
Theoretically, yes. Practice, not. Most kernels (incl. linux) doesn't allow you a second bind() to an already allocated port. It weren't a really big patch to make this allowed.
Conceptionally, we should differentiate between socket and port. Sockets are bidirectional communication endpoints, i.e. "things" where we can send and receive bytes. It is a conceptional thing, there is no such field in a packet header named "socket".
Port is an identifier which is capable to identify a socket. In case of the TCP, a port is a 16 bit integer, but there are other protocols as well (for example, on unix sockets, a "port" is essentially a string).
The main problem is the following: if an incoming packet arrives, the kernel can identify its socket by its destination port number. It is a most common way, but it is not the only possibility:
Sockets can be identified by the destination IP of the incoming packets. This is the case, for example, if we have a server using two IPs simultanously. Then we can run, for example, different webservers on the same ports, but on the different IPs.
Sockets can be identified by their source port and ip as well. This is the case in many load balancing configurations.
Because you are working on an application server, it will be able to do that.
I guess none of the answers tells every detail of the process, so here it goes:
Consider an HTTP server:
It asks the OS to bind the port 80 to one or many IP addresses (if you choose 127.0.0.1, only local connections are accepted. You can choose 0.0.0.0 to bind to all IP addresses (localhost, local network, wide area network, both IP versions)).
When a client connects to that port, it WILL lock it up for a while (that's why the socket has a backlog: it queues a number of connection attempts, because they ARE NOT instantaneous).
The OS then chooses a random port and transfer that connection to that port (think of it as a temporary port that will handle all the traffic from now on).
The port 80 is then released for the next connection (first, it will accept the first one in the backlog).
When client or server disconnects, the random port is held open for a while (CLOSE_WAIT in the remote side, TIME_WAIT in the local side). That allows flushing some lost packets along the path. The default time for that state is 2 * MSL seconds (and it WILL consume memory while is waiting).
After that waiting, that random port is free again to receive other connections.
So, TCP cannot even share a port amongst two IP's!
No. It is not possible to share the same port at a particular instant. But you can make your application such a way that it will make the port access at different instant.
Absolutely not, because even multiple connections may shave same ports but they'll have different IP addresses
When I open TCP with the server (on 7 layer of OSI), the layer 5 create socket with port number and IP.
I want to know if this socket include my IP/the server IP, and my (random) port or the server port (e.g. 80 for HTTP)
And when I open TCP with server we open TCP together
So it's mean we have common socket?
When I open TCP with the server (on 7 layer of OSI)
Forget about OSI. It is obsolete, and TCP/IP doesn't follow it. It has its own layer model.
The layer 5 create socket with port number and IP
TCP creates it at the TCP layer.
I want to know if this socket include my IP/the server IP, and my (random) port or the server port (80 for HTTP for ex.)
All of the above.
And when I open TCP with server we open TCP together So it's mean we have common socket?
No. A socket is an endpoint of a connection. There are two ends, and two sockets.
TCP is a Layer 4 as it is called - or a Transport Layer, so ignore the OSI model for the time being.
Generally - 'a socket' is just an end point without any identity. The socket gets it's identity when you bind to an address or connect to an address.
When you bind to an address - you only get your local port and local IP address in it's end point, but not the remote IP and port address. As such such socket is not very useful unless you listen on it. This is typically done on the server. Also note that you can bind to 'All Addresses on the machine' and then you really don't have any one end-point per se.
When you connect to a server (a TCP server # port 80 say), your OS TCP/IP stack makes use of a local IP address and chooses a random port to connect to a sever socket (like say one listening above). This is when all the 4 addresses come into picture. This socket is a connected socket and all 4 values will be present.