I have a lot (ten of thousands) of connected mobile devices which are maintaining an opened connection to a server. If my understanding of the 64k connection limitation is correct, you cannot have more than 64k (because of the TCP/IP protocol) connections to a single port of a server per client IP (because of the range of ephemeral ports on the client side).
But most of the time, you are in a context where these devices are connected through a network provider which use NAT to translate addresses. (for example, a smartphone won't have a static IP address).
So in this context, my server will see the same ip address and nothing garantee that the source port won't be the same in 2 different clients.
My question is maybe dumb but there it is : how can my server identify the correct connection if we think of a connection as the 5-tuple (protocol, server port, server ip, client ip, client port) in this situation ? Is there a risk of losing a connection or conflicts between 2 different clients ?
my server will see the same ip address and nothing guarantees that the source port won't be the same in 2 different clients [...] Is there a risk of losing a connection or conflicts?
No, that's the job of the router performing the NAT: keeping the IP:port combinations at one side linked to the ones on the other side.
So:
Client | IP | Src | < NAT > | IP | Src | Dest | Dst
======================================================
1 | .1 | 42 | <-----> | .3 | 1 | Server | 80
2 | .2 | 84 | <-----> | .3 | 2 | Server | 80
Given two clients, with (source IP 10.0.0.1, source port 42) and (source IP 10.0.0.2, source port 84) wish to connect to your server at port 80, then NAT will translate their IP:port pair to a pair that is valid on the other (right) side of the NAT (e.g. 11.0.0.3), by giving them a unique source port (on that side of the NAT). It will keep this translation in memory in order to be able to send packets both ways.
You'll see that the tuples on the right side of the NAT (so what your server sees) are unique:
11.0.0.3:1 - Server:80
11.0.0.3:2 - Server:80
If the router determines that the possible tuples towards your server have exhausted (so after 11.0.0.3:65535 - Server:80), it may refuse to open new connections to it.
Related
I have a Windows 10 machine, and I would like to access a database which is set on another machine outside local network.
Is there any possibility of achieving that using postgresql?
Thank's a lot, and I'd appreciate your effort to help me overcome this situation.
It is possible, provided that:
The firewall of your local network allows outgoing connections to the PostgreSQL listen port (usually 5432).
The firewall of the other network allows incoming connections to the PostgreSQL listen port (usually 5432).
The firewall of the PostgreSQL server allows connection on its listen port (usually 5432).
The PostgreSQL server is configured to accept network connections.
You can use a network scanner such as Nmap to test things, thing to do is to get a laptop on the customer's network, and scan from there. If you can connect to the PostgreSQL from an address on the same subnet, then you know there is nothing else needed on the PostgreSQL server, and so your attention need to be on the customer's firewall. This is where things can get difficult, and you'll need to work with whoever controls that firewall / router.
Chances are that the customer's network is on an RFC 1918 subnet. If this is the case the firewall / router will need to be configured to port forward like this:
public internet
|
----public address--port nnn--
| |
| firewall |
| |
|-----rfc 1918 address--------|
|
|
|
----rfc 1918 address--port 5432--
| |
| PostgreSQL server |
| |
|--------------------------------|
Three computers connected to the same router will have the same public IP. If these computers send a request to my server, will they all have the same PORT as well or are there exceptions?
EDIT: When I get requests from the browser, the PORT is different for each connection it creates. Does the browser client just pick a random port that is available on the router?
Three computers connected to the same router will have the same public IP.
Correct, from the server's perspective, not from the client's perspective.
If these computers send a request to my server, will they all have the same PORT as well or are there exceptions?
No, they will not have the same Port on the router (they may on each client PC, though).
A TCP connection is uniquely identified by the tuple {protocol, local-ip, local-port, remote-ip, remote-port}. So, when multiple TCP connections have the same {remote-ip, remote-port} (IOW, when multiple clients are connected to the same server), then each {local-ip, local-port} must be unique. And vice versa, when multiple TCP connections have the same {local-ip, local-port} (IOW, when a client connects to multiple servers), then each {remote-ip, remote-port} must be unique.
When passing through a router, each TCP connection as seen on the client side will be {TCP, lan-ip, lan-port, server-ip, server-port}, while on the server side each connection will be seen as {TCP, listen-ip, listen-port, client-ip, client-port}, where {client-ip, client-port} will be the router's {public-ip, public-port}, so each {public-ip, public-port} must be unique.
So, multiple clients connecting to the same server through a router simply cannot use the same outgoing port on the router, otherwise the server would not be able to differentiate between the connections.
When I get requests from the browser, the PORT is different for each connection it creates.
Correct.
Does the browser client just pick a random port that is available on the router?
No, nor does the browser care that a router is present. The browser creates a local socket endpoint and binds it to an available {local-ip, local-port} and then uses it to connect to the server's {server-ip, server-port}. The packets go to the OS, the OS sends them to the router, the router opens its own available {public-ip, public-port} for each new connection and then forwards those packets to the server. When the server sends packets back, the router will receive those packets on its public NIC, forward them to the appropriate client OS, which will pass them to the appropriate socket endpoint.
-------------
| Client PC A |
-------------
{tcp, client-lan-ip, client-lan-port, server-ip, server-port}
/|\
|
\|/
{tcp, router-lan-ip, router-lan-port, client-lan-ip, client-lan-port}
--------
| Router |
--------
{tcp, router-public-ip, router-public-port, server-ip, server-port}
/|\
|
\|/
{tcp, listen-ip, listen-port, router-public-ip, router-public-port}
--------
| Server |
--------
I am learning TCP/IP basics. I made a server-client chat application in which server opens a port 1024 and client can send message to it. I am a bit confused about the contents of TCP/IP packets exchanged by server and client. If client sends a message to server, it goes as packets via ethernet. In the ethernet frame from client, data field is encoded in TCP/IP format. In the TCP/IP frame, destination port will be 1024. But what will be source port's value ? Client is opening no port. Only server opens a port. Also I would like to know if there is any way to monitor these TCP/IP packets sent and received in PC.
Don't forget there's multiple layers involve here. TCP, IP and Ethernet are on different ones even if they are often used in conjunction. The separation is important to keep in mind. Ethernet (layer 2) is a protocol that connects individual computers together, but it doesn't care what IP addresses they have. IP connects computers over a much larger scale, it can be routed and sent over a variety of "layer two" network technologies where Ethernet is but one of those.
The great thing about IETF internet protocols is they're all thoroughly documented so you can find out how they work internally. In the case of TCP, which operates on top of IP, the port numbers are in the TCP layer. IP itself doesn't care about them, it's only concerned about source and destination addresses.
The key is right here in the diagram that describes the TCP header:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source Port | Destination Port |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Acknowledgment Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Data | |U|A|P|R|S|F| |
| Offset| Reserved |R|C|S|S|Y|I| Window |
| | |G|K|H|T|N|N| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | Urgent Pointer |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Options | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| data |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Both source and destination port must be populated. This is a key component of how your system's IP stack tracks which packets pertain to which connections.
Normally when you write code that connects to a server your connection originates from a (somewhat) random source port. When you create a server process that listens on a port, then that port can be automatically assigned or set specifically.
For services like HTTP you'd want that port pinned to 80 if you want other clients to connect to that service, so automatic assignment is of no help. Sometimes automatic assignment is preferable so there's no conflicts.
You can monitor all of this with tools like tcpdump or Wireshark among many others. They can dig into the various layers and show what's going on.
Port number is a logical entity/number used to identify a process running in server/client. Just like your server application has a port number ( which you decided ), client application will also have some port number associated with it, as assigned by the OS. Type in netstat -ab in cmd prompt, you can see the port number associated with your client application, in the list of processes and corresponding port numbers given by the command.
I am aware of this and this stackoverflow questions which answers known methods to achieve "Reliable Multicast" but off-late I have come across some websites which mentions even routers should also be programmed to handle custom protocols which are designed over UDP, is that true?
Basically I want to use Multicast for my application and I want don't want to impose any restriction of changing router for configuring custom protocol to handle UDP in reliable way , for example I was thinking for implementing/using PGM protocol over UDP to handle multicast but someone said that router should also have support for PGM which restricts me in providing solution since customers should change infrastructure for my solution which is unwarranted.
Please let me know if there is any solution which I can implement to handle UDP packets in reliable way without any changes to network infrastructure.
Thanks in advance.
EDIT:
I don't mean to say that I don't want to enable multicast in router, I would definitely enable multicast routing in router. When I read about PGM implementation some one said even router should be PGM capable which I thought is different router than commercially available routers in stores. Is my understanding wrong?
If you can't or don't want to configure routers to forward multicast traffic or otherwise handle a third party protocol, you'll need to tunnel multicast traffic over a unicast link. UFTP is capable of multicast tunneling via the use of a UFTP proxy server.
From the man page:
The proxy can run in one of three modes: a server proxy, a client
proxy, or response proxy.
A server proxy is typically local to a server and acts as the upstream
end of a multicast tunnel. It listens on the public multicast address
(and private multicast address when specified) and forwards downstream
packets to a specific address downstream. Upstream packets are
forwarded back where the announcement originated from.
A client proxy is typically local to one or more clients and forms the
downstream end of a multicast tunnel. It receives unicast data from
one or more server proxies and forwards downstream traffic to the
multicast address specified in the packet header. Upstream traffic
from clients is gathered and forwarded back where the announcement
came from as an aggregated response.
Below is a diagram of a typical configuration where a server sends multicast messages to the local network, and one or more server proxies unicast the messages to a corresponding client proxy, which in turn multicasts the messages to its local network.
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
x Network A x
x ---------- x
x | Server | x
x ---------- x
x | x
x | multicast x
x | x
x |----------------------------------------- x
x | | | x
x v v v x
x ---------------- ---------------- ---------- x
x | Server Proxy | | Server Proxy | | Client | x
x ---------------- ---------------- ---------- x
x | | x
x | unicast | unicast x
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
| |
| ------------
| |
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxx
x | Network B x x | Network C x
x v x x v x
x ---------------- x x ---------------- x
x | Client Proxy | x x | Client Proxy | x
x ---------------- x x ---------------- x
x | x x | x
x | multicast x x | multicast x
x | x x | x
x |------------- x x |------------ x
x | | x x | | x
x v v x x v v x
x ---------- ---------- x x ---------- ---------- x
x | Client | | Client | x x | Client | | Client | x
x ---------- ---------- x x ---------- ---------- x
x x x x
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxx
These proxies are also capable of working in a NATed environment:
If a client proxy is behind a firewall, the proxy can send a heartbeat
message to the upstream proxy to make a pinhole in the firewall that
the upstream server proxy can connect to. If the client proxy is also
NATed, the upstream server proxy may not know the IP/port of the
client proxy, so the server proxy can be configured to wait for a
heartbeat message, and use the IP/port the heartbeat came from as its
downstream address. If the server proxy is also behind a firewall or
NAT, a second server proxy on a machine with a publicly accessible IP
can be inserted between the first server proxy and the client proxy.
In this case, the first server proxy is set up to use the second as
its downstream address, and the second server proxy is set up to use
the first heartbeat it receives from a client proxy as its downstream
address.
I'm the author of this software, so if you need pointers regarding how to set this up, send me an email via the link at the bottom of the UFTP page and we'll see what we can do.
Update:
In the case of PGM, it can be configured to run at either the application layer (i.e. on top of UDP) or at the transport layer (i.e. directly on top of IP). If PGM is run at the transport layer, that's where you might need to worry about the router having special support for it. Conversely, UFTP runs strictly at the application layer.
If you use multicast you need to add multicast support requirement to the networking infrastructure. Unless multicast routing is enabled on networking devices (multicast routers) multicast will be available only within one LAN.
The only reliable way to go through Internet infrastructure is uni-cast.
Update: Multicast routing is performed on Network layer and multicast routers don't need to know anything about transport/application layers. In some situation knowledge about application layer is required on network layer but almost all that situation is related to NAT ALG (Application Level Gateway).
By the way one of possible solution to pass multicast through the Internet is tunneling protocols (GRE, IP over IP and so on).
What i require is that, I will open an UDP server listening in X port(local machine) and a machine(public IP) can send UDP packet to me. My machine doesn't have a public IP. Basically I need stun.
I am testing stuntman server/client project. I run stuntman-server in a server(public ip). Run client in my system (local ip). I asked for mapped ip/port for 9999 port.
./stunclient --mode full --protocol udp --localport 9999 stun.server.ip
Stun server returns an IP and port. What i did then, open an UDP server (using java) in my local system and start listening in 9999 port and send an UDP message from other machine (which has public IP) to mapped IP/port returned by stun server. But i can't receive any data. You can assume that my server/client code (written in java) is working fine in local network.
Flow:
My machine ->>>>>stun request for 9999 port and my ip ------> stun server
My machine <<<<<<<<<<<<<<<<<<mapped ip/port <<<<<<<<<<<<<<< stun server
My machine : Run JAVA udp server socket in 9999 port
My machine <<<<<<<<<<<<<<<<<<<UDP message to mapped ip/port<<<<<< other public machine
xxxxxxxxxxxxxxxxxxxNot workingxxxxxxxxxxxxxxxxxxxxxxxx
You didn't publish the results of your stunclient run, but I imagine it looked something like the following:
$ stunclient --mode full --localport 9999 stunserver.stunprotocol.org
Binding test: success
Local address: 192.168.1.8:9999
Mapped address: 1.2.3.4:9999
Behavior test: success
Nat behavior: Endpoint Independent Mapping
Filtering test: success
Nat filtering: Address and Port Dependent Filtering
I'm going to guess that your Behavior Test is "Endpoint Independent" and the Filtering test was "Address and Port Dependent" as those are the most common in the home and mostly matches with what you described above. (aka as "port restricted NAT").
In any case, this means you have created a port mapping between yourself and the STUN server. In the example above, my public IP address is 1.2.3.4. And is common, but not always the case, my local port (9999) is the same as the public port.
Internally, your NAT keeps a logical table such as the following:
------------------------------------------------------------------------------------
|| LOCAL IP | LOCAL PORT || EXT PORT || REMOTE IP | REMOTE PORT ||
||================================================================================||
|| 192.168.1.8 | 9999 || 9999 || 107.23.150.92 | 3478 ||
------------------------------------------------------------------------------------
Because you sent out a packet from port 9999 to the stun server (107.23.150.92), the NAT creates a port mapping entry in it's table for several minutes. When a packet arrives on the NAT/router from the Internet, it consults the table. When the response came back from the STUN server's IP:port, the NAT was able to forward it to your computer behind the NAT based on the "remote" fields in the table above.
But there is no port mapping between you and the "other public machine" that you are hoping to receive data from. Let's assume that the IP address of that other machine is 2.4.6.8 and it is attempting to send from it's local port 8888. The NAT still doesn't have anything in the table to map traffic from 2.4.6.8:8888 to a host behind the NAT. So when traffic arrive at a NAT from an a host not in the table, the NAT only knows to drop the packet on the floor. There is a NAT classification known as "Cone NAT" where this would work, but those aren't as common.
In your case, there is an easy workaround. After obtaining a port mapping from the STUN server, send another datagram from your same local port (9999) to the remote host (and remote port) that you want to receive data from. The remote host can simply ignore this datagram, but it effectively creates another port mapping entry on your NAT
------------------------------------------------------------------------------------
|| LOCAL IP | LOCAL PORT || EXT PORT || REMOTE IP | REMOTE PORT ||
||================================================================================||
|| 192.168.1.8 | 9999 || 9999 || 107.23.150.92 | 3478 ||
|| 192.168.1.8 | 9999 || 9999 || 2.4.6.8 | 8888 ||
------------------------------------------------------------------------------------
That simple 1-byte data packet to 2.4.6.8:8888 allows the NAT to forward traffic back from that address to your host behind the NAT.
In other words, using your own network flow nomenclature:
My machine:9999 ---->[STUN BINDING REQUEST]--->stun server:3478
My machine:9999 <----[STUN BINDING RESPONSE mapped IP:port]<--- stun server:3478
My machine:9999 [Open socket on port 9999]
My machine:9999 ---->[1 byte datagram] -------> 'other:8888'
My machine:9999 <---- [UDP to public IP:port obtained in step 2]<----'other:8888'
Typically, in a normal P2P flow, both endpoints work with a STUN server to discover their port mapping. And then use another service on to exchange IP:port information between each other. From what you describe, you are manually exchanging these values between your programs, which is fine for testing.
If the other machine is on the public internet, you technically do not need STUN. The first machine (behind a NAT) can just send directly to the remote IP and port to say, "send me some data". The remote side just inspects the peer address and port of this message to decide where to send back to. The port mapping has already been created. Some RTSP clients assume the server is public
My answer on the basics of socket NAT traversal is here.
I happen to know the developer of STUNTMAN. He's a reasonably nice guy, good looking, and very smart. They also say him and I look alike and have near identical spelling with our names. You can always mail him directly if you have questions about STUN and NAT traversal.