Using UDP between remote server and client - sockets

I am currently working on a project that needs a UDP communication between a server Amazon Frankfurt) and several hosts (Arduino). I need to be able to send JSON data from server to a host over UDP and it needs to work in real time (time is a real constraint here). Also, periodically a host will speak to server to inform server about its status.
The problem I'm facing is that although I can ping the server, when I create the UDP socket with a port, I get this error (for any port):
errno99: cannot assign requested address
I also believe that since the hosts are in different countries, I can only get router's IP. Hosts will be given to users therefore it is impossible to alter user's router by hand. I need to use something that can handle these by code.
I have been searching on Hole Punching and other techniques. I cannot decide what suits more to my scenario here. Sorry for my lack of network information, any help or guidance as "read or search these techniques" are appreciated.
This code runs on server:
import socket
import time
#Host's public IP
UDP_IP_ADDRESS = "178.243.98.86"
UDP_PORT_NO = 13000
Destination = (UDP_IP_ADDRESS,UDP_PORT_NO)
Message = "Hello, Server"
senderSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
while True:
senderSock.sendto(Message, Destination)
time.sleep(1);
And this code runs on my machine:
import socket
#Server's public IP
UDP_IP_ADDRESS = "172.31.46.111"
UDP_PORT_NO = 13000
recieverSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
recieverSock.bind((UDP_IP_ADDRESS, UDP_PORT_NO))
while True:
data, addr = recieverSock.recvfrom(1024)
print "Message: ", data

Server
# Host's public IP
UDP_IP_ADDRESS = "178.243.98.86"
You cannot rely on 178.243.98.86. Your arduinos are behind one or several NATs. With NAT, when a packet leaves a network, your packets source and/or destination addresses (and most likely ports) get rewritten. When this conversion happens, a device responsible for NAT remembers this conversion, but only for a limited period of time. The period usually defaults to 24h.
In order to overcome this issue, it's up to clients to initiate conversations with the server. Introduce a new message type to your protocol saying something like hey it's me, client #34. The server should be associating source IP and port of the received message with client #34 for some period of time, like hours.
In this case the server binds a socket to an address and port as well as a client. And beware that hey it's me, client #34 sent over UDP is not encrypted and is extremely vulnerable to record and replay attacks. I believe an emergency system should be encrypted.
Client
# Server's public IP
UDP_IP_ADDRESS = "172.31.46.111"
172.31.46.111 cannot be server's public IP, because it is an address from an IPv4 private space. You probably meant to say "host's private address". It makes sense to bind to such an address. You get an error most likely because you have incorrectly determined the IP address of your host.
There are several ways to get the correct IP. If you use DHCP (you use it if you've never though about what IPs to use in your SOHO), log into your nearest router's admin interface and check the DCHP Pool. Alternatively, Linux and Windows have terminal commands to do the same. For Arduino, you have probably assigned it a static address. If so, bind to it.

You also seem to open a tcp socket on the server, and connect to it from client via udp socket. Is this intentional?
gsn

Related

Lua Networking - Passing data through a 'closed' port

This might be a bit weird to explain, but I'll try my best.
I have a Lua program that's intended to serve some data through the network. Specifically, the internet. The data the program is actually transmitting are only strings stored within UDP packets. Generalized, this is how the program operates:
The first client launches the program and specifies that they are the 'host' of the connection. The program opens a connection on UDP port 6000 and the main loop listens for any packets received on said port.
The second client launches the program and specifies that they are to connect to the 'host' on port 6000. The user enters the IP, and the client opens a UDP connection using a random port between 6050 and 7000
When the client successfully connects to the server, they send a 'connection' packet, simply containing a '202 OK' string. The 'host' receives this and registers the new client
Now that the connection has been initialized, the programs can send data between each other using the registered data.
Now, on a local network this program works fine. The purpose of the 'host' mode is to have multiple clients connect to the host and have the host relay packets from one clients to all the currently registered clients. Port selections are arbitrary and random port selection from the client was simply to allow debugging and testing from a single computer. This has been tested between two and more computers on a physical network, and worked successfully. However, when I attempt to run this over the internet it's a no go. I know that the ports are closed and that's why it's not working. But seeing as I'm going to be distributing this program (privately) I can't expect every person to open ports on their router (or know how to). Security is not currently a concern with the program, and should be disregarded in the current state. That being said, I recognise there's the potential for a lot to go wrong with the use of this program through the network and I accept that. Onto the main question, how can I have the host and client communicate over the internet without having to open ports?
I'll elaborate - for example, browsers. Although the technology is quite different to what I'm doing, it's easier to paint a picture - the browser requests data from a web server, and it gets sent back to the client. But wait, if the router is in it's default state (I hope) all the ports are closed? So how does the client receive this data if the port is closed?
I hope this makes some kind of sense and I don't sound like a complete fool.
I managed to find some suitable libraries and utilities to be able to communicate through the internet (NAT traversal is now a term I am familiar with), those libraries being that supplied by NMAP. These libraries include an implementation for STUN in LUA, among HEAPS of other useful networking-related libraries and scripts.
To actually answer my own question (very simply), the clients and servers are behind what's known as a NAT gateway. Due to the limitations of addresses of IPv4, NAT gateways were implemented to bypass this limitation of IPv4 (a total of about 4.2 billion addresses) by separating the clients' internal network from the external network - in this case the internet. The NAT is supplied with a single IP address, and the NAT then supplies all of its users within the internal network with an IP respective to the network they're on. As such, the devices cannot directly be accessed without forwarding connections from the NAT gateway (generally the router) to the client. However, when using UDP connections the NAT gateway opens a port for the purposes of this connection which gets closed after the connection dies. This port that is opened differs from what is specified by the client when they open the connection, which is where the STUN methods come in. STUN allows the host to find the port that the client is connecting from and send data back to this port so the user can receive it. Bear in mind this is an EXTREMELY simple explanation of how the technology works, and I'd suggest reading up on the Wiki and some of the Request for Comments for STUN.

Obtaining the source IP and port of an INADDR_ANY client socket before the TCP three-way handshake?

I'm on Windows 7, using bind before connect with SO_REUSEADDR, and setting the local address structure to IP address INADDR _ANY and port 0 (zero), in order to let the operating system select the source details for a client socket.
Firstly, I've read that it's not possible to get the source IP before connecting to the server, since it's being chosen at this point and several addresses can be valid. But the port is selected before the connection, so is there a way to get it? (getsockname() looks like to not work).
Secondly, about the source IP, is there a way to get it before a packet is sent to the server? I need the specific time between the moment the OS selected the source IP and the moment it starts the three-way handshake. The connect() function dominates the both.
I'm on Windows 7, using bind before connect with SO_REUSEADDR
Why are you using SO_REUSEADDR in this situation? You don't need it, and it makes no sense for what you are attempting. SO_REUSEADDR should typically only be used for a listening socket, not a connecting socket.
setting the local address structure to IP address INADDR _ANY and port 0 (zero), in order to let the operating system select the source details for a client socket.
It is meaningless to bind() a client socket to INADDR_ANY:0. You can (and should) omit such a bind() completely and leave the socket unbound until connect() is called. The only time you should ever bind() a client socket is if you want to bind it to a specific local IP and/or Port. But you are not doing that in this situation, so get rid of it.
Firstly, I've read that it's not possible to get the source IP before connecting to the server, since it's being chosen at this point and several addresses can be valid.
Correct, unless you bind() to a specific source IP.
But the port is selected before the connection
Both source IP and source port are selected by connect() if the socket is unbound, or bound to source IP INADDR_ANY and/or source port 0. So you have no opportunity to query either value before connect() has selected them.
so is there a way to get it? (getsockname() looks like to not work).
getsockname() is exactly what you need. Just make sure you are not calling it until connect() has successfully connected to the server first. This is stated in the getsockname() documentation:
This call is especially useful when a connect call has been made without doing a bind first; the getsockname function provides the only way to determine the local association that has been set by the system.
...
The getsockname function does not always return information about the host address when the socket has been bound to an unspecified address, unless the socket has been connected with connect or accept (for example, using ADDR_ANY). A Windows Sockets application must not assume that the address will be specified unless the socket is connected. The address that will be used for the socket is unknown unless the socket is connected when used in a multihomed host. If the socket is using a connectionless protocol, the address may not be available until I/O occurs on the socket.
Secondly, about the source IP, is there a way to get it before a packet is sent to the server?
For TCP, you can retrieve the selected source IP using getsockname() immediately after connect() has successfully connected to the server. Not before.
I need the specific time between the moment the OS selected the source IP and the moment it starts the three-way handshake.
It is not possible to determine that detail. No socket application should ever need that detail. Why do you need it?
I'm on Windows 7, using bind before connect with SO_REUSEADDR, and setting the local address structure to IP address INADDR _ANY and port 0 (zero), in order to let the operating system select the source details for a client socket.
Why? That's exactly what happens if you don't call bind() at all, during connect(). Binding a client socket to INADDR_ANY isn't correct in any case. Setting SO_REUSEADDR doesn't make sense either without specifying a non-zero port number. Just remove all this.
Firstly, I've read that it's not possible to get the source IP before connecting to the server, since it's being chosen at this point and several addresses can be valid.
Correct.
But the port is selected before the connection, so is there a way to get it?
Yes. getsockname().
(getsockname() looks like to not work)
Doesn't work how?
Secondly, about the source IP, is there a way to get it before a packet is sent to the server?
You can get it with getsockname() as soon as connect() has succeeded, but this involves sending packets to the server.
I need the specific time between the moment the OS selected the source IP and the moment it starts the three-way handshake. The connect() function dominates the both.
Bad luck.

What does it mean to connect to a certain port?

For example, when you make an ssh connection, you are connected to port 22. What happens then? On a very high level brief overview, I know that if port 22 is open on the other end and if you can authenticate to it as a certain user, then you get a shell on that machine.
But I don't understand how ports tie into this model of services and connections to different services from remote machines? Why is there a need for so many specific ports running specific services? And what exactly happens when you try to connect to a port?
I hope this question isn't too confusing due to my naive understanding. Thanks.
Imagine your server as a house with 65536 doors. If you want to visit family "HTTP", you go to door 80. If you were to visit family "SMTP", you would visit door no. 25.
Technically, a port is just one of multiple possible endpoints for outgoing/incomming connections. Many of the port numbers are assigned to certain services by convention.
Opening/establishing a connection means (when the transport protocol is TCP, which are most of the “classical” services like HTTP, SMTP, etc.) that you are performing a TCP handshake. With UDP (used for things like streaming and VoIP), there's no handshake.
Unless you want to understand the deeper voodoo of IP networks, you could just say, that's about it. Nothing overly special.
TCP-IP ports on your machine are essentially a mechanism to get messages to the right endpoints.
Each of the possible 65536 ports (16 total bits) fall under certain categories as designated by the Internet Assigned Numbers Authority (IANA).
But I don't understand how ports tie into this model of services and
connections to different services from remote machines? Why is there a
need for so many specific ports running specific services?
...
And what exactly happens when you try to connect to a port?
Think of it this way: How many applications on your computer communicate with other machines? Web browser, e-mail client, SSH client, online games, etc. Not to mention all of the stuff running under the hood.
Now think: how many physical ports do you have on your machine? Most desktop machines have one. Occasionally two or three. If a single application had to take complete control over your network interface nothing else would be able to use it! So TCP ports are a way of turning 1 connection into 65536 connections.
For example, when you make an ssh connection, you are connected to
port 22. What happens then?
Think of it like sending a package. Your SSH client in front of you needs to send information to a process running on the other machine. So you supply the destination address in the form of "user#[ip or hostname]" (so that it knows which machine on the network to send it to), and "port 22" (so it gets to the right application running on the machine). Your application then packs up a TCP parcel and stamps a destination and a return address and sends it to the network.
The network finds the destination computer and delivers the package. So now it's at the right machine, but it still needs to get to the right application. What do you think would happen if your SSH packet got delivered to an e-mail client? That's what the port number is for. It effectively tells your computer's local TCP mailman where to make the final delivery. Then the application does whatever it needs to with the data (such as verify authentication) and sends a response packet using your machine's return address. The back and forth continues as long as the connection is active.
Hope that helps. :)
The port is meant to allow applications on TCP/IP to exchange data. Each machine on the internet has one single address which is its IP. The port allows different applications on one machine to send and receive data with multiple servers on the network/internet. Common application like ftp and http servers communicate on default ports like 21 and 80 unless network administrators change those default ports for security reasons

Communicating between networks using sockets

I have a question about network connections among computers.
I've made some applications where messages pass through the Internet (via sockets) to make a connection between two devices. However, a strong condition is that two devices must be connected to the same network.
Can anyone give me a trick how to create a communication using sockets between two computers even if they are connected to different netwkorks?
Thank you in advance.
Here is a great tutorial on how to use sockets and general networking
(in java) http://www.thenewboston.org/watch.php?cat=25&number=38
In order to communicate between two diffrent networks over the internet, you will need to do something called port forwarding. What that does is that when your public IP of your network receives a packet with a spesific port number. The router knows where to send that packet to which local IP.
If you dont port forward and receive some data. The router doesent know where to send the packet. Therefore it discards it, which means others wont be able to connect to you.
You will only need to port forward the network with your server (using the example i linked). How you do that is by logging in to your router, and say that a port which the server uses gets forwarded to the IP of the PC hosting the server.
On the other network (client) you will need to change the IP address of which the client shall connect to. That IP address needs to be your public IP of your server's network. You can find that by connecting to the server's network and go to: http://www.whatsmyip.org/ . Keep in mind that public IP addresses may change over time.
Hope this helped!
-Kad

Error when using two different user agents

I have 2 sip clients on the same computer.
Both of them is registering to a server that is running on port 5060.
For the first client the UDP is on port 5060 and for the other is 5061. When I come from one client to another, after the ringing part i receive the error:
only one usage of each socket address is normally permited.
Got any ideas why I got this error?
Your server and client are both trying to use port 5060, hence the error message. Change the first client to use 5062 or something else.
Also, 5061 is normally used for secured SIP (normal listening port + 1 in the proxy/server). Do not use it for the second client.
It means you're clients are both trying to claim the same socket for the communication channel, or the server is trying to reclaim the socket given to client A, to reuse it for client B.
The software handeling the socket, should be smart enough to rely on the OS to assign port numbers instead of hardcoding the port numbers in the code, this is a 100% guarantee for socket issues.