How do I do TCP hole punching? - sockets

Question is below. Here is my current test code which did not succeed.
static void Main(string[] args)
{
if (args.Count() != 3)
{
Console.WriteLine("Bad args");
}
var ep = new IPEndPoint(IPAddress.Parse(args[0]), int.Parse(args[1]));
var lp = new IPEndPoint(IPAddress.Any, int.Parse(args[2]));
var s = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
s.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
s.Bind(lp);
var c = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
c.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
c.Bind(lp);
Task.Run(() => { try { c.Connect(ep); } catch { } });
s.Listen(10);
var v = s.Accept();
v.Close();
}
How do I do TCP hole punching? I am testing using a remote server. I'm running wget local_public_ip:port/test. I have my router setup for port 80 so it doesn't need a hole punch. My code got a connection. Now I try on other ports and I can't exactly figure out how to punch the hole.
What I have done is (C# code)
var l = new TcpListener(8090);
l.Start();
try { var o = new TcpClient(); o.Connect("myserverip", 123); }
catch(Exception ex) {}
var e = l.AcceptSocket();
Console.WriteLine(e.RemoteEndPoint.AddressFamily);
I thought maybe I need to setup the local endpoint on the out tcp connection.
TcpClient(new System.Net.IPEndPoint(new System.Net.IPAddress(bytearray), port));
I made a mistake and got this exception
The requested address is not valid in its context
Fixing up the byte array to 192,168,1,5 it appears to make outgoing connects correctly. Now that I have a out connection to the remote IP using my listening port I thought wget would be able to connect to me. It wasn't the case
How do I do TCP hole punching?

I'd use the "sequential hole punching technique" detailed in http://www.bford.info/pub/net/p2pnat/index.html. It seems much simpler to do that simultaneous connections and socket reuse. It is not necessary for hole punching to do anything exactly simultaneously (that is a meaningless notion in distributed systems anyway).
I have implemented hole punching. My router seems not to like it. Wireshark shows the outbound hole punching SYN is correct but the remote party can't get through to me. I verifies all ports with TcpView.exe and disabled all firewalls. Must be a router issue. (It is a strange and invasive router.)
class HolePunchingTest
{
IPEndPoint localEndPoint;
IPEndPoint remoteEndPoint;
bool useParallelAlgorithm;
public static void Run()
{
var ipHostEntry = Dns.GetHostEntry("REMOTE_HOST");
new HolePunchingTest()
{
localEndPoint = new IPEndPoint(IPAddress.Parse("LOCAL_IP"), 1234),
remoteEndPoint = new IPEndPoint(ipHostEntry.AddressList.First().Address, 1235),
useParallelAlgorithm = true,
}.RunImpl();
}
void RunImpl()
{
if (useParallelAlgorithm)
{
Parallel.Invoke(() =>
{
while (true)
{
PunchHole();
}
},
() => RunServer());
}
else
{
PunchHole();
RunServer();
}
}
void PunchHole()
{
Console.WriteLine("Punching hole...");
using (var punchSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
{
EnableReuseAddress(punchSocket);
punchSocket.Bind(localEndPoint);
try
{
punchSocket.Connect(remoteEndPoint);
Debug.Assert(false);
}
catch (SocketException socketException)
{
Console.WriteLine("Punching hole: " + socketException.SocketErrorCode);
Debug.Assert(socketException.SocketErrorCode == SocketError.TimedOut || socketException.SocketErrorCode == SocketError.ConnectionRefused);
}
}
Console.WriteLine("Hole punch completed.");
}
void RunServer()
{
using (var listeningSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
{
EnableReuseAddress(listeningSocket);
listeningSocket.Bind(localEndPoint);
listeningSocket.Listen(0);
while (true)
{
var connectionSocket = listeningSocket.Accept();
Task.Run(() => ProcessConnection(connectionSocket));
}
}
}
void ProcessConnection(Socket connectionSocket)
{
Console.WriteLine("Socket accepted.");
using (connectionSocket)
{
connectionSocket.Shutdown(SocketShutdown.Both);
}
Console.WriteLine("Socket shut down.");
}
void EnableReuseAddress(Socket socket)
{
if (useParallelAlgorithm)
socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
}
}
You can try both values for useParallelAlgorithm. Both should work.
This code is for the server. It punches a hole into the local NAT. You can then connect from the remote side using any client that allows to pick the local port. I used curl.exe. Apparently, telnet on Windows does not support binding to a port. wget apparently neither.
Verify that the ports are correct on both sides using TcpView or Process Explorer. You can use Wireshark to verify packets. Set a filter like tcp.port = 1234.
When you "call out" to punch a hole you enable the tuple (your-ip, your-port, remote-ip, remote-port) to communicate. This means that all further communication must use those values. All sockets (inbound or outbound) must use these exact port numbers. In case you aren't aware: outgoing connections can control the local port as well. This is just uncommon.

Related

Unity TCP client: connection to a server

I have a hardware that streams data to a windows server, and I have IP and port of this server. Now I want to connect unity app as a TCP client to this server. I have tried many solutions, but still have hard time to figure out best way to do this. Any help is highly appreciated. One issue that I have with current tutorials is that most of them define a unity server and try to connect another unity app as client and mostly based on localhost. However, I just want to connect a unity app as a TCP client to a server which I have IP and port.
Warm regards,
enter image description here
First of all, at the top of your script:
using System;
using System.Net.Sockets;
Then, create an instance of TcpClient and begin the connection:
socket = new TcpClient
{
// dataBufferSize is in bytes
ReceiveBufferSize = dataBufferSize,
SendBufferSize = dataBufferSize
};
socket.BeginConnect(ip, port, ConnectCallback, socket);
In ConnectCallback():
private void ConnectCallback(IAsyncResult asyncResult)
{
socket.EndConnect(asyncResult);
if (!socket.Connected) return;
// stream is a NetworkStream you should declare somewhere
stream = socket.GetStream();
// receiveBuffer is a byte[] you should declare somewhere
receiveBuffer = new byte[dataBufferSize];
stream.BeginRead(receiveBuffer, 0, dataBufferSize, ReceiveCallback, null);
}
In ReceiveCallback():
private void ReceiveCallback(IAsyncResult asyncResult)
{
try
{
int byteLength = stream.EndRead(asyncResult);
if (byteLength <= 0)
{
// Disconnect client
return;
}
// Transfer data from receiveBuffer to data variable for handling
byte[] data = new byte[byteLength];
Array.Copy(receiveBuffer, data, byteLength);
// Handle data in any way you want to
// BeginRead again so you can keep receiving data
stream.BeginRead(receiveBuffer, 0, dataBufferSize, ReceiveCallback, null);
}
catch (Exception e)
{
Console.WriteLine($"Error receiving TCP data: {e.Message}");
}
}
I got this information from this tutorial.

Xamarin Forms How to change Port or IPAddress of socket connection

I have a UWP (soon to be MacOS also) application that listens for incoming messages. The user can configure which IP Address and Port to listen on. Once the socket connection is listening, the user can also go back into the settings and change the IP Address or Port. I am trying to figure out how to shut down the existing listener and restart it using the new Port / IP Address when the user changes the values. Here is my code that starts the listener. Any help would be appreciated.
private static Socket iobj_listener;
public async static Task StartListening()
{
try
{
Debug.WriteLine("Point 1");
IPEndPoint localEndPoint = new IPEndPoint(ViewModelObjects.AppSettings.ServerIPAddress, ViewModelObjects.AppSettings.ServerPort);
// Create a TCP/IP socket.
iobj_listener = new Socket(ViewModelObjects.AppSettings.ServerIPAddress.AddressFamily,
SocketType.Stream, ProtocolType.Tcp);
// Bind the socket to the local endpoint and listen for incoming connections.
iobj_listener.Bind(localEndPoint);
iobj_listener.Listen(100);
ViewModelObjects.AppSettings.ListeningOnSocket = true;
while (true)
{
Debug.WriteLine("Point 2");
// Set the event to nonsignaled state.
allDone.Reset();
// Start an asynchronous socket to listen for connections.
Debug.WriteLine("Waiting for a connection on " + ViewModelObjects.AppSettings.ServerIPAddress.ToString() + "...");
iobj_listener.BeginAccept(
new AsyncCallback(AcceptCallback),
iobj_listener);
// Wait until a connection is made before continuing.
allDone.WaitOne();
}
}
catch (Exception e)
{
Debug.WriteLine(e.ToString());
}
finally
{
Debug.WriteLine("Point 3");
ViewModelObjects.AppSettings.ListeningOnSocket = false;
}
}
SO I could not find any quick answers so had to kind of figure this out on my own. If you see anything wrong with this, please let me know.
First of all I declared an e_Num as follows
public enum ge_SocketStatus
{
e_NotListening = 0,
e_Listening = 1,
e_Restart = 2
}
Then I added a StopListening function to my class that handles all my Socket communications and set the socket status to not listening as follows:
public static async Task StopListening()
{
try
{
if (iobj_listener.Connected)
{
//Wait till the connection ends or 30 seconds - this is so any last messages can be processed.
await Task.Delay(30000);
}
ViewModelObjects.AppSettings.SocketStatus = ge_SocketStatus.e_NotListening;
iobj_listener.Close(1);
}
catch (Exception ex)
{
App.AppException(ex);
}
}
I then use the value of this enum to know when to end the loop:
public async static Task StartListening()
{
try
{
Debug.WriteLine("Point 1");
IPEndPoint localEndPoint = new IPEndPoint(ViewModelObjects.AppSettings.ServerIPAddress, ViewModelObjects.AppSettings.ServerPort);
// Create a TCP/IP socket.
iobj_listener = new Socket(ViewModelObjects.AppSettings.ServerIPAddress.AddressFamily,
SocketType.Stream, ProtocolType.Tcp);
// Bind the socket to the local endpoint and listen for incoming connections.
iobj_listener.Bind(localEndPoint);
iobj_listener.Listen(100);
ViewModelObjects.AppSettings.SocketStatus = ge_SocketStatus.e_Listening;
while (ViewModelObjects.AppSettings.SocketStatus == ge_SocketStatus.e_Listening)
{
Debug.WriteLine("Point 2");
// Set the event to nonsignaled state.
allDone.Reset();
// Start an asynchronous socket to listen for connections.
Debug.WriteLine("Waiting for a connection on " + ViewModelObjects.AppSettings.ServerIPAddress.ToString() + "...");
iobj_listener.BeginAccept(
new AsyncCallback(AcceptCallback),
iobj_listener);
// Wait until a connection is made before continuing.
allDone.WaitOne();
}
}
catch (Exception e)
{
Debug.WriteLine(e.ToString());
}
finally
{
Debug.WriteLine("Point 3");
}
}
This line above
while (ViewModelObjects.AppSettings.SocketStatus == ge_SocketStatus.e_Listening)
used to be
while (true)
so the loop would never end.
One gotcha I found is in the AcceptCallback used in the BeginAccept function of my socket. In this code, I also had to detect if the socket was connected because this function is called one last time after the StartListening loop exits. At the point the socket is not connected so trying to do anything with is, such as EndAccept, causes the application to throw an exception. Below you can see where I added the line
if (listener.Connected)
in order to stop the code from crashing after I had closed the connection.
public static void AcceptCallback(IAsyncResult ar)
{
// Signal the main thread to continue.
allDone.Set();
// Get the socket that handles the client request.
Socket listener = (Socket)ar.AsyncState;
//If we have shut down the socket don't do this.
if (listener.Connected)
{
Socket handler = listener.EndAccept(ar);
// Create the state object.
StateObject state = new StateObject();
state.workSocket = handler;
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReadCallback), state);
}
}
Once all StopListening function ends and everything from the sockets is disconnected, I can call start listening again and open the socket on a different IPAddress and or Port.
I hope this helps as I could not find a good solution to this.

Connection not working from Android to UWP on desktop - ZeroMq (NetMq)

I tried some examples of ZMQ on C++,C# and Python. I am trying to have Request-Reply pattern to connect Android device to PC running UWP with Xamarin forms.
Below is the Requestor code:
public void HelloWorld()
{
var timer = new Timer(60000);
timer.Start();
timer.Elapsed += (sender, args) =>
{
this.Cancel = true;
timer.Stop();
};
// Create
const string endpoint = "tcp://PC_ip:3245";
using (var request = new RequestSocket())
{
request.Bind(endpoint);
Thread.Sleep(2000);
while (!Cancel)
{
request.SendFrame("Requester says hello");
var reply = request.ReceiveFrameString();
Debug.WriteLine("Gets reply {0}",reply);
}
}
}
Reply socket code:
public void HelloWorld()
{
var timer = new Timer(60000);
const string endpoint = "tcp://PC_ip:3245";
timer.Start();
timer.Elapsed += (sender, args) =>
{
timer.Stop();
Cancel = true;
};
using (var replierSocket = new ResponseSocket())
{
replierSocket.Connect(endpoint);
Thread.Sleep(2000);
while (!Cancel)
{
var replyFromRequester = replierSocket.ReceiveFrameString();
Debug.WriteLine("Got reply {0}", replyFromRequester);
replierSocket.SendFrame("Response socket say hello");
}
}
}
Cancel is boolean
I went through some questions posted on this and added delay and these connection code blocks only trigger after button clicks on app.
While debugging , request.ReceiveFrameString() replierSocket.ReceiveFrameString(); are not even hit.
I am new to network programming , I understand that for REQ/REP pattern the code has to be in particular order which I traced and fixed I believe and turned off firewall on my PC so that firewall wont block my incoming connections from Android device.
PC_ip stands for IPv4 address I got from ipconfig /all for my wifi. I also tried external ip of my machine from sites like whatsmyip.org at ResponseSocket but I still dont get response between devices.
Please let me know what am I doing wrong.
Issue replication repository : GitHub/me/XamZeroMq

Cloud Service for incoming TCP connections hangs

I'm developing a cloud service (worker role) for collecting data from a number of instruments. These instruments reports data randomly every minute or so. The service itself is not performance critical and doesn't need to be asynchronous. The instruments are able to resend their data up to an hour on failed connection attempt.
I have tried several implementations for my cloud service including this one:
http://msdn.microsoft.com/en-us/library/system.net.sockets.tcplistener.stop(v=vs.110).aspx
But all of them hang my cloud server sooner or later (sometimes within an hour).
I suspect something is wrong with my code. I have a lot of logging in my code but I get no errors. The service just stops to receive incoming connections.
In Azure portal it seems like the service is running fine. No error logs and no suspicious cpu usage etc.
If I restart the service it will run fine again until it hangs next time.
Would be most grateful if someone could help me with this.
public class WorkerRole : RoleEntryPoint
{
private LoggingService _loggingService;
public override void Run()
{
_loggingService = new LoggingService();
StartListeningForIncommingTCPConnections();
}
private void StartListeningForIncommingTCPConnections()
{
TcpListener listener = null;
try
{
listener = new TcpListener(RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["WatchMeEndpoint"].IPEndpoint);
listener.Start();
while (true)
{
_loggingService.Log(SeverityLevel.Info, "Waiting for connection...");
var client = listener.AcceptTcpClient();
var remoteEndPoint = client.Client != null ? client.Client.RemoteEndPoint.ToString() : "Unknown";
_loggingService.Log(SeverityLevel.Info, String.Format("Connected to {0}", remoteEndPoint));
var netStream = client.GetStream();
var data = String.Empty;
using (var reader = new StreamReader(netStream, Encoding.ASCII))
{
data = reader.ReadToEnd();
}
_loggingService.Log(SeverityLevel.Info, "Received data: " + data);
ProcessData(data); //data is processed and stored in database (all resources are released when done)
client.Close();
_loggingService.Log(SeverityLevel.Info, String.Format("Connection closed for {0}", remoteEndPoint));
}
}
catch (Exception exception)
{
_loggingService.Log(SeverityLevel.Error, exception.Message);
}
finally
{
if (listener != null)
listener.Stop();
}
}
private void ProcessData(String data)
{
try
{
var processor = new Processor();
var lines = data.Split('\n');
foreach (var line in lines)
processor.ProcessLine(line);
processor.ProcessMessage();
}
catch (Exception ex)
{
_loggingService.Log(SeverityLevel.Error, ex.Message);
throw new Exception(ex.InnerException.Message);
}
}
}
One strange observation i just did:
I checked the log recently and no instrument has connected for the last 30 minutes (which indicates that the service is down).
I connected to the service myself via a TCP client i've written myself and uploaded some test data.
This worked fine.
When I checked the log again my test data had been stored.
The strange thing is, that 4 other instruments had connected about the same time and send their data successfully.
Why couldn't they connect by themself before I connected with my test client?
Also, what does this setting in .csdef do for an InputEndpoint, idleTimeoutInMinutes?
===============================================
Edit:
Since a cuple of days back my cloud service has been running successfully.
Unfortunately this morning last log entry was from this line:
_loggingService.Log(SeverityLevel.Info, String.Format("Connected to {0}", remoteEndPoint));
No other connections could be made after this. Not even from my own test TCP client (didn't get any error though, but no data was stored and no new logs).
This makes me think that following code causes the service to hang:
var netStream = client.GetStream();
var data = String.Empty;
using (var reader = new StreamReader(netStream, Encoding.ASCII))
{
data = reader.ReadToEnd();
}
I've read somewhere that StremReader's ReadToEnd() could hang. Is this possible?
I have now changed this piece of code to this:
int i;
var bytes = new Byte[256];
var data = new StringBuilder();
const int dataLimit = 10;
var dataCount = 0;
while ((i = netStream.Read(bytes, 0, bytes.Length)) != 0)
{
data.Append(Encoding.ASCII.GetString(bytes, 0, i));
if (dataCount >= dataLimit)
{
_loggingService.Log(SeverityLevel.Error, "Reached data limit");
break;
}
dataCount++;
}
Another explanation could be something hanging in the database. I use the SqlConnection and SqlCommand classes to read and write to my database. I always close my connection afterwards (finally block).
SqlConnection and SqlCommand should have default timeouts, right?
===============================================
Edit:
After some more debugging I found out that when the service wasn't responding it "hanged" on this line of code:
while ((i = netStream.Read(bytes, 0, bytes.Length)) != 0)
After some digging I found out that the NetStream class and its read methods could actually hang. Even though MS declares otherwise.
NetworkStream read hangs
I've now changed my code into this:
Thread thread = null;
var task = Task.Factory.StartNew(() =>
{
thread = Thread.CurrentThread;
while ((i = netStream.Read(bytes, 0, bytes.Length)) != 0)
{
// Translate data bytes to a ASCII string.
data.Append(Encoding.ASCII.GetString(bytes, 0, i));
}
streamReadSucceeded = true;
});
task.Wait(5000);
if (streamReadSucceeded)
{
//Process data
}
else
{
thread.Abort();
}
Hopefully this will stop the hanging.
I'd say that part of your problem is you are processing your data on the thread that listens for connections from clients. This would prevent new clients from connecting if another client has started a long running operation of some type. I'd suggest you defer your processing to worker threads thus freeing the "listener" thread to accept new connections.
Another problem you could be experiencing, if your service throws an error, then the service will stop accepting connections as well.
private static void ListenForClients()
{
tcpListener.Start();
while (true)
{
TcpClient client = tcpListener.AcceptTcpClient();
Thread clientThread = new Thread(new ParameterizedThreadStart(HandleClientComm));
clientThread.Start(client);
}
}
private static void HandleClientComm(object obj)
{
try
{
using(TcpClient tcpClient = (TcpClient)obj)
{
Console.WriteLine("Got Client...");
using (NetworkStream clientStream = tcpClient.GetStream())
using (StreamWriter writer = new StreamWriter(clientStream))
using(StreamReader reader = new StreamReader(clientStream))
{
//do stuff
}
}
}
catch(Exception ex)
{
}
}

I cannot make UDP Ports work on a Windows Azure Virtual Machine

I cannot receive a UDP packet on a Windows Azure Virtual Machine. I have done the following:
On the Virtual Machine, via Windows Firewall, I opened up Port 1234* both Inbound and Outbound for both UDP and TCP protocols. I did not add any IP exclusions. The rule should apply to Domain, Private and Public profiles. I am allowing Block Edge Traversal.
In the Azure Management Portal, I added Endpoints for my Virtual Machine instance. I added both UDP and TCP protocol endpoints. Public and Private port numbers are both 1234.
I have two test programs that I wrote: UDPSender and UDPReceiver. Using two computers on my local network, the test programs successfully sent a packet between them. (Edit: I also used UDPTester Android App to successfully send a 'trans-ISP' message to my PC running the UDPReceiver.)
Moving UDPReceiver to my Virtual Machine, I cannot successfully receive a message.
Did I miss anything in my Azure Endpoint configuration? Please Help!
* Port Number changed to protect the innocent.
Test Program Code Below...
UDPSender:
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
textMessage.Text = "Knock, knock";
textIP.Text = "xxx.xxx.xxx.xxx";
textPort.Text = "1234";
}
private void buttonSend_Click(object sender, EventArgs e)
{
UdpClient udpClient = new UdpClient(textIP.Text, Convert.ToInt32(textPort.Text));
Byte[] sendBytes = Encoding.ASCII.GetBytes(textMessage.Text);
try
{
udpClient.Send(sendBytes, sendBytes.Length);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
}
UDPReceiver:
private static void Main(string[] args)
{
//Creates a UdpClient for reading incoming data.
UdpClient receivingUdpClient = new UdpClient(1234);
while (true)
{
//Creates an IPEndPoint to record the IP Address and port number of the sender.
// The IPEndPoint will allow you to read datagrams sent from any source.
System.Net.IPEndPoint RemoteIpEndPoint = new System.Net.IPEndPoint(System.Net.IPAddress.Any, 0);
try
{
// Blocks until a message returns on this socket from a remote host.
Byte[] receiveBytes = receivingUdpClient.Receive(ref RemoteIpEndPoint);
string returnData = Encoding.ASCII.GetString(receiveBytes);
string messageOut = String.Format("[{0},{1}]#[{2}]: {3}",
RemoteIpEndPoint.Address.ToString(),
RemoteIpEndPoint.Port.ToString(),
DateTime.Now,
returnData.ToString());
Console.WriteLine(messageOut);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
}
The code looks correct. I compared it against my implementation. Here are the key sections for reference:
Your csdef should have the correct port protocol. You said you did this through the portal, but confirm the settings saved:
<Endpoints>
<InputEndpoint name="UdpEndpoint" protocol="udp" port="8080" localPort="8080" />
</Endpoints>
(from https://github.com/ytechie/LogHub/blob/master/Microsoft.LogHub.Cloud/ServiceDefinition.csdef)
And listening on the correct port was as easy as:
var endPoint = new IPEndPoint(IPAddress.Any, 0);
var udp = new UdpClient(8080);
(from https://github.com/ytechie/LogHub/blob/master/Microsoft.LogHub/Global.asax.cs)
Feel free to take a look at my implementation and look for differences. I did notice that you're using the synchronous version of "Receive", but that shouldn't matter.
I'm also curious if this is PaaS or IaaS. In either case, you'll need to hit against the load balanced endpoint, and not an internal endpoint which would be inaccessible from the internet.