A friend of mine came to me with a problem: when using the NetworkStream class on the server end of the connection, if the client disconnects, NetworkStream fails to detect it.
Stripped down, his C# code looked like this:
List<TcpClient> connections = new List<TcpClient>();
TcpListener listener = new TcpListener(7777);
listener.Start();
while(true)
{
if (listener.Pending())
{
connections.Add(listener.AcceptTcpClient());
}
TcpClient deadClient = null;
foreach (TcpClient client in connections)
{
if (!client.Connected)
{
deadClient = client;
break;
}
NetworkStream ns = client.GetStream();
if (ns.DataAvailable)
{
BinaryFormatter bf = new BinaryFormatter();
object o = bf.Deserialize(ns);
ReceiveMyObject(o);
}
}
if (deadClient != null)
{
deadClient.Close();
connections.Remove(deadClient);
}
Thread.Sleep(0);
}
The code works, in that clients can successfully connect and the server can read data sent to it. However, if the remote client calls tcpClient.Close(), the server does not detect the disconnection - client.Connected remains true, and ns.DataAvailable is false.
A search of Stack Overflow provided an answer - since Socket.Receive is not being called, the socket is not detecting the disconnection. Fair enough. We can work around that:
foreach (TcpClient client in connections)
{
client.ReceiveTimeout = 0;
if (client.Client.Poll(0, SelectMode.SelectRead))
{
int bytesPeeked = 0;
byte[] buffer = new byte[1];
bytesPeeked = client.Client.Receive(buffer, SocketFlags.Peek);
if (bytesPeeked == 0)
{
deadClient = client;
break;
}
else
{
NetworkStream ns = client.GetStream();
if (ns.DataAvailable)
{
BinaryFormatter bf = new BinaryFormatter();
object o = bf.Deserialize(ns);
ReceiveMyObject(o);
}
}
}
}
(I have left out exception handling code for brevity.)
This code works, however, I would not call this solution "elegant". The other elegant solution to the problem I am aware of is to spawn a thread per TcpClient, and allow the BinaryFormatter.Deserialize (née NetworkStream.Read) call to block, which would detect the disconnection correctly. Though, this does have the overhead of creating and maintaining a thread per client.
I get the feeling that I'm missing some secret, awesome answer that would retain the clarity of the original code, but avoid the use of additional threads to perform asynchronous reads. Though, perhaps, the NetworkStream class was never designed for this sort of usage. Can anyone shed some light?
Update: Just want to clarify that I'm interested to see if the .NET framework has a solution that covers this use of NetworkStream (i.e. polling and avoiding blocking) - obviously it can be done; the NetworkStream could easily be wrapped in a supporting class that provides the functionality. It just seemed strange that the framework essentially requires you to use threads to avoid blocking on NetworkStream.Read, or, to peek on the socket itself to check for disconnections - almost like it's a bug. Or a potential lack of a feature. ;)
Is the server expecting to be sent multiple objects over the same connection? IF so I dont see how this code will work, as there is no delimiter being sent that signifies where the first object starts and the next object ends.
If only one object is being sent and the connection closed after, then the original code would work.
There has to be a network operation initiated in order to find out if the connection is still active or not. What I would do, is that instead of deserializing directly from the network stream, I would instead buffer into a MemoryStream. That would allow me to detect when the connection was lost. I would also use message framing to delimit multiple responses on the stream.
MemoryStream ms = new MemoryStream();
NetworkStream ns = client.GetStream();
BinaryReader br = new BinaryReader(ns);
// message framing. First, read the #bytes to expect.
int objectSize = br.ReadInt32();
if (objectSize == 0)
break; // client disconnected
byte [] buffer = new byte[objectSize];
int index = 0;
int read = ns.Read(buffer, index, Math.Min(objectSize, 1024);
while (read > 0)
{
objectSize -= read;
index += read;
read = ns.Read(buffer, index, Math.Min(objectSize, 1024);
}
if (objectSize > 0)
{
// client aborted connection in the middle of stream;
break;
}
else
{
BinaryFormatter bf = new BinaryFormatter();
using(MemoryStream ms = new MemoryStream(buffer))
{
object o = bf.Deserialize(ns);
ReceiveMyObject(o);
}
}
Yeah but what if you lose a connection before getting the size? i.e. right before the following line:
// message framing. First, read the #bytes to expect.
int objectSize = br.ReadInt32();
ReadInt32() will block the thread indefinitely.
Related
I'm writing a server application in D, who should be able to manage n connections simultaneously.
To achieve this i am using std.socket.Socket.select. This works fine. But I can't bind session specific data to the socket and i don't see any way to do this, cause Socket does not allow to save a handle to user specific data. After
Socket.select(socketSet, null, null);
I'm able to get all affected sockets, but I can't assign this sockets to my user specific session data. What's my mistake? Is it possible to reach my goal in this way? Or should I choose another way for my requirements?
My relevant code:
ushort port = 5010;
stoprequest = false;
auto listener = new TcpSocket();
assert(listener.isAlive);
listener.blocking = false;
listener.bind(new InternetAddress(port));
listener.listen(10);
enum MAX_CONNECTIONS = 100;
auto socketSet = new SocketSet(MAX_CONNECTIONS + 1);
Socket[] reads;
Session[] sessions;
while (true)
{
socketSet.add(listener);
foreach (session; sessions)
socketSet.add(session.socket);
Socket.select(socketSet, null, null);
for (size_t i = 0; i < reads.length; i++)
{
if (socketSet.isSet(reads[i]))
{
// Now i should acces to session related data, but how?
char[1024] buf;
auto datLength = reads[i].receive(buf[]);
if (datLength == Socket.ERROR)
writeln("Connection error.");
else if (datLength != 0)
{
writefln("Received %d bytes from %s: \"%s\"", datLength, reads[i].remoteAddress().toString(), buf[0..datLength]);
continue;
}
else { // Error Handling. Shortened, since unimportant for the example}
reads[i].close();
reads = reads.remove(i);
i--;
}
}
if (socketSet.isSet(listener))
{
Socket sn = null;
sn = listener.accept();
if (reads.length < MAX_CONNECTIONS)
{
Session session = new Session();
session.socket = sn;
sessions ~= session;
}
else { // Error Handling for too many connection. Shortened, since unimportant for the example}}
}
socketSet.reset();
}
The hint to use poll() was helpful. After reading https://daniel.haxx.se/docs/poll-vs-select.html I think that both variants work and neither of them are the real thing. For an efficient way, I should better deal with libev. Fortunately, efficiency is not my problem in this particular project. For this reason I will use select(), because i found out, that accessing handle gives me a unique number which can be passed to a own lookup table. This allows me to assign session data to a socket. So I prefer to stick with the encapsulated functionality of std.socket.Socket and don't work around it.
My concrete question can therefore be answered with :
Use Socket.handle to identify the socket and manage session related
data
A few other alternatives you can consider:
1) use a subclass of Socket. You can make your own class that inherits from it and adds more stuff.
2) The poll function is found in import core.sys.posix.poll;, and you can pass socket.handle to that as well. But note it will not work on Windows without modification.
or indeed 3) do your own lookup table, that works too.
Note that the std.socket.Socket is a very thin wrapper around the bsd socket api, just internally it does conveniently handle the slight differences between Windows and posix. Still it is pretty easy to adapt code to use the other apis with it (or tutorials on C language stuff to D) since it is all basically the same thing - and literally the same functions if you import core.sys stuff.
I am having Problems useing the Indy TIdTCPClient.
I want to call a function, everytime if there is Data available on the socket. For this I have a Thread calling IdTCPClient->Socket->Readable(100).
The function itself looks like this:
TMemoryStream *mStream = new TMemoryStream;
int len = 0;
try
{
if(!Form1->IdTCPClient2->Connected())
Form1->IdTCPClient2->Connect();
mStream->Position = 0;
do
{
Form1->IdTCPClient2->Socket->ReadStream(mStream, 1);
}
while(Form1->IdTCPClient2->Socket->Readable(100));
len = mStream->Position;
mStream->Position = 0;
mStream->Read(Buffer, len);
}catch(Exception &Ex) {
Form1->DisplaySSH->Lines->Add(Ex.Message);
Form1->DisplaySSH->GoToTextEnd();
}
delete mStream;
It will not be called directly within the thread, but the thread triggers an event, which is calling this function. Which means I am using Readable(100) twice, without reading data in betwee.
So since I dont know how many bytes I have to read I thought I can read one byte, check if there is more available and then read another byte.
The Problem here is that the do while loop doesnt loop, it just runs once.
I am guessing that Readable does not quite wokt the way I need it to.
Is there any other way to receive all the bytes available in the Socket?
You should not be using Readable() directly in this situation. That call reports whether the underlying socket has pending unread data in its internal kernel buffer. That does not take into account that the TIdIOHandler may already have unread data in its InputBuffer that is left over from a previous read operation.
Use the TIdIOHandler::CheckForDataOnSource() method instead of TIdIOHandler::Readable():
TMemoryStream *mStream = new TMemoryStream;
try
{
if (!Form1->IdTCPClient2->Connected())
Form1->IdTCPClient2->Connect();
mStream->Position = 0;
do
{
if (Form1->IdTCPClient2->IOHander->InputBufferIsEmpty())
{
if (!Form1->IdTCPClient2->IOHander->CheckForDataOnSource(100))
break;
}
Form1->IdTCPClient2->IOHandler->ReadStream(mStream, Form1->IdTCPClient2->IOHandler->InputBuffer->Size, false);
/* alternatively:
Form1->IdTCPClient2->IOHandler->InputBuffer->ExtractToStream(mStream);
*/
}
while (true);
// use mStream as needed...
}
catch (const Exception &Ex) {
Form1->DisplaySSH->Lines->Add(Ex.Message);
Form1->DisplaySSH->GoToTextEnd();
}
delete mStream;
Or, you can alternatively use TIdIOHandler::ReadBytes() instead of TIdIOHandler::ReadStream(). If you set its AByteCount parameter to -1, it will return only the bytes that are currently available (if the InputBuffer is empty, ReadBytes() will wait up to the ReadTimeout interval for the socket to receive any new bytes) 1:
try
{
if (!Form1->IdTCPClient2->Connected())
Form1->IdTCPClient2->Connect();
TIdBytes data;
do
{
if (Form1->IdTCPClient2->IOHander->InputBufferIsEmpty())
{
if (!Form1->IdTCPClient2->IOHander->CheckForDataOnSource(100))
break;
}
Form1->IdTCPClient2->IOHandler->ReadBytes(data, -1, true);
/* alternatively:
Form1->IdTCPClient2->IOHandler->InputBuffer->ExtractToBytes(data, -1, true);
*/
}
while (true);
// use data as needed...
}
catch (const Exception &Ex) {
Form1->DisplaySSH->Lines->Add(Ex.Message);
Form1->DisplaySSH->GoToTextEnd();
}
1: make sure you are using an up-to-date snapshot of Indy 10. Prior to Oct 6 2016, there was a logic bug in ReadBytes() when AByteCount=-1 that didn't take the InputBuffer into account before checking the socket for new bytes.
I'll give you a little bit of context first:
I have a class that is supposed to write in a socket some querys and then read the answers.
Here it is the code that is supposed to read from the inputStream:
private String getDataFromInputStream() throws IOException
{
StringBuilder sb = new StringBuilder();
InputStream stream = this.socket.getInputStream();
byte[] buff = new byte[1024];
int bytesRead = 0;
while(bytesRead >= 0)
{
bytesRead = stream.read(buff);
if (bytesRead > 0) sb.append(new String(buff, 0, bytesRead));
}
return sb.toString();
}
If the returned result is OK, this method works great, but if, for some reason, the InputStream.read keeps waiting for input it, of course, blocks.
So I changed that code to this one:
private String getDataFromInputStream() throws IOException
{
StringBuilder sb = new StringBuilder();
InputStream stream = this.socket.getInputStream();
byte[] buff = new byte[1024];
int bytesRead = 0;
while(stream.available() > 0)
{
bytesRead = stream.read(buff);
if (bytesRead > 0) sb.append(new String(buff, 0, bytesRead));
}
return sb.toString();
}
But here's the new problem: The second piece of code doesn't read the whole response. It just returns a fragment of the full response.
I know that the available() method returns 0 because the stream doesn't have more data to be read inside the buffer but there is more data to be read from the socket.
So how can I make a method flexible enought to read the whole thing even if it takes some time to the Stream to buffer the response and, also doesn't block when expecting input?
I need to get the whole response if the query succeeded and to close the input so it won't block, if the command failed and the socket is expecting any input.
Since you're using a socket, you'll need the source to tell you how much data it wants to send. That may be present in the first few bytes that is sent (in which case you wait for that specific number of bytes first) otherwise you'll have to modify the source to provide that information. This is what TCP and UDP packets do as well, where the header to the packets include the length of the data so that the client knows how many bytes to wait for.
I am trying to implement Facebook X_FACEBOOK_PLATFORM SASL mechanism so I could integrate Facebook Chat to my application over XMPP.
Here is the code:
var ak = "my app id";
var sk = "access token";
var aps = "my app secret";
using (var client = new TcpClient())
{
client.Connect("chat.facebook.com", 5222);
using (var writer = new StreamWriter(client.GetStream())) using (var reader = new StreamReader(client.GetStream()))
{
// Write for the first time
writer.Write("<stream:stream xmlns=\"jabber:client\" xmlns:stream=\"http://etherx.jabber.org/streams\" version=\"1.0\" to=\"chat.facebook.com\"><auth xmlns=\"urn:ietf:params:xml:ns:xmpp-sasl\" mechanism=\"X-FACEBOOK-PLATFORM\" /></stream:stream>");
writer.Flush();
Thread.Sleep(500);
// I am pretty sure following works or at least it's not what causes the error
var challenge = Encoding.UTF8.GetString(Convert.FromBase64String(XElement.Parse(reader.ReadToEnd()).Elements().Last().Value)).Split('&').Select(s => s.Split('=')).ToDictionary(s => s[0], s => s[1]);
var response = new SortedDictionary<string, string>() { { "api_key", ak }, { "call_id", DateTime.Now.Ticks.ToString() }, { "method", challenge["method"] }, { "nonce", challenge["nonce"] }, { "session_key", sk }, { "v", "1.0" } };
var responseString1 = string.Format("{0}{1}", string.Join(string.Empty, response.Select(p => string.Format("{0}={1}", p.Key, p.Value)).ToArray()), aps);
byte[] hashedResponse1 = null;
using (var prov = new MD5CryptoServiceProvider()) hashedResponse1 = prov.ComputeHash(Encoding.UTF8.GetBytes(responseString1));
var builder = new StringBuilder();
foreach (var item in hashedResponse1) builder.Append(item.ToString("x2"));
var responseString2 = Convert.ToBase64String(Encoding.UTF8.GetBytes(string.Format("{0}&sig={1}", string.Join("&", response.Select(p => string.Format("{0}={1}", p.Key, p.Value)).ToArray()), builder.ToString().ToLower()))); ;
// Write for the second time
writer.Write(string.Format("<response xmlns=\"urn:ietf:params:xml:ns:xmpp-sasl\">{0}</response>", responseString2));
writer.Flush();
Thread.Sleep(500);
MessageBox.Show(reader.ReadToEnd());
}
}
I shortened and shrunk the code as much as possible, because I think my SASL implementation (whether it works or not, I haven't had a chance to test it yet) is not what causes the error.
I get the following exception thrown at my face: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine.
10053
System.Net.Sockets.SocketError.ConnectionAborted
It happens every time I try to read from client's stream for the second time. As you can see i pause a thread here so Facebook server has enough time to answer me, but I used asynchronous approach before and I encountered the exact same thing, so I decided to try it synchronously first. Anyway actual SASL mechanism implementation really shouldn't cause this because if I don't try to authenticate right away, but I send the request to see what mechanisms server uses and select that mechanism in another round of reading and writing, it fails, but when I send mechanism selection XML right away, it works and fails on whatever second I send.
So the conclusion is following: I open the socket connection, write to it, read from it (first read works both sync and async), write to it for the second time and try to read from it for the second time and here it always fails. Clearly then, problem is with socket connection itself. I tried to use new StreamReader for second read but to no avail. This is rather unpleasant since I would really like to implement facade over NetworkStream with "Received" event or something like Send(string data, Action<string> responseProcessor) to get some comfort working with that stream, and I already had the implementation, but it also failed on second read.
Thanks for your suggestions.
Edit: Here is the code of facade over NetworkStream. Same thing happens when using this asynchronous approach, but couple of hours ago it worked, but for second response returned same string as for first. I can't figute out what I changed in a meantime and how.
public void Send(XElement fragment)
{
if (Sent != null) Sent(this, new XmppEventArgs(fragment));
byte[] buffer = new byte[1024];
AsyncCallback callback = null;
callback = (a) =>
{
var available = NetworkStream.EndRead(a);
if (available > 0)
{
StringBuilder.Append(Encoding.UTF8.GetString(buffer, 0, available));
NetworkStream.BeginRead(buffer, 0, buffer.Length, callback, buffer);
}
else
{
var args = new XmppEventArgs(XElement.Parse(StringBuilder.ToString()));
if (Received != null) Received(this, args);
StringBuilder = new StringBuilder();
// NetworkStream.BeginRead(buffer, 0, buffer.Length, callback, buffer);
}
};
NetworkStream.BeginRead(buffer, 0, buffer.Length, callback, buffer);
NetworkStreamWriter.Write(fragment);
NetworkStreamWriter.Flush();
}
The reader.ReadToEnd() call consumes everything until end-of-stream, i.e. until TCP connection is closed.
I am learning android but I can't get past the InputStream.read().
This is just a socket test - the server sends back two bytes when it receives a connection and I know that this working fine. All I want to do is read these values. The b = data.read reads both values in turn but then hangs, it never returns the -1 value which is what expect it to. Also it does not throw an exception.
Any ideas?
Thanks.
protected void startLongRunningOperation() {
// Fire off a thread to do some work that we shouldn't do directly in the UI thread
Thread t = new Thread() {
public void run() {
try {
Log.d("Socket", "try connect ");
Socket sock = new Socket("192.168.0.12", 5001);
Log.d("socket", "connected");
InputStream data = sock.getInputStream();
int b = 0;
while (b != -1) {
b = data.read();
}
data.close();
} catch (Exception e) {
Log.d("Socket", e.toString());
}
}
};
t.start();
}
Reaching the end of the stream is a special state. It doesn't happen just because there is nothing left to read. If the stream is still open, but there's nothing to be read, it will "hang" (or block) as you've noticed until a byte comes across.
To do what you want, the server either needs to close/end the stream, or you need to use:
while (data.available() > 0) {
..
When the number of available bytes is zero, there's nothing sitting in the stream buffer to be read.
On the other hand, if you know that there should only ever be two bytes to read, and that's the end of your data, then just read the two bytes and move on (i.e. don't use a while loop). The reason to use a while loop here would only be if you weren't sure how many total bytes to expect.