I am learning android but I can't get past the InputStream.read().
This is just a socket test - the server sends back two bytes when it receives a connection and I know that this working fine. All I want to do is read these values. The b = data.read reads both values in turn but then hangs, it never returns the -1 value which is what expect it to. Also it does not throw an exception.
Any ideas?
Thanks.
protected void startLongRunningOperation() {
// Fire off a thread to do some work that we shouldn't do directly in the UI thread
Thread t = new Thread() {
public void run() {
try {
Log.d("Socket", "try connect ");
Socket sock = new Socket("192.168.0.12", 5001);
Log.d("socket", "connected");
InputStream data = sock.getInputStream();
int b = 0;
while (b != -1) {
b = data.read();
}
data.close();
} catch (Exception e) {
Log.d("Socket", e.toString());
}
}
};
t.start();
}
Reaching the end of the stream is a special state. It doesn't happen just because there is nothing left to read. If the stream is still open, but there's nothing to be read, it will "hang" (or block) as you've noticed until a byte comes across.
To do what you want, the server either needs to close/end the stream, or you need to use:
while (data.available() > 0) {
..
When the number of available bytes is zero, there's nothing sitting in the stream buffer to be read.
On the other hand, if you know that there should only ever be two bytes to read, and that's the end of your data, then just read the two bytes and move on (i.e. don't use a while loop). The reason to use a while loop here would only be if you weren't sure how many total bytes to expect.
Related
I am having Problems useing the Indy TIdTCPClient.
I want to call a function, everytime if there is Data available on the socket. For this I have a Thread calling IdTCPClient->Socket->Readable(100).
The function itself looks like this:
TMemoryStream *mStream = new TMemoryStream;
int len = 0;
try
{
if(!Form1->IdTCPClient2->Connected())
Form1->IdTCPClient2->Connect();
mStream->Position = 0;
do
{
Form1->IdTCPClient2->Socket->ReadStream(mStream, 1);
}
while(Form1->IdTCPClient2->Socket->Readable(100));
len = mStream->Position;
mStream->Position = 0;
mStream->Read(Buffer, len);
}catch(Exception &Ex) {
Form1->DisplaySSH->Lines->Add(Ex.Message);
Form1->DisplaySSH->GoToTextEnd();
}
delete mStream;
It will not be called directly within the thread, but the thread triggers an event, which is calling this function. Which means I am using Readable(100) twice, without reading data in betwee.
So since I dont know how many bytes I have to read I thought I can read one byte, check if there is more available and then read another byte.
The Problem here is that the do while loop doesnt loop, it just runs once.
I am guessing that Readable does not quite wokt the way I need it to.
Is there any other way to receive all the bytes available in the Socket?
You should not be using Readable() directly in this situation. That call reports whether the underlying socket has pending unread data in its internal kernel buffer. That does not take into account that the TIdIOHandler may already have unread data in its InputBuffer that is left over from a previous read operation.
Use the TIdIOHandler::CheckForDataOnSource() method instead of TIdIOHandler::Readable():
TMemoryStream *mStream = new TMemoryStream;
try
{
if (!Form1->IdTCPClient2->Connected())
Form1->IdTCPClient2->Connect();
mStream->Position = 0;
do
{
if (Form1->IdTCPClient2->IOHander->InputBufferIsEmpty())
{
if (!Form1->IdTCPClient2->IOHander->CheckForDataOnSource(100))
break;
}
Form1->IdTCPClient2->IOHandler->ReadStream(mStream, Form1->IdTCPClient2->IOHandler->InputBuffer->Size, false);
/* alternatively:
Form1->IdTCPClient2->IOHandler->InputBuffer->ExtractToStream(mStream);
*/
}
while (true);
// use mStream as needed...
}
catch (const Exception &Ex) {
Form1->DisplaySSH->Lines->Add(Ex.Message);
Form1->DisplaySSH->GoToTextEnd();
}
delete mStream;
Or, you can alternatively use TIdIOHandler::ReadBytes() instead of TIdIOHandler::ReadStream(). If you set its AByteCount parameter to -1, it will return only the bytes that are currently available (if the InputBuffer is empty, ReadBytes() will wait up to the ReadTimeout interval for the socket to receive any new bytes) 1:
try
{
if (!Form1->IdTCPClient2->Connected())
Form1->IdTCPClient2->Connect();
TIdBytes data;
do
{
if (Form1->IdTCPClient2->IOHander->InputBufferIsEmpty())
{
if (!Form1->IdTCPClient2->IOHander->CheckForDataOnSource(100))
break;
}
Form1->IdTCPClient2->IOHandler->ReadBytes(data, -1, true);
/* alternatively:
Form1->IdTCPClient2->IOHandler->InputBuffer->ExtractToBytes(data, -1, true);
*/
}
while (true);
// use data as needed...
}
catch (const Exception &Ex) {
Form1->DisplaySSH->Lines->Add(Ex.Message);
Form1->DisplaySSH->GoToTextEnd();
}
1: make sure you are using an up-to-date snapshot of Indy 10. Prior to Oct 6 2016, there was a logic bug in ReadBytes() when AByteCount=-1 that didn't take the InputBuffer into account before checking the socket for new bytes.
Okay this is my first question here on Stack Overflow, so bare over with it if I'm not asking properly.
Basically I'm trying to code some asynchronous sockets using std.socket, but I'm not sure if I've understood the concept correct. I've only ever worked with asynchronous sockets in C# and in D it seem to be on a much lower level. I've researched a lot and looked up a lot of code, documentation etc. both for D and C/C++ to get an understanding, however I'm not sure if I understand the concept correctly and if any of you have some examples. I tried looking at splat, but it's very outdated and vibe seems to be too complex just for a simple asynchronous socket wrapper.
If I understood correctly there is no poll() function in std.socket so you'd have to use SocketSet with a single socket on select() to poll the status of the socket right?
So basically how I'd go about handling the sockets is polling to get the read status of the socket and if it has a success (value > 0) then I can call receive() which will return 0 for disconnection else the received value, but I'd have to keep doing this until the expected bytes are received.
Of course the socket is set to nonblocked!
Is that correct?
Here is the code I've made up so far.
void HANDLE_READ()
{
while (true)
{
synchronized
{
auto events = cast(AsyncObject[int])ASYNC_EVENTS_READ;
foreach (asyncObject; events)
{
int poll = pollRecv(asyncObject.socket.m_socket);
switch (poll)
{
case 0:
{
throw new SocketException("The socket had a time out!");
continue;
}
default:
{
if (poll <= -1)
{
throw new SocketException("The socket was interrupted!");
continue;
}
int recvGetSize = (asyncObject.socket.m_readBuffer.length - asyncObject.socket.readSize);
ubyte[] recvBuffer = new ubyte[recvGetSize];
int recv = asyncObject.socket.m_socket.receive(recvBuffer);
if (recv == 0)
{
removeAsyncObject(asyncObject.event_id, true);
asyncObject.socket.disconnect();
continue;
}
asyncObject.socket.m_readBuffer ~= recvBuffer;
asyncObject.socket.readSize += recv;
if (asyncObject.socket.readSize == asyncObject.socket.expectedReadSize)
{
removeAsyncObject(asyncObject.event_id, true);
asyncObject.event(asyncObject.socket);
}
break;
}
}
}
}
}
}
So basically how I'd go about handling the sockets is polling to get the read status of the socket
Not quite right. Usually, the idea is to build an event loop around select, so that your application is idle as long as there are no network or timer events that need to be handled. With polling, you'd have to check for new events continuously or on a timer, which leads to wasted CPU cycles, and events getting handled a bit later than they occur.
In the event loop, you populate the SocketSets with sockets whose events you are interested in. If you want to be notified of new received data on a socket, it goes to the "readable" set. If you have data to send, the socket should be in the "writable" set. And all sockets should be on the "error" set.
select will then block (sleep) until an event comes in, and fill the SocketSets with the sockets which have actionable events. Your application can then respond to them appropriately: receive data for readable sockets, send queued data for writable sockets, and perform cleanup for errored sockets.
Here's my D implementation of non-fiber event-based networking: ae.net.asockets.
I have implemented a TCP server using boost::asio. This server uses basic_stream_socket::read_some function to read data. I know that read_some does not guarantee that supplied buffer will be full before it returns.
In my project I am sending strings separated by a delimiter(if that matters). At client side I am using WinSock::send() function to send data. Now my problem is on server side I am not able to get all the strings which were sent from client side. My suspect is that read_some is receiving some data and discarding leftover data for some reason. Than again in next call its receiving another string.
Is it really possible in TCP/IP ?
I tried to use async_receive but that is eating up all my CPU, also since buffer has to be cleaned up by callback function its causing serious memory leak in my program. (I am using IoService::poll() to call handler. That handler is getting called at a very slow rate compared to calling rate of async_read()).
Again I tried to use free function read but that will not solve my purpose as it blocks for too much time with the buffer size I am supplying.
My previous implementation of the server was with WinSock API where I was able to receive all data using WinSock::recv().
Please give me some leads so that I can receive complete data using boost::asio.
here is my server side thread loop
void
TCPObject::receive()
{
if (!_asyncModeEnabled)
{
std::string recvString;
if ( !_tcpSocket->receiveData( _maxBufferSize, recvString ) )
{
LOG_ERROR("Error Occurred while receiving data on socket.");
}
else
_parseAndPopulateQueue ( recvString );
}
else
{
if ( !_tcpSocket->receiveDataAsync( _maxBufferSize ) )
{
LOG_ERROR("Error Occurred while receiving data on socket.");
}
}
}
receiveData() in TCPSocket
bool
TCPSocket::receiveData( unsigned int bufferSize, std::string& dataString )
{
boost::system::error_code error;
char *buf = new char[bufferSize + 1];
size_t len = _tcpSocket->read_some( boost::asio::buffer((void*)buf, bufferSize), error);
if(error)
{
LOG_ERROR("Error in receiving data.");
LOG_ERROR( error.message() );
_tcpSocket->close();
delete [] buf;
return false;
}
buf[len] ='\0';
dataString.insert( 0, buf );
delete [] buf;
return true;
}
receiveDataAsync in TCP Socket
bool
TCPSocket::receiveDataAsync( unsigned int bufferSize )
{
char *buf = new char[bufferSize + 1];
try
{
_tcpSocket->async_read_some( boost::asio::buffer( (void*)buf, bufferSize ),
boost::bind(&TCPSocket::_handleAsyncReceive,
this,
buf,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred) );
//! Asks io_service to execute callback
_ioService->poll();
}
catch (std::exception& e)
{
LOG_ERROR("Error Receiving Data Asynchronously");
LOG_ERROR( e.what() );
delete [] buf;
return false;
}
//we dont delete buf here as it will be deleted by callback _handleAsyncReceive
return true;
}
Asynch Receive handler
void
TCPSocket::_handleAsyncReceive(char *buf, const boost::system::error_code& ec, size_t size)
{
if(ec)
{
LOG_ERROR ("Error occurred while sending data Asynchronously.");
LOG_ERROR ( ec.message() );
}
else if ( size > 0 )
{
buf[size] = '\0';
emit _asyncDataReceivedSignal( QString::fromLocal8Bit( buf ) );
}
delete [] buf;
}
Client Side sendData function.
sendData(std::string data)
{
if(!_connected)
{
return;
}
const char *pBuffer = data.c_str();
int bytes = data.length() + 1;
int i = 0,j;
while (i < bytes)
{
j = send(_connectSocket, pBuffer+i, bytes-i, 0);
if(j == SOCKET_ERROR)
{
_connected = false;
if(!_bNetworkErrNotified)
{
_bNetworkErrNotified=true;
emit networkErrorSignal(j);
}
LOG_ERROR( "Unable to send Network Packet" );
break;
}
i += j;
}
}
Boost.Asio's TCP capabilities are pretty well used, so I would be hesitant to suspect it is the source of the problem. In most cases of data loss, the problem is the result of application code.
In this case, there is a problem in the receiver code. The sender is delimiting strings with \0. However, the receiver fails to proper handle the delimiter in cases where multiple strings are read in a single read operation, as string::insert() will cause truncation of the char* when it reaches the first delimiter.
For example, the sender writes two strings "Test string\0" and "Another test string\0". In TCPSocket::receiveData(), the receiver reads "Test string\0Another test string\0" into buf. dataString is then populated with dataString.insert(0, buf). This particular overload will copy up to the delimiter, so dataString will contain "Test string". To resolve this, consider using the string::insert() overload that takes the number of characters to insert: dataString.insert(0, buf, len).
I have not used the poll function before. What I did is create a worker thread that is dedicated to processing ASIO handlers with the run function, which blocks. The Boost documentation says that each thread that is to be made available to process async event handlers must first call the io_service:run or io_service:poll method. I'm not sure what else you are doing with the thread that calls poll.
So, I would suggest dedicating at least one worker thread for the async ASIO event handlers and use run instead of poll. If you want that worker thread to continue to process all async messages without returning and exiting, then add a work object to the io_service object. See this link for an example.
The following code listens for an incoming sms, takes all the spaces out of the sms then emails the edited sms. Everything works fine, except that the app fails to send an email. Can anyone see what I am doing wrong and help me?
new Thread() {
public void run() {
try {
DatagramConnection _dc =
(DatagramConnection)Connector.open("sms://");
for(;;) { //'For-Loop' used to listen continously for incoming sms's
Datagram d = _dc.newDatagram(_dc.getMaximumLength());
_dc.receive(d); //The sms is received
byte[] bytes = d.getData();
String address = d.getAddress(); //The address of the sms is put on a string.
String msg = new String(bytes); //The body of the sms is put on a string.
String msg2 = (replaceAll(msg, " ","")) ; //
Store store = Session.getDefaultInstance().getStore();
Folder[] folders = store.list(Folder.SENT);
Folder sentfolder = folders[0]; //Retrieve the sent folder
Message in = new Message(sentfolder);
Address recipients[] = new Address[1];
recipients[0]= new Address("me#yahoo.com", "user");
in.addRecipients(Message.RecipientType.TO, recipients);
in.setSubject("Incoming SMS"); //The subject of the message is added
in.setContent("You have just received an SMS from: " + address + "/n" + "Message: " + msg2); //Here the body of the message is formed
in.setPriority(Message.Priority.HIGH); //The priority of the message is set.
Transport.send(in); //The message is sent
in.setFlag(Message.Flag.OPENED, true);
Folder folder = in.getFolder(); //The message is deleted from the sent folder
folder.deleteMessage(in);
}
}catch (Exception me) { //All Exceptions are caught
}
}
};
public static String replaceAll(String front, String pattern, String back) {
if (front == null)
return "";
StringBuffer sb = new StringBuffer(); //A StringBufffer is created
int idx = -1;
int patIdx = 0;
while ((idx = front.indexOf(pattern, patIdx)) != -1) {
sb.append(front.substring(patIdx, idx));
sb.append(back);
patIdx = idx + pattern.length();
}
sb.append(front.substring(patIdx));
return sb.toString();
}
Thanks
This isn't really an answer to the problem, just an elaboration on my comment above, that might help.
Make sure do something in your exception catch block, so that problems in the code don't go unnoticed. It's possible that your code is not encountering any exceptions, but in order for us to help, we need to try to eliminate potential problems, and since you say the code isn't working, but you have an empty exception handler, that's an easy area to fix first.
the simplest handler is just:
try {
// try sending sms here
} catch (Exception e) {
e.printStackTrace();
}
If you can run this in the debugger (which I highly suggest), then you can now put a breakpoint on the e.printStackTrace() line, and see if it ever gets hit. If it does, inspect the value of e and tell us what it is.
Normally, in my programs, I don't actually use e.printStackTrace() in catch handlers, but I have a logging class that takes strings, and maybe a log level (e.g. info, warning, error, verbose), and writes to a log file. The log file can be attached to emails the users send to tech support, or can be disabled for production if you only want to use the feature while developing.
Anyway, start with a simple printStackTrace() and see if it ever gets hit. Then, report back.
Edit: from the symptoms you describe in the comments after your question, it seems like it's a possibility that
String msg2 = (replaceAll(msg, " ","")) ; //
is throwing an exception, and therefore never letting you get to where you'd send the email. I can't see anything wrong with your implementation of replaceAll() upon initial inspection, but that might be a place to look. Has that implementation been thoroughly unit-tested?
Also, I think you have a "/n" in your code where you probably want a "\n", right?
A friend of mine came to me with a problem: when using the NetworkStream class on the server end of the connection, if the client disconnects, NetworkStream fails to detect it.
Stripped down, his C# code looked like this:
List<TcpClient> connections = new List<TcpClient>();
TcpListener listener = new TcpListener(7777);
listener.Start();
while(true)
{
if (listener.Pending())
{
connections.Add(listener.AcceptTcpClient());
}
TcpClient deadClient = null;
foreach (TcpClient client in connections)
{
if (!client.Connected)
{
deadClient = client;
break;
}
NetworkStream ns = client.GetStream();
if (ns.DataAvailable)
{
BinaryFormatter bf = new BinaryFormatter();
object o = bf.Deserialize(ns);
ReceiveMyObject(o);
}
}
if (deadClient != null)
{
deadClient.Close();
connections.Remove(deadClient);
}
Thread.Sleep(0);
}
The code works, in that clients can successfully connect and the server can read data sent to it. However, if the remote client calls tcpClient.Close(), the server does not detect the disconnection - client.Connected remains true, and ns.DataAvailable is false.
A search of Stack Overflow provided an answer - since Socket.Receive is not being called, the socket is not detecting the disconnection. Fair enough. We can work around that:
foreach (TcpClient client in connections)
{
client.ReceiveTimeout = 0;
if (client.Client.Poll(0, SelectMode.SelectRead))
{
int bytesPeeked = 0;
byte[] buffer = new byte[1];
bytesPeeked = client.Client.Receive(buffer, SocketFlags.Peek);
if (bytesPeeked == 0)
{
deadClient = client;
break;
}
else
{
NetworkStream ns = client.GetStream();
if (ns.DataAvailable)
{
BinaryFormatter bf = new BinaryFormatter();
object o = bf.Deserialize(ns);
ReceiveMyObject(o);
}
}
}
}
(I have left out exception handling code for brevity.)
This code works, however, I would not call this solution "elegant". The other elegant solution to the problem I am aware of is to spawn a thread per TcpClient, and allow the BinaryFormatter.Deserialize (née NetworkStream.Read) call to block, which would detect the disconnection correctly. Though, this does have the overhead of creating and maintaining a thread per client.
I get the feeling that I'm missing some secret, awesome answer that would retain the clarity of the original code, but avoid the use of additional threads to perform asynchronous reads. Though, perhaps, the NetworkStream class was never designed for this sort of usage. Can anyone shed some light?
Update: Just want to clarify that I'm interested to see if the .NET framework has a solution that covers this use of NetworkStream (i.e. polling and avoiding blocking) - obviously it can be done; the NetworkStream could easily be wrapped in a supporting class that provides the functionality. It just seemed strange that the framework essentially requires you to use threads to avoid blocking on NetworkStream.Read, or, to peek on the socket itself to check for disconnections - almost like it's a bug. Or a potential lack of a feature. ;)
Is the server expecting to be sent multiple objects over the same connection? IF so I dont see how this code will work, as there is no delimiter being sent that signifies where the first object starts and the next object ends.
If only one object is being sent and the connection closed after, then the original code would work.
There has to be a network operation initiated in order to find out if the connection is still active or not. What I would do, is that instead of deserializing directly from the network stream, I would instead buffer into a MemoryStream. That would allow me to detect when the connection was lost. I would also use message framing to delimit multiple responses on the stream.
MemoryStream ms = new MemoryStream();
NetworkStream ns = client.GetStream();
BinaryReader br = new BinaryReader(ns);
// message framing. First, read the #bytes to expect.
int objectSize = br.ReadInt32();
if (objectSize == 0)
break; // client disconnected
byte [] buffer = new byte[objectSize];
int index = 0;
int read = ns.Read(buffer, index, Math.Min(objectSize, 1024);
while (read > 0)
{
objectSize -= read;
index += read;
read = ns.Read(buffer, index, Math.Min(objectSize, 1024);
}
if (objectSize > 0)
{
// client aborted connection in the middle of stream;
break;
}
else
{
BinaryFormatter bf = new BinaryFormatter();
using(MemoryStream ms = new MemoryStream(buffer))
{
object o = bf.Deserialize(ns);
ReceiveMyObject(o);
}
}
Yeah but what if you lose a connection before getting the size? i.e. right before the following line:
// message framing. First, read the #bytes to expect.
int objectSize = br.ReadInt32();
ReadInt32() will block the thread indefinitely.