How can I determine what the numeric timeout value is that is causing the below stack trace?
...
File "/usr/lib/python2.7/httplib.py", line 548, in read
s = self._safe_read(self.length)
File "/usr/lib/python2.7/httplib.py", line 647, in _safe_read
chunk = self.fp.read(min(amt, MAXAMOUNT))
File "/usr/lib/python2.7/socket.py", line 380, in read
data = self._sock.recv(left)
timeout: timed out
After importing my modules, the result of socket.getdefaulttimeout() is None (note that this isn't the same situation as what produced the above, since getting those requires an 8-hour stress run on the system).
My code is not setting any timeout values (default or otherwise) AFAICT. I have not yet been able to find any hint that 3rd party libraries are doing so either.
Obviously there's some timeout somewhere in the system. I want to know the numeric value, so that I can have the system back off as it is approached.
This is python 2.7 under ubuntu 12.04.
Edit:
The connection is to localhost (talking to CouchDB listening on 127.0.0.1), so NAT shouldn't be an issue. The timeout only occurs when the DB is under heavy load, which implies to me that it's only when the DB is getting backed up and cannot respond quickly enough to requests, which is why I would like to know what the timeout is, so I can track response times and throttle incoming requests when the response time gets over something like 50% of the timeout.
Not knowing anything more my guess would be that NAT tracking expires due to long inactivity and unfortunately in most cases you won't be able to discover exact timeout value. Workaround would be to introduce some sort of keep alive packets to your protocol if there's such a possibility.
Related
I'm trying to create a server that sets up a Unix socket and listens for clients which send/receive data. I've made a small repository to recreate the problem.
The server runs and it can receive data from the clients that connect, but I can't get the server response to be read from the client without an error on the server.
I have commented out the offending code on the client and server. Uncomment both to recreate the problem.
When the code to respond to the client is uncommented, I get this error on the server:
thread '' panicked at 'called Result::unwrap() on an Err value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', src/main.rs:77:42
MRE Link
Your code calls set_read_timeout to set the timeout on the socket. Its documentation states that on Unix it results in a WouldBlock error in case of timeout, which is precisely what happens to you.
As to why your client times out, the likely reason is that the server calls stream.read_to_string(&mut response), which reads the stream until end-of-file. On the other hand, your client calls write_all() followed by flush(), and (after uncommenting the offending code) attempts to read the response. But the attempt to read the response means that the stream is not closed, so the server will wait for EOF, and you have a deadlock on your hands. Note that none of this is specific to Rust; you would have the exact same issue in C++ or Python.
To fix the issue, you need to use a protocol in your communication. A very simple protocol could consist of first sending the message size (in a fixed format, perhaps 4 bytes in length) and only then the actual message. The code that reads from the stream would do the same: first read the message size and then the message itself. Even better than inventing your own protocol would be to use an existing one, e.g. to exchange messages using serde.
Is it possible to implement the equivalent of Socket.Poll in async/await paradigm (or BeginXXX/EndXXX async pattern)?
A method which would act like NetworkStream.ReadAsync or Socket.BeginReceive but:
leave the data in the socket buffer
complete after the specified interval of time if no data arrived (leaving the socket in connected state so that the polling operation can be retried)
I need to implement IMAP IDLE so that the client connects to the mail server and then goes into waiting state where it received data from the server. If the server does not send anything within 10 minutes, the code sends ping to the server (without reconnecting, the connection is never closed), and starts waiting for data again.
In my tests, leaving the data in the buffer seems to be possible if I tell Socket.BeginReceive method to read no more than 0 bytes, e.g.:
sock.BeginReceive(b, 0, 0, SocketFlags.None, null, null)
However, not sure if it indeed will work in all cases, maybe I'm missing something. For instance, if the remote server closes the connection, it may send a zero-byte packet and not sure if Socket.BeginReceive will act identically to Socket.Poll in this case or not.
And the main problem is how to stop socket.BeginReceive without closing the socket.
I'm trying to run the following client and server socket example code in matlab:
http://www.mathworks.com/help/instrument/using-tcpip-server-sockets.html
This is my code.
Server:
t=tcpip('0.0.0.0', 9994, 'NetworkRole', 'server');
fopen(t);
data=fread(t, t.BytesAvailable, 'double');
plot(data);
Client:
data=sin(1:64);
t=tcpip('localhost', 9994, 'NetworkRole', 'client');
fopen(t);
fwrite(t, data, 'double');
This is what happens: I run the server code-> The program waits for the connection from the client-> I run the client code ->In the server console I get:
Error using icinterface/fread (line 163)
SIZE must be greater than 0.
Error in socketTentativaMatlab (line 3)
data=fread(t, t.BytesAvailable, 'double');
What am I doing wrong? It looks like the server doesn't wait for the client to send anything to try to read the data, so there's no data to read (It waits for the client connection thought).
Edit1:
Ok, I'm sending chars now, so we know for sure that t.BytesAvaiable = number of elements.
I have been able to successfully receive synchronously in the following way (this is server code, client code is the same but I send chars now and pause 1 second after establishing the connection with the server):
t=tcpip('0.0.0.0', 30000, 'NetworkRole', 'server');
fopen(t);
data=strcat(fread(t, 1, 'uint8')');
if get(t,'BytesAvailable') > 1
data=strcat(data,fread(t, t.BytesAvailable, 'uint8')');
end
data
This is because I suspected that bytesAvaiable is the number of bytes left to read after attempting to read at least once... this doesn't seem very logical, but it apparently is what happens. Since I have to read at least once to know how many bytes the message has...I choose to read 1 byte only the first time. I then read what's left, if there is something left...
I can make this work between matlab processes, but I can't do it between C++ and matlab. The C++ client successfully connects to the matlab server, and can send the data without problems or errors. However, on the matlab server side, I can't read it.
Something seems very wrong with all this matlab tcpip implementation!
Edit2:
If I properly close all the sockets in both client and server (basically don't let the program exit with open sockets), the above code seams to work consistently. I went to console and typed "netstat" to see all the connections ...It turns out since I was leaving open sockets, some connections were in the FIN_WAIT_2 state, which apparently rendered the ports of those connections unusable. Eventually the connection times out definitely, but it takes a minute or more, so, it's really best practice to just make sure the sockets are always properly closed.
I don't understand thought what is the logic behind t.BytesAvaiable... it doesn't seam to make much sense the way it is. If I loop and wait for it to become greater then 0, it eventually happens, but this is not the way things are supposed to be with synchronous sockets. My code lets one do things synchronously, even though I don't understand why t.BytesAvaiable isn't properly set the first time.
Final server code:
t=tcpip('0.0.0.0', 30000, 'NetworkRole', 'server');
fopen(t);
data=strcat(fread(t, 1, 'uint8'));
if get(t,'BytesAvailable') > 1
data=strcat(data,fread(t, t.BytesAvailable, 'uint8')');
end
fclose(t);
Final client code:
Your typical socket client, implemented in any language, but you will have to make sure that between successive calls of send() method/function (or between calling connect() and send()), at least 100ms (lower number seam to be risky) are ellapsed.
You are right, the server doesn't appear to be waiting for the client, even though the default mode of communication is synchronous. You can implement the waiting yourself, for example by inserting
while t.BytesAvailable == 0
pause(1)
end
before the read.
However, I've found that there are more problems – it's weird that the code from the MathWorks site is so bad – namely, t.BytesAvailable gives a number of bytes, while fread expects a number of values, and since one double value needs 8 bytes it has to say
data=fread(t, floor(t.BytesAvailable / 8), 'double');
Moreover, if the client writes the data immediately after opening the connection, I've found that the server simply overlooks them. I was able to fix this by inserting a pause(1) in the client code, like this
data=sin(1:64);
t=tcpip('localhost', 9994, 'NetworkRole', 'client');
fopen(t);
pause(1)
fwrite(t, data, 'double');
My impression is that Matlab's implementation of TCP/IP server client communication is quite fragile and needs a lot of workarounds...
I am trying to set a timeout on write operations when using GCDAsyncSocket. The code is pretty simple and is the following.
[iAsyncSocket writeData:bytesToSend withTimeout:3.0 tag:0];
Then I disable the Internet connection on my Mac and wait for write timeout to occur, but nothing happens. I don't get a disconnection with a GCDAsyncSocketWriteTimeoutError error as I should.
I have also validated that my server stops, as expected, receiving the messages after I turn off the Internet connection.
I have looked inside the source code and I have found out that the writeTimer, that is responsible for firing a write timeout event, is always cancelled (function endCurrentWrite is called). Tracing back to where the timer is cancelled, I ended up at the following line of code.
ssize_t result = write(socketFD, buffer, (size_t)bytesToWrite);
The write system call always returns the total number of bytes that I am sending, as if the socket manages to send the data although there is no Internet connection. Is this logical?
Has anyone come up with the same problem or seen similar behaviour? Or has anyone managed to set a write timeout for a GCDAsyncSocket?
Thanks a lot.
I'm writing an internal service that needs to touch a mod_perl2 instance for a long-running-process. The job is fired from a HTTP POST, and them mod_perl handler picks it up and does the work. It could take a long time, and is ready to be handled asynchronously, so I was hoping I could terminate the HTTP connection while it is running.
PHP has a function ignore_user_abort(), that when combined with the right headers, can close the HTTP connection early, while leaving the process running (this technique is mentioned here on SO a few times).
Does Perl have an equivalent? I haven't been able to find one yet.
Ok, I figured it out.
Mod_perl has the 'opposite' problem of PHP here. By default, mod_perl processes are left open, even if the connection is aborted, where PHP by default closes the process.
The Practical mod_perl book says how to deal with aborted connections.
(BTW, for the purposes of this specific problem, a job queue was lower on the list than a 'disconnecting' http process)
#setup headers
$r->content_type('text/html');
$s = some_sub_returns_string();
$r->connection->keepalive(Apache2::Const::CONN_CLOSE);
$r->headers_out()->{'Content-Length'} = length($s);
$r->print($s);
$r->rflush();
#
# !!! at this point, the connection will close to the client
#
#do long running stuff
do_long_running_sub();
You may want to look at using a job queue for this. Here is one provided by Zend that will let you start background processing jobs. There should be a number of these to choose from for php and perl.
Here's another thread that talks about this problem and an article on some php options. I'm not perl monk, so I'll leave suggestions on those tools to others.