How do I check if a TCP socket is still open without blocking?
If I am using usocket, then reading from a socket stream will signal end of file if the socket is closed, but will block it otherwise, which I don't want. I have tried using (listen stream) to check in advance if the socket will be blocked, but (listen) returns nil on a closed socket, which does not help here. I have also tried checking (usocket::state socket), which gives :READ on a closed socket, but it also sometimes gives :READ on an open socket even if there is not anything there. I am using sbcl, so I've tried using (sb-bsd-sockets:socket-open-p), but that gives T on a closed socket, also not helpful. The only command that seems to be helpful is (wait-for-input), namely it will return almost immediately when called on a closed socket, even when a longer timeout is given. At the same time, (listen) will still give nil, and that taken together seems to indicate a closed socket. This way looks rather hackish to me.
Is there a better way?
This isn't specifically a Lisp problem. It's a problem inherent to TCP itself.
The only way in TCP to tell if a connection is still open is to try to send data over it, or to use TCP's keepalive feature (which sends data over the socket for you).
Linux, Windows, and MacOS all have the SO_KEEPALIVE socket option (see setsockopt(2), which implements TCP keepalive. There are other associated sockopts, such as TCP_KEEPIDLE, TCP_KEEPINTVL, and TCP_KEEPCNT, but they aren't portable between operating systems, and in Lisp you'll have the added challenge of finding the actual values of the C preprocessor constants, and of providing a pointer for the option_value argument.
I haven't had any luck using SO_KEEPALIVE on Linux, and I haven't tried using it on any other operating system.
It's a bug in SBCL. The stream you get from (usocket:socket-stream s) and the one you get from calling cl:open are of the same type, so they behave the same way.
You get the correct behavior if you close a stream using cl:close:
(with-open-file (in "/dev/zero" :element-type '(unsigned-byte 8))
(close in)
(listen in))
And that correct behavior is an error. (On SBCL the type of the error is SB-INT:CLOSED-STREAM-ERROR)
If you simulate the other end of a socket closing by closing the underlying file descriptor with a POSIX call, you get the same NIL return value as with a real socket:
(with-open-file (in "/dev/zero" :element-type '(unsigned-byte 8))
(sb-posix:close (slot-value in 'sb-impl::fd))
(listen in)) ;; NIL
Whatever listen is doing is failing to notice that the underlying file descriptor is closed.
The blocking behavior is also identical:
(with-open-file (in "/dev/zero" :element-type '(unsigned-byte 8))
(sb-posix:close (slot-value in 'sb-impl::fd))
(read-byte in)) ;; hangs, Emacs interrupt doesn't work
In the source code for listen, I saw comments that claim that select(2) is used to implement listen, but I couldn't find the actual call to select. That call to select, if it's happening, is returning -1, and errno is not being consulted to find out why. It's likely that the -1 return code is being missed.
This affects the latest version of SBCL that Debian ships, but I don't know about the latest released version of SBCL.
Related
I am trying to play with a Swank server from the command line (with no specific production purpose, merely trying to understand things with some hacky ideas in mind for later) and I can't figure out what I can type from a telnet session for getting any usable answer; for instance how could I evaluate (+ 1 1)?
A previous question tells:
It is indeed possible to communicate with a swank server if you familiarize yourself with the swank protocol, which seems to be underdocumented (see e. g. here: https://github.com/astine/swank-client/blob/master/swank-description.markdown). However, this exposes a TCP socket over a network, which could be unsafe. Once I tried that, too, but I was not satisfied with the IPC speed.
and the link actually gives some hints, but unfortunately no example. I couldn't get any result.
Here is what I tried; after having separately started a server with
(swank:create-server :port 4005 :dont-close t :style NIL)
I tried things like:
~ $ telnet localhost:4005
00001e(swank:listener-eval (+ 1 2))
and got the following error message:
" "Reader error in file #<string-input stream from \"(swan...\" 0x6045ce0640>, position 20:
Cannot find the external symbol LISTENER-EVAL in #<\"SWANK\" package>.")Connection closed by foreign host
What would be in that case the string I should type?
I managed to make this work:
00003e(:emacs-rex (swank:interactive-eval "(+ 1 2)") "cl-user" t 8)
Notice how there is first an :emacs-rex keyword, and how the form to evaluate is in a string. The 8 is just an arbitrary value here. The reply first starts with 00A1C8(:indentation-update ...), followed by:
000031(:return (:ok "=> 3 (2 bits, #x3, #o3, #b11)") 8)
I'm trying to communicate from a Lisp script to another program by using TCP/IP sockets (with sbcl and the usocket library in a Linux system). Through some online sources I have managed to put together the following simple code:
(require 'asdf)
(require 'usocket)
(defun start-client (message)
"Connects to server."
(usocket:with-client-socket (socket stream "0.0.0.0" 30000)
(format stream message)
(force-output stream)))
(start-client "Hello!~%")
This code lets me send a message, (I have tested it and it works). My problem is that I need to split this code in two different functions, one for opening the socket connection and another to send different messages at different times. Also I need to add an additional function to receive messages from the other program. However, as I'm quite new with Lisp I have failed to do so.
The best way (I think) would be to have your entire script in the scope of with-client-socket. You might have something like a main function where this would fit. This avoids resource leaks. You might want to use a dynamic variable to avoid passing the socket stream manually through function arguments to wherever it is needed.
Otherwise, you have to manage the closing of the socket yourself. Any call path that might lead to program termination needs to be protected by some unwind-protect that closes the socket using usocket:socket-close. For that, you open the socket using usocket:socket-connect with the same arguments as you used for usocket:with-client-socket. (You can take a look at the source for usocket:with-client-socket and usocket:with-connected-socket to see the interactions taking place.)
In order to be able to write to the socket stream (obtainable through (usocket:socket-stream socket)) and close the socket, you need to remember it somewhere, e. g. by binding a dynamic variable.
The usocket FAQ suggests that the way I should do this is by reading from a socket-stream and checking for an end-of-file result. That works in the case where I've got one thread active per socket, but it doesn't seem to satisfy for the case where I'm trying to service multiple sockets in the same thread.
Consider something like
(defparameter *socket* (socket-listen "127.0.0.1" 123456))
(defparameter *client-connections*
(list (socket-accept *socket*)
(socket-accept *socket*)
(socket-accept *socket*)
(socket-accept *socket*)))
For this exercise, assume that I've actually got four clients connecting there. It seems like the way to go about serving them from one thread is something like
(wait-for-input *client-connections*)
(loop for sock in *client-connections*
for stream = (socket-stream sock)
when (listen stream)
do (let ((line (read-line stream nil :eof)))
(if (eq line :eof)
(progn (delete sock *client-connections*)
(socket-close sock))
(handle sock line))))
Except that this won't work, because a disconnected socket still returns nil to listen, and an attempt to read from an active socket with no messages will block but wait-for-intput returns immediately when there's a closed socket in the mix, even when no other socket has a message ready (though it seems to fail to specify which sockets caused it to return).
In the situation where no client has spoken in a little while, and third client disconnects, there doesn't seem to be a good way of finding that out and closing that specific socket connection. I'd have to read them in sequence, except that since read blocks on no input, that would cause the thread to wait until the first two clients both sent a message.
The solutions I've got in mind, but haven't found after some determined googling, are (in descending order of preference):
A function otherwise equivalent to listen that returns t if a read on the targets' stream would return an end-of-file marker. (Replacing listen above with this notional function would let the rest of it work as written)
A function otherwise equivalent to wait-for-input that returns a list of closed sockets that cause it to trip. (In this case, I could iterate through the list of closed sockets, check that they're actually closed with the suggested read technique, and close/pop them as needed)
A function otherwise equivalent to wait-for-input that returns the first closed socket that caused it to trip. (As #2, but slower, because it prunes at most one inactive connection per iteration)
Keeping track of how long its been since I've received input from each socket connection, and closing them out regardless after a certain period of inactivity. (Which I'd probably want to do anyway, but doing just this would potentially keep a bunch of dead connections around much longer than necessary)
A function that attempts to read-char from a stream with an instant timeout, returns t if it encounters an :eof, and unread-chars anything else (returning nil after either timing out or unreading). (Which is a last resort since it seems like it would be trivially easy to break in a non-obvious-but-lethal way).
Also, if I'm thinking about this in precisely the wrong way, point that out too.
It turns out that the thing I mention as Option 2 above exists.
wait-for-input defaults to returning the full list of tracked connections for memory management purposes (someone was reportedly very concerned about consing new lists for the result), but it has a &key parameter that tells it to just return the connections that have something to say.
(wait-for-input (list conn1 conn2 conn3 conn4) :ready-only t)
is what I was looking for there. This returns all the ready connections, not just ones that are going to signal end-of-file, so the loop still needs to handle both cases. Something like
(loop for sock in (wait-for-input *client-connections* :ready-only t)
for stream = (socket-stream sock)
do (let ((line (read-line stream nil :eof)))
(if (eq line :eof)
(progn (delete sock *client-connections*)
(socket-close sock))
(handle sock line))))
should do nicely.
I'm trying to choose between pipes and unix sockets for an IPC mechanism.
Both support the select() and epoll() functions which is great.
Now, pipes have a 4kB (as of today) "atomic" write, which is guaranteed by the Linux Kernel.
Does such a feature exists in the case of unix sockets? I couldn't find any document stating this explicitely.
Say I use a UNIX socket and I write x bytes of data from my client. Am I sure that these x bytes will be written on the server end of the socket when my server's select() cracks?
On the same subject, would using SOCK_DGRAM ensure that writes are atomic (if such a guarantee is possible), since datagrams are supposed to be single well-defined messages?
What would then be the difference using SOCK_STREAM as a transfer mode?
Thanks in advance.
Pipes
Yes the non-blocking capacity is usually 4KB, but for maximum portability you'd probably be better off using the PIPE_BUF constant. An alternative is to use non-blocking I/O.
More information than you want to know in man 7 pipe.
Unix datagram sockets
Writes using the send family of functions on datagram sockets are indeed guaranteed to be atomic. In the case of Linux, they're reliable as well, and preserve ordering. (which makes the recent introduction of SOCK_SEQPACKET a bit confusing to me) Much information about this in man 7 unix.
The maximum datagram size is socket-dependent. It's accessed using getsockopt/setsockopt on SO_SNDBUF. On Linux systems, it ranges between 2048 and wmem_max, with a default of wmem_default. For example on my system, wmem_default = wmem_max = 112640. (you can read them from /proc/sys/net/core) Most relevant documentation about this is in man 7 socket around the SO_SNDBUF option. I recommend you read it yourself, as the capacity doubling behavior it describes can be a bit confusing at first.
Practical differences between stream and datagram
Stream sockets only work connected. This mostly means they can only communicate with one peer at a time. As streams, they're not guaranteed to preserve "message boundaries".
Datagram sockets are disconnected. They can (theoretically) communicate with multiple peers at a time. They preserve message boundaries.
[I suppose the new SOCK_SEQPACKET is in between: connected and boundary-preserving.]
On Linux, both are reliable and preserve message ordering. If you use them to transmit stream data, they tend to perform similarly. So just use the one that matches your flow, and let the kernel handle buffering for you.
Crude benchmark comparing stream, datagram, and pipes:
# unix stream 0:05.67
socat UNIX-LISTEN:u OPEN:/dev/null &
until [[ -S u ]]; do :;done
time socat OPEN:large-file UNIX-CONNECT:u
# unix datagram 0:05.12
socat UNIX-RECV:u OPEN:/dev/null &
until [[ -S u ]]; do :;done
time socat OPEN:large-file UNIX-SENDTO:u
# pipe 0:05.44
socat PIPE:p,rdonly=1 OPEN:/dev/null &
until [[ -p p ]]; do :;done
time socat OPEN:large-file PIPE:p
Nothing statistically significant here. My bottleneck is likely reading large-file.
Say I use a UNIX socket and I write x
bytes of data from my client. Am I
sure that these x bytes will be
written on the server end of the
socket when my server's select()
cracks?
If you are using AF_UNIX SOCK_STREAM socket, there is no such guarantee, that is, data written in one write/send() may require more than one read/recv() call on the receiving side.
On the same subject, would using
SOCK_DGRAM ensure that writes are
atomic (if such a guarantee is
possible), since datagrams are
supposed to be single well-defined
messages?
On there other hand, AF_UNIX SOCK_DGRAM sockets are required to preserve the datagram boundaries and be reliable. You should get EMSGSIZE error if send() can not transmit the datagram atomically. Not sure what happens for write() as the man page does not say that it can report EMSGSIZE (although man pages sometimes do not list all errors returned). I would try overflowing the receiver's buffer with big sized datagrams to see which errors exactly send/write() report.
One advantage of using UNIX sockets over pipes is the bigger buffer size. I don't remember exactly what is the limit of pipe's kernel buffer, but I remember not having enough of it and not being able to increase it (it is a hardcoded kernel constant). fast_producer_process | slow_consumer_process was orders of magnitude slower than fast_producer_process > file && file > slow_consumer_process due to insufficient pipe buffer size.
New version of SBCL 1.0.28 running on debian breaks AllegroServe 1.2.47 on incoming connection with following error:
aserve-accept-6: 05/26/09 - 21:11:01 - accept: error 0 on accept invalid
keyword argument: :AUTO-CLOSE (valid keys are
:INPUT, :OUTPUT, :ELEMENT-TYPE, :EXTERNAL-FORMAT,
:BUFFERING, :TIMEOUT).
Portable AllegroServe page does make a mention of this problem. However, no google searches turn up anything of use for this problem.
Any ideas as to how to move forward with this problem, or alternatively, links pointing to places where this has been dealt with?
After some mucking around, I've come up with the following solution:
In my source files, after I declare my package, compile/load the appropriate modules but before I declare anything in my package, I added the following code:
(defmethod sb-bsd-sockets:socket-make-stream ((socket sb-bsd-sockets:socket)
&key input output
(element-type 'character)
(buffering :full)
(external-format :default)
timeout
(auto-close t))
"Default method for SOCKET objects. An ELEMENT-TYPE of :DEFAULT
will construct a bivalent stream. Acceptable values for BUFFERING
are :FULL, :LINE and :NONE. Streams will have no TIMEOUT
by default.
The stream for SOCKET will be cached, and a second invocation of this
method will return the same stream. This may lead to oddities if this
function is invoked with inconsistent arguments \(e.g., one might request
an input stream and get an output stream in response\)."
(let ((stream
(and (slot-boundp socket 'stream) (slot-value socket 'stream))))
(unless stream
(setf stream (sb-sys:make-fd-stream
(sb-bsd-sockets:socket-file-descriptor socket)
:name "a socket"
:dual-channel-p t
:input input
:output output
:element-type element-type
:buffering buffering
:external-format external-format
:timeout timeout
:auto-close auto-close)))
(setf (slot-value socket 'stream) stream)
(sb-ext:cancel-finalization socket)
stream))
(It's basically a lift from what is in the sb-bsd-sockets/socket.lisp with the auto-close key added to the argument list)
This way I avoid modifying or patching system files, and basically hook into the sb-bsd-sockets package directly.
So far, it seems to be working as it should. Basic testing via successive calls to (room) shows me that there's no obvious memory leaks, and the performance is as expected.
Please feel free to comment on this kludge, and if you think it might affect the stability of my system in unexpected ways.