I wrote a Kernel module that deals with socket-based TCP connections. Everything works great except one specific use case. I’d appreciate if somebody advise me how to solve the problem described below.
I have:
Kernel module which is a device registered using
misc_register().
User space application that communicates with this module using the standard file i/o functions: open,
close, ioctl, etc.
The exact scenario looks like this:
Load the module using insmod.
Open the associated device from user application using the standard open() function
call ioctl() that performs the following actions in the Kernel module (insignificant code lines omitted):
`
...
sock_create(PF_INET, SOCK_STREAM, 0, sock);
...
flags = O_NONBLOCK;
sock_map_fd(*sock, flags);
...
kernel_connect (sock, (struct sockaddr *)server_addr, sizeof(struct sockaddr_in), sock->file->f_flags);
...
`
All functions return successfully. The TCP connection is established successfully. After that tere can be also reads/writes on this connection but it doesn’t influence the problem.
If the application finishes naturally or I interrupt it by sending SIGINT the connection is closed nicely - with FIN exchange etc. On SIGKILL it issues TCP as I expect. No problems so far.
Now I would like to close this socket w/o stopping application. I try to do it by calling sock_release() in my Kernel module via another ioctl call. Upon this call the TCP connection is also closed nicely. However now the Kernel gets stuck when my application finishes or is interrupted!
I suspect that the Kernel somehow is not “informed” that the socket is closed. It tries to close it again and fails once the socket memory structure is de-allocated.
Did somebody use sockets from Kernel modules and had similar problems?
Can you recommend an alternative way to work with TCP sockets from Kernel modules?
Alternative ways to close sockets from within Kernel?
Thank you very much in advance.
After Kernel code investigation I found out that in case you map socket to a file using sock_map_fd() function it is not enough to call sock_release(). This function doesn't release the file descriptor associated wit the socket. In case you really need to map a Kernel socket to a file keep the file descriptor returned by sock_map_fd() and use sys_close() function to close the socket and clean up the associated file. Note that when the device file descriptor is closed all sockets created in the module and associated with files are also closed automatically.
Alternatively you can just avoid mapping socket to a file descriptor. The socket basic functionality will stay ok even without the mapping. In this case sock_release() works perfectly.
Related
I am attempting to run MOOS-IvP on a Beaglebone Black
On attempting to run the MOOS database it continuously throws the exception
"Exception Thrown in listen loop: Error Listening To Socket. Operation not supported"
This software runs on a Raspberry Pi
Any ideas what might be the issue?
I have found the problem and fixed it.
When the socket is created it needs to be TCP. However when getprotobyname(_sName) is called in the XPCGetProtocol class to lookup the correct protocol number in /etc/protocols it returns the value of the previous time it was called, which was when a UDP socket was setup.
To fix it I simply called the function twice, the second time it returns the correct value.
I am not sure why it would return incorrect the first time but this works!
I also encountered this error while working with a BeagleBone Black running Ubuntu 14.04. However, the solution of running the request twice did not work. More troubleshooting led me to determine that the one that was supposed to be a TCP socket was opened after another process had opened a UDP socket. The structure returned by getprotobyname() is a pointer to a static location that does not change from call to call, but does get updated with the protocol details (see here although for another Unix os). Therefore, the second call by another process overwrites the original details.
This then gets tested during socket creation in the constructor of XPCSocket, and results in creation of a UDP socket where it should have been a TCP socket. This could probably be fixed by adding a lock to this function, but I took the non-blocking approach to initialize the requested protocol using the string the constructor was called with (_sProtocol) instead of the one returned in the socketProtocol structure. In addition, I modified the XPCGetProtocol class to store the protocol number in a member variable that would not be changed upon subsequent calls to getprotobyname().
My modifications can be found here.
if I add an event for a specific socket to event loop,
for example, a TCP connection socket.
then it may happen that the socket is closed,
then how will libevent act?
can it detect this?
thanks!
EDIT: I think I misinterpreted your question at first.
If you mean that the socket is closed from the remote end
As per the documentation you can use the event_new() and event_add() calls to register interest in a socket. Make sure you specify EV_READ since you are interested in when the socket is closed.
Remember that there is no difference in file descriptor readiness between data available for reading and a closed socket. Normally you must read the socket to find out which condition is true, but if you don't want to read the socket then you can look here for a hint.
If you mean that the socket is locally closed (the file descriptor was closed
Using a file descriptor after it has been closed is never defined and can always lead to undefined results. This is not specific to libevent. Before you close a file descriptor, you must make sure that no other thread in your program is using it, and you must make sure that no other part of your program is going to try using it in the future. That means unregistering the file descriptor from libevent at the same time that you close it.
Say one part of a program writes some stuff to a socket and another part of the same program reads stuff from that same socket. If an external tool writes to that very same socket, how would I differentiate who wrote what to the socket (using the part that reads it)? Would using a named pipe work?
If you are talking about TCP the situation you describe is impossible, because connections are 1-to-1. If you mean UDP, you can get the sender address by setting the appropriate flags in the recvmesg() function.
I'm writing a Unix domain socket server for Linux.
A peculiarity of Unix domain sockets I quickly found out is that, while creating a listening Unix socket creates the matching filesystem entry, closing the socket doesn't remove it. Moreover, until the filesystem entry is removed manually, it's not possible to bind() a socket to the same path again : bind() fails with EADDRINUSE if the path it is given already exists in the filesystem.
As a consequence, the socket's filesystem entry needs to be unlink()'ed on server shutdown to avoid getting EADDRINUSE on server restart. However, this cannot always be done (i.e.: server crash). Most FAQs, forum posts, Q&A websites I found only advise, as a workaround, to unlink() the socket prior to calling bind(). In this case however, it becomes desirable to know whether a process is bound to this socket before unlink()'ing it.
Indeed, unlink()'ing a Unix socket while a process is still bound to it and then re-creating the listening socket doesn't raise any error. As a result, however, the old server process is still running but unreachable : the old listening socket is "masked" by the new one. This behavior has to be avoided.
Ideally, using Unix domain sockets, the socket API should have exposed the same "mutual exclusion" behavior that is exposed when binding TCP or UDP sockets : "I want to bind socket S to address A; if a process is already bound to this address, just complain !" Unfortunately this is not the case...
Is there a way to enforce this "mutual exclusion" behavior ? Or, given a filesystem path, is there a way to know, via the socket API, whether any process on the system has a Unix domain socket bound to this path ? Should I use a synchronization primitive external to the socket API (flock(), ...) ? Or am I missing something ?
Thanks for your suggestions.
Note : Linux's abstract namespace Unix sockets seem to solve this issue, as there is no filesystem entry to unlink(). However, the server I'm writing aims to be generic : it must be robust against both types of Unix domain sockets, as I am not responsible for choosing listening addresses.
I know I am very late to the party and that this was answered a long time ago but I just encountered this searching for something else and I have an alternate proposal.
When you encounter the EADDRINUSE return from bind() you can enter an error checking routine that connects to the socket. If the connection succeeds, there is a running process that is at least alive enough to have done the accept(). This strikes me as being the simplest and most portable way of achieving what you want to achieve. It has drawbacks in that the server that created the UDS in the first place may actually still be running but "stuck" somehow and unable to do an accept(), so this solution certainly isn't fool-proof, but it is a step in the right direction I think.
If the connect() fails then go ahead and unlink() the endpoint and try the bind() again.
I don't think there is much to be done beyond things you have already considered. You seem to have researched it well.
There are ways to determine if a socket is bound to a unix socket (obviously lsof and netstat do it) but they are complicated and system dependent enough that I question whether they are worth the effort to deal with the problems you raise.
You are really raising two problems - dealing with name collisions with other applications and dealing with previous instances of your own app.
By definition multiple instances of your pgm should not be trying to bind to the same path so that probably means you only want one instance to run at a time. If that's the case you can just use the standard pid filelock technique so two instances don't run simultaneously. You shouldn't be unlinking the existing socket or even running if you can't get the lock. This takes care of the server crash scenario as well. If you can get the lock then you know you can unlink the existing socket path before binding.
There is not much you can do AFAIK to control other programs creating collisions. File permissions aren't perfect, but if the option is available to you, you could put your app in its own user/group. If there is an existing socket path and you don't own it then don't unlink it and put out an error message and letting the user or sysadmin sort it out. Using a config file to make it easily changeable - and available to clients - might work. Beyond that you almost have to go some kind of discovery service, which seems like massive overkill unless this is a really critical application.
On the whole you can take some comfort that this doesn't actually happen often.
Assuming you only have one server program that opens that socket.
Then what about this:
Exclusively create a file that contains the PID of the server process (maybe also the path of the socket)
If you succeed, then write your PID (and socket path) there and continue creating the socket.
If you fail, the socket was created before (most likely), but the server may be dead. Therefore read the PID from the file that exists, and then check that such a process still exists (e.g. using the kill with 0-signal):
If a process exists, it may be the server process, or it may be an unrelated process
(More steps may be needed here)
If no such process exists, remove the file and begin trying to create it exclusively.
Whenever the process terminates, remove the file after having closed (and removed) the socket.
If you place the socket and the lock file both in a volatile filesystem (/tmp in older ages, /run in modern times, then a reboot will clear old sockets and lock files automatically, most likely)
Unless administrators like to play with kill -9 you could also establish a signal handler that tries to remove the lock file when receiving fatal signals.
Am a c-coder for a while now - neither a newbie nor an expert. Now, I have a certain daemoned application in C on a PPC Linux. I use PHP's socket_connect as a client to connect to this service locally. The server uses epoll for multiplexing connections via a Unix socket. A user submitted string is parsed for certain characters/words using strstr() and if found, spawns 4 joinable threads to different websites simultaneously. I use socket, connect, write and read, to interact with the said webservers via TCP on their port 80 in each thread. All connections and writes seems successful. Reads to the webserver sockets fail however, with either (A) all 3 threads seem to hang, and only one thread returns -1 and errno is set to 104. The responding thread takes like 10 minutes - an eternity long:-(. *I read somewhere that the 104 (is EINTR?), which in the network context suggests that ...'the connection was reset by peer'; or (B) 0 bytes from 3 threads, and only 1 of the 4 threads actually returns some data. Isn't the socket read/write thread-safe? I use thread-safe (and reentrant) libc functions such as strtok_r, gethostbyname_r, etc.
*I doubt that the said webhosts are actually resetting the connection, because when I run a single-threaded standalone (everything else equal) all things works perfectly right, but of course in series not parallel.
There's a second problem too (oops), I can't write back to the client who connect to my epoll-ed Unix socket. My daemon application will hang and hog CPU > 100% for ever. Yet nothing is written to the clients end. Am sure the client (a very typical PHP socket application) hasn't closed the connection whenever this is happening - no error(s) detected either. Any ideas?
I cannot figure-out whatever is wrong even with Valgrind, GDB or much logging. Kindly help where you can.
Yes, read/write are thread-safe. But beware of gethostbyname() and getservbyname() if you're using them - they return pointers to static data, and may not be thread-safe.
errno 104 is ECONNREFUSED (not EINTR). Use strerror or perror to get the textual error message (like 'Connection reset by peer') for a particular errno code.
The best way to figure out what's going wrong is often to do very detailed logging - log the results of every operation, plus details like the IP address/port connecting to, the number of bytes read/written, the thread id, and so forth. And, of course, make sure your logging code is thread-safe :-)
Getting an ECONNRESET after 10 minutes sounds like the result of your connection timing out. Either the web server isn't sending the data or your app isn't receiving it.
To test the former, hookup a program like Wireshark to the local loopback device and look for traffic to and from the port you are using.
For the later, take a look at the epoll() man page. They mention a scenario where using edge triggered events could result in a lockup, because there is still data in the buffer, but no new data comes in so no new event is triggered.