I'm modifying an app in order to replace its use of select() with kqueue. select() allows one to poll for exceptional conditions:
int select(int nfds,
fd_set *restrict readfds,
fd_set *restrict writefds,
fd_set *restrict errorfds, <---- this thing here
struct timeval *restrict timeout
);
After reading the kqueue documentation it looks like there's no way to do that. There's EVFILT_READ and EVFILT_WRITE but nothing along the lines of EVFILT_ERROR/EVFILT_EXCEPTIONAL. Is it possible to poll for exceptional conditions, and if so, how?
There is no such thing as exceptional state on FreeBSD, to quote man 2 select:
The only exceptional condition detectable is out-of-band data received on a socket.
So your question boils down to “How can I detect OOB-data on a socket with kqueue”, which I, to be honest, cannot answer without doing some research.
Related
I never really got the idea behind this parameter, what is it good for? I also noticed this parameter is ignored in WinSock2, why is that?
Do Unix systems use this parameter or do they ignore it as well?
Windows' implementation of select() uses linked lists internally, so it doesn't need to use the nfds parameter for anything.
On other OS's, however, the fd_set struct is implemented to hold an array of bits (one bit per socket). For example, here is how it is declared (in sys/_types/_fd_def.h) under MacOS/X:
typedef struct fd_set {
__int32_t fds_bits[__DARWIN_howmany(__DARWIN_FD_SETSIZE, __DARWIN_NFDBITS)];
} fd_set;
... and in order to do the right thing, the select() call will have to loop over the bits in the array to see what they contain. By supplying select() with the nfds parameter, we tell the select() implementation that it only needs to iterate over the first (nfds) bits of the array, rather than always having to iterate over the entire array on every call. This allows select() to be more efficient than it would otherwise be.
Currently in my degree we're starting to work with sockets.
I Have a couple of questions regarding polling for input from sockets,
using the select() function.
int select( int nfds,
fd_set *readfds,
fd_set *writefds,
fd_set *exceptfds,
const struct timespec *timeout);
We give select "nfds" param, which would normally would
be the maximum sockets number we would like to monitor. How can i watch only one specific socket instead of the range of 0 to nfds_val sockets ?
What are the file descriptors objects that we use? what is their purpose,
and why can't we just point "select" to the relevant socket structure?
I've read over the forum regarding Blocking and Non-Blocking mode of select, but couldn't understand the meaning or uses of each, nor how to implement such, would be glad if someone could explain.
Last but not least (only for the time being :D ) - When binding a socketaddr_in to socket number, why does one needs to cast to socketaddr * and not leave it as sockaddr_in * ?
I mean except for the fact that bind method expects this kind of pointer type ;)
Would appreciate some of the experts answers here :)
Thank you guys and have a great week!
We give select "nfds" param, which would normally would be the maximum sockets number we would like to monitor. How can i watch only one specific socket instead of the range of 0 to nfds_val sockets ?
Edit. (sorry, the previous text here was wrong) Just provide your socket descriptor + 1. I'm pretty sure it doesn't mean OS will check all the descriptors in [0, 1... descriptor] range.
What are the file descriptors objects that we use? what is their purpose, and why can't we just point "select" to the relevant socket structure?
File descriptors are usually integer values given to the user by OS. OS uses descriptors to control physical and logical resources - one file descriptor means OS has given you something file-like to control. Since Berkeley Sockets have read and write operations defined, they are file-like and socket objects essentially are plain file descriptors.
Answering why can't we just point "select" to the relevant socket structure? - we actually can. What exactly to pass to select depends on OS and language. In C you place your socket descriptor (plain int value most probably) into a fd_set. fd_set is then passed to select.
Edit.
An tiny example for Linux:
fd_set set;
FD_ZERO(&set);
FD_SET(socket_fd, &set);
// check if socket_fd is ready for reading
result = select(socket_fd + 1, &set, NULL, NULL, NULL);
if (result == -1) report_error(errno);
Docs.
Windows has similar code.
I've read over the forum regarding Blocking and Non-Blocking mode of select, but couldn't understand the meaning or uses of each, nor how to implement such, would be glad if someone could explain.
A blocking operation makes your thread wait until it is done. It's 99% of functions you use. If there are sockets ready for some IO, blocking select will return something immediately. It there are no such sockets, the thread will wait for them. Non-blocking select, in the latter case, won't wait and will return -1 (error).
As an example, try to implement single threaded server that is capable of working with multiple clients, including long operations like file transfer happening simultaneously. You definitely don't want to use blocking socket operations in this case.
Last but not least (only for the time being :D ) - When binding a socketaddr_in to socket number, why does one needs to cast to socketaddr * and not leave it as sockaddr_in * ? I mean except for the fact that bind method expects this kind of pointer type ;)
Probably due to historical reasons, but I'm not sure. And there seems to be a fine answer on SO already.
I'm currently studying OS's and i came across a question that i cant quite figure it out.
I have this piece of pseudo-code:
for(;;) {
mask = teskmask;
select(MAXSOCKS,&mask,0,0);
if(FD_ISSET(strmfd,&mask)) {
clilen=sizeof(cliaddr);
newfd=accept(strmfd,(struct sockaddr*)&cliaddr,&clilen);
echo(newfd);
close(newfd);
}
if (FD_ISSET(dgrmfd,&mask)) echo (dgrmfd);
}
Note: consider MAXSOCKS to be defined as whatever (doesn't matter here), strmfd to be a Stream socket, dgrmfd as a datagram socket, clilen is the size of client address,
echo(newfd); is just a function to echo whats in the socket.
So my question is:
what is/whats for the testmask and mask and how is it used here?
I know that select blocks the process until some socket is available to read/write or exceptions.
The simplest way to get answer to your question is just to read select(2) manual page, which says:
int select(int nfds,
fd_set *readfds, fd_set *writefds, fd_set *exceptfds,
struct timeval *timeout);
or a quite similar declaration. So "testmask" is fd_set. Select() gets these masks on input and refills them on output, unless an error is signaled.
In all Unix systems I've seen fd_set is bit array up to FD_SETSIZE bits, and it could be hard to extend this to larger value, so a descriptor greater or equal to FD_SETSIZE can't be select()ed without target-specific hacking. So poll() is preferred for a large descriptor set. (And, platform-specific solutions as kqueue (*BSD, MacOSX), epoll (Linux), /dev/poll (Solaris) are better if relatively small subset of all descriptors is ready on each cycle; for Linux it's ~30% when poll() is more efficient than epoll-based scheme, measured for systems ~3 years ago.)
In Windows systems, fd_set is array of socket numbers (integers) with unlimited value, but limited total count (AFAIR, 64).
Using only standard macros to deal with fd_set gives code which is portable between Unix and Windows. But this code could be inefficient for both in different manner.
Actually the proper way to copy the select mask is to use the FD_COPY macro.
I have just posted an article about the difference between select, poll and epoll here: http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system-architects/ - I suggest you take a look on it as it covers many things you asked, and there's a chance it covers something you missed.
I'm trying to do some IP agnostic coding and as suggested by various sources I tried to use sockaddr_storage. However all the API calls (getaddrinfo, getnameinfo) still depend on struct sockaddr. And casting between them isn't exactly a good option, gves rise to a lot of other problems.
And casting to sockaddr_in and sockaddr_in6 separately sort of defeats the purpose of me trying to use sockaddr_storage.
Anybody who has effectively used sockaddr_storage in devloping a simple client server socket application.
The problem with jointly doing IPV6 and IPV4 programming is that a pure sockaddr struct itself is not big enough to hold a sockaddr_in6. So if you need to blindly pass around an address that could be either sockaddr_in or sockaddr_in6, sockaddr_storage is a bit easier to use.
At the end of the day, whether you are using sockaddr_in, sockaddr_in6, or sockaddr_storage, you'll have to cast those pointers to make a call to sendto, recvfrom, connect, accept, and many other socket functions. It's just a known nuance of socket programming. Just let go of the feeling of doing something unsafe. Your code will be ok.
Now when writing network code that is meant to work for both IPV4 and IPV6, you can easily get into a trap of having an abundance of switch statements to handle the different network types. Code then gets messy like the following:
if (addr.ss_family == AF_INET)
sendto(sock, buffer, len, 0, (sockaddr*)&addr, sizeof(sockaddr_in))
else (addr.ss_family == AF_INET6)
sendto(sock, buffer, len, 0, (sockaddr*)&addr, sizeof(sockaddr_in6));
And then that type of "if family == AF_INET" expression can easily start to repeat itself over and over. That's what you want to avoid.
Assuming you are using C++, you'll find that an abstraction class for a socket address object is incredibly useful. I have an example on github here and here. The CSocketAddress class is backed by a union of {sockaddr, sockaddr_in, sockaddr_in6} and can be constructed with a sockaddr_storage. If I had known about sockaddr_storage before I had started this class, I would have used that instead of the union thing. In any case, it allows me to write code as follows:
CSocketAddress addr;
...
sendto(sock, buffer, len, 0, addr.GetSockAddr(), addr.GetSockAddrLength());
Likewise, an "accept" statement looks like this:
sockaddr_storage addrstorage = {};
int len = sizeof(sockaddr_storage);
accept(sock, (sockaddr*)&addrstorage, &len);
CSocketAdddress addr(addrstorage); // construct an address object to pass around everywhere else
This was incredibly helpful for the code paths that call bind, send, and recv. Now my STUN server and client code paths no longer have to know anything about the family type of the socket address. They just work with "CSocketAddress" objects. The only IPV4 and IPV6 specific code is during the client and server initialization - when the address objects are actually constructed. Fortunately, that was partially abstracted out as well.
You might also want to peruse the helper functions here. There's some more useful stuff for resolving hostnames, enumerating adapters, etc... in an IP agnostic way. It's Linux code, but some of it should map ok to Windows and winsock.
I am almost done adding TCP support to this code base. In the process of adding support for SOCK_STREAM, I have not had to make a single change nor add any new code to deal with the differences in IPV4 and IPV6 address structures.
I don't generally see the need for struct sockaddr_storage. It's purpose is to allocate enough space for the sockaddr structure of any given protocol, but how often do you need to do that in IP-version-agnostic code? Generally you call getaddrinfo() and it gives you a bunch of struct sockaddr *s, and you don't care whether they're sockaddr_in or sockaddr_in6, you just pass them along as-is to bind() and connect() (no casting required).
In typical client/server code, the main place I can think of where struct sockaddr_storage is useful is to reserve space for the second parameter to accept(). In this case I agree it's ugly to have to cast it to struct sockaddr * once for accept() and again for getnameinfo(). But I can't see a way around those casts. This is C. Structure inheritance always involves lots of casts.
The prototype is:
int select (int nfds,
fd_set *read-fds,
fd_set *write-fds,
fd_set *except-fds,
struct timeval *timeout);
I've been struggling to understand this function for quite some time. My question is, if it checks all the file descriptors from 0 to nfds-1, and will modify the read-fds, write-fds and except-fds when return, why do I need to use FD_SET to add file descriptors to the set at the begining, it will check all the file descriptors anyway, or not?
It won't check from 0 to nfds-1. The first argument just provides an upper bound on how large, numerically, the file descriptors used are. This is because the set itself might be represented as a bitvector, without a way to know how many bits are actually used. Specifying this as a separate argument helps select() avoid checking file descriptors that are not in use.
Also, a descriptor that is not in e.g. the read set when you call select() is not being checked at all, so it cannot appear in the set when the call returns, either.
I once had the same doubt as yours. You can look at following question and answers:
Query on Select System Call