Questions about select() - sockets

Considering the following class method:
void TCPServer::listenWithTimeout() {
fd_set descrSet;
struct timeval timeout = {1, 0};
while (listeningThread.active) {
std::cout << "TCP Server thread is listening..." << std::endl;
FD_ZERO(&descrSet);
FD_SET(mySocketFD, &descrSet);
int descrAmount = select(mySocketFD + 1, &descrSet, NULL, NULL, &timeout);
if (descrAmount > 0) {
assert(FD_ISSET(mySocketFD, &descrSet));
int threadIndx = findAvailableThread();
if (threadIndx < 0) {
std::cout << "Connection not accepted: no threads available..." << std::endl;
} else {
joinThread(threadIndx);
int newSocketFD = accept(mySocketFD, (struct sockaddr *) &mySocket, &clieAddrssLen);
std::cout << "TCP Client connected..." << std::endl;
connectionThreads[threadIndx].thread = std::thread(TCPServer::startTCPConnectionWithException, std::ref(*this), threadIndx, newSocketFD);
connectionThreads[threadIndx].active = true;
}
}
}
std::cout << "TCP Server thread is terminating..." << std::endl;
}
Here are some question:
when there are not available threads (findAvailableThreads() returns -1), is it a normal behaviour that select() doesn't wait its timeout and so the while loop iterates really fast until a new thread is available?
if yes, how could I avoid these really fast iterations? Instead of using something like a simple sleep() at line 13 inside the if branch, is there a way to let select() restore its timeout? Or even, is there a way to completely reject the incoming connection pending?

when there are not available threads (findAvailableThreads() returns -1), is it a normal behaviour that select() doesn't wait its timeout and so the while loop iterates really fast until a new thread is available?
Yes, because under that condition, you are not calling accept(), so you are not changing the listening socket's state. It will remain in a readable state as long as it has a client connection waiting to be accept()'ed.
if yes, how could I avoid these really fast iterations?
Call accept() before checking for an available thread. If no thread is available, close the accepted connection.
Instead of using something like a simple sleep() at line 13, inside the if branch, is there a way to let select() restore its timeout?
The only way is to accept() the connection that put the listening socket into a readable state, so it has a chance to go back to a non-readable state. The timeout will not apply again until the socket is no longer in a readable state.
Or even, is there a way to completely reject the incoming connection pending?
The only way is to accept() it first, then you can close() it if needed.
Try this:
void TCPServer::listenWithTimeout() {
fd_set descrSet;
while (listeningThread.active) {
std::cout << "TCP Server thread is listening..." << std::endl;
FD_ZERO(&descrSet);
FD_SET(mySocketFD, &descrSet);
struct timeval timeout = {1, 0};
int descrAmount = select(mySocketFD + 1, &descrSet, NULL, NULL, &timeout);
if (descrAmount > 0) {
assert(FD_ISSET(mySocketFD, &descrSet));
int newSocketFD = accept(mySocketFD, (struct sockaddr *) &mySocket, &clieAddrssLen);
if (newSocketFD != -1) {
int threadIndx = findAvailableThread();
if (threadIndx < 0) {
close(newSocketFD);
std::cout << "Connection not accepted: no threads available..." << std::endl;
} else {
joinThread(threadIndx);
std::cout << "TCP Client connected..." << std::endl;
connectionThreads[threadIndx].thread = std::thread(TCPServer::startTCPConnectionWithException, std::ref(*this), threadIndx, newSocketFD);
connectionThreads[threadIndx].active = true;
}
}
}
}
std::cout << "TCP Server thread is terminating..." << std::endl;
}

Related

runtime error with mongocxx::options::find

It works fine when I do query without option
document condition;
condition << "age" << 22;
auto cursor = collection_.find(condition.view());
for (auto&& doc : cursor)
{
std::cout << bsoncxx::to_json(doc) << std::endl;
}
but once I use mongocxx::options::find opts
document condition;
condition << "age" << 22;
mongocxx::options::find opts{};
opts.limit(1);
auto cursor = collection_.find(condition.view(), opts);
for (auto&& doc : cursor)
{
std::cout << bsoncxx::to_json(doc) << std::endl;
}
read access permission conflicts error occured
enter image description here
I recompiled mongocxx and built a new project, the error didn't appear. Maybe there is something wrong with my previous environment.

SCTP: What should be the sctp_status.sstate value of an SCTP socket after succesful connect() call?

I'm trying to connect to a remote peer (which I don't have directory access other than connecting to it via socket and ping) via SCTP. Assuming that I have connected succesfully, what should be the value of my sctp_status.sstate if I try calling getsocktopt()? Mine is SCTP_COOKIE_ECHOED(3) according to sctp.h. Is it correct? Shouldn't it be SCTP_ESTABLISHED?
Because I tried sending message to the remote peer with this code:
ret = sctp_sendmsg (connSock, (void *) data, (size_t) strlen (data), (struct sockaddr *) &servaddr, sizeof (servaddr), 46, 0, 0, 0, 0);
It returned the number of bytes I tried sending. Then when I tried catching if there's any response:
ret = sctp_recvmsg (connSock, (void *) reply, sizeof (reply), NULL,
NULL, NULL, &flags);
It returns -1 with errno of ECONNRESET(104). What are the possible mistakes in my code, or maybe in my flow? Did I miss something?
Thanks in advance for answering. Will gladly appreciate that. :)
Update: Here down below is my client code in connecting to the remote peer. It's actually a node addon for me to use since SCTP is not fully supported in node. Using lksctp-tools package to include the headers.
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <netinet/sctp.h>
#include <arpa/inet.h>
#include <signal.h>
#define MAX_BUFFER 1024
int connSock = 0;
int connect(char host[], int port, char remote_host[], int remote_port, int timeout) {
int ret, flags;
fd_set rset, wset;
struct sockaddr_in servaddr;
struct sockaddr_in locaddr;
struct sctp_initmsg initmsg;
struct timeval tval;
struct sctp_status status;
socklen_t opt_len;
errno = 0;
connSock = socket (AF_INET, SOCK_STREAM, IPPROTO_SCTP);
flags = fcntl(connSock, F_GETFL, 0);
fcntl(connSock, F_SETFL, flags | O_NONBLOCK);
if (connSock == -1)
{
return (-1);
}
memset(&locaddr, 0, sizeof(locaddr));
locaddr.sin_family = AF_INET;
locaddr.sin_port = htons(port);
locaddr.sin_addr.s_addr = inet_addr(host);
ret = bind(connSock, (struct sockaddr *)&locaddr, sizeof(locaddr));
if (ret == -1)
{
return (-1);
}
memset (&initmsg, 0, sizeof (initmsg));
initmsg.sinit_num_ostreams = 5;
initmsg.sinit_max_instreams = 5;
initmsg.sinit_max_attempts = 10;
ret = setsockopt(connSock, IPPROTO_SCTP, SCTP_INITMSG, &initmsg, sizeof(initmsg));
if (ret == -1)
{
return (-1);
}
memset (&servaddr, 0, sizeof (servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons (remote_port);
servaddr.sin_addr.s_addr = inet_addr (remote_host);
if((ret = connect (connSock, (struct sockaddr *) &servaddr, sizeof (servaddr))) < 0)
if (errno != EINPROGRESS)
return (-1);
if (ret == 0) {
fcntl(connSock, F_SETFL, flags);
return 0;
}
FD_ZERO(&rset);
FD_SET(connSock, &rset);
wset = rset;
tval.tv_sec = timeout;
tval.tv_usec = 0;
ret = select(connSock+1, &rset, &wset, NULL, timeout ? &tval : NULL);
if (ret == 0) {
close(connSock);
errno = ETIMEDOUT;
return(-1);
}
else if (ret < 0) {
return(-1);
}
fcntl(connSock, F_SETFL, flags);
opt_len = (socklen_t) sizeof(struct sctp_status);
getsockopt(connSock, IPPROTO_SCTP, SCTP_STATUS, &status, &opt_len);
printf ("assoc id = %d\n", status.sstat_assoc_id);
printf ("state = %d\n", status.sstat_state);
printf ("instrms = %d\n", status.sstat_instrms);
printf ("outstrms = %d\n", status.sstat_outstrms);
return 0;
}
int sendMessage(char remote_host[], int remote_port, char data[]) {
int ret, flags;
struct sockaddr_in servaddr;
char reply[1024];
errno = 0;
memset (&servaddr, 0, sizeof (servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons (remote_port);
servaddr.sin_addr.s_addr = inet_addr (remote_host);
printf("\nSending %s (%li bytes)", data, strlen(data));
ret = sctp_sendmsg (connSock, (void *) data, (size_t) strlen (data),
(struct sockaddr *) &servaddr, sizeof (servaddr), 46, 0, 0, 0, 0);
if (ret == -1)
{
printf("\nError sending errno(%d)", errno);
return -1;
}
else {
ret = sctp_recvmsg (connSock, (void *) reply, sizeof (reply), NULL,
NULL, NULL, &flags);
if (ret == -1)
{
printf("\nError receiving errno(%d)", errno);
return -1;
}
else {
printf("\nServer replied with %s", reply);
return 0;
}
}
}
int getSocket() {
return connSock;
}
I don't know if there's anything significant I need to set first before connecting that I missed out. I got the snippet from different sources so it's quite messy.
Another update, here's the tshark log of that code when executed:
3336.919408 local -> remote SCTP 82 INIT
3337.006690 remote -> local SCTP 810 INIT_ACK
3337.006727 local -> remote SCTP 774 COOKIE_ECHO
3337.085390 remote -> local SCTP 50 COOKIE_ACK
3337.086650 local -> remote SCTP 94 DATA
3337.087277 remote -> local SCTP 58 ABORT
3337.165266 remote -> local SCTP 50 ABORT
Detailed tshark log of this here.
Looks like the remote sent its COOKIE_ACK chunk but my client failed to set its state to ESTABLISHED (I double checked the sstate value of 3 here).
If the association setup processes completed the state should be SCTP_ESTABLISHED. SCTP_COOKIE_ECHOED indicated that association has not completely established. It means that originating side (your localhost in this case) has sent (once or several times) COOKIE_ECHO chunk which has not been acknowledged by COOKIE_ACK from remote end.
You can send messages in this state (SCTP will simply buffer it until it get COOKIE_ACK and resend it later on).
It is hard to say what went wrong based on information you provided. At this stage it is probably will be worth diving into wireshark trace, to see what remote side is replying on your COOKIE_ECHO.
Also if you can share your client/server side code that might help to identify the root cause.
UPDATE #1:
It should be also noted that application can abort association them self (e.g. if this association is not configured on that server). If you trying to connect to the random server (rather than your specific one) that is quite possible and actually makes sense in your case. In this case state of association on your side is COOKIE_ECHOED because COOKIE_ACK has not arrived yet (just a race condition). As I said previously SCTP happily accepts your data in this state and just buffers it until it receives COOKIE_ACK. SCTP on remote side sends COOKIE_ACK straight away, even before the application received execution control in accept(). If application decided to terminate the association in ungraceful way, it will send ABORT (that is your first ABORT in wireshark trace). Your side has not received this ABORT yet and sends DATA chunk. Since remote side considers this association as already terminated it cannot process DATA chunk, so it treats it as out of the blue (see RFC 4960 chapter 8.4) and sends another ABORT with t-bit set to 1.
I guess this is what happened in your case. You can confirm it easily just by looking into wireshark trace.

Why is RIOSend slower than classic winapi socket send?

I am testing two apps with TCP sockets. First app uses new RIO socket calls, the second app uses classic Winsock socket calls. Both apps send a single string to a server. I made them both use non-blocking sockets and disabled the Nagle algorithm. When I measure the time it takes to execute the sends, contrary to my expectations the classic socket is faster. What is the reason for this?
Here is my code:
/*
Classic socket
*/
SOCKET ConnectSocket = INVALID_SOCKET;
// ... socket initialization
// Make socket non-blocking and disable Nagle's algorithm
unsigned long mode = 1;
ioctlsocket(ConnectSocket, FIONBIO, &mode);
int isOn = 1;
setsockopt(ConnectSocket, IPPROTO_TCP, TCP_NODELAY, (char *) &isOn, sizeof(int) );
// Measure exec time
t1 = __rdtscp(&dum);
iResult = send(ConnectSocket, sendbuf, (int)strlen(sendbuf), 0);
t2 = __rdtscp(&dum);
cout << (t2 - t1) / 3000.0f << " 10^-6 sec" << endl;
/*
RIO socket
*/
SOCKET s = WSASocket(
AF_INET,
SOCK_STREAM,
IPPROTO_TCP,
NULL,
0,
WSA_FLAG_REGISTERED_IO);
// ... socket initialization
// Make socket non-blocking and disable Nagel's algorithm
unsigned long mode = 1;
ioctlsocket(s, FIONBIO, &mode);
int isOn = 1;
setsockopt(s, IPPROTO_TCP, TCP_NODELAY, (char *)&isOn, sizeof(int));
// RIO socket is event-based
hEvent = WSACreateEvent();
RIO_NOTIFICATION_COMPLETION type;
type.Type = RIO_EVENT_COMPLETION;
type.Event.EventHandle = hEvent;
type.Event.NotifyReset = TRUE;
RIO_CQ complQueue = rioFuncTable.RIOCreateCompletionQueue(1000, &type);
RIO_RQ requestQueue = rioFuncTable.RIOCreateRequestQueue(
s,
10,
1,
10,
1,
complQueue,
complQueue,
NULL);
// Init rio buffers
char *pSndBuffer = new char[sndBufferSize + 1]();
std::copy(message, message + sndBufferSize, pSndBuffer); // put message to a server here
RIO_BUFFERID sndBufID = rioFuncTable.RIORegisterBuffer(pSndBuffer, sndBufferSize);
RIO_BUF sndBuffer;
sndBuffer.BufferId = sndBufID;
sndBuffer.Offset = 0;
sndBuffer.Length = sndBufferSize;
// Measure exec time
t1 = __rdtscp(&dum);
rioFuncTable.RIOSend(requestQueue, &sndBuffer, 1, 0, NULL);
t2 = __rdtscp(&dum);
cout << (t2 - t1) / 3000.0f << " 10^-6 sec" << endl;
RioSend() sends for real. send() memcopies the buffer to socket's tx buffer.

pselect problems with FD_ISSET on cygwin

I'm running cygwin and use pselect to monitor a socket and filedescriptors for child processes.
According to some example that I found here http://www.linuxprogrammingblog.com/code-examples/using-pselect-to-avoid-a-signal-race and the man pages pselect should return the number of filedescriptors which are set in the mask fields (http://linux.die.net/man/2/pselect).
Now when I connect to my server pselect returns, which is fine. But when I test for the filedescriptors using FD_ISSET they always return true:
FD_ZERO(&readers);
FD_ZERO(&writers);
FD_ZERO(&exceptions);
FD_SET(fileno(stdin), &readers);
FD_SET(socket, &readers);
pret = pselect(FD_SETSIZE, &readers, &writers, &exceptions, NULL, &mSignalMask);
, &readers, &writers, &exceptions, NULL, &mSignalMask);
if(pret <= 0)
{
// ignore for now
continue;
}
if(FD_ISSET(fileno(stdin), &readers))
{
string s;
cin >> s;
cout << "stdin: " << s << endl; // blocks because the code always gets here even when
// pselect returns because of a childsignal without any data.
continue;
}
if(FD_ISSET(socket, &readers))
{
accept(); // blocks because the code always gets here even when
// pselect returns because of a childsignal without any data.
cout << "task created connection from " <<task->getClientHost() << endl;
continue;
}
Found the problem myself. FD_ISSET can ONLY be used if the result from pselect is > 0, otherweise the returnvalue from FD_ISSET is as before the call. So best treat it as undefined when pselect returns <= 0;

Flexible socket application

I'm writing a game wich playing on LAN with socket. I use 4 bytes length prefix to know how many data in the rest like this:
void trust_recv(int sock, int length, char *buffer)
{
int recved = 0;
int justRecv;
while(recved < length) {
justRecv = recv(sock, buffer + recved, length - recved, 0);
if (justRecv < 0) return;
recved += justRecv;
}
}
void onDataArrival(int sock)
{
int length;
char *data;
trust_recv(sock, 4, (char *) &length);
data = new char[length];
trust_recv(sock, length, data);
do_somethings_with_data(data);
}
The problem is if someone (an intruder or hacker for example) sends data with other format (maybe only 2 bytes or the length of the rest lower than 4 bytes prefix value) or an network problem, my application will be go to "not responding" state and have to close (because I use blocking socket). How to make my socket application more flexible but don't swith socket to non-blocking mode to pass this issue? (Or any ideas for organize data or algorithms as well)
You can set a receive timeout, during the socket setup phase, with setsockopt() call and SO_RCVTIMEO parameter;
struct timeval tv;
tv.tv_sec =8;
tv.tv_usec = 0 ;
if (setsockopt (your_sock_fd, SOL_SOCKET, SO_RCVTIMEO, (char *)&tv, sizeof tv)
perror(“setsockopt error”);
then test the return of recv() and his errno
if (justRecv < 0)
{
if (errno == EAGAIN)
perror("TIMEOUT!");
return;
}