C select is overwriting timeout value [duplicate] - sockets

This question already has answers here:
Is timeout changed after a call to select in c?
(5 answers)
Closed 25 days ago.
In a very simple C program, using select() to check any new read data on a socket, when I use the optional timeout parameter, it is being overwritten by select(). It looks like it resets it to values of seconds and microseconds it actually waited, so when data is coming sooner than the timeout, it will have much smaller values, leading to smaller and smaller timeouts unless timeout is reset, when select() is called in a loop.
I could not find any information on this behavior in select() description. I am using Linux Ubuntu 18.04 in my testing. It looks like I have to reset the timeout value every time before calling select() to keep the same timeout?
The code snippet is this:
void *main_udp_loop(void *arg)
{
struct UDP_CTX *ctx = (UDP_CTX*)arg;
fd_set readfds = {};
struct sockaddr peer_addr = { 0 };
int peer_addr_len = sizeof(peer_addr);
while (1)
{
struct timeval timeout;
timeout.tv_sec = 0;
timeout.tv_usec = 850000; // wait 0.85 second.
FD_ZERO(&readfds);
FD_SET(ctx->udp_socketfd, &readfds);
int activity = select( ctx->udp_socketfd + 1 , &readfds , NULL , NULL , &timeout);
if ((activity < 0) && (errno != EINTR))
{
printf("Select error: Exiting main thread\n");
return NULL;
}
if (timeout.tv_usec != 850000)
{
printf ("Timeout changed: %ld %ld\n", (long)timeout.tv_sec, (long)timeout.tv_usec);
}
if (activity == 0)
{
printf ("No activity from select: %ld \n", (long)time(0));
continue;
}
...
}

This is documented behavior in the Linux select() man page:
On Linux, select() modifies timeout to reflect the amount of time not slept; most other implementations do not do this. (POSIX.1 permits either behavior.) This causes problems both when Linux code which reads timeout is ported to other operating systems, and when code is ported to Linux that reuses a struct timeval for multiple select()s in a loop without reinitializing it. Consider timeout to be undefined after select() returns.
So, yes, you have to reset the timeout value every time you call select().

Related

TLS connection problem in iOS due to Socket File descriptor value increase more than 1023

We are using RESIProcate Lib in iOS, here in socket.hxx select() was used for getting FD value.
we are facing issue that FD value is getting increased in every new TLS connection establishment.
socket.hxx -->> https://github.com/resiprocate/resiprocate/blob/master/rutil/Socket.hxx
int select(struct timeval& tv)
{
return numReady = ::select(size, &read, &write, &except, &tv);
}
int selectMilliSeconds(unsigned long ms)
{
struct timeval tv;
tv.tv_sec = (ms/1000);
tv.tv_usec = (ms%1000)*1000;
return select(tv);
}
from Linux manual it is written that:
select() can monitor only file descriptors numbers that are less than FD_SETSIZE (1024)—an unreasonably low limit for many modern applications—and this limitation will not change.
(https://www.man7.org/linux/man-pages/man2/select.2.html)
we faced this problem with iOS platform. when size param in select() is getting more that 1024 value.
due to this:
resip_assert(read.fd_count < FD_SETSIZE); // Ensure there is room to add new FD
#endif
FD_SET(fd, &read);
size = ( int(fd+1) > size) ? int(fd+1) : size;
tls connection is getting interrupted and not proceeding further.
So is there any way in the current resip to overcome this situation if FD value becomes more than 1023?
in internalTransport.cxx -> fd = ::socket(ipVer == V4 ? PF_INET : PF_INET6, SOCK_STREAM, 0);
when fd is becoming more than 1023, we are facing the above-mentioned problem.
Is there any way to control this value so that it may always stay under 1023 in iOS?
one more thing:
is there any relation with server and client connection time period, which may cause the FD value increase on the client side in each new set of TCP/TLS connections by using the socket method even though the previous socket was closed?

STM32 Keil - Can not access target while debugging (AT Command UART)

I am trying to communicate with GSM module via UART communication. I could get message from the module as I expected. However when it comes to while loop (it is empty), debug session ends with "can not access target" error. Stepo by step, I am going to share my code:
Function 1 is AT_Send. (Note: Some of variables are declared globally.)
int AT_Send(UART_HandleTypeDef *huart, ATHandleTypedef *hat, unsigned char *sendBuffer, uint8_t ssize, unsigned char *responseBuffer, uint8_t rsize) {
if (HAL_UART_Transmit_IT(huart,sendBuffer,ssize) != HAL_OK) {
return -1;
}
while ((HAL_UART_GetState(huart) & HAL_UART_STATE_BUSY_TX) == HAL_UART_STATE_BUSY_TX) {
continue;
}
//;HAL_Delay(1000);
if (strstr((char*)receiveBuffer,(char*)responseBuffer) != NULL) {
rxIndex = 0;
memset(command, 0, sizeof(command));
return 0;
}
rxIndex = 0;
memset(command, 0, sizeof(command));
return 1;
}
Second function is AT_Init function. It sends AT to get OK response. At this point on, if I am not wrong, I am opening receive interrrupt and I am trying to get 1 byte.
int AT_Init(UART_HandleTypeDef *huart, ATHandleTypedef *hat)
{
HAL_UART_Receive_IT(huart,&rData,1);
tx = AT_Send(huart,hat,"AT\r",sizeof("AT\r\n"),"OK\r\n",sizeof("OK\r\n"));
return tx;
}
After these two functions, I am calling receive IT function in the call back while there are data on the bus.
void HAL_UART_RxCpltCallback(UART_HandleTypeDef *huart)
{
if (huart->Instance == USART1){
command[rxIndex] = rData;
rxIndex++;
if((rxIndex == 2) && (strstr((char*)command,"\r\n") != NULL)) {
rxIndex = 0;
} else if (strstr((char*)command,"\r\n") != NULL) {
memcpy(receiveBuffer, command, sizeof(command));
rxIndex = 0;
memset(command,0,sizeof(command));
}
HAL_UART_Receive_IT(&huart1,&rData,1);
}
}
Moreover, I am going to send a few HTTP commands simultaneously if I can get rid of this problem.
Can anyone share his/her knowledge?
Edit: Main function is shown below
tx = AT_Init(&huart1,&hat);
while (1)
{
HAL_GPIO_TogglePin(GPIOB,GPIO_PIN_3);
HAL_Delay(500);
}
Edit 2: I had replaced uart channel by USART2, and debugger worked. I suppose that it is related to hardware. Still, I am curious about possible reasons that cause to this problem.
The question doesn't mention on which µC the program is running, I only see the "stm32" tag. Similarly, we don't know which debug protocol is used (JTAG or SWD?).
Still, I dare to guess that the toggle command for GPIO port PB3 in the main loop is causing the observations: On many (most? all?) STM32 controllers, PB3 is used as JTDO pin, which is needed for JTAG debug connections.
Please make sure to configure the debug connection to SWD (without SWO, i.e., neither SWV is correct). It may also help to check the wiring of the debug cable, the fast toggle at the PB3/JTDO line may influence the signal levels on some neighbouring SWD lines if the wiring is low quality or a fast SWD speed has been chosen.
My hypothesis can be falsified by removing all actions to PB3. If the problem remains, I'm wrong.

Detecting CAN bus errors under socketCAN linux driver

Our products are using a well known CANopen stack, which uses socketCAN, on an embedded Beaglebone Black based system running under Ubuntu 14.04 LTS. But for some reason, even though the stack we're using will detect when the CAN bus goes into a PASSIVE state or even a BUS OFF state, it never indicates when the CAN bus recovers from errors and goes out of a PASSIVE or warning state, and enters a non error state.
If I were to query the socketCAN driver directly (via ioctl calls), would I be able to detect when the CAN bus goes in and out of a warning state (which is less than 127 errors), in and out of a PASSIVE state (greater than 127 errors) or goes BUS OFF (greater than 255 errors)?
I'd like to know if I'd be wasting my time doing this or is there a better way to detect, accurately and in real-time, all conditions of a CAN bus?
I have only a partial solution to that problem.
As you are using socketCAN, the interface is seen as a standard network interface, on which we can query the status.
Based on How to check Ethernet in Linux? (replace "eth0" by "can0"), you can check the link status.
This is not real-time, but can be executed in a periodic thread to check the bus state.
So while this is an old question, I just happened to stumble upon it (while searching for something only mildly related).
SocketCAN provides all the means for detecting error frames OOB.
Assuming your code looks similar to this:
int readFromCan(int socketFd, unsigned char* data, unsigned int* rxId) {
int32_t bytesRead = -1;
struct can_frame canFrame = {0};
bytesRead = (int32_t)read(socketFd, &canFrame, sizeof(can_frame));
if (bytesRead >= 0) {
bytesRead = canFrame.can_dlc;
if (data) {
memcpy(data, canFrame.data, readBytes);
}
if (rxId) {
*rxId = canFrame.can_id; // This will come in handy
}
}
return bytesRead;
}
void doStuffWithMessage() {
int32_t mySocketFd = fooGetSocketFd();
int32_t receiveId = 0;
unsigned char myData[8] = {0};
int32_t dataLength = 0;
if ((dataLength = readFromCan(mySocketFd, myData, &receiveId) == -1) {
// Handle error
return;
}
if (receiveId & CAN_ERR_MASK != 0) {
// Handle error frame
return;
}
// Do stuff with your data
}

Is select() + non-blocking write() possible on a blocking pipe or socket?

The situation is that I have a blocking pipe or socket fd to which I want to write() without blocking, so I do a select() first, but that still doesn't guarantee that write() will not block.
Here is the data I have gathered. Even if select() indicates that
writing is possible, writing more than PIPE_BUF bytes can block.
However, writing at most PIPE_BUF bytes doesn't seem to block in
practice, but it is not mandated by the POSIX spec.
That only specifies atomic behavior. Python(!) documentation states that:
Files reported as ready for writing by select(), poll() or similar
interfaces in this module are guaranteed to not block on a write of up
to PIPE_BUF bytes. This value is guaranteed by POSIX to be at least
512.
In the following test program, set BUF_BYTES to say 100000 to block in
write() on Linux, FreeBSD or Solaris following a successful select. I
assume that named pipes have similar behavior to anonymous pipes.
Unfortunately the same can happen with blocking sockets. Call
test_socket() in main() and use a largish BUF_BYTES (100000 is good
here too). It's unclear whether there is a safe buffer size like
PIPE_BUF for sockets.
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <limits.h>
#include <stdio.h>
#include <sys/select.h>
#include <unistd.h>
#define BUF_BYTES PIPE_BUF
char buf[BUF_BYTES];
int
probe_with_select(int nfds, fd_set *readfds, fd_set *writefds,
fd_set *exceptfds)
{
struct timeval timeout = {0, 0};
int n_found = select(nfds, readfds, writefds, exceptfds, &timeout);
if (n_found == -1) {
perror("select");
}
return n_found;
}
void
check_if_readable(int fd)
{
fd_set fdset;
FD_ZERO(&fdset);
FD_SET(fd, &fdset);
printf("select() for read on fd %d returned %d\n",
fd, probe_with_select(fd + 1, &fdset, 0, 0));
}
void
check_if_writable(int fd)
{
fd_set fdset;
FD_ZERO(&fdset);
FD_SET(fd, &fdset);
int n_found = probe_with_select(fd + 1, 0, &fdset, 0);
printf("select() for write on fd %d returned %d\n", fd, n_found);
/* if (n_found == 0) { */
/* printf("sleeping\n"); */
/* sleep(2); */
/* int n_found = probe_with_select(fd + 1, 0, &fdset, 0); */
/* printf("retried select() for write on fd %d returned %d\n", */
/* fd, n_found); */
/* } */
}
void
test_pipe(void)
{
int pipe_fds[2];
size_t written;
int i;
if (pipe(pipe_fds)) {
perror("pipe failed");
_exit(1);
}
printf("read side pipe fd: %d\n", pipe_fds[0]);
printf("write side pipe fd: %d\n", pipe_fds[1]);
for (i = 0; ; i++) {
printf("i = %d\n", i);
check_if_readable(pipe_fds[0]);
check_if_writable(pipe_fds[1]);
written = write(pipe_fds[1], buf, BUF_BYTES);
if (written == -1) {
perror("write");
_exit(-1);
}
printf("written %d bytes\n", written);
}
}
void
serve()
{
int listenfd = 0, connfd = 0;
struct sockaddr_in serv_addr;
listenfd = socket(AF_INET, SOCK_STREAM, 0);
memset(&serv_addr, '0', sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);
serv_addr.sin_port = htons(5000);
bind(listenfd, (struct sockaddr*)&serv_addr, sizeof(serv_addr));
listen(listenfd, 10);
connfd = accept(listenfd, (struct sockaddr*)NULL, NULL);
sleep(10);
}
int
connect_to_server()
{
int sockfd = 0, n = 0;
struct sockaddr_in serv_addr;
if((sockfd = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
perror("socket");
exit(-1);
}
memset(&serv_addr, '0', sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons(5000);
if(inet_pton(AF_INET, "127.0.0.1", &serv_addr.sin_addr) <= 0) {
perror("inet_pton");
exit(-1);
}
if (connect(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0) {
perror("connect");
exit(-1);
}
return sockfd;
}
void
test_socket(void)
{
if (fork() == 0) {
serve();
} else {
int fd;
int i;
int written;
sleep(1);
fd = connect_to_server();
for (i = 0; ; i++) {
printf("i = %d\n", i);
check_if_readable(fd);
check_if_writable(fd);
written = write(fd, buf, BUF_BYTES);
if (written == -1) {
perror("write");
_exit(-1);
}
printf("written %d bytes\n", written);
}
}
}
int
main(void)
{
test_pipe();
/* test_socket(); */
}
Unless you wish to send one byte at a time whenever select() says the fd is ready for writes, there is really no way to know how much you will be able to send and even then it is theoretically possible (at least in the documentation, if not in the real world) for select to say it's ready for writes and then the condition to change in the time between select() and write().
Non blocking sends are the solution here and you don't need to change your file descriptor to non blocking mode to send one message in non-blocking form if you change from using write() to send(). The only thing you need to change is to add the MSG_DONTWAIT flag to the send call and that will make the one send non-blocking without altering your socket's properties. You don't even need to use select() at all in this case either since the send() call will give you all the information you need in the return code - if you get a return code of -1 and the errno is EAGAIN or EWOULDBLOCK then you know you can't send any more.
The Posix section you cite clearly states:
[for pipes] If the O_NONBLOCK flag is clear, a write request may cause the thread to block, but on normal completion it shall return nbyte.
[for streams, which presumably includes streaming sockets] If O_NONBLOCK is clear, and the STREAM cannot accept data (the STREAM write queue is full due to internal flow control conditions), write() shall block until data can be accepted.
The Python documentation you quoted can therefore only apply to non-blocking mode only. But as you're not using Python it has no relevance anyway.
The answer by ckolivas is the correct one but, having read this post, I thought I could add some test data for interest's sake.
I quickly wrote a slow reading tcp server (sleeping 100ms between reads) which did a read of 4KB on each cycle. Then a fast writing client which I used for testing various scenarios on write. Both were using select before read (server) or write (client).
This was on Linux Mint 18 running under a Windows 7 VM (VirtualBox) with 1GB of memory assigned.
For the blocking case
If a write of a "certain number of bytes" became possible, select returned and the write either completed in total immediately or blocked until it completed. On my system, this "certain number of bytes" was at least 1MB. On the OP's system, this was clearly much less (less than 100,000).
So select did not return until a write of at least 1MB was possible. There was never a case (that I saw) where select would return if a smaller write would subsequently block. Thus select + write(x) where x was 4K or 8K or 128K never write blocked on this system.
This is all very well of course but this was an unloaded VM with 1GB of memory. Other systems would be expected to be different. However, I would expect that writes below a certain magic number (PIPE_BUF perhaps), issued subsequent to a select, would never block on all POSIX compliant systems. However (again) I don't see any documentation to that effect so one can't rely on that behaviour (even though the Python documentation clearly does). As the OP says, it's unclear whether there is a safe buffer size like PIPE_BUF for sockets. Which is a pity.
Which is what ckolivas' post says even though I'd argue that no rational system would return from a select when only a single byte was available!
Extra information:
At no point (in normal operation) did write return anything other than the full amount requested (or an error).
If the server was killed (ctrl-c), the client side write would immediately return a value (usually less than was requested - no normal operation!) with no other indication of error. The next select call would return immediately and the subsequent write would return -1 with errno saying "Connection reset by peer". Which is what one would expect - write as much as you can this time, fail the next time.
This (and EINTR) appears to be the only time write returns a number > 0 but less than requested.
If the server side was reading and the client was killed, the server continued to read all available data until it ran out. Then it read a zero and closed the socket.
For the non-blocking case:
The behaviour below some magic value is the same as above. select returns, write doesn't block (of course) and the write completes in its totality.
My issue was what happens otherwise. The send(2) man page says that in non-blocking mode, send fails with EAGAIN or EWOULDBLOCK. Which might imply (depending on how you read it) that it's all or nothing. Except that it also says select may be used to determine when it is possible to send more data. So it can't be all or nothing.
Write (which is the same as send with no flags), says it can return less than requested. This nitpicking seems pedantic but the man pages are the gospel so I read them as such.
In testing, a non-blocking write with a value larger than some particular value returned less than requested. This value wasn't constant, it changed from write to write but it was always pretty large (> 1 to 2MB).

Checking sockets using select() - Winsock

I have been exploring on function select() to check if some sockets are ready to read and I must admit that I'm a bit confused. The MSDN says "The select function returns the total number of socket handles that are ready and contained in the fd_set structures".
Suppose I have 3 sockets and 2 sockets are ready, select() returns 2, but this gives me no information which 2 of these 3 sockets are ready to read so how can I check it?
On stack overflow I came across this: When select returns, it has updated the sets to show which file descriptors have become ready for read/write/exception
So I put breakpoints in my program to track my fd_set structure. What I have realized is that ( just one socket in fd_set): If socket is ready to read select():
returns 1
leaves fd_count (The number of sockets in the set) untouched
leaves fd_array (An array of sockets that are in the set.) untouched
If client did not send any data addressed to that socket select():
returns 0
decreases fd_count to 0
leaves fd_array untouched
If I call select() again and client again sent no data:
return -1 (I think this is because of the fd_count value - 0)
I guess I miss some crucial rules how select() works and what this function does but I can't figure out it.
Here is some code snippet to show what I do to call select():
CServer::CServer(char *ipAddress,short int portNumber)
{ // Creating socket
ServerSocket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
if (ServerSocket == INVALID_SOCKET)
std::cout << "I was not able to create ServerSocket\n";
else
std::cout << "ServerSocket created successfully\n";
// Initialization of ServerSocket Address
ServerSocketAddress.sin_family = AF_INET;
ServerSocketAddress.sin_addr.S_un.S_addr = inet_addr(ipAddress);
ServerSocketAddress.sin_port = htons(portNumber);
// Binding ServerSocket to ServerSocket Address
if (bind(ServerSocket, (SOCKADDR*)&ServerSocketAddress, sizeof(ServerSocketAddress)) == 0)
std::cout << "Binding ServersSocket and ServerSocketAddress ended with success\n";
else
std::cout << "There were problems with binding ServerSocket and ServerSocket Address\n";
// Initialization of the set of sockets
ServerSet.fd_count = 1;
ServerSet.fd_array[0] = ServerSocket;
}
In main :
CServer Server(IP_LOOPBACK_ADDRESS, 500);
tmp = select(0, &Server.ServerSet, NULL, NULL, &TimeOut);
Should't the fd_array be filled with 0 values after the select() call, when there is no socket that can be read?
You're suppose to use the FD_SET macro and friends. You're not doing that.