Getting problems with sockets and select - sockets

I am implementing a socket programming project in C. I am using
select()
for waiting for data from client. I have two UDP sockets and select call is always ignoring one of my sockets. Can anybody briefly describe where should I start looking for it? this is what my server is doing
waitThreshold.tv_sec = 5000;
waitThreshold.tv_usec = 50;
if(sd > sd1)
max_sd = (sd + 1);
else if(sd1 > sd)
max_sd = (sd1 + 1);
FD_ZERO(&read_sds);
FD_SET(sd, &read_sds);
FD_SET(sd1, &read_sds);
ret = select(max_sd, &read_sds, NULL, NULL, &waitThreshold);
if(ret <0)
{
printf("\nSelect thrown an exception\n");
return 0;
}
else if(FD_ISSET(sd, &read_sds))
{
// code for socket one
}
else if(FD_ISSET(sd1, &read_sds))
{
// code for socket two
}

You wrote else if , so just one of them will run.

Generally speaking, when pooling multiple sockets using select() you want to use a for loop instead of branching the code with IFs. Also take note of the fact that select CHANGES the fd_set arguments (the read, write and error file descriptor sets - 2nd, 3rd and 4th arguments) and that you need to re-set them before each select(). A pretty general code layout for selecting sockets that have data to read with multiple concurrent connections would be something like this:
FD_ZERO(&master_sds);
FD_ZERO(&read_sds);
for (i=0; i<number_of_sockets); i++){
FD_SET(sd[i], &master_sds);
if sd[i] > max_sd {
max_sd=sd[i];
}
}
for(;;){
read_sds=master_sds;
ret = select(max_sd, &read_sds, NULL, NULL, &waitThreshold);
if(ret<0){
printf("\nSelect thrown an exception\n");
return 0;
}
for(i=0; i<max_sd; i++){
if FD_ISSET(sd[i], &read_fds){
// code for socket i
}
}
}
You might not want to have an endless loop to pool the sockets for data, you can insert some condition like receiving specific data on one of the sockets or specific user input as an exit condition. Hope this helps.

Related

Zerocopy TCP message sending from kernel module error EFAULT

I am working on a kernel module which receives data over DMA from an FPGA and stores it in a ring buffer allocated with dma_alloc_attrs(dev, size, &data->dma_addr, GFP_KERNEL, DMA_ATTR_FORCE_CONTIGUOUS). Everytime when new data is available in the ring buffer, a completion is fired.
In the same kernel module, I am running a TCP server and during the lifetime of the kernel module only one client (on a different machine) connects to the server(and stays connected). A separate thread in the kernel module sends data received in the ring buffer to the connected client whenever the completion was fired. The idea behind having a tcp server in the kernel space is to get rid of the unnecessary context switches from kernel space and user space whenever the data should be sent to the client, thus increasing performance. So far everything works, but the performance isn't as expected (on the TCP side).
After looking a bit into how to increase performance, i found the ZEROCOPY option.
I changed the settings of the server socket to set the SO_ZEROCOPY flag: kernel_setsockopt(socket, SOL_SOCKET, SO_ZEROCOPY, (char *)&one, sizeof(one)) and the implementation of the sending to client to:
static DEFINE_MUTEX(tcp_send_mtx);
static int send(struct socket *sock, const char *buf,
const size_t length, unsigned long flags)
{
struct msghdr msg;
struct kvec vec;
int len, written = 0;
int left = length;
if(sock == NULL)
{
printk(KERN_ERR MODULE_NAME ": tcp server send socket is NULL\n");
return -EFAULT;
}
msg.msg_name = 0;
msg.msg_namelen = 0;
msg.msg_control = NULL;
msg.msg_controllen = 0;
msg.msg_flags = MSG_ZEROCOPY;
repeat_send:
vec.iov_len = left;
vec.iov_base = (char *)buf + written;
len = kernel_sendmsg(sock, &msg, &vec, left, left);
if((len == -ERESTARTSYS) || (!(flags & MSG_DONTWAIT) && (len == -EAGAIN)))
goto repeat_send;
if(len > 0)
{
written += len;
left -= len;
if(left)
goto repeat_send;
}
return written?written:len;
}
Note the msg.msg_flags = MSG_ZEROCOPY; assignment in the send function.
Now when i am trying to use this, I am getting EFAULT(-14) error code from kernel_sendmsg just by adding the MSG_ZEROCOPY flag.
UPDATE:
I understand now that the ZEROCOPY flag is wrongly used in the kernel space since it's designed to remove the additional copy between the user-space and kernel-space.
My initial problem still exists. TCP transfer is still slow and the ring buffer overflows when the DMA transfer speed exceeds 120mb/s. The thread that forwards the messages to the client is not able to send the 8kb messages faster than 120mb/s.
Anyone knows what is wrong here? Maybe that the idea is wrong in the first place

Implementation of Poll Mechanism in Char Device Driver

Hello Dear participants of stackoverflow,
I'm new to kernel space development and still in the beginning of the road.
I developed a basic char device driver that can read open close etc . But couldn't find a proper source and how to tutorial for Poll/select mechanism sample.
I've written the sample code for poll function below:
static unsigned int dev_poll(struct file * file, poll_table *wait)
{
poll_wait(file,&dev_wait,wait);
if (size_of_message > 0 ){
printk(KERN_INFO "size_of_message > 0 returning POLLIN | POLLRDNORM\n");
return POLLIN | POLLRDNORM;
}
else {
printk(KERN_INFO "dev_poll return 0\n");
return 0;
}
}
It works fine but couldn't undestand a few things.
When I call select from user space program as
struct timeval time = {5,0 } ;
select(fd + 1 , &readfs,NULL,NULL,&time);
the dev_poll function in driver called once and return zero or POLLIN in order to buffer size . And then never called again. In user space , after 5 seconds the program continue if dev_poll returned 0.
What I couldn't understand is here , How the driver code will decide and let user space program if there is something in buffer that is readable withing this 5 seconds , if it's called once and returned immediately.
Is there anyway in kernel module to gather information of timeval parameter that comes from userspace ?
Thank you from now on.
Regards,
Call poll_wait() actually places some wait object into a waitqueue, specified as a second parameter. When wait object is fired (via waitqueue's wake_up or similar function), the poll function is evaluated again.
Kernel driver needn't to bother about timeouts: when time is out, the wait object will be removed from the waitqueue automatically.
Hello dear curious people like me about poll . I came up with a solution.
From another topic on stackowerflow a guy said that the poll_function is called multiple times if kernel need to last situation. So basically I implement that code .
when poll called call wait_poll(wait_queue_head);
when device have buffered data(this is usually in driver write function).
call wake_up macro with wait_queue_head paramater.
So after this step poll function of driver is called again .
So here you can return whatever you want to return. In this case POLLIN | POLLRDNORM..
Here is my sample code for write and poll in the driver.
static unsigned int dev_poll(struct file * file, poll_table *wait)
{
static int dev_poll_called_count = 0 ;
dev_poll_called_count ++;
poll_wait(file,&dev_wait,wait);
read_wait_queue_length++;
printk(KERN_INFO "Inside dev_poll called time is : %d read_wait_queue_length %d\n",dev_poll_called_count,read_wait_queue_length);
printk(KERN_INFO "After poll_wait wake_up called\n");
if (size_of_message > 0 ){
printk(KERN_INFO "size_of_message > 0 returning POLLIN | POLLRDNORM\n");
return POLLIN | POLLRDNORM;
}
else {
printk(KERN_INFO "dev_poll return 0\n");
return 0;
}
}
static ssize_t dev_write(struct file *filep, const char *buffer, size_t len, loff_t *offset){
printk(KERN_INFO "Inside write \n");;
int ret;
ret = copy_from_user(message, buffer, len);
size_of_message = len ;
printk(KERN_INFO "EBBChar: Received %zu characters from the user\n", size_of_message);
if (ret)
return -EFAULT;
message[len] = '\0';
printk(KERN_INFO "gelen string %s", message);
if (read_wait_queue_length)
{
wake_up(&dev_wait);
read_wait_queue_length = 0;
}
return len;
}

read() hangs while watching for event0 using inotify

I am in need to use /dev/input/event0 to be watched for key events.I have used inotify_add_watch(), but the read() call hangs.But If I cat /dev/input/event0 I can see some events.Please let me know what is wrong.Below is my code snippet
/creating the INOTIFY instance/
fd = inotify_init();
/*checking for error*/
if ( fd < 0 ) {
perror( "inotify_init" );
}
/*adding the /dev/input/event0 to watch list.*/
wd = inotify_add_watch(fd, "/dev/input/event0", IN_ALL_EVENTS);
if (wd < 0){
perror("inotify_add_watch");
exit(-1);
}
for (;;) {
length = read(fd, buffer,EVENT_BUF_LEN);
printf("length = %d",length);
if (length == 0)
perror("read() from inotify fd returned 0!");
if (length < 0)
perror("read");
printf("Read %ld bytes from inotify fd\n", (long) numRead);
You haven't explained why you think you need to use inotify for this.
I'm assuming that you just want to programmatically test whether an event is ready.
You can do something like:
int fd = open("/dev/input/event0", O_RDONLY|O_NONBLOCK);
struct pollfd pfd; // see man 2 poll
pfd.fd = fd;
pfd.events = POLLIN;
if (poll(&pfd, 1, &ts, 1000 /* milliseconds */) > 0) {
// reading from fd now will not block
}
This will wait for up to 1 second (1000 milliseconds) for an event to be ready to read. You can change the timeout to whatever you need. You can also use 0 to test whether there is data available immediately without waiting.
read() is a blocking function and waits for file update, unless specifically mentioned.
If you want it to be non-blocking, simply add a flag.
file_descriptor = inotify_init1(IN_NONBLOCK);
https://linux.die.net/man/2/inotify_init1

using select() to detect connection close

As described in other posts, I'm trying to use select() in socket programming to detect closed connections. See the following code which tries to detect closed connections by select() and a following check on whether recv() returns 0. Before the while loop starts, there are two established TCP connections already. In our controlled experiment, the first connection always closes after about 15 seconds and the second about 30 seconds.
Theoretically (as described by others), when they get closed, select() should return (twice in our case) which allows us to detect both close events. The problem we face is that the select() now only returns once and never again, which allows us to detect ONLY the first connection close event. If the code for one IP it works fine but not for two or more connections.
Anyone has any ideas or suggestions? Thanks.
while (1)
{
printf("Waiting on select()...\n");
if ((result = select(max + 1, & readset, NULL, NULL, NULL)) < 0)
{
printf("select() failed");
break;
}
if (result > 0)
{
i=0;
while(i<max+1)
{
if (FD_ISSET(i, &readset))
{
result = recv(i, buffer, sizeof(buffer), 0);
if (result == 0)
{
close(i);
FD_CLR(i, &readset);
if (i == max)
{
max -= 1;
}
}
}
i++;
}
}
}
select() modifies readset to remove socket(s) that are not readable. Every time you call select(), you have to reset and fill readset with your latest list of active sockets that you want to test, eg:
fd_set readset;
int max;
while (1)
{
FD_ZERO(&readset);
max = -1;
// populate readset from list of active sockets...
// set max according...
printf("Waiting on select()...\n");
result = select(max + 1, &readset, NULL, NULL, NULL);
if (result < 0)
{
printf("select() failed");
break;
}
if (result == 0)
continue;
for (int i = 0; i <= max; ++i)
{
if (FD_ISSET(i, &readset))
{
result = recv(i, buffer, sizeof(buffer), 0);
if (result <= 0)
{
close(i);
// remove i from list of active sockets...
}
}
}
}

Sending UDP packet in Linux Kernel

For a project, I'm trying to send UDP packets from Linux kernel-space. I'm currently 'hard-coding' my code into the kernel (which I appreciate isn't the best/neatest way) but I'm trying to get a simple test to work (sending "TEST"). It should be mentioned I'm a newbie to kernel hacking - I'm not that clued up on many principles and techniques!
Every time my code gets run the system hangs and I have to reboot - no mouse/keyboard response and the scroll and caps lock key lights flash together - I'm not sure what this means, but I'm assuming it's a kernel panic?
The repeat_send code is unnecessary for this test code, yet when it's working I want to send large messages that may require multiple 'send's - I'm not sure that if could be a cause of my issues?
N.B. This code is being inserted into neighbour.c of linux-source/net/core/ origin, hence the use of NEIGH_PRINTK1, it's just a macro wrapper round printk.
I'm really banging my head against a brick wall here, I can't spot anything obvious, can anyone point me in the right direction (or spot that blindingly obvious error!)?
Here's what I have so far:
void mymethod()
{
struct socket sock;
struct sockaddr_in addr_in;
int ret_val;
unsigned short port = htons(2048);
unsigned int host = in_aton("192.168.1.254");
unsigned int length = 5;
char *buf = "TEST\0";
struct msghdr msg;
struct iovec iov;
int len = 0, written = 0, left = length;
mm_segment_t oldmm;
NEIGH_PRINTK1("forwarding sk_buff at: %p.\n", skb);
if ((ret_val = sock_create(PF_INET, SOCK_DGRAM, IPPROTO_UDP, &sock)) < 0) {
NEIGH_PRINTK1("Error during creation of socket; terminating. code: %d\n", ret_val);
return;
}
memset(&addr_in, 0, sizeof(struct sockaddr_in));
addr_in.sin_family=AF_INET;
addr_in.sin_port = port;
addr_in.sin_addr.s_addr = host;
if((ret_val = sock.ops->bind(&sock, (struct sockaddr *)&addr_in, sizeof(struct sockaddr_in))) < 0) {
NEIGH_PRINTK1("Error trying to bind socket. code: %d\n", ret_val);
goto close;
}
memset(&msg, 0, sizeof(struct msghdr));
msg.msg_flags = 0;
msg.msg_name = &addr_in;
msg.msg_namelen = sizeof(struct sockaddr_in);
msg.msg_iov = &iov;
msg.msg_iovlen = 1;
msg.msg_control = NULL;
msg.msg_controllen = 0;
repeat_send:
msg.msg_iov->iov_len = left;
msg.msg_iov->iov_base = (char *)buf + written;
oldmm = get_fs();
set_fs(KERNEL_DS);
len = sock_sendmsg(&sock, &msg, left);
set_fs(oldmm);
if (len == -ERESTARTSYS)
goto repeat_send;
if (len > 0) {
written += len;
left -= len;
if (left)
goto repeat_send;
}
close:
sock_release(&sock);
}
Any help would be hugely appreciated, thanks!
You may find it easier to use the netpoll API for UDP. Take a look at netconsole for an example of how it's used. The APIs you're using are more intended for userspace (you should never have to play with segment descriptors to send network data!)
Run your code when you're in a text mode console (i.e. press Ctrl+Alt+F1 to go to the text console). This way a kernel panic will print out the stack trace and any extra information about what went wrong.
If that doesn't help you, update your question with the stack trace.
I'm not much of a Linux Kernel developer, but can you throw some printk's in there and watch dmesg before it goes down? Or have you thought about hooking up with a kernel debugger?
I think you should try to put all variables outside mymethod() function and make them static. Remember, that the size of kernel stack is limited do 8KiB, so to much of/too big local variables may cause stack overflow and system hangup.