Flexible socket application - sockets

I'm writing a game wich playing on LAN with socket. I use 4 bytes length prefix to know how many data in the rest like this:
void trust_recv(int sock, int length, char *buffer)
{
int recved = 0;
int justRecv;
while(recved < length) {
justRecv = recv(sock, buffer + recved, length - recved, 0);
if (justRecv < 0) return;
recved += justRecv;
}
}
void onDataArrival(int sock)
{
int length;
char *data;
trust_recv(sock, 4, (char *) &length);
data = new char[length];
trust_recv(sock, length, data);
do_somethings_with_data(data);
}
The problem is if someone (an intruder or hacker for example) sends data with other format (maybe only 2 bytes or the length of the rest lower than 4 bytes prefix value) or an network problem, my application will be go to "not responding" state and have to close (because I use blocking socket). How to make my socket application more flexible but don't swith socket to non-blocking mode to pass this issue? (Or any ideas for organize data or algorithms as well)

You can set a receive timeout, during the socket setup phase, with setsockopt() call and SO_RCVTIMEO parameter;
struct timeval tv;
tv.tv_sec =8;
tv.tv_usec = 0 ;
if (setsockopt (your_sock_fd, SOL_SOCKET, SO_RCVTIMEO, (char *)&tv, sizeof tv)
perror(“setsockopt error”);
then test the return of recv() and his errno
if (justRecv < 0)
{
if (errno == EAGAIN)
perror("TIMEOUT!");
return;
}

Related

How to deal with SOCKET in select method?

I saw this example in IBM docs on how to use select method for a server program, I would like to something similar to that on windows without using vectors and unordered_map but the problem I am facing is that windows uses SOCKET for socket descriptors and linux uses int although I can cast the windows socket into an integer it is not recommended and the socket value is bigger than FD_SETSIZE, that being said the for loop ends before reaching the server socket descriptor and the function ends up being kind of useless
int Server::handler()
{
int iResult;
timeval timeout;
fd_set activeFdSet;
fd_set readFdSet;
FD_ZERO(&activeFdSet);
FD_SET(serv_sock, &activeFdSet);
printf("FD_SETSIZE=%d\n", FD_SETSIZE);
// listen for incoming connections
iResult = listen(serv_sock, SOMAXCONN);
timeout.tv_sec = 5;
timeout.tv_usec= 0;
while (1)
{
readFdSet = activeFdSet;
printf("\tCopied activefdset to readfdset\n");
int res = select(FD_SETSIZE, &readFdSet, NULL,NULL,&timeout);
for (int i=0;i<FD_SETSIZE; i++)
{
if (FD_ISSET(i , &readFdSet)) // check socket descriptor
{
if (i == (int)serv_sock)
{
// accept connections to the server
}
else // client socket
{
// receive from client
}
}
}
}
return 0;
}
what is the best way to deal with the server socket in a for loop without using vectors or any other similar concepts
On non-Windows platforms, sockets are represented with file descriptors, which are basically indexes into a files table. That is why you can use int sockets as loop counters.
However, that is not the case with Windows sockets. A SOCKET is an opaque handle to an actual kernel object, so you can't use SOCKETs as loop counters, like you are trying to do. And do not cast them to int.
You really have no choice but to store the accepted sockets in an array or other container and then iterate through that instead, especially if you want the code to be portable across platforms, and particularly if you want to handle more than FD_SETSIZE number of clients (in which case, you should be using (e)poll() or other asynchronous socket I/O mechanism instead of select()), eg:
int Server::handler()
{
int iResult;
timeval timeout;
fd_set readFdSet;
int maxFd;
// listen for incoming connections
iResult = listen(serv_sock, SOMAXCONN);
timeout.tv_sec = 5;
timeout.tv_usec= 0;
while (1)
{
FD_ZERO(&readFdSet);
FD_SET(serv_sock, &readFdSet);
#ifdef WIN32
maxFd = -1; // not used on Windows
#else
maxFd = serv_sock;
#endif
for (each client_sock in list)
{
FD_SET(client_sock, &readFdSet);
#ifndef WIN32
if (client_sock > maxFd) maxFd = client_sock;
#endif
}
#endif
int res = select(maxFd+1, &readFdSet, NULL, NULL, &timeout);
if (res < 0) ... // error handling as needed...
if (FD_ISSET(serv_sock, &readFdSet))
{
// accept connections to the server, add to clients list
}
for (each client_sock in list)
{
if (FD_ISSET(client_sock, &readFdSet)) // check socket descriptor
{
// receive from client
}
}
}
return 0;
}
That being said, on Windows only, you can rely on Microsoft's documented implementation detail that fd_set has fd_count and fd_array[] members, so you can just iterate through the fd_set's internal array directly, eg:
int Server::handler()
{
int iResult;
timeval timeout;
fd_set readFdSet;
int maxFd;
// listen for incoming connections
iResult = listen(serv_sock, SOMAXCONN);
timeout.tv_sec = 5;
timeout.tv_usec= 0;
while (1)
{
FD_ZERO(&readFdSet);
FD_SET(serv_sock, &readFdSet);
#ifdef WIN32
maxFd = -1; // not used on Windows
#else
maxFd = serv_sock;
#endif
for (each client_sock in list)
{
FD_SET(client_sock, &readFdSet);
#ifndef WIN32
if (client_sock > maxFd) maxFd = client_sock;
#endif
}
#endif
int res = select(maxFd+1, &readFdSet, NULL, NULL, &timeout);
if (res < 0) ... // error handling as needed...
#ifdef WIN32
for (int i = 0; i < readFdSet.fd_count; ++i)
#else
for (int client_sock = 0; client_sock <= maxFd; ++client_sock)
#endif
{
#ifdef WIN32
SOCKET client_sock = readFdSet.fd_array[i];
#else
if (!FD_ISSET(client_sock, &readFdSet)) // check socket descriptor
continue;
#endif
if (client_sock == serv_sock)
{
// accept connections to the server, add to clients list
}
else // client socket
{
// receive from client
}
}
}
return 0;
}

What buffer collects the data sent through TCP sockets on localhost?

I have a client and server connected through TCP sockets on localhost.
I check with getsockopt that the server's SO_SNDBUF is small and the client's SO_RCVBUF is small (in my case both are 64KB)
I send twenty 500KB buffers from the server to the client, but in the client I've added a sleep for 500ms after each recv and I've capped the client receive buffer to 1MB.
What I observe is that the server very quickly rids itself of the 10MB of data which then arrives at the client in the next several seconds. 7-8MB are consistently in the "ether" in my experiments.
My question is what is this "ether"? It's obviously some buffer somewhere but can one tell which buffer it is?
Here is my test program.
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <thread>
#include <cstdio>
#include <vector>
#include <cstdlib>
#define PROXY 0
static std::vector<uint8_t> getRandomBuf() {
std::vector<uint8_t> buf;
buf.reserve(500 * 1024);
for (size_t i = 0; i < buf.capacity(); ++i) buf.push_back(rand() % 256);
return buf;
}
int server() {
auto sd = socket(AF_INET, SOCK_STREAM, 0);
if (sd < 0) return puts("socket fail");
sockaddr_in srv = {};
srv.sin_family = AF_INET;
srv.sin_addr.s_addr = INADDR_ANY;
srv.sin_port = htons(7654);
int enable = 1;
if (setsockopt(sd, SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(int)) < 0) {
return puts("setsockopt fail");
}
if (bind(sd, (sockaddr*)&srv, sizeof(srv)) < 0) {
return puts("bind fail");
}
listen(sd, 3);
puts("listening...");
sockaddr_in client;
socklen_t csz = sizeof(client);
auto sock = accept(sd, (sockaddr*)&client, &csz);
if (sock < 0) return puts("accept fail");
{
int data;
socklen_t size = sizeof(data);
getsockopt(sock, SOL_SOCKET, SO_SNDBUF, &data, &size);
printf("accepted: %d\n", int(data));
}
for (int i=0; i<20; ++i) {
auto buf = getRandomBuf();
puts("Server sending blob");
send(sock, buf.data(), buf.size(), 0);
puts(" Server completed send of blob");
}
while (true) std::this_thread::yield();
return close(sock);
}
int client() {
int sd = socket(AF_INET, SOCK_STREAM, 0);
if (sd < 0) return puts("socket fail");
sockaddr_in client = {};
client.sin_family = AF_INET;
client.sin_addr.s_addr = inet_addr("127.0.0.1");
#if PROXY
client.sin_port = htons(9654);
#else
client.sin_port = htons(7654);
#endif
if (connect(sd, (sockaddr*)&client, sizeof(client)) < 0) {
return puts("connect fail");
}
{
int data;
socklen_t size = sizeof(data);
getsockopt(sd, SOL_SOCKET, SO_RCVBUF, &data, &size);
printf("connected: %d\n", int(data));
}
std::vector<uint8_t> buf(1024*1024);
while (true) {
auto s = recv(sd, buf.data(), buf.size(), 0);
if (s <= 0) {
puts("recv fail");
break;
}
printf("Client received %.1f KB\n", double(s)/1024);
#if !PROXY
std::this_thread::sleep_for(std::chrono::milliseconds(500));
#endif
}
return close(sd);
}
int main() {
std::thread srv(server);
std::this_thread::sleep_for(std::chrono::milliseconds(300)); // give time for the server to start
client();
srv.join();
return 0;
}
Note that in the test program there is a #define PROXY 0.
In another experiment with PROXY set to 1, I ditch the sleep and instead connect the client to a throttling proxy (Charles) and throttle the bandwidth to 400KB/s. In this case the server rids itself of the 10MB almost immediately and they arrive in course of ~20 seconds on the client. I assume that the proxy is buffering, though I don't see a configuration in this particular one for the buffer size.
This is all done hunting for another (likely bufferbloat) issue in which the server sends 10MB with 20 packets from Denver to Amsterdam over an Internet connection which does indeed have a 400KB/s bandwidth. In this case the server, much like the throttling proxy example from above, rids itself of the 10MB almost immediately, and they arrive over the next 20 seconds on the client, leading to 20 second delays for any subsequent messages. Had they not left the server, I would've been able to reorder the packets and send higher-priority ones in-between the ones from the 10MB blob, and not have the client suffer a 20 second delay due to network clog.

Problem in reading packets from tunnel using read()

I have been trying to receive and process the packets from tunnel. There are separate blocks for processing v4 and v6 packets. If the packet does not fall under the either of the categories, they will be dropped. For me, every packets are being dropped during execution. When I used wireshark to capture the packets from the tunnel, I noticed the difference in packet size, i.e., length of the packet. For example, when the length of a received packet in Wireshark is 60 whereas the program prints it 64 as length. I noticed the 4 bytes difference in all packets. I am unable to find out, what I am doing wrong here? Would anyone help me. I also attached the screen of wireshark and program execution for perusal.
Image: Captured packets from tunnel through wireshark and program
#define MTU 1600
void processPacket(const uint8_t *packet, const size_t len) {
//1st octet identifies the IP version
uint8_t version = (*packet) >> 4;
//...
printf("IP version - %d\n", version);
if (version == 4 ) {
//ipv4 packet process ...
} else if (version == 6) {
//ipv6 packet process ...
} else {
//drop packet
printf("Unknown IP version, drop packet\n");
}
}
int main() {
struct ifreq ifr;
int fd;
uint8_t *buffer = (uint8_t *)(malloc(MTU));
ssize_t len;
if ( (fd = open("/dev/net/tun", O_RDWR)) == -1 ) {
perror("Unable to open /dev/net/tun");
exit(EXIT_FAILURE);
}
memset(&ifr, 0, sizeof(ifr));
ifr.ifr_flags = IFF_TUN;
strncpy(ifr.ifr_name, "tun0", IFNAMSIZ);
if ( (err = ioctl(fd, TUNSETIFF, (void *) &ifr)) == -1 ) {
perror("Error encountered during ioctl TUNSETIFF");
close(fd);
exit(EXIT_FAILURE);
}
printf("Device tun0 opened\n");
while(1) {
len = read(fd, buffer, MTU);
printf("Read %lu bytes from tun0\n", len);
processPacket(buffer, len);
}
printf("\nPress any key to exit...");
getchar();
close(fd);
}
The tunnel device pre-pends the IP packet with additional information, so the first byte is not the IP version. If you don't need it, you can add IFF_NO_PI to ifr_flags. See kernel documentation.

SCTP: What should be the sctp_status.sstate value of an SCTP socket after succesful connect() call?

I'm trying to connect to a remote peer (which I don't have directory access other than connecting to it via socket and ping) via SCTP. Assuming that I have connected succesfully, what should be the value of my sctp_status.sstate if I try calling getsocktopt()? Mine is SCTP_COOKIE_ECHOED(3) according to sctp.h. Is it correct? Shouldn't it be SCTP_ESTABLISHED?
Because I tried sending message to the remote peer with this code:
ret = sctp_sendmsg (connSock, (void *) data, (size_t) strlen (data), (struct sockaddr *) &servaddr, sizeof (servaddr), 46, 0, 0, 0, 0);
It returned the number of bytes I tried sending. Then when I tried catching if there's any response:
ret = sctp_recvmsg (connSock, (void *) reply, sizeof (reply), NULL,
NULL, NULL, &flags);
It returns -1 with errno of ECONNRESET(104). What are the possible mistakes in my code, or maybe in my flow? Did I miss something?
Thanks in advance for answering. Will gladly appreciate that. :)
Update: Here down below is my client code in connecting to the remote peer. It's actually a node addon for me to use since SCTP is not fully supported in node. Using lksctp-tools package to include the headers.
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <netinet/sctp.h>
#include <arpa/inet.h>
#include <signal.h>
#define MAX_BUFFER 1024
int connSock = 0;
int connect(char host[], int port, char remote_host[], int remote_port, int timeout) {
int ret, flags;
fd_set rset, wset;
struct sockaddr_in servaddr;
struct sockaddr_in locaddr;
struct sctp_initmsg initmsg;
struct timeval tval;
struct sctp_status status;
socklen_t opt_len;
errno = 0;
connSock = socket (AF_INET, SOCK_STREAM, IPPROTO_SCTP);
flags = fcntl(connSock, F_GETFL, 0);
fcntl(connSock, F_SETFL, flags | O_NONBLOCK);
if (connSock == -1)
{
return (-1);
}
memset(&locaddr, 0, sizeof(locaddr));
locaddr.sin_family = AF_INET;
locaddr.sin_port = htons(port);
locaddr.sin_addr.s_addr = inet_addr(host);
ret = bind(connSock, (struct sockaddr *)&locaddr, sizeof(locaddr));
if (ret == -1)
{
return (-1);
}
memset (&initmsg, 0, sizeof (initmsg));
initmsg.sinit_num_ostreams = 5;
initmsg.sinit_max_instreams = 5;
initmsg.sinit_max_attempts = 10;
ret = setsockopt(connSock, IPPROTO_SCTP, SCTP_INITMSG, &initmsg, sizeof(initmsg));
if (ret == -1)
{
return (-1);
}
memset (&servaddr, 0, sizeof (servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons (remote_port);
servaddr.sin_addr.s_addr = inet_addr (remote_host);
if((ret = connect (connSock, (struct sockaddr *) &servaddr, sizeof (servaddr))) < 0)
if (errno != EINPROGRESS)
return (-1);
if (ret == 0) {
fcntl(connSock, F_SETFL, flags);
return 0;
}
FD_ZERO(&rset);
FD_SET(connSock, &rset);
wset = rset;
tval.tv_sec = timeout;
tval.tv_usec = 0;
ret = select(connSock+1, &rset, &wset, NULL, timeout ? &tval : NULL);
if (ret == 0) {
close(connSock);
errno = ETIMEDOUT;
return(-1);
}
else if (ret < 0) {
return(-1);
}
fcntl(connSock, F_SETFL, flags);
opt_len = (socklen_t) sizeof(struct sctp_status);
getsockopt(connSock, IPPROTO_SCTP, SCTP_STATUS, &status, &opt_len);
printf ("assoc id = %d\n", status.sstat_assoc_id);
printf ("state = %d\n", status.sstat_state);
printf ("instrms = %d\n", status.sstat_instrms);
printf ("outstrms = %d\n", status.sstat_outstrms);
return 0;
}
int sendMessage(char remote_host[], int remote_port, char data[]) {
int ret, flags;
struct sockaddr_in servaddr;
char reply[1024];
errno = 0;
memset (&servaddr, 0, sizeof (servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons (remote_port);
servaddr.sin_addr.s_addr = inet_addr (remote_host);
printf("\nSending %s (%li bytes)", data, strlen(data));
ret = sctp_sendmsg (connSock, (void *) data, (size_t) strlen (data),
(struct sockaddr *) &servaddr, sizeof (servaddr), 46, 0, 0, 0, 0);
if (ret == -1)
{
printf("\nError sending errno(%d)", errno);
return -1;
}
else {
ret = sctp_recvmsg (connSock, (void *) reply, sizeof (reply), NULL,
NULL, NULL, &flags);
if (ret == -1)
{
printf("\nError receiving errno(%d)", errno);
return -1;
}
else {
printf("\nServer replied with %s", reply);
return 0;
}
}
}
int getSocket() {
return connSock;
}
I don't know if there's anything significant I need to set first before connecting that I missed out. I got the snippet from different sources so it's quite messy.
Another update, here's the tshark log of that code when executed:
3336.919408 local -> remote SCTP 82 INIT
3337.006690 remote -> local SCTP 810 INIT_ACK
3337.006727 local -> remote SCTP 774 COOKIE_ECHO
3337.085390 remote -> local SCTP 50 COOKIE_ACK
3337.086650 local -> remote SCTP 94 DATA
3337.087277 remote -> local SCTP 58 ABORT
3337.165266 remote -> local SCTP 50 ABORT
Detailed tshark log of this here.
Looks like the remote sent its COOKIE_ACK chunk but my client failed to set its state to ESTABLISHED (I double checked the sstate value of 3 here).
If the association setup processes completed the state should be SCTP_ESTABLISHED. SCTP_COOKIE_ECHOED indicated that association has not completely established. It means that originating side (your localhost in this case) has sent (once or several times) COOKIE_ECHO chunk which has not been acknowledged by COOKIE_ACK from remote end.
You can send messages in this state (SCTP will simply buffer it until it get COOKIE_ACK and resend it later on).
It is hard to say what went wrong based on information you provided. At this stage it is probably will be worth diving into wireshark trace, to see what remote side is replying on your COOKIE_ECHO.
Also if you can share your client/server side code that might help to identify the root cause.
UPDATE #1:
It should be also noted that application can abort association them self (e.g. if this association is not configured on that server). If you trying to connect to the random server (rather than your specific one) that is quite possible and actually makes sense in your case. In this case state of association on your side is COOKIE_ECHOED because COOKIE_ACK has not arrived yet (just a race condition). As I said previously SCTP happily accepts your data in this state and just buffers it until it receives COOKIE_ACK. SCTP on remote side sends COOKIE_ACK straight away, even before the application received execution control in accept(). If application decided to terminate the association in ungraceful way, it will send ABORT (that is your first ABORT in wireshark trace). Your side has not received this ABORT yet and sends DATA chunk. Since remote side considers this association as already terminated it cannot process DATA chunk, so it treats it as out of the blue (see RFC 4960 chapter 8.4) and sends another ABORT with t-bit set to 1.
I guess this is what happened in your case. You can confirm it easily just by looking into wireshark trace.

MSG_PROXY not working to provide/specify alternate addresses for transparent proxying

I'm trying to write a transparent proxy that translates arbitrary UDP packets to a custom protocol and back again. I'm trying to use transparent proxying to read the incoming UDP packets that need translation, and to write the outgoing UDP packets that have just been reverse-translated.
My setup for the socket I use for both flavors of UDP sockets is as follows:
static int
setup_clear_sock(uint16_t proxy_port)
{
struct sockaddr_in saddr;
int sock;
int val = 1;
socklen_t ttllen = sizeof(std_ttl);
sock = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP);
if (sock < 0)
{
perror("Failed to create clear proxy socket");
return -1;
}
if (getsockopt(sock, IPPROTO_IP, IP_TTL, &std_ttl, &ttllen) < 0)
{
perror("Failed to read IP TTL option on clear proxy socket");
return -1;
}
if (setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val)) < 0)
{
perror("Failed to set reuse address option on clear socket");
return -1;
}
if (setsockopt(sock, IPPROTO_IP, IP_TRANSPARENT, &val, sizeof(val)) < 0)
{
perror("Failed to set transparent proxy option on clear socket");
return -1;
}
saddr.sin_family = AF_INET;
saddr.sin_port = htons(proxy_port);
saddr.sin_addr.s_addr = INADDR_ANY;
if (bind(sock, (struct sockaddr *) &saddr, sizeof(saddr)) < 0)
{
perror("Failed to bind local address to clear proxy socket");
return -1;
}
return sock;
}
I have two distinct, but possibly related problems. First, when I read an incoming UDP packet from this socket, using this code:
struct sock_double_addr_in
{
__SOCKADDR_COMMON (sin_);
in_port_t sin_port_a;
struct in_addr sin_addr_a;
sa_family_t sin_family_b;
in_port_t sin_port_b;
struct in_addr sin_addr_b;
unsigned char sin_zero[sizeof(struct sockaddr) - __SOCKADDR_COMMON_SIZE - 8
- sizeof(struct in_addr) - sizeof(in_port_t)];
};
void
handle_clear_sock(void)
{
ssize_t rcvlen;
uint16_t nbo_udp_len, coded_len;
struct sockaddr_in saddr;
struct sock_double_addr_in sdaddr;
bch_coding_context_t ctx;
socklen_t addrlen = sizeof(sdaddr);
rcvlen = recvfrom(sock_clear, &clear_buf, sizeof(clear_buf),
MSG_DONTWAIT | MSG_PROXY,
(struct sockaddr *) &sdaddr, &addrlen);
if (rcvlen < 0)
{
perror("Failed to receive a packet from clear socket");
return;
}
....
I don't see a destination address come back in sdaddr. The sin_family_b, sin_addr_b, and sin_port_b fields are all zero. I've done a block memory dump of the structure in gdb, and indeed the bytes are coming back zero from the kernel (it's not a bad placement of the field in my structure definition).
Temporarily working around this by hard-coding a fixed IP address and port for testing purposes, I can debug the rest of my proxy application until I get to the point of sending an outgoing UDP packet that has just been reverse-translated. That happens with this code:
....
udp_len = ntohs(clear_buf.u16[2]);
if (udp_len + 6 > decoded_len)
fprintf(stderr, "Decoded fewer bytes (%u) than outputting in clear "
"(6 + %u)!\n", decoded_len, udp_len);
sdaddr.sin_family = AF_INET;
sdaddr.sin_port_a = clear_buf.u16[0];
sdaddr.sin_addr_a.s_addr = coded_buf.u32[4];
sdaddr.sin_family_b = AF_INET;
sdaddr.sin_port_b = clear_buf.u16[1];
sdaddr.sin_addr_b.s_addr = coded_buf.u32[3];
if (sendto(sock_clear, &(clear_buf.u16[3]), udp_len, MSG_PROXY,
(struct sockaddr *) &sdaddr, sizeof(sdaddr)) < 0)
perror("Failed to send a packet on clear socket");
}
and the packet never shows up. I've checked the entire contents of the sdaddr structure I've built, and all fields look good. The UDP payload data looks good. There's no error coming back from the sendto() syscall -- indeed, it returns zero. And the packet never shows up in wireshark.
So what's going on with my transparent proxying? How do I get this to work? (FWIW: development host is a generic x86_64 ubuntu 14.04 LTS box.) Thanks!
Alright, so I've got half an answer.
It turns out if I just use a RAW IP socket, with the IP_HDRINCL option turned on, and build the outgoing UDP packet in userspace with a full IP header, the kernel will honor the non-local source address and send the packet that way.
I'm now using a third socket, sock_output, for that purpose, and decoded UDP packets are coming out correctly. (Interesting side note: the UDP checksum field must either be zero, or the correct checksum value. Anything else causes the kernel to silently drop the packet, and you'll never see it go out. The kernel won't fill in the proper checksum for you if you zero it out, but if you specify it, it will verify that it's correct. No sending UDP with intentionally bad checksums this way.)
So the first half of the question remains: when I read a packet from sock_clear with the MSG_PROXY flag to recvfrom(), why do I not get the actual destination address in the second half of sdaddr?