How to make server ignores sent data from client after client get timeout on getting server response? - sockets

I'm using socket with O_NONBLOCK, select, keep alive connection and something like HTTP.
each server connection and client side uses a buffer to get all sent data until complete message be received
How to working:
client send data "A"
client try receive response from server
server receive but don't reply immediately
client gets timeout
server send response "A" (but client don't receive now)
another request:
client send data "B"
server send response "B"
client receive "AB" response instead only "B"
the problem is that client receives previous buffer message
source code bellow:
Server.cpp base class:
bool Server::start()
{
struct sockaddr_in client_addr;
SOCKET client_socket, max_sock;
Connection* conn;
int addrlen = sizeof(struct sockaddr_in);
std::list<Connection*>::iterator it, itr;
int response;
fd_set fdset, read_fds;
max_sock = m_socket;
FD_ZERO(&fdset);
FD_SET(m_socket, &fdset);
onStart();
while(true)
{
// make a copy of set
read_fds = fdset;
// wait for read event
response = select(max_sock + 1, &read_fds, NULL, NULL, NULL);
if(response == -1)
break;
// check for new connections
if(FD_ISSET(m_socket, &read_fds))
{
response--;
// accept connections
client_socket = accept(m_socket, (struct sockaddr*)&client_addr, &addrlen);
if(client_socket != INVALID_SOCKET)
{
conn = new Connection(*this, client_socket, &client_addr);
m_connections.push_front(conn);
// add connection to set for wait for read event
FD_SET(client_socket, &fdset);
// allow select new sock from select funcion
if(max_sock < client_socket)
max_sock = client_socket;
}
}
// check for received data from clients
it = m_connections.begin();
while(it != m_connections.end() && response > 0)
{
conn = *it;
// verify if connection can be readed
if(FD_ISSET(conn->getSocket(), &read_fds))
{
response--;
conn->receive();
if(!conn->isConnected())
{
FD_CLR(conn->getSocket(), &fdset);
// remove connection from list
itr = it;
it++;
m_connections.erase(itr);
delete conn;
continue;
}
}
it++;
}
}
onFinish(response >= 0);
return response >= 0;
}
main.cpp Server implementation:
void ClientConnection::onReceive(const void * data, size_t size)
{
const char *str, *pos = NULL;
HttpParser* p;
buffer->write(data, size);
do
{
str = (const char*)buffer->data();
if(contentOffset == 0)
{
pos = strnstr(str, buffer->size(), "\r\n\r\n");
if(pos != NULL)
{
contentOffset = pos - str + 4;
p = new HttpParser((const char*)buffer->data(), contentOffset);
contentLength = p->getContentLength();
delete p;
}
}
if(buffer->size() - contentOffset < contentLength || contentOffset == 0)
return;
proccessRequest();
keepDataStartingOf(contentOffset + contentLength);
}
while(buffer->size() > 0);
}
client side code is a simple recv send with timeout
any idea how to solve?

The first thing that comes to mind is to make the client's timeout large enough that the client won't timeout unless the server is actually dead... but I'm sure you've already thought of that. :)
So assuming that's not a good enough fix, the next thing to try is to have the client send an ID number with each request it sends. The ID number can be generated with a simple counter (e.g. for the client's first request, it tags the request with 0, for the second it tags it with 1, etc). The server, when sending its reply, will include that same ID number with the reply.
When the client receives a reply, it compares the ID number in the reply data against the current value of its counter. If the two numbers are the same, it processes the data. If not, it ignores the data. Et voila!

Related

Failure to send data from client to server in ESP-01 WiFi

Using the ESP8266WiFi library, I have two ESP-01's/ESP8266's connected over WiFi. It works perfectly when the client sends a request (all non HTML!) to the server (using port 5000 - to prevent any confusion with HTTP, FTP etc.). But I cannot get the client to receive an answer back from the server. Now, in the ESP8266WiFi library (3.0.2) there is a note that server.write() is not implemented, and that I should use server.accept() instead of server.available(); though I did not see any applicable examples using server.accept(), but I see many examples using client.print() so I try to follow those - to no avail, yet. What I am doing is the following: 1. establish connectivity to the WiFi; 2. have the client connect to the server and send two bytes to the server. 3. Do a digital write to a pin of the server-ESP8266.(this toggles a relay, which works fine) 4. write back from server to client that the digital write has been done. On the client side, after writing to the server, I run in a loop for some 10 seconds trying to receive something from the server, which never comes. Then I cycle back to the beginning, and the client asks to toggle the relay again - this runs nicely for hours.
Any insights here on what I should do differently are highly appreciated. I really want to be able to get some acknowledgement back to the client once the server has toggled the relay. Or if someone has a working example with server.accept() - I would try that too.
Client side code:
int pin_value;
uint8_t ip[4];
void setup()
{
Serial.begin(115200);
ip[0]=10;
ip[1]=0;
ip[2]=0;
ip[3]=6;
//We connect to the WiFi network
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
//Wait until connected
while (WiFi.status() != WL_CONNECTED){
delay(500);
Serial.print(".");
}
Serial.print("Client - ");
Serial.println("WiFi connected");
}
void loop(){
//Variable that we will use to connect to the server
WiFiClient client;
//if not able to connect, return.
if (!client.connect(ip, SERVER_PORT)){ return; }
// We create a buffer to put the send data
uint8_t buffer[Protocol::BUFFER_SIZE];
//We put the pin number in the buffer
// whose state we want to send
buffer[Protocol::PIN] = RELAY;
//put the current state of the pin in the send buffer
buffer[Protocol::VALUE] = pin_value;
//We send the data to the server
client.write(buffer, Protocol::BUFFER_SIZE);
// try to read the answer from the server for about 10 seconds
int nr_of_tries = 10000;
while (client.connected() && nr_of_tries > 0)
{if (client.available())
{ String line = client.readStringUntil('\n');
nr_of_tries = 0;
Serial.print("line= ");
Serial.println(line);
}
else
{delay(1);
nr_of_tries=nr_of_tries-1;
}
}
Serial.print("nr of tries= ");
Serial.println(nr_of_tries);
Serial.print("connected: ");
Serial.println(client.connected());
client.flush();
client.stop();
Serial.println(" change sent");
if (pin_value == 0)
{pin_value =1;
Serial.println("Pin_value set to 1");
}
else
{pin_value=0;
Serial.println("Pin_value set to 0");}
delay(10000);
}
Server side code:
WiFiServer server(SERVER_PORT);
void setup()
{
Serial.begin(115200); // must have the same baud rate as the serial monitor
pinMode(RELAY,OUTPUT);
digitalWrite(RELAY, LOW);
// Connect to the WiFi network
Serial.println();
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED){
delay(500);
Serial.print(".");
}
Serial.println("Server - ");
Serial.println("WiFi connected");
// Set this ESP to behave as a WiFi Access Point
// WiFi.mode(WIFI_AP);
// set SSID and Password to connect to this ESP
// WiFi.softAP(SSID, PASSWORD);
// Start the server
server.begin();
Serial.println("Server started");
// Output of the IP address
Serial.print("Use this IP to connect: ");
Serial.println(WiFi.localIP());
}
void loop()
{
// Check if there is any client connecting
WiFiClient client = server.available();
if (client)
{
//Serial.println("Client detected");
//If the client has data he wants to send us
//check for a second or so as transmission can take time
int nr_of_tries = 1000;
while(!client.available() && nr_of_tries > 0)
{ nr_of_tries=nr_of_tries-1;
delay(1);
}
if (client.available())
{
// Serial.println(" Client data");
// create a buffer to put the data to be received
uint8_t buffer[Protocol::BUFFER_SIZE];
// We put the data sent by the client in the buffer
// but do not read more than the buffer length.
int len = client.read(buffer, Protocol::BUFFER_SIZE);
// retrieve which pin number the client sent
int pinNumber = buffer[Protocol::PIN];
Serial.print("Pin Number: ");
Serial.println(pinNumber);
// retrieve the value of this pin
int value = buffer[Protocol::VALUE];
Serial.print("Value: ");
Serial.println(value);
// Set the pin indicated by the received pin number in output mode
// but only if the pin is the GPIO0 pin!
if (pinNumber == RELAY)
{ pinMode(pinNumber, OUTPUT);
// Set the pin indicated by the received pin number to the passed value
digitalWrite(pinNumber, value);
}
// tell the client that the relay has been set or reset.
size_t i;
if (value == 0) {
i=server.println("Set");
Serial.print("i= ");
Serial.println(i);
}
else {
i=server.println("Reset");
Serial.print("i= ");
Serial.println(i);
}
}
}
//Close the connection with the client
//client.stop();
}
Common definitions:
#include <ESP8266WiFi.h>
const char* ssid = "blablabla";
const char* password = "blublublu";
#define SERVER_PORT 5000
#define RELAY 0
//Protocol that the Server and Client will use to communicate
enum Protocol{
PIN, // Pin whose state you want to change
VALUE, // State to which the pin should go (HIGH = 1 or LOW = 0)
BUFFER_SIZE // The size of our protocol. IMPORTANT: always leave it as the last item of the enum
};
Solved! By changing server.println("Set"); into client.println("Set") and doing the same for the transmission of "Reset" a few lines lower in the server side code it works!

asio server reads data only once

Here is my server side read code.
void ServerSession::doRead()
{
//note sbuf_ is std::string
asio::async_read_until(socket_, asio::dynamic_buffer(sbuf_), "\n",
[this](std::error_code ec, std::size_t length)
{
if(!ec || ec == asio::error::eof)
{
printf("length = %lu [S] received str size = %lu, Client sent : %s\n", length, sbuf_.size(), sbuf_.data());
if(sbuf_.size() > 0)
{
std::string msg{sbuf_};
addMessageToQueue(std::move(msg));
sbuf_.clear();
}
}
else
{
socket_.close(); //force close the socket upon read error.
}
});
}
I run it and connect using a TCP client, I send some text say "A 1" server receives it correctly. but when I send a next string say "B 12" it doesn't receive it.
I tried multiple connections. for all the connections that I establish with server, server receives first string that client sends, and after that there is a silence. I added many log statements in the code, but I am not able to see them when I try to send second string.

Asynchronous sending data using kqueue

I have a server written in plain-old C accepting TCP connections using kqueue on FreeBSD.
Incoming connections are accepted and added to a simple connection pool to keep track of the file handle.
When data is received (on EVFILT_READ), I call recv() and then I put the payload in a message queue for a different thread to process it.
Receiving and processing data this way works perfect.
When the processing thread is done, it may need to send something back to the client. Since the processing thread has access to the connection pool and can easily get the file handle, I'm simply calling send() from the processing thread.
This works 99% of the time, but every now and then kqueue gives me a EV_EOF flag, and the connection is dropped.
There is a clear correlation between the frequency of the calls to send() and the number of EV_EOF errors, so I have a feeling the EV_EOF due to some race condition between my kqueue thread and the processing thread.
The calls to send() always returns the expected byte count, so I'm not filling up the tx buffer.
So my question; Is it acceptable to call send() from a separate thread as described here? If not, what would be the right way to send data back to the clients asynchronously?
All the examples I find calls send() in the same context as the kqueue loop, but my processing threads may need to send back data at any time - even minutes after the last received data from the client - so obviously I can't block the kqueue loop for that time..
Relevant code snippets:
void *tcp_srvthread(void *arg)
{
[[...Bunch of declarations...]]
tcp_serversocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
...
setsockopt(tcp_serversocket, SOL_SOCKET, SO_REUSEADDR, &i, sizeof(int));
...
err = bind(tcp_serversocket, (const struct sockaddr*)&sa, sizeof(sa));
...
err = listen(tcp_serversocket, 10);
...
kq = kqueue();
EV_SET(&evSet, tcp_serversocket, EVFILT_READ | EV_CLEAR, EV_ADD, 0, 0, NULL);
...
while(!fTerminated) {
timeout.tv_sec = 2; timeout.tv_nsec = 0;
nev = kevent(kq, &evSet, 0, evList, NLIST, &timeout);
for (i=0; i<nev; i++) {
if (evList[i].ident == tcp_serversocket) { // new connection?
socklen = sizeof(addr);
fd = accept(evList[i].ident, &addr, &socklen); // accept it
if(fd > 0) { // accept ok?
uidx = conn_add(fd, (struct sockaddr_in *)&addr); // Add it to connected controllers
if(uidx >= 0) { // add ok?
EV_SET(&evSet, fd, EVFILT_READ | EV_CLEAR, EV_ADD, 0, 0, (void*)(uint64_t)(0x00E20000 | uidx)); // monitor events from it
if (kevent(kq, &evSet, 1, NULL, 0, NULL) == -1) { // monitor ok?
conn_delete(uidx); // ..no, so delete it from my list also
}
} else { // no room on server?
close(fd);
}
}
else Log(0, "ERR: accept fd=%d", fd);
}
else
if (evList[i].flags & EV_EOF) {
[[ ** THIS IS CALLED SOMETIMES AFTER CALLING SEND - WHY?? ** ]]
uidx = (uint32_t)evList[i].udata;
conn_delete( uidx );
}
else
if (evList[i].filter == EVFILT_READ) {
if((nr = recv(evList[i].ident, buf, sizeof(buf)-2, 0)) > 0) {
uidx = (uint32_t)evList[i].udata;
recv_data(uidx, buf, nr); // This will queue the message for the processing thread
}
}
}
else {
// should not get here.
}
}
}
The processing thread looks something like this (obviously there's a lot of data manipulation going on in addition to what's shown) :
void *parsethread(void *arg)
{
int len;
tmsg_Queue mq;
char is_ok;
while(!fTerminated) {
if((len = msgrcv(msgRxQ, &mq, sizeof(tmsg_Queue), 0, 0)) > 0) {
if( process_message(mq) ) {
[[ processing will find the uidx of the client and build the return data ]]
send( ctl[uidx].fd, replydata, replydataLen, 0 );
}
}
}
}
Appreciate any ideas or nudges in the right direction. Thanks.
EV_EOF
If you write to a socket after the peer closed the reading part of it, you will receive a RST, which triggered EVFILT_READ with EV_EOF set.
Async
You should try aio_read and aio_write.

TCP Listener which should accept 100 threads per second

I have written code for TcpListener in c# which is supposed to receive request from client socket and process the request (send the processed request to our another web service to get final response) then parse the response and send the response back to client socket which initiated the request.
Code snippet below.
Code works fine when receive few requests at a time but now in order to move this to cloud and accept multiple request. We are testing this functionality by running JMeter test on same.
We are getting throughput like 4 when we hit 100 threads per seconds (end to end test - client system to server socket to our web service and back) which should be at least 30 to match client requirement.
If we omit the end to end flow and just send back to hardcode response from server socket itself we are seeing throughput 700.
To find the root cause I have added delay while sending hardcore response (same which we need to communicate with our web service) and I can see same behavior i.e. throughput drastically downgrades = 4/3.8
It means when TcpListener is busy processing existing request it may not attend the next requests (may be I am wrong in assumption - please correct if so)
Please have a look at code and help me increasing the performance .
public void StartTCPServer()
{
Logger.Write_Info_Log("In StartTCPServer - inPort : " + AESDK_CONFIG.PORT_NO, 1, log);
try
{
// Data buffer for incoming data.
byte[] bytes = new Byte[1024];
// Establish the local endpoint for the socket.
IPEndPoint localEndPoint = new IPEndPoint(IPAddress.Any, AESDK_CONFIG.PORT_NO);
// Create a TCP/IP socket.
Socket listener = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
// Bind the socket to the local endpoint and listen for incoming connections.
listener.Bind(localEndPoint);
listener.Listen(100);
while (true)
{
// Set the event to nonsignaled state.
allDone.Reset();
// Start an asynchronous socket to listen for connections.
listener.BeginAccept(
new AsyncCallback(AcceptCallback),
listener);
// Wait until a connection is made before continuing.
allDone.WaitOne();
}
}
catch (Exception Ex)
{
Logger.Write_Fatal_Log("Exception in Start Listening : " + Ex.Message, 1, log);
}
}
public void AcceptCallback(IAsyncResult ar)
{
// Signal the main thread to continue.
allDone.Set();
// Get the socket that handles the client request.
Socket listener = (Socket)ar.AsyncState;
Socket handler = listener.EndAccept(ar);
// Create the state object.
StateObject state = new StateObject();
state.workSocket = handler;
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReadCallback), state);
}
public void ReadCallback(IAsyncResult ar)
{
String strdata = String.Empty;
// Retrieve the state object and the handler socket
// from the asynchronous state object.
StateObject state = (StateObject)ar.AsyncState;
Socket handler = state.workSocket;
// Read data from the client socket.
int bytesRead = handler.EndReceive(ar);
if (bytesRead > 0)
{
// There might be more data, so store the data received so far.
state.sb.Append(Encoding.ASCII.GetString(
state.buffer, 0, bytesRead));
// Check for end-of-file tag. If it is not there, read
// more data.
strdata = state.sb.ToString();
if (!string.IsNullOrEmpty(strdata))
{
// All the data has been read from the client.
if (strdata.Contains("<<CheckConnection>>"))
{
log.Info(GlobalVar.gThreadNo(GlobalVar.gintCurrentThread) + "Data Received: " + strdata);
byte[] byData1 = System.Text.Encoding.UTF8.GetBytes("<<CheckConnectionAlive>>");
Send(handler, "<<CheckConnectionAlive>>");
}
else
{
Interlocked.Increment(ref m_clientCount);
//Process incoming requests here and send response back to client
string strResponse = GetRequest(strdata, m_clientCount);
Send(handler, strResponse);
}
}
else
{
// Not all data received. Get more.
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReadCallback), state);
}
}
}
private static void Send(Socket handler, String data)
{
// Convert the string data to byte data using ASCII encoding.
byte[] byteData = Encoding.ASCII.GetBytes(data);
// Begin sending the data to the remote device.
handler.BeginSend(byteData, 0, byteData.Length, 0,
new AsyncCallback(SendCallback), handler);
}
private static void SendCallback(IAsyncResult ar)
{
try
{
// Retrieve the socket from the state object.
Socket handler = (Socket)ar.AsyncState;
// Complete sending the data to the remote device.
int bytesSent = handler.EndSend(ar);
Console.WriteLine("Sent {0} bytes to client.", bytesSent);
handler.Shutdown(SocketShutdown.Both);
handler.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}

epoll_wait() wakes up on EPOLLIN even I have consumed all reader buffer

Hello there I am getting mad with this issue since I'm using a simple pattern.
Ok, I have a infinite while in which I use epoll_wait on server socket and already connected sockets. All went ok if a new socket send a connection request; my problem is when connected sockets (right now I am just using one socket sending 390k packet) sends data: doesn't matter if I use EPOLLONESHOT or EPOLLET, after consumed all request buffer on that socket, rearming socket or receiving EAGAIN on recv(), epoll_wait always wakes up again with wrong buffer!
My server works with threadpool, but right now is just one thread that does all work (to simplify testing):
while (TRUE) {
int num = epoll_wait(efd, events, MAX_EVENTS, -1);
for (int i = 0; i < num; i++) { // ciclo epoll()
if (events[i].events & EPOLLERR || events[i].events & EPOLLHUP || !(events[i].events & EPOLLIN)) {
fprintf (stderr, "epoll error on socket: closed\n");
s = epoll_ctl(efd, EPOLL_CTL_DEL, events[i].data.fd, NULL);
}
else if (events[i].data.fd == serverSocket) {
while (TRUE) {
newInfoClient = server->m_bgServer->AcceptClient(&newSocketClient);
if (newInfoClient == NULL) { // nessun client
break;
}
else {
printf("\nSocket accettato: %d", newSocketClient);
s = CSocket::MakeNonBlocking(newSocketClient);
if (s == -1)
abort();
event.data.fd = newSocketClient;
event.events = EPOLLIN | EPOLLET;
s = epoll_ctl(efd, EPOLL_CTL_ADD, newSocketClient, &event);
if (s == -1) {
abort();
}
}
}
else {
AbstractTcpServerGame::DownloadTcpRequest(client);
}
}
}
I have just omitted some checks and other internal codes.
AbstractTcpServerGame::DownloadTcpRequest(...)
This function is just a recv in loop to rescues my own header, get buffer body and just to verify empty buffer outside loop I call a simple recv() that returns -1 (errno=EAGAIN or EWOULDBLOCK).
After this when I rearm the socket with epoll_ctr() in DownloadTcpRequest() in EPOLLONESHOT case, when it returns epoll_wait() wakes up again on the same socket!! this is my execution log:
New socket (6) request (errno 11) <--- when epoll_wait() emits EPOLLIN on socket 6
Download of 18 bytes (socket 6) <-- inside AbstractTcpServerGame::DownloadTcpRequest()
Download of 380k (socket 6) <-- another recv() loop to rescue body request
------------------- empty buffer on socket 6 ----------- <-- dummy recv to show empty buffer
New socket (6) request (errno 11)
Download of 18 bytes (socket 6)
Download of -1556256155 (socket 6)
Error on socket 6 (bad::alloc exception)
Client sends 398k (18 header + body) and all data are correctly received as shown above, but rearming socket or using EPOLLET, epoll_wait() generates another request and I don't know where those are taken infact are not correct!