I have a simple program which transfers data from a client to a server using rdma. I have 5 qps and the length of send queue and receive queue are both 50. Under this way, my program runs well, it can thansfers 1million requests. When i set th length of send queue and receive queue from 50 to 500, then I got wrong work completion. The server side gets IBV_WC_LOC_ACCESS_ERR(8) of wc.status and the client side gets IBV_WC_REM_ACCESS_ERR(10) of wc.status.
I want to know why I get above wrong wc.status.
Related
I am working on a project wherein I have multiple push sockets( created dynamically ) connected to a single pull socket. Here, each push socket sends a unique kind of data to the pull socket and the data size for each messages is less than 1000Bytes and each push socket sends upto 20,000 to 30,0000 such messages. During the peak load the number of push sockets can scale upto 100. So here, there are 100 push sockets connecting to one PULL receiver and each push socket sending 20-30K of messages with size almost 1000Bytes.
Now, On the PULL side receiver I process these each unique packets and put it into the respective queue, For Ex: If I have 50 PUSH sockets sending in data than on my receiver side there will be 50 queues processing the data. This is required as each queue holds unique set of data which needs to be processed uniquely. Here, I cannot have multiple PULL receivers as the data would then be routed in a round-robin fashion and one set of unique data might go into another one which I don't need.
Here, the problem I am facing is sometimes I see I don't receive all the data sent by the PUSH sockets. I believe there is some data loss that I happening. In order to control this I have setup HWM on both the sides. I tried the below combinations:
1) SNDHWM set to 100 and RCVHWM set to 100
2) SNDHWM set to 10 and RCVHWM set to 10
3) SNDHWM set to 10 and RCVHWM set to 100
Apart from the above configs I have also set the context threads on sender side to be 3 and on the receiver side to be 1. Here, on the sender side there is only one context created from which multiple push sockets are created dynamically as in when the data receives.
But, in all these combinations I find still some packets are missing.
Can someone guide me as in which parameters I need to configure in ZMQ so that I don't end up in packet loss and all the packets I receive on my receiver side. Also, I somehow believe my receiver side processing might be slow due to which this could have happened.
Please pen your thoughts here and guide me if possible.
How come every site explains that in SSE a single connection stays opened between client and server "With SSE, a client sends a standard HTTP request asking for an event stream, and the server responds initially with a standard HTTP response and holds the connection open"
And then, when server decides it can send data to the client while what I am trying to implement SSE I see on fiddler requests being sent every couple of seconds
For me it feels like long polling and not a one single connection kept opened.
Moreover, It is not that the server decides to send data to the client and it sends it but it sends data only when the client sends next request
If i respond with "retry: 10000" even tough something has happened that the server wants to notify right now, will get to the client only on the next request (in 10 seconds from now) which for me does not really looks like connection that is kept opened and server sends data as soon as he wants to
Your server is closing the connection immediately. SSE has a built-in retry function for when the connection is lost, so what you are seeing is:
Client connects to server
Server myteriously dies
Client waits two seconds then auto-reconnects
Server myteriously dies
Client waits two seconds then auto-reconnects
...
To fix the server-side script, you want to go against everything your parents taught you about right and wrong, and deliberately create an infinite loop. So, it will end up looking something like this:
validate user, set up database connection, etc.
while(true){
get next bit of data
send it to client
flush
sleep 2 seconds
}
Where get next bit of data might be polling a DB table for new records since the last poll, or scan a file system directory for new files, etc.
Alternatively, if the server-side process is a long-running data analysis, your script might instead look like this:
validate user, set-up, etc.
while(true){
calculate next 1000 digits of pi
send them to client
flush
}
This assumes that the calculate line takes at least half a second to run; any more frequently and you will start to clog up the socket with lots of small packets of data for no benefit (the user won't notice that they are getting 10 updates/second instead of 2 updates/second).
I am working upon an application based on client and server making use of sockets,in this i am sending data from client to receiver and have made sleep call for 10 sec in server side.Now,when i am sending data from client 1000000 times the server receives it very slowly and the client is printing the values but it is also taking some time in doing so.So, i need to clear following points:
-In both client and server when the values are being displayed there is no loss of data on either side.Does this means that the recv call which is on server side is blocking?
-Secondly,is there any good documentation which could help me to understand better the blocking and non blocking concept of the send and recv calls which are used in sockets programming.
I have send Udp packet to the Server. If the server is OK then I can receive the response packet nicely but when the server is down then I did not get any response packet. Anybody can help me that how can I send my packet to server multiple time when fail to receive the response packet. Moreover, want to keep alive the connection with server. Thanks in advance.
Well,
After you've sent the packet, you wait for the ACK (response) package from the server. You could use the DatagramSocket.setSoTimeout() to an appropriate time, if you get the Timeout Exception increment a counter, if that counter is less than 2/3 send the packet again and repeat these steps. If the counter is bigger than 2/3 the server is down, just quit.
According to Java documentation, receive will block until a package is received or a timeout has expired.
To keep the connection alive you need to implement a ping-pong. In another thread of your program, you send a Keep-Alive packet (any small packet will do) and wait for a response. I suggest using a different port number for this purpose so that these packets won't mess up with the normal data packets. These packets can be send every 2 seconds o 2 minutes depends on your particular needs. When the thread receives the ACK packet it will update a private time variable with the current time, for example:
lastTimeSeen = System.currentTimeMillis();
put a method in your thread class to access the value of that variable.
I have a strange problem on one of my clients workstation. I have a simple application that exchanges some data over network between two endpoints.
Basically the transaction goes like this:
Client A listens for incomming connection
Client B connects to A and sends some data
Client A read this data for further processing
Now the strange part is that client A does not receive whole data (sometimes it a part of buffer sometimes it is empty).
The A client uses WSAEventSelect function and waits for FD_READ to read data sent by B and for FD_CLOSE to detect disconnection.
Usually ( everytime except this one particular client) the FD_READ is signaled, data is processed and after that FD_CLOSE is signaled and all is fine, but here instead FD_READ i receive FD_CLOSE.
Can someone tell me how this is possible? Another thing is that program was working fine for about a year and suddenly it crashed.
Now the strange part is that client A does not receive whole data (sometimes it a part of buffer sometimes it is empty).
There's nothing strange about that, that's how TCP works, except that you will never receive zero bytes in blocking mode.
Usually ( everytime except this one particular client) the FD_READ is signaled, data is processed and after that FD_CLOSE is signaled and all is fine, but here instead FD_READ i receive FD_CLOSE.
Note that FD_READ can be signalled any number of times, not just once. You're not guaranteed to receive an entire message in a single read.
Can someone tell me how this is possible?
The client has closed the connection.
Quoting http://msdn.microsoft.com/en-us/library/windows/desktop/ms741576%28v=vs.85%29.aspx
"An application should check for remaining data upon receipt of FD_CLOSE to avoid any possibility of losing data."
So if the error code associated with the FD_CLOSE notification is 0, you should check to see if you still have data to read, that might be where your missing data is.
If the error code is NOT 0, then there was an error and the missing data is probably lost.