Scenario: I am running a rest server(nghttp2) with two APIs with 4 threads.
/something : takes some time to process
/anything : takes no time to process
Now, in the client side I am creating one session(which is essentially one TCP connection) and making two async requests, first to /something and next to /anything consecutively. The behavior I notice is that server doesn't process the second request until the the first one is finished. In the packet capture I can see nice implementation of HTTP2 multiplexing. But isn't this head of line blocking? Or is it that my expectation that the requests should be processed parallely rather in a interleaving fashion even if they are from the same TCP connection is wrong?
Note: If I create two different session or TCP connection for each request then they are processed parallely.
Related
I'm using 0MQ to let multiple processes talk to each other (IPC sockets, but should also work via TCP across different nodes). My code is similar to a client/server pattern, but REQ/REP sockets are not enough. Here is a sample conversation. See below for further details.
Process A
Process B
open socket
not started
start process B
-
-
open socket, connect to A
-
send hello (successful start, socket information)
request work
-
-
do work
-
send response (work result 1)
-
send response (work result 2)
-
send unsolicited message
-
send response (work finished)
request termination
-
Actually, A is (even though doing all the requests) closer to be the server component, since it is constantly running. Based on external triggers, A starts a sort of plugin process B.
Every request needs to be answered by a finished response. Before that, N (between 0 and an arbitrary upper bound) responses can be sent from B.
A new request can be sent from A even when the current request is still ongoing (no finished message received). If relevant, the code could be updated to buffer the requests.
B sends an initial message which is not preceded by a request from A.
B can send other messages (logging) anywhere in between, also not preceded by a request.
Optional: A single socket in A should handle multiple plugin processes B, C, D...
A DEALER/ROUTER combination would probably match all requirements, but might be a bit too much. Process B will only ever connect to a single peer. And without the optional requirement above, the same would be true for process A as well. So I'm a bit hesitant to use DEALER and ROUTER sockets which are both able to handle multiple peers.
I am acting as server which receives multiple requests from client in socket and handles in a thread.
Should i set any parameter in TCP level to set maximum number of requests a connection can handle simultaneously?
because in my server side ,if processing the request is slow i observe that other requests are queued up (client says request has been sent but i receive it late)
Kindly guide me
If it takes a long time to do the work and you want to handle multiple connections simultaneously, you have to change how you do things.
If you are actively using a lot of CPU during processing a long request, you'll need multiple threads. That's the only way to actually get more CPU time / second -- assuming you have multiple cores available.
If you are waiting on things like file IO, then you can instead use asynchronous processing to handle the requests on a single thread, but just handle a little piece at a time.
Setting a maximum number of TCP connections won't help you handle more processes more quickly. It will just reject connections and not even allow a first-come first-served type of behavior - it will just be random if a specific client ever gets through or not.
I have application where I am listening on multiple sockets using select. If I start processing request that came in from Socket A and in the meanwhile if another request on socket B arrives then I want to know how long does socket B request had to wait before I could get it. Since this is a single threaded application I cannot spawn a new thread and go back to select to monitor again and instantly start processing request from socket B.
Is there a 'C' api available to get me this metric or is this just not possible to get?
There is no a straightforward way how to measure the interval between the 'data ready' time and 'data read' time because there is not any timestamp written together with the data. Moreover the situation is even more complex because a stream oriented socket may receive several data segments till select is closed and the it is not what interval should be measured.
If the application data processing is longer than packet processing in the kernel the you can do a reasonable measurement in following way:
print current time and some unique data id based on application protocol when select wakes up due to socket B data availability.
log any packet received for the socket B. You can use either a network traffic capture tool like wireshark or tcpdump. Or you can configure an iptables firewall rule (if it is running on linux) with target -j LOG.
Write a simple script/program that correlates the captured packets and the application log and subtract received and start processing time.
Of course the idea above does ignore the kernel processing time. If you really need exact time I have to introduce a new thread to your application.
How come every site explains that in SSE a single connection stays opened between client and server "With SSE, a client sends a standard HTTP request asking for an event stream, and the server responds initially with a standard HTTP response and holds the connection open"
And then, when server decides it can send data to the client while what I am trying to implement SSE I see on fiddler requests being sent every couple of seconds
For me it feels like long polling and not a one single connection kept opened.
Moreover, It is not that the server decides to send data to the client and it sends it but it sends data only when the client sends next request
If i respond with "retry: 10000" even tough something has happened that the server wants to notify right now, will get to the client only on the next request (in 10 seconds from now) which for me does not really looks like connection that is kept opened and server sends data as soon as he wants to
Your server is closing the connection immediately. SSE has a built-in retry function for when the connection is lost, so what you are seeing is:
Client connects to server
Server myteriously dies
Client waits two seconds then auto-reconnects
Server myteriously dies
Client waits two seconds then auto-reconnects
...
To fix the server-side script, you want to go against everything your parents taught you about right and wrong, and deliberately create an infinite loop. So, it will end up looking something like this:
validate user, set up database connection, etc.
while(true){
get next bit of data
send it to client
flush
sleep 2 seconds
}
Where get next bit of data might be polling a DB table for new records since the last poll, or scan a file system directory for new files, etc.
Alternatively, if the server-side process is a long-running data analysis, your script might instead look like this:
validate user, set-up, etc.
while(true){
calculate next 1000 digits of pi
send them to client
flush
}
This assumes that the calculate line takes at least half a second to run; any more frequently and you will start to clog up the socket with lots of small packets of data for no benefit (the user won't notice that they are getting 10 updates/second instead of 2 updates/second).
I'm writing a server in python that needs to take requests from clients, queue the requests, execute them one at a time, then tell the clients that their particular request has been processed.
Currently the way I've approached it is using a TCP socket server -- however, I'm not sure how to make it so that only one request is being executed at a time from a queue?
The way I would like for it to look:
Client1 -> (a) -> Server
Client2 -> (b) -> Server
Client3 -> (c) -> Server
Server makes queue |a, b, c|
Execute a first. Done? Tell Client 1
Execute b second. Done? Tell Client 2
Execute c third. Done? Tell Client 3
From what I understand, if I have the server recv the client's request, execute it, and respond, that may happen at the same time in different threads. I only want one thread executing all the tasks (because I anticipate many tasks coming in and it'd be slow if everyone was running one at the same time). How do I accomplish that?
There are tons of ways to skin it, but a solution is going to look something like the below:
Client -> Client-Mediator (TCP Port) <--> Server Mediator -> (ServerQ) <- Task Process
The flow would be like this:
Client Process:
Client creates a client mediator on a tcp socket.
Sends whatever info it needs over the port.
Server Mediator receives the request
Creates a response Q for the Task Process
Places the request on the Server Q (command + responseQ)
Wait for response on responseQ
No response after X time timeout ?
Once response comes, read and send response over tcp port.
Server Process:
Reads from Server Q.
Processes command
Write the response to the response Q
Components involved
Client - Simple process that sends requests for tasks to be completed.
Client-Mediator - Creates a connection to the server process.
Server-Mediator - Accepts a client request for task processing, enqueues tasks and waits for response.
Task Process - Reads from ServerQ and waits for a task to come in.
Okay so what Nix said was right but I wasn't sure how to make that exactly happen (my question was how to go about actually making this)
As it turns out I had to start 2 threads: one that executes from the queue, and the other being the main server handler. The server handler spawns threads for each new connection, and the client blocks after sending a request / if the request is successfully queued. This means that the queue needs to be thread-safe / protected with a semaphore or mutex. In the case of python, there is a multiprocessing.Queue class that handles that for you. Whenever a task is executed, the execution thread does a notifyAll() which causes all sleeping threads to wake up and check if their requested task is done. I use a condition variable for that.