I have seen many a web framework provide a non-blocking web server, I just want to know what it means.
a blocking web-server is similar to a phone call. you need to wait on-line to get a response and continue; where as a non-blocking web-server is like a sms service. you sms your request,do your things and react when you receive an sms back!
Using a blocking socket, execution will wait (ie. "block") until the full socket operation has taken place. So, you can process any results/responses in your code immediately after. These are also called synchronous sockets.
A non-blocking socket operation will allow execution to resume immediately and you can handle the server's response with a callback or event. These are called asynchronous sockets.
Non-blocking generally means event driven, multiplexing all activity via an event driven system in a single thread, as opposed to using multiple threads.
Related
The question is about Play framework specifically although concept is generic. I guess that the blocked client is listening to a socket which is tracked on the server side and passed around with the Future[Result] so that when the Future finishes, then the response is written to the socket and then the socket is closed.
Can someone share more concrete explanation with references?
Quoting from:
https://www.playframework.com/documentation/2.6.18/ScalaAsync
The web client will be blocked while waiting for the response, but
nothing will be blocked on the server, and server resources can be
used to serve other clients.
Note that Play does not manage how to address the client. This is managed by TCP. Basically (as a simple analogy) you can think of a client, like a web browser, as making a telephone call to the server. When the clients makes a request, one of it's sockets gets connected to a particular socket on the server - this is a persistent connection between the sockets for the duration of the request/response. Play's underlying server (Netty for older versions or Akka Http for v2.6+) will accept the incoming request from the socket and assign it a thread. Play will do the work and the resulting Response will get mapped back to the correct socket by the server. The TCP server will manage the mapping between response and the socket, not Play.
As others have noted, the reference to blocking is essentially to do with the way Play Action's are intended to work (non-blocking). They take the request, wrap whatever work you have coded in a Future, and hand this off to get completed at some point in the near future (it might be a different thread that completes the Future, or it could even end up being the same thread). The point is that the creation of the Future is quick and so the thread that handled the request gets returned quickly to the pool so it can pick up another request to work on. If you have heard about Reactive Programming then this is essentially the idea being keeping an Application Responsive.
The web client will be blocked while waiting for the response, but
nothing will be blocked on the server, and server resources can be
used to serve other clients.
So the client might be blocked on it's end whilst waiting for the response to come back through it's socket (unless it too is making async calls), but the idea is that the thread pool handling the requests in Play will not be blocked because of the way they create a Future and hand the completion of this back to Play so they can go back to handle other requests.
There is a bit more to it but hopefully this gives a bit more context to that particular statement from Play's docs.
NGNIX uses epoll notification to know if there is any data on the socket to read.
Let assume:
There are two requests to the server.
nginx is notificated about this two requests and starts to:
receive the first request
parse ist headers
check the boudary (body size)
send the first request to upstream server
etc.
nginx is singe-threaded and can do only one operation at the same time.
But what happens with the second request?
Does nginx receive the second request during parsing the first one?
Or it begins to handle the second request after getting the first done?
Or something else that I don't understand.
If 1. is correct than I don't understand how it is possible within a single thread.
If 2. is correct than how can nginx be so fast? because nginx handles all incoming requests sequentially. At any given time only ONE request handling is possible.
Please help me to understand.
Thanks
Nginx is not a single threaded application. It does not start a thread for each connection but it starts several worker threads during start. The nginx architecture is well described in the http://www.aosabook.org/en/nginx.html.
Actually a single threaded non-blocking application is the most efficient design for a single processor hardware. When we have only one CPU and the application is completely non-blocking the application can fully utilize the CPU power. Non-blocking application means that application does not call any function that might wait for an event. All IO operation are asynchronous. That means application does not call simple read() from socket because the call might wait till data is available. Non-blocking application uses some mechanism how to notify application that data is available and it can call read() without risk that the call will wait for something. So ideal non-blocking application needs only one thread for one CPU in the system. As nginx uses non-blocking calls the processing in multiple threads has no meaning because there would be no CPU to execute additional threads.
The real data receiving from a network card to a buffer is done in the kernel when network card issue an interrupt. Then nginx gets a request in a buffer and process it. It has no meaning to start processing another request till the current request processing is done or till the current request processing requires an action that might block (for example disk read).
I am building socket web server with Netty 5.0. I came through WebSocketServer example (https://github.com/netty/netty/tree/master/example/src/main/java/io/netty/example/http/websocketx/server).
But I can't understand how to send events to sockets from separate thread. So I have a thread which each second loads some data from external resource. This is StockThread which receives stock data. After receiving data the thread should send events to sockets. What is best practise to do this?
It am using following approach: inside StockThread I store list of ChannelHandlerContext. After receving data I just call write() method of ChannelHandlerContext. So write() method is called from StockThread. Is it okay or there is more appropriate way for this?
Yes, ChannelHandlerContext is thread-safe and can be cached, so this way of usage is completely ok.
See note from "Netty In Action" book, that proves my words:
You can keep the ChannelHandlerContext for later use,
such as triggering an event outside the handler methods,
even from a different Thread.
Let's say, I have a server with many connected clients via TCP, i have a socket for every client and i have a sending and receiving thread for every client. Is it safe and possible to call send function at the same time as it will not call send function for same socket.
If it's safe and ok, Can i stream data to clients simultaneously without blocking send function for other clients ?
Thank you very much for answers.
Yes it is possible and thread-safe. You could have tested it, or worked out for yourself that IS, IIS, SQL Server etc. wouldn't work very well if it wasn't.
Assuming this is Windows from the tag of "Winsock".
This design (having a send/receive thread for every single connected client), overall, is not going to scale. Hopefully you are aware of that and you know that you have an extremely limited number of clients (even then, I wouldn't write it this way).
You don't need to have a thread pair for every single client.
You can serve tons of clients with a single thread using non-blocking IO and read/write ready notifications (either with select() or one of the varieties of Overlapped IO such as completion routines or completion ports). If you use completion ports you can set a pool of threads to handle socket IO and queue the work for your own worker thread or threads/threadpool.
Yes, you can send and receive to many sockets at once from different threads; but you shouldn't need those extra threads because you shouldn't be making blocking calls to send/recv at all. When you make a non-blocking call the amount that could be written immediately is written and the function returns, you then note how much was sent and ask for notification when the socket is next writable.
I think you might want to consider a different approach as this isn't simple stuff; if you're using .Net you might get by with building this with TcpListener or HttpListener (both of which use completion ports for you), though be aware that you can't easily disable Nagle's algorithm with those so if you need interactivity (think of the auto-complete on Google's search page) then you probably won't get the performance you want.
The network lib I'm writing needs to send and receive messages through a TCP socket. Messages can be sent or received any time, i.e should work as a full duplex channel.
I was able to implement such scenario using two threads: main thread calling send() and a dedicated thread mostly blocked at recv() call.
My question is: is it possible to implement the same scenario with a single thread? I.e. by registering some callback function?
As a side note: I need implement this scenario in C++, Java and Python.
Yes, it possible. You need to use an API that allows multiplexed I/O. Under C/C++ and Python you can use select() and non-blocking I/O, so that the only networking call you ever block in is select(). There is also poll() and epoll() and a number of other similar APIs that accomplish the same thing, with varying degrees of efficiency and portability. Java has non-blocking I/O APIs also.
Another possibility is to use asynchronous I/O, where you tell the OS to start an I/O transaction and it notifies you (by some mechanism) when it has finished the operation. I'm not familiar with that style of network programming, however, so I won't try to give details.
In general, a library would do this by providing an interface into the application's main loop / event loop.
For example, most single-threaded network-using applications would have a main loop that blocks in select(), poll() or similar. Your library would just return a file descriptor that the application should include in its select() / poll() call, and a callback function that the application should call when that file descriptor is readable.