I am wring a small http server which is using the Microsoft Windows WinSock API.
Do I need to apply multithreaded logic when handling multiple users?
Currently Windows sends a message when there is a network event and each message
carried (in wParam) the socket to be used in either send() or recv().
When client A connects and requests a couple of files usually a number of socket
are created by Winsock. My server then get a message that "send this file to
socket 123" and later "send that file to socket 456"
When another client connect it too gets a few sockets, say 789 and 654.
My server then respond to requests to send data using supplied socket number. It
does not have to know who wants the file since the correct file has to be sent to
the right socket.
I do not know whether Windows itself uses multiple threads when handling
accepting connection and sending the message down to my program.
So my question is:
Do I need to apply multithreaded logic when handling multiple users? And if so at
what point should I create a thread?
You typically use a thread per socket. And if you are accepting connections, a thread in a loop to block, waiting for an incoming connection socket. You then create a new thread and pass this socket handle to the new thread to handle. When that connection is closed and done with, simply let that thread terminate (or join). This is the basis of a threaded server.
in psudo code...
loop {
socket = accept();
new ThreadHandler( socket )
}
Using a single thread to handle multiple sockets is tricky, mainly because the thread can block (stop, waiting) while its writing, or more often, reading from a socket. It's not for the faint hearted.
For most applications, there is no point in using multiple threads to handle network connections. I've made a small writeup in an answer to this question.
Multiple threads become useful when handling the received data requires an unpredictable amount of CPU time, for example in database servers, or when the program structure does not allow for requests to be handled asynchronously.
There is also a third option, the "worker pool". A single thread handles all incoming connections and deserializes incoming requests, and then passes off work items to a pool of threads that handle one item at a time.
This way, simply opening a connection does not yet consume the resources needed for an entire thread, and system load is implicitly limited by the number of threads in the pool.
Related
I have a client/server application implemented with a thread pool of worker.
In server's main I create a listener socket and then for each client I get a fd with accept() to handle client's requests.
My question is: do I need a lock on this fd to read client's requests and to write the answers and data?
Nope. Listener sockets are coordination points, you don't need to explicitly coordinate. It's not even clear what "locking" a socket means; sockets don't "lock". They may or my not "block" depending on how they are configured - and this will determine if your operations return immediately, after a given timeout, or wait indefinitely to complete.
When you accept() on a listen socket successfully, the result is a second socket - this is the one that you use to communicate with the client, not the listener socket.
Of course if your application is multi-threaded, your /threads/ may need to lock or mutex to get proper behavior. Details will vary depending on your actual design and how socket IO states interact with threads can get very complicated / prickly. You'd need to post code for more concrete answers.
There are many more details to even single-threaded socket IO. See https://cis.temple.edu/~giorgio/old/cis307s96/readings/docs/sockets.html
This question is not for a concrete implementation of how this is done. It is more about the concept and design of sending information over Internet with some kind of protocol - either TCP or UDP. I know only that sockets are needed, but I am wondering about the rest. For example after a connection is made and you send the information through that, but how does the other end listen for a specific port and does it listen constantly?
Is listening done in a background thread waiting for information to be received? (In order to be able to do other things/processing while waiting for information)
So in essence, I think a real world example of how such an application works on a high level would be enough to explain the data flow. For example sending files in Skype or something similar.
P.S. Most other questions on similar topics are about a concrete implementation or a bug that someone has.
What I currently do in an application is the following using POSIX sockets with the TCP Protocol:
Most important thing is: The most function are blocking functions. So when you tell your server to wait for client connection, the function will block until a connection is established (if you need a server that handles multiple clients at once, you need to use threading!)
Server listens for specific port until a client connects. After the connect, you will get a new socket file descriptor to communicate with the client whilst the initial socket can listen to new connections. My server then creats a new thread to handle that client whilst waiting for new connections on the initial socket. In the new thread the server waits for a request command from the Client (e.g. Request Login Token). After a request was received by the server, the server will gather its informations, packs it together using Googles Protocol Buffers and sends it to the client. The client now either tells the server to terminate the session (if every data is received by the client that it needs) or send another request.
Thats basically the idea in my server. The bigger problem is the way you transmit and receive data. E.g. you cant send structs or classes (at least not via C++) over the wire, you need some kind of serializer and you have to make sure the other part knows how much to receive. So what i do is, first send a 4byte integer over the wire containing the size of the incomming package, then send the package itself using a serializer (in my case Googles Protocol buffers). The other side waits for 4 byte to be available, knowing that this will be the size of the incomming package. After 4 bytes are received, the program waits for exact that amount of data being available on the socket, when available, read the data out of the buffer and deserialize it. When the socket is not receiving data for 30 seconds, trigger a timeout and terminate the connection.
What you always need to be aware of is the endianess of the systems. E.g. a big endian system (e.g. PowerPC) and a little endian system (e.g. x86) will have problems when you send an integer directly over the wire. For example a
0001
on the x86, is a
1000
on the Power PC, thus making a 8 out of a 1. So you should always use functions like ntohl, an htonl, which will convert data from and to host byte order from and to network byte order (network byte order is always big endian).
Hope this kind of helps. I could also provide some code to you if that would help.
I am just starting to learn sockets and client/servers. I am not clear on the following concept. Assume non-blocking sockets.
Assume I have a server application, and I have 1000 clients trying to talk to it, I think it is very realistic. Assume the client and server talk via sockets.
1- Does this mean that with every client, there is a separate socket connection? (Do we have 1000 sockets, or one socket with 1000 connections?
2- Does every socket connection belong to a separate thread? If Yes, How can we limit number of threads as it can get out of control?
Assuming you're using TCP, then every connection is over a separate socket. The operating system allocates them using file descriptors.
When using a protocol like UDP, this need not be the case, and won't be unless you write the code to do make it happen.
Threading? It depends on how you build the server. You don't need threads to be a part of a server at all and you can (obviously) have multiple threads with just a single connection. One common way of doing things, however, is to hand the socket returned by accept() to a new thread, yes.
If you don't have an interest in threads--for example, if the server only performs very quick tasks and creating a thread is just wasting time--you can use select() to poll the sockets and determine which ones need attention. Some servers use a combination of threading and polling to try to maximize throughput.
I'm weighing up how to implement a TCP based server (in C) - the server will accept a connection from a client, receive commands from the client, and then send the response. Pretty simple stuff - but the processing of the command must be done by another thread in the system, which introduces a bit of concurrency to the mix.
So I'm trying to decide whether to handle all TCP comms in one thread, using non-blocking sockets and select(), or to use blocking sockets and two separate comms threads (one for sending, one for receiving).
My concern about the latter is handling socket synchronisation - if I close the socket in the send thread, what happens in the receive thread (or vice versa) .. and how to deal with this and clean up in the correct manner.
Any advice would be much appreciated.
You do not need separate receive and send threads for a client. When the client is accepted, create one thread that handles all of the I/O for that client, both receiving and sending (especially since you are implementing a command/response protocol). But if you do choose to use separate threads, closing a socket in one thread will cause detectable errors in the other thread that is using the same socket. Simply terminate each thread when a socket error occurs, and then decide which thread is going to be responsible for closing the socket.
However, if you need to handle a high number of concurrent clients then threading is not the best choice. Asynchronous I/O using non-blocking sockets (or on Windows, using I/O Completion Ports) is better, as it requires a smaller number of threads.
Let's say, I have a server with many connected clients via TCP, i have a socket for every client and i have a sending and receiving thread for every client. Is it safe and possible to call send function at the same time as it will not call send function for same socket.
If it's safe and ok, Can i stream data to clients simultaneously without blocking send function for other clients ?
Thank you very much for answers.
Yes it is possible and thread-safe. You could have tested it, or worked out for yourself that IS, IIS, SQL Server etc. wouldn't work very well if it wasn't.
Assuming this is Windows from the tag of "Winsock".
This design (having a send/receive thread for every single connected client), overall, is not going to scale. Hopefully you are aware of that and you know that you have an extremely limited number of clients (even then, I wouldn't write it this way).
You don't need to have a thread pair for every single client.
You can serve tons of clients with a single thread using non-blocking IO and read/write ready notifications (either with select() or one of the varieties of Overlapped IO such as completion routines or completion ports). If you use completion ports you can set a pool of threads to handle socket IO and queue the work for your own worker thread or threads/threadpool.
Yes, you can send and receive to many sockets at once from different threads; but you shouldn't need those extra threads because you shouldn't be making blocking calls to send/recv at all. When you make a non-blocking call the amount that could be written immediately is written and the function returns, you then note how much was sent and ask for notification when the socket is next writable.
I think you might want to consider a different approach as this isn't simple stuff; if you're using .Net you might get by with building this with TcpListener or HttpListener (both of which use completion ports for you), though be aware that you can't easily disable Nagle's algorithm with those so if you need interactivity (think of the auto-complete on Google's search page) then you probably won't get the performance you want.