I'm weighing up how to implement a TCP based server (in C) - the server will accept a connection from a client, receive commands from the client, and then send the response. Pretty simple stuff - but the processing of the command must be done by another thread in the system, which introduces a bit of concurrency to the mix.
So I'm trying to decide whether to handle all TCP comms in one thread, using non-blocking sockets and select(), or to use blocking sockets and two separate comms threads (one for sending, one for receiving).
My concern about the latter is handling socket synchronisation - if I close the socket in the send thread, what happens in the receive thread (or vice versa) .. and how to deal with this and clean up in the correct manner.
Any advice would be much appreciated.
You do not need separate receive and send threads for a client. When the client is accepted, create one thread that handles all of the I/O for that client, both receiving and sending (especially since you are implementing a command/response protocol). But if you do choose to use separate threads, closing a socket in one thread will cause detectable errors in the other thread that is using the same socket. Simply terminate each thread when a socket error occurs, and then decide which thread is going to be responsible for closing the socket.
However, if you need to handle a high number of concurrent clients then threading is not the best choice. Asynchronous I/O using non-blocking sockets (or on Windows, using I/O Completion Ports) is better, as it requires a smaller number of threads.
Related
I have a client/server application implemented with a thread pool of worker.
In server's main I create a listener socket and then for each client I get a fd with accept() to handle client's requests.
My question is: do I need a lock on this fd to read client's requests and to write the answers and data?
Nope. Listener sockets are coordination points, you don't need to explicitly coordinate. It's not even clear what "locking" a socket means; sockets don't "lock". They may or my not "block" depending on how they are configured - and this will determine if your operations return immediately, after a given timeout, or wait indefinitely to complete.
When you accept() on a listen socket successfully, the result is a second socket - this is the one that you use to communicate with the client, not the listener socket.
Of course if your application is multi-threaded, your /threads/ may need to lock or mutex to get proper behavior. Details will vary depending on your actual design and how socket IO states interact with threads can get very complicated / prickly. You'd need to post code for more concrete answers.
There are many more details to even single-threaded socket IO. See https://cis.temple.edu/~giorgio/old/cis307s96/readings/docs/sockets.html
I am just starting to learn sockets and client/servers. I am not clear on the following concept. Assume non-blocking sockets.
Assume I have a server application, and I have 1000 clients trying to talk to it, I think it is very realistic. Assume the client and server talk via sockets.
1- Does this mean that with every client, there is a separate socket connection? (Do we have 1000 sockets, or one socket with 1000 connections?
2- Does every socket connection belong to a separate thread? If Yes, How can we limit number of threads as it can get out of control?
Assuming you're using TCP, then every connection is over a separate socket. The operating system allocates them using file descriptors.
When using a protocol like UDP, this need not be the case, and won't be unless you write the code to do make it happen.
Threading? It depends on how you build the server. You don't need threads to be a part of a server at all and you can (obviously) have multiple threads with just a single connection. One common way of doing things, however, is to hand the socket returned by accept() to a new thread, yes.
If you don't have an interest in threads--for example, if the server only performs very quick tasks and creating a thread is just wasting time--you can use select() to poll the sockets and determine which ones need attention. Some servers use a combination of threading and polling to try to maximize throughput.
Let's say, I have a server with many connected clients via TCP, i have a socket for every client and i have a sending and receiving thread for every client. Is it safe and possible to call send function at the same time as it will not call send function for same socket.
If it's safe and ok, Can i stream data to clients simultaneously without blocking send function for other clients ?
Thank you very much for answers.
Yes it is possible and thread-safe. You could have tested it, or worked out for yourself that IS, IIS, SQL Server etc. wouldn't work very well if it wasn't.
Assuming this is Windows from the tag of "Winsock".
This design (having a send/receive thread for every single connected client), overall, is not going to scale. Hopefully you are aware of that and you know that you have an extremely limited number of clients (even then, I wouldn't write it this way).
You don't need to have a thread pair for every single client.
You can serve tons of clients with a single thread using non-blocking IO and read/write ready notifications (either with select() or one of the varieties of Overlapped IO such as completion routines or completion ports). If you use completion ports you can set a pool of threads to handle socket IO and queue the work for your own worker thread or threads/threadpool.
Yes, you can send and receive to many sockets at once from different threads; but you shouldn't need those extra threads because you shouldn't be making blocking calls to send/recv at all. When you make a non-blocking call the amount that could be written immediately is written and the function returns, you then note how much was sent and ask for notification when the socket is next writable.
I think you might want to consider a different approach as this isn't simple stuff; if you're using .Net you might get by with building this with TcpListener or HttpListener (both of which use completion ports for you), though be aware that you can't easily disable Nagle's algorithm with those so if you need interactivity (think of the auto-complete on Google's search page) then you probably won't get the performance you want.
I am wring a small http server which is using the Microsoft Windows WinSock API.
Do I need to apply multithreaded logic when handling multiple users?
Currently Windows sends a message when there is a network event and each message
carried (in wParam) the socket to be used in either send() or recv().
When client A connects and requests a couple of files usually a number of socket
are created by Winsock. My server then get a message that "send this file to
socket 123" and later "send that file to socket 456"
When another client connect it too gets a few sockets, say 789 and 654.
My server then respond to requests to send data using supplied socket number. It
does not have to know who wants the file since the correct file has to be sent to
the right socket.
I do not know whether Windows itself uses multiple threads when handling
accepting connection and sending the message down to my program.
So my question is:
Do I need to apply multithreaded logic when handling multiple users? And if so at
what point should I create a thread?
You typically use a thread per socket. And if you are accepting connections, a thread in a loop to block, waiting for an incoming connection socket. You then create a new thread and pass this socket handle to the new thread to handle. When that connection is closed and done with, simply let that thread terminate (or join). This is the basis of a threaded server.
in psudo code...
loop {
socket = accept();
new ThreadHandler( socket )
}
Using a single thread to handle multiple sockets is tricky, mainly because the thread can block (stop, waiting) while its writing, or more often, reading from a socket. It's not for the faint hearted.
For most applications, there is no point in using multiple threads to handle network connections. I've made a small writeup in an answer to this question.
Multiple threads become useful when handling the received data requires an unpredictable amount of CPU time, for example in database servers, or when the program structure does not allow for requests to be handled asynchronously.
There is also a third option, the "worker pool". A single thread handles all incoming connections and deserializes incoming requests, and then passes off work items to a pool of threads that handle one item at a time.
This way, simply opening a connection does not yet consume the resources needed for an entire thread, and system load is implicitly limited by the number of threads in the pool.
I'm developping a little data processor in c++ over UDP sockets, and have a thread (just one, and apart the sockets) that process the info received from them.
My problem happens when i need to receive info from multiple clients in the socket at the same time.
How could i do something like:
Socket foo;
/* init socket vars and attribs */
while (serving){
thread_processing(foo_info);
}
for multiple clients (many concurrent access) in c++?
I'm using winsocks atm on win32, but just get standard blocking udp sockets working. No gui, it's a console app.
I'll appreciate so much an example or pointer to one ;).
Thanks in advance.
UDP socket is able to receive datagrams from multiple clients with the recvfrom() function. Just block on receive, read the request, process it, send the reply, repeat. You don't even need a thread unless processing takes a very long time (in that case a thread connected with two queues, in- and out-, would work).
I would suggest this is best tackled by putting the requests in a queue and letting the other thread work off the queue. This decouples the socket receive from the process and thus allows you to scale to more listeners and processing threads if your requirements change.