How can we remove close_wait state of the socket without restarting the server? - sockets

We have written an application in which client-server communication is used with the IOCP concept.
Client connects to the server through wireless access points.
When temporary disconnection happens in the network, this can lead a CLOSE_WAIT state.This could indicate that the
client properly closed the connection. But the server still has its socket open.
If there are too many instances of the port (to which the server and client were talking) were in CLOSE_WAIT state then at the highest peak ,server stop functioning thus rejecting the connection.That is totally frustrating.In this case, user has to restart the server to wipe out all the close_wait state by clearing the memory.When server restart,client again try to connect to the server.Server calls accept command again,But before accepting a new connection ,previous connection should be closed at server side,How can we do that ?
How can we remove close_wait state of the socket without restarting the server ?
Is there any alternate way to avoid server restart ?
We also came to know that,If all of the available ephemeral ports are allocated to client applications then the
client experiences a condition known as TCP/IP port exhaustion. When TCP/IP port exhaustion occurs, client port
reservations cannot be made and errors will occur in client applications that attempt to connect to a server via TCP/IP sockets.
if this is happening then we need to increase the upper range of ephemeral ports that are dynamically allocated to client TCP/IP socket connections.
Reference :
http://msdn.microsoft.com/en-us/library/aa560610%28v=bts.10%29.aspx
Let us know if this alternate way is useful or not ?
Thanks in advance.
Regards
Amey

Fix the server code.
The server should be reading with a timeout, and if the timeout expires it should close the socket.

Related

Understanding the requisites that allow bittorrent peers to connect to each other via TCP

BitTorrent peers connect with each other via TCP (mainly). When a peer A tries to connect to peer B, does peer B also needs to try to connect with A simultaneously so the TCP 3-way handshake happens and they form a connection? If not, why?
Also, I have been studying three bittorrent client implementations. While they start TCP connections with the obtained peers, I noticed none of them opens a TCP socket to listen on the port they are announcing to the tracker. Does it mean no one can initiate connection to them? Shouldn't they create such TCP socket?
When a peer A tries to connect to peer B, does peer B also needs to try to connect with A simultaneously so the TCP 3-way handshake happens and they form a connection? If not, why?
Connection setup is a general TCP feature, not specific to bittorrent. One side initiates the connection by calling connect on an unconnected socket and the other side has a listening socket configured on which it calls accept in a loop to create create connection-specific sockets for each accepted incoming connection.
There is a simultaneous open flow for connection setup but that's rarely relevant and the connect/accept flow is used by bittorrent clients.
I noticed none of them opens a TCP socket to listen on the port they are announcing to the tracker.
They generally do and should unless process privileges are insufficient to bind a particular port or another process is already listening on it, in which case they should log a warning at least.
If you used a portscan then you may be seeing firewall or NATs getting in the way rather than the client not having a listening socket open. Instead you could use something like netstat (may need some additional arguments, depending on OS) to show listening sockets.
If they truly do not have a listening socket open then yes, that would be a problem since they could not accept incoming connections and only talk to a more limited set of clients (those that do).
Bittorrent being a peer-to-peer protocol means that clients should be equals (peers) which means they should be equally capable of initiating and accepting connections.

syn flooding attack on a telnet server that listens on multiple ports

I want to implement a telnet server which listens on different ports for different applications. How to do it in a clean and efficient way?
Suppose I am able to do it, i.e my telnet server listens on port 23 and 12345. If an attacker launches an SYN flooding attack against the telnet server on my telnet server at port 23. What will happen to another port - 12345, when the attack is successful? Is it still accessible?
A SYN flood attack is fully handled in the OS kernel. A server doing accept will only return from accept if the three-way-handshake to establish the TCP connection was already successful, which is not the case with SYN flooding.
But, SYN flooding will affect the memory usage of the system. If this will only affect the single socket or will affect the system in general depends on the OS and maybe its configuration. But it should not actually matter if there is a process handling multiple sockets vs. multiple processes each handling a single socket.

How to handle TCP keepalive in application

I have a TCP application running on VxWorks. I have SO_KEEPALIVE option set for my TCP connections. My application keep track of all TCP connection and put it into a link list.
If client is idle for long time, we see that connection is closing down. Connection is not listed in netstat output.
As the connection is closed by TCP stack, resources allocated for that connection are not cleaned up. Can you please help me figure out how does application get notified if connection is closed due to keep-alive's failures.
TCP keepalive is intended primarily to prevent network routers from shutting the TCP connection down during long periods of inactivity, not to prevent your OS or application from shutting down the connection when it deems appropriate.
In most TCP/IP implementations, you can determine if a connection has been closed by attempting to read from it.
From this reference : http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/overview.html
I quote :
This procedure is useful because if the other peers lose their connection (for example by rebooting) you will notice that the connection is broken, even if you don't have traffic on it. If the keepalive probes are not replied to by your peer, you can assert that the connection cannot be considered valid and then take the correct action.
If you have a server for instance and a lot of clients can connect to it, without sending regularly, you might end up in a situation with clients that are no longer there. A client may have rebooted and this goes undetected because a FIN is never sent in that case.
For cases like this the keepalive exists.
From TCP point of view there is nothing special with a keep alive. And hence if the peer fails to ack a keepalive, you will receive 0 bytes on your socket and you'll have to close your end of the socket. Which is the only corrective action you can do at that moment.
As the connection is closed by TCP stack, resources allocated for that connection are not cleaned up.
Only if you never use the connection again.
If client is idle for long time, we see that connection is closing down. Connection is not listed in netstat output.
Make up your mind. Either you see it or you don't. What you will see is the port in CLOSE_WAIT in netstat.
Can you please help me figure out how does application get notified if connection is closed due to keep-alive's failures.
Next time you use the connection for read or write you will get an ECONNRESET.

TCP connection between client and server gone wrong

I establish a TCP connection between my server and client which runs on the same host. We gather and read from the server or say source in our case continuously.
We read data on say 3 different ports.
Once the source stops publishing data or gets restarted , the server/source is not able to publish data again on the same port saying port is already bind. The reason given is that client still has established connection on those ports.
I wanted to know what could be the probable reasons of this ? Can there be issue since client is already listening on these ports and trying to reconnect again and again because we try this reconnection mechanism. I am more looking for reason on source side as the same code in client sides when source and client are on different host and not the same host works perfectly fine for us.
Edit:-
I found this while going through various article .
On the question of using SO_LINGER to send a RST on close to avoid the TIME_WAIT state: I've been having some problems with router access servers (names withheld to protect the guilty) that have problems dealing with back-to-back connections on a modem dedicated to a specific channel. What they do is let go of the connection, accept another call, attempt to connect to a well-known socket on a host, and the host refuses the connection because there is a connection in TIME_WAIT state involving the well-known socket. (Stevens' book TCP Illustrated, Vol 1 discusses this problem in more detail.) In order to avoid the connection-refused problem, I've had to install an option to do reset-on-close in the server when the server initiates the disconnection.
Link to source:- http://developerweb.net/viewtopic.php?id=2941
I guess i am facing the same problem: 'attempt to connect to a well-known socket on a host, and the host refuses the connection'. Probable fix mention is 'option to do reset-on-close in the server when the server initiates the disconnection'. Now how do I do that ?
Set the SO_REUSEADDR option on the server socket before you bind it and call listen().
EDIT The suggestion to fiddle around with SO_LINGER option is worthless and dangerous to your data in flight. Just use SO_RESUSEADDR.
You need to close the socket bound to that port before you restart/shutdown the server!
http://www.gnu.org/software/libc/manual/html_node/Closing-a-Socket.html
Also, there's a timeout time, which I think is 4 minutes, so if you created a TCP socket and close it, you may still have to wait 4 minutes until it closes.
You can use netstat to see all the bound ports on your system. If you shut down your server, or close your server after forking on connect, you may have zombie processes which are bound to certain ports that do not close and remain active, and thus, you can't rebind to the same port. Show some code.

Connectivity issues with SSL Socket Server

Socket Server with SSLStream some times refuses new connections from clients.
I used the telent hostname port, and it says Connecting To host...
Could not open connection to the host, on port 6002: Connect failed
I used netstat -a , and I see TCP status as
TCP 0.0.0.0:6002 host:0 LISTENING
I also see the service as listening in tcpview too.
The error I see on client side is connection refused with error code 10061.
The same socket server was accepting new connections and just runs fine without any issues.But after some time the above issue happens.its random.
When I restart the sockets it just works fine and accepts conenctions, which I don;t want to do it frequently.becasue this disconnects clients, who are already connected.
Could somebody help me to trouble shoot this?
Thanks.
Where are you running netstat? On the server?
Try connecting to the socket from localhost (from the server itself) using the destination IP address 127.0.0.1
Do the same test with the network IP of the server.
My guess is that the firewall is preventing external access or a router in between is preventing the connection.
It works for a while and then stops. Few options I can think of:
Some firewall on the way does some kind of throttling
You open and close too many connections too quickly. In this case you exhaust the ephemeral ports on the client (usually) and/or on the server. If you do netstat -a you will see a lot of sockets in TIME_WAIT state, try this both on client and server. Solution here is to reuse connections (best). Or increase the number of ephemeral ports (registry setting). But this will take you only so far.
You have a bug in your server and it stops accepting new connections after a while.