(Running on VS2017, Win7 x64)
I am confused about the point of SO_REUSEADDR and SO_EXCLUSIVEADDRUSE. And yes, I've read the MSDN documentation, but I'm obviously not getting it.
I have the following simple code in two separate processes. As expected, because I enable SO_REUSEADDR on both sockets, the second process's bind succeeds. If I don't enable this on any one of these sockets, the second bind will not succeed.
#define PORT 5150
SOCKET sockListen;
if ((sockListen = WSASocket(AF_INET, SOCK_STREAM, 0, NULL, 0, WSA_FLAG_OVERLAPPED)) == INVALID_SOCKET)
{
printf("WSASocket() failed with error %d\n", WSAGetLastError());
return 1;
}
int optval = 1;
if (setsockopt(sockListen, SOL_SOCKET, `SO_REUSEADDR`, (char*)&optval, sizeof(optval)) == -1)
return -1;
SOCKADDR_IN InternetAddr;
InternetAddr.sin_family = AF_INET;
InternetAddr.sin_addr.s_addr = inet_addr("10.15.20.97");
InternetAddr.sin_port = htons(PORT);
if (::bind(sockListen, (PSOCKADDR)&InternetAddr, sizeof(InternetAddr)) == SOCKET_ERROR)
{
printf("bind() failed with error %d\n", WSAGetLastError());
return 1;
}
So doesn't having to enable SO_REUSEADDR for both sockets make SO_EXCLUSIVEADDRUSE unnecessary - if I don't want anyone to foricibly bind to my port, I just don't enable SO_REUSEADDR in that process?
The only difference I can see is that if I enable SO_EXCLUSIVEADDRUSE in the first process, then attempt a bind in the second process, that second bind will fail with
a) WSAEADDRINUSE if I don't enable SO_REUSEADDR in that second process
b) WSAEACCES if I do enable SO_REUSEADDR in that second process
So I tried enabling both SO_EXCLUSIVEADDRUSE and SO_REUSEADDR in the first process but found that whichever one I attempted second failed with WSAEINVAL.
Note also that I have read this past question but what that says isn't what I'm seeing: it states
A socket with SO_REUSEADDR can always bind to exactly the same source
address and port as an already bound socket, even if the other socket
did not have this option set when it was bound
Now if that were the case then I can definitely see the need for SO_EXCLUSIVEADDRUSE.
I'm pretty sure I'm doing something wrong but I cannot see it; can someone clarify please?
As stated in the docs, SO_EXCLUSIVEADDRUSE became available on Windows NT4 SP4; before that there was only SO_REUSEADDR. So both being present has (also) historical reasons.
I think of SO_REUSEADDR as the intention to share an address (which is only really useful for UDP multicast. For unicast or TCP it really doesn´t do much since the bahaviour is non-deterministic for both sockets).
SO_EXCLUSIVEADDRUSE is a security measure to avoid my (server) application´s traffic being hijacked / rendered useless by a later binding to the same IP/port.
As I see it, you need SO_REUSEADDR for UDP multicats, and you need SO_EXCLUSIVEADDRUSE as a security measure for server applications.
Related
Consider the following sequence:
Client app (web browser) opens several TCP connections to different web servers;
Ethernet cable then becomes disconnected;
Client app is then closed;
Ethernet cable remains disconnected for a few hours;
Ethernet cable is reconnected;
I see "TCP keep-alive" packets (every 60 seconds, for hours) from a few of the servers to which the long-closed client app had connected!
Normally, when an app is closing, the app would initiate the closure of each open socket, and the TCP layer would then attempt to send a FIN packet to each remote endpoint. If it is physically possible to send the FIN packet, and such sending actually happens, then the local endpoint goes from the ESTABLISHED state to the FINWAIT_1 state (and awaits receiving an ACK from the remote endpoint, etc.). But, if the physical link is broken, then the TCP local endpoint can't send that FIN, and the server still assumes the TCP connection still exists (and the client-side call to the "close" function would block indefinitely until the physical link was reestablished, assuming the socket were set to blocking mode, right?).
In any case, upon reconnecting the Ethernet cable after some time with all conventional networked apps (e.g., web browsers) long closed, I am receiving "TCP Keep-Alive" packets from three separate web servers at precisely 60-second intervals for HOURS!
Wireshark shows the local port numbers to which these TCP Keep-Alive packets are being sent, but neither TCPView nor netstat -abno show those local port numbers being used by any application. Looking at the "TCP/IP" property of every single running process using Process Explorer also does not show any matching port numbers. I don't think the ports are being held because of a zombie "process record" (of, say, the web browser process) due to any ongoing child process (e.g., plugin app), but I'm not sure if my observations with TCPView/netstat/Process Explorer were sufficient to rule out this possibility.
Given the identities of the remote web servers (e.g., Akamai servers), I believe the connections were established by "recent" use of a web browser. But, these keep-alives keep coming from those three web servers, even though the browser had been closed, and the physical link had been broken for hours.
If the connections appeared in TCPView, I could simply select them and manually close them. However, the client-side TCP endpoints seem long gone.
Meanwhile, I am baffled why the servers are retrying so many times to get a reply to their keep-alive packets.
TCP keep-alive behavior is typically controlled by three parameters: \
(1) Time to wait until the next "burst" or "probe" attempts;
(2) Time interval between sending each keep-alive packet during a single "probe" attempt;
(3) The maximum number of "probe" attempts before the "burst" is considered a failure (and the TCP connection is consequently considered permanently broken).
For the TCP keep-alive packets I am seeing from the three different servers, the time interval between "probe" retries is exactly 60 seconds. But, it seems like the maximum number of "probe" retries is infinite, which seems like a really bad choice for any server!
Although I am curious about how this relentless stream of keep-alives was created and sustained, I am more immediately interested in how I might use a client-side application to force the server-side endpoints to close, given that there aren't existing local TCP endpoints receiving those keep-alive packets.
My rough idea is to create an app which creates a TCP-mode socket, binds (with port-number reuse allowed) to the port number to which the incoming keep-alives are directed, and then call "open" followed by "close", hoping that the server endpoint will make the TCP state transitions to reach the closed state one way or another! Another way might be to create a raw-mode socket, and receive the TCP keep-alive packet (which is just an ACK), and then form and send an appropriate FIN packet (with proper sequence number, etc., to pick up where the long-terminated client app evidently left off), and then receive an ACK and FIN before sending the final ACK.
One final note -- and I know there will be eye-rolling and ridicule: the working environment here is Windows XP SP3 running in VirtualBox on Windows 7! So, I'd prefer code or an open-source app which could achieve the goal (closing half-open TCP connection) within Windows XP SP3. Sure, I could restart the snapshot, which might close the connections -- but I am more interested in learning how to get more information about the state of network connections, and what I can do to handle this kind of TCP state problem.
I succeeded in provoking the closing of each apparent half-open TCP connection by writing a simple program (full code appears below) which binds a local socket to the port to which the server believes it is already connected, attempts to establish a new connection, and then closes the connection.
(Note: If the connection succeeds, I make an HTTP GET request, just because the phantom TCP keep-alives in my case are apparently originating from plain HTTP servers, and I was wondering what response I might get back. I think the "send" and "recv" calls could be removed without affecting the ability of the code to achieve the desired result.)
In the following code, the src_port_num variable represents the client-side port number (currently unused) to which the server is sending "TCP keep-alive" packets, and dst_ip_cstr is the IP address of the server (e.g., an Akamai web server), and dst_port_num is the port number (which, in my situation, happens to be a plain HTTP server at port 80).
CAUTION! By sharing this code I do not mean to imply that its theory of operation can be rigorously explained by an understanding of the TCP protocol specification. I just guessed that claiming an abandoned local port to which a remote endpoint is sending TCP keep-alive packets, and attempting to establish a new connection to that very same remote endpoint, would, one way or another, prod the remote endpoint to close the stale half-open connection -- and it happened to work for me.
#define _CRT_SECURE_NO_WARNINGS
#include <stdio.h>
#include <winsock2.h>
#pragma comment(lib, "ws2_32.lib")
void main()
{
// Local IP and port number
char * src_ip_cstr = "10.0.2.15";
int src_port_num = 4805;
// Remote IP and port number
char * dst_ip_cstr = "23.215.100.98";
int dst_port_num = 80;
int res = 0;
WSADATA wsadata;
res = WSAStartup( MAKEWORD(2,2), (&(wsadata)) );
if (0 != res) { printf("WSAStartup() FAIL\n"); return; }
printf( "\nSRC IP:%-16s Port:%d\nDST IP:%-16s Port:%d\n\n",
src_ip_cstr, src_port_num, dst_ip_cstr, dst_port_num );
sockaddr_in src;
memset( (void*)&src, 0, sizeof(src) );
src.sin_family = AF_INET;
src.sin_addr.S_un.S_addr = inet_addr( src_ip_cstr );
src.sin_port = htons( src_port_num );
sockaddr_in dst;
memset( (void*)&dst, 0, sizeof(dst) );
dst.sin_family = AF_INET;
dst.sin_addr.S_un.S_addr = inet_addr( dst_ip_cstr );
dst.sin_port = htons( dst_port_num );
int s = socket( PF_INET, SOCK_STREAM, IPPROTO_TCP );
if ((-1) == s) { printf("socket() FAIL\n"); return; }
int val = 1;
res = setsockopt( s, SOL_SOCKET, SO_REUSEADDR,
(const char*)&val, sizeof(val) );
if (0 != res) { printf("setsockopt() FAIL\n"); return; }
res = bind( s, (sockaddr*)&src, sizeof(src) );
if ((-1) == res) { printf("bind() FAIL\n"); return; }
res = connect( s, (sockaddr*)&dst, sizeof(dst) );
if ((-1) == res) { printf("connect() FAIL\n"); return; }
char req[1024];
sprintf( req, "GET / HTTP/1.1\r\nHost: %s\r\nAccept: text/html\r\n"
"Accept-Language: en-us,en\r\nAccept-Charset: US-ASCII\r\n\r\n",
dst_ip_cstr );
printf("REQUEST:\n================\n%s\n================\n\n", req );
res = send( s, (char*)&req, strlen(req), 0 );
if ((-1) == res) { printf("send() FAIL\n"); return; }
const int REPLY_SIZE = 4096;
char reply[REPLY_SIZE];
memset( (void*)&reply, 0, REPLY_SIZE );
res = recv( s, (char*)&reply, REPLY_SIZE, 0 );
if ((-1) == res) { printf("recv() FAIL\n"); return; }
printf("REPLY:\n================\n%s\n================\n\n", reply );
res = shutdown( s, SD_BOTH );
res = closesocket( s );
res = WSACleanup();
}
HILARIOUS / SHAMEFUL / FASCINATING DISCLOSURES
As I mentioned in my original question, I observed these "TCP keep-alive" packets with Wireshark within VirtualBox running Windows XP SP3, where the host OS was Windows 7.
When I woke up this morning and looked at the phenomenon again with a cup of coffee and fresh eyes, with the "TCP keep-alive" packets still appearing every 60 seconds even after 24 hours, I made a hilarious discovery: These packets continued to arrive from the three different IP addresses, precisely at 60-second intervals (but staggered for the three IPs), even when I disconnected the Ethernet cable from the Internet! My mind was blown!
So, although the three IP addresses did correspond to real-world web servers to which my web browser connected long ago, the TCP keep-alive packets were clearly originating from some local software component.
This revelation, as shocking as it was, did not change my thinking about the situation: from my client-side software perspective, there were "server-side" half-open TCP connections that I wanted to provoke to closing.
Within VirtualBox, choosing "Devices" -> "Network" -> "Connect Network Adapter" toggles the virtual network adapter on or off, as if a virtual Ethernet cable were connected or disconnected. Toggling to a disconnected state caused the phantom TCP keep-alive packets to stop arriving to Wireshark. Subsequently toggling to a connected state caused the TCP keep-alive packets to resume arriving in Wireshark.
Anyway, I sometimes needed to run the code above TWICE to succeed in closing the half-open connection. When running the code a first time, Wireshark would show a packet with an annotation "[TCP ACKed unseen segment]", which is just the kind of TCP gas-lighting confusion I hoped to create, haha! Because the new client endpoint is unexpected by the remote endpoint, the call to "connect" hangs for maybe 30 seconds before failing. For a couple of the zombie/phantom half-open connections, running the program just once was enough to also cause an RST packet.
I needed to modify the program repeatedly to change the combination of local port number, remote IP, and remote port number, to match each phantom TCP keep-alive packet I observed in Wireshark. (I leave implementing user-friendly command-line parameters to the dear reader (that's you!).) After a few rounds of modifying and running the program, all zombie keep-alive packets were stopped. "Silence of the Packets", one might say.
EPILOGUE
[In tuxedo, martini glass in hand, gazing wistfully at the ocean from the deck of a yacht, in the company of fellow hackers] "I never did figure out where those zombie packets came from... Was it the 'VirtualBox Host-Only Network' virtual Ethernet adapter? Only the Oracle knows!"
There is nothing you need to do to close the remote socket, it is already built into the TCP protocol. If the system receives TCP packets which don't create a new connection (i.e. have SYN set) and don't belong to any established connection, it will reply with a RST packet. This way the peer will know that the endpoint is no longer there and abandon the connection.
While programming for sockets I came across a doubt on usage of setsocketopt(). If we provide
setsockopt( socket_no, SOL_SOCKET,SO_REUSEADDR , (char *) &optval, (socklength) sizeof( optval ) );//To reuse addr
Followed by another socket option for the same socket,
setsockopt( socket_no, IPPROTO_IPV6,IPV6_V6ONLY , (char *) &optval, (socklength) sizeof( optval ) );//To use only IPv6
(1)Does setting socket option again, removes the reuse address option which is set already ?
In few situation, there is a need to close and reopen a static port immediately without any delay.
(2)Does closing and reopening a port immediately causes problem ?
(3)If closing and reopening a port immediately causes problem, can it be avoided using SO_REUSEPORT/SO_REUSEADDR in socket option. As it overcomes time_wait set by TCP protocol. Or is there some alternative to overcome this problem ?
(1)Does setting socket option again, removes the reuse address option which is set already ?
No. Each socket option is independent.
(2)Does closing and reopening a port immediately causes problem ?
No, not unless there was at least one TCP connection to or from that port recently.
(3)If closing and reopening a port immediately causes problem, can it be avoided using SO_REUSEPORT/SO_REUSEADDR in socket option. As it overcomes time_wait set by TCP protocol. Or is there some alternative to overcome this problem ?
It has no effect on time_wait. It just lets you re-open the port immediately. The existing connection is unaffected and continues to timeout normally, it just doesn't prevent you from re-opening the port.
I'm having what doesn't seem to be a problem but I don't understand the error. Basically, I've been trying to work on code that "gracefully" shuts down a tcp socket on both ends. At the end of my program, on both sides, I shutdown my sockets before closing them. On a quick restart, which is the thing I'm trying to solve, the much worse problem of crashing because the sockets are lingering on both sides doesn't happen anymore. The shutdown/then close seems to work on that front.
However, i get "Address already in use" still, which usually made it so that I couldn't connect. Now I'm able to connect after that error just fine. I've read a lot on the subject of graceful shutdown, reuse address, and the like. And I guess my question is, how, if the socket error'd on bind ("Address already in use"), after a successful open, is it possibly able to connect to the endpoint? In other words, if the address is actually already in use, how is the connection being made? Also, of note, reuse address doesn't work in this situation. Because I'm using the same socket settings, local/remote addresses and ip.
Failing to bind() the socket to an address does not invalidate the underlying socket. As such, the connect() operation will continue with an unbound socket, deferring to the kernel to bind to a local endpoint.
Here is a complete example demonstrating this behavior:
#include <boost/asio.hpp>
#include <boost/bind.hpp>
// This example is not interested in all handlers, so provide a noop function
// that will be passed to bind to meet the handler concept requirements.
void noop() {}
int main()
{
using boost::asio::ip::tcp;
// Create all I/O objects.
boost::asio::io_service io_service;
tcp::acceptor acceptor(io_service, {tcp::v4(), 0});
tcp::socket server(io_service, tcp::v4());
// Open socket1, binding to a random port.
tcp::socket socket1(io_service, {boost::asio::ip::address_v4::any(), 0});
tcp::socket socket2(io_service); // non-open
// Explicitly open client2, which will bind it to the any address.
boost::system::error_code error;
socket2.open(tcp::v4(), error);
assert(!error);
assert(socket2.local_endpoint().port() == 0);
// Attempt to bind socket2 to socket1's address will fail with
// an already in use error, leaving socket2 bound to the any endpoint.
// (e.g. a failed bind does not have side effects on the socket)
socket2.bind(socket1.local_endpoint(), error);
assert(error == boost::asio::error::address_in_use);
assert(socket2.local_endpoint().port() == 0);
// Connect will defer to the kernel to bind the socket.
acceptor.async_accept(server, boost::bind(&noop));
socket2.async_connect(acceptor.local_endpoint(),
[&error](const boost::system::error_code& ec) { error = ec; });
io_service.run();
io_service.reset();
assert(!error);
assert(socket2.local_endpoint().port() != 0);
}
I am facing this problem of Binding to socket.
1st instance works properly i.e.
socket() returns success and hence forth bind() and listen(), accept() and hence recv() - All fine till here.
2nd instance throw error while binding "Address already in use"
I went through all the post earlier on this and i dont see any specific solution provided on the same.
My code is as below :-
if((status = getaddrinfo(NULL,"8080",&hints,&servinfo))!=0){
ALOGE("Socket:: getaddrinfo failed %s\n",strerror(errno));
return NULL;
}
server_sockfd = socket(servinfo->ai_family, servinfo->ai_socktype, servinfo->ai_protocol);
if(server_sockfd == -1) {
ALOGE("Socket:: Scoket System Call failed %s\n",strerror(errno));
return NULL;
}
if ((setsockopt(server_sockfd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(int))) < 0)
{
ALOGE("Socket:: setsockopt failed %s\n",strerror(errno));
return NULL;
}
ret = bind(server_sockfd, servinfo->ai_addr,servinfo->ai_addrlen);
if(ret!=0) {
ALOGE("Socket:: Error Binding on socket %s\n",strerror(errno));
return NULL;
}
This code runs on android platform.
I have properly closed each session before opening a new session as below :-
ret = shutdown(client_sockfd,0);
if(ret != 0)
ALOGE("Socket:: Shutdown Called%s\n",strerror(errno));
I tried with close as well but it did not work.
Surprisingly the error does not disappear even when we try to open the socket after long time (as per TIME_WAIT logic)
Could anyone please guide me to proper call or API or Logic(in code and not on command line apart from directly killing the process) to handle this situation ?
A socket is one half a channel of communication between two computers over a network on a particular port. (the other half is the corresponding socket on the other computer)
Error is very clear I suppose in this case. As mentioned Address already in use, so the the socket you are trying to connect in the second attempt is already used (port was already occupied) -> maybe due to first socket connection.
To investigate further check another SO question here and here
You can't share a TCP listening port between two processes even with SO_REUSEADDR.
NB shutdown() does not close a TCP session. It half-closes it. You have to close the socket.
The purpose of the application is to listen for a specific UDP multicast and then to forward the data to any TCP clients connected to the server. The code works fine, but I have a problem with the sockets not closing after the TCP clients disconnects. A socketsniffer utility shows the the sockets remain open and all the UDP data continues to be forwarded to the clients. The problem I believe is with the "if ($write->connected())" block as it always return true, even if the TCP client is no longer connected. I use standard Windows Telnet to connect to the server and to see the data. When I close telnet, the TCP socket is suppose to close on the server.
Any reason why connected() show the connections as active even if they are not? Also, what alternative should I use then?
Code:
#!/usr/bin/perl
use IO::Socket::Multicast;
use IO::Socket;
use IO::Select;
my $tcp_port = "4550";
my $tcp_socket = IO::Socket::INET->new(
Listen => SOMAXCONN,
LocalAddr => '0.0.0.0',
LocalPort => $tcp_port,
Proto => 'tcp',
ReuseAddr => 1,
);
use Socket qw(IPPROTO_TCP TCP_NODELAY);
setsockopt( $tcp_socket, IPPROTO_TCP, TCP_NODELAY, 1);
use constant GROUP => '239.2.0.81';
use constant PORT => '6550';
my $udp_socket= IO::Socket::Multicast->new(Proto=>'udp',LocalPort=>PORT);
$udp_socket->mcast_add(GROUP) || die "Couldn't set group: $!\n";
my $read_select = IO::Select->new();
my $write_select = IO::Select->new();
$read_select->add($tcp_socket);
$read_select->add($udp_socket);
## Loop forever, reading data from the UDP socket and writing it to the
## TCP socket(s).
while (1) {
## No timeout specified (see docs for IO::Select). This will block until a TCP
## client connects or we have data.
my #read = $read_select->can_read();
foreach my $read (#read) {
if ($read == $tcp_socket) {
## Handle connect from TCP client. Note that UDP connections are
## stateless (no accept necessary)...
my $new_tcp = $read->accept();
$write_select->add($new_tcp);
}
elsif ($read == $udp_socket) {
## Handle data received from UDP socket...
my $recv_buffer;
$udp_socket->recv($recv_buffer, 1024, undef);
## Write the data read from UDP out to the TCP client(s). Again, no
## timeout. This will block until a TCP socket is writable.
my #write = $write_select->can_write();
foreach my $write (#write) {
## Make sure the socket is still connected before writing.
if ($write->connected()) {
$write->send($recv_buffer);
}
else {
$write_select->remove($write);
close $write;
}
}
}
}
}
I don't know anything about perl, or perl sockets for that matter, but I can't think of a socket API that provides a way to know if a socket is connected. In fact, I'm pretty sure TCP doesn't actually have a way of knowing immediately like this. This suggests to me that connected() is not telling you what you think it is telling you. (I have no idea, but I'll bet it is telling you whether you've called connect/accept or not)
Usually sockets tell you they've become disconnected by reading or writing zero bytes - you might want to check the return value of write to see if it ever returns zero
Thanks for the feedback. I have found a solution that seems to work well for me(Thanks Stewart). It was as simple as checking the return value:
$resultsend = $write->send($recv_buffer);
if (!$resultsend) {
$write_select->remove($write);
close $write;
}
TCP sockets are closed now after a client disconnects.
The IO::Socket connected method just tells you whether the socket is in a connected state, i.e., whether you called "connect" on it after you created it. As Stewart said, there is no general way to tell if the other end of a TCP socket dropped off, and whether you are "still connected".
Off the top of my head, try:
ReuseAddr => 0
I'm more than certain the IO::Socket's magic is what's causing your issues.