Related
I've followed the code in this answer to create a pair of programs which send and receive datagrams via a Unix socket.
What's awkward about this: On the side which creates the first socket (i.e. the "server"), I can't use calls to send, recv, read or write because there is no destination set (those calls fail with "Destination address required" error.
I've tried working around this by adding an initial call to recvfrom and using the address given back through there, but it never has the correct value (on OSX at least). It also doesn't work to use sendto since we don't know the client address.
The way which I have got it working is roughly following this process:
Start server program, which:
Calls socket and bind to create the server socket.
It waits here.
Start client program, which:
Calls socket and bind to create the client socket.
It knows the path to the server socket and calls connect.
This side is now set up correctly.
Server program:
Accepts the path to the client socket via stdin
Copies the path to a struct sockaddr_un and uses that to call connect (as in the linked answer).
This is pretty awkward! If I was doing this with SOCK_STREAM sockets, I could use listen and accept; the flow is much more straight-forward without the server needing to know the client's socket path.
Is there a more elegant way of getting these sockets connected?
SOCK_DGRAM (UDP) sockets are "Connectionless", so you cannot "connect" the two sockets. They only send packets to the designated destination address and the client simply captures it. So you'll to first decide if you are going to use a SOCK_DGRAM (UDP) or SOCK_STREAM (TCP).
If you are using UDP sockets the client side socket need not connect, you simply sendto the destination address (Server in this case) after creating and binding.
So if you need a dedicated connected connection you are better off using TCP socket. Or if you are using this over the internet the closest thing you can find for UDP is Hole punching.
One way to solve the problem:
Your messages probably have common header.
Add address information of sender to the header.
Then your server can respond to the correct client by using sendto.
Pseudo example:
void handle_my_message(const my_message_t *msg)
{
struct sockaddr_un client_address = msg->header.sender;
my_message_response_t response_msg;
... handle the message and fill the response...
// Send response message
sendto(fd, &response_msg, sizeof(response_msg), 0,
(struct sockaddr*)&client_address, sizeof(client_address));
}
This way your server programs does not need to keep book of connections.
Instead of struct sockaddr_un in the header you maybe should use something smaller and more portable format, that can be converted to struct sockaddr_un.
You should also bind the client side socket to an address. If the client socket is bound (i.e. has its own name), then you don't need an out-of-band mechanism to communicate the client's address to the server. The OS sends it along with each datagram.
Sample code for client (in python because it's quick and easy to prototype -- should be easy to translate to the equivalent C):
#!/usr/bin/env python3
import os
import socket
server_addr = "/tmp/ux_server"
client_addr = "/tmp/ux_client"
if os.path.exists(client_addr):
os.remove(client_addr)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.bind(client_addr)
for n in range(5):
data = "Hello " + str(n)
data = data.encode()
print("Sent '{}' to {}".format(data, server_addr))
sock.sendto(data, server_addr)
data, addr = sock.recvfrom(16000)
print("Got '{}' back from {}".format(data, addr))
Furthermore, you can execute a connect on the client side. Since it's a datagram socket, that doesn't actually create a connection between the two but it does fix the address of the server endpoint, relieving you of the need to provide the server address on every send (i.e. you can use simple send rather than sendto).
For completeness, here's the echo server corresponding to the above:
#!/usr/bin/env python3
import os
import socket
server_addr = "/tmp/ux_server"
if os.path.exists(server_addr):
# Bind will fail if endpoint exists
os.remove(server_addr)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.bind(server_addr)
while True:
data, addr = sock.recvfrom(16000)
print("Got '{}' from {}".format(data, addr))
sock.sendto(data, addr)
EDIT
Hmm... I see now that you say you're already binding the client socket, and then connecting to the server side. But that means you simply need to have the server use recvfrom once initially to obtain the client's address. The OS will send the address along and you don't need to use an out-of-band mechanism.
The downside to connecting the socket is that if the client goes down, the server won't know that unless it attempts to send, but the client won't be able to reconnect because the server's socket is already connected. That's why datagram servers typically use recvfrom and sendto for all messages.
Updated server with initial recvfrom followed by connect:
#!/usr/bin/env python3
import os
import socket
server_addr = "/tmp/ux_server"
if os.path.exists(server_addr):
# Bind will fail if endpoint exists
os.remove(server_addr)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.bind(server_addr)
client_addr = None
while True:
if client_addr:
data = sock.recv(16000)
else:
data, client_addr = sock.recvfrom(16000)
sock.connect(client_addr)
print("Got '{}' from {}".format(data, client_addr))
sock.send(data)
Updated client with connected socket.
#!/usr/bin/env python3
import os
import socket
server_addr = "/tmp/ux_server"
client_addr = "/tmp/ux_client"
if os.path.exists(client_addr):
os.remove(client_addr)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.bind(client_addr)
sock.connect(server_addr)
for n in range(5):
data = ("Hello " + str(n)).encode()
print("Sent '{}'".format(data))
sock.send(data)
data = sock.recv(16000)
print("Got '{}' back".format(data))
Consider the following sequence:
Client app (web browser) opens several TCP connections to different web servers;
Ethernet cable then becomes disconnected;
Client app is then closed;
Ethernet cable remains disconnected for a few hours;
Ethernet cable is reconnected;
I see "TCP keep-alive" packets (every 60 seconds, for hours) from a few of the servers to which the long-closed client app had connected!
Normally, when an app is closing, the app would initiate the closure of each open socket, and the TCP layer would then attempt to send a FIN packet to each remote endpoint. If it is physically possible to send the FIN packet, and such sending actually happens, then the local endpoint goes from the ESTABLISHED state to the FINWAIT_1 state (and awaits receiving an ACK from the remote endpoint, etc.). But, if the physical link is broken, then the TCP local endpoint can't send that FIN, and the server still assumes the TCP connection still exists (and the client-side call to the "close" function would block indefinitely until the physical link was reestablished, assuming the socket were set to blocking mode, right?).
In any case, upon reconnecting the Ethernet cable after some time with all conventional networked apps (e.g., web browsers) long closed, I am receiving "TCP Keep-Alive" packets from three separate web servers at precisely 60-second intervals for HOURS!
Wireshark shows the local port numbers to which these TCP Keep-Alive packets are being sent, but neither TCPView nor netstat -abno show those local port numbers being used by any application. Looking at the "TCP/IP" property of every single running process using Process Explorer also does not show any matching port numbers. I don't think the ports are being held because of a zombie "process record" (of, say, the web browser process) due to any ongoing child process (e.g., plugin app), but I'm not sure if my observations with TCPView/netstat/Process Explorer were sufficient to rule out this possibility.
Given the identities of the remote web servers (e.g., Akamai servers), I believe the connections were established by "recent" use of a web browser. But, these keep-alives keep coming from those three web servers, even though the browser had been closed, and the physical link had been broken for hours.
If the connections appeared in TCPView, I could simply select them and manually close them. However, the client-side TCP endpoints seem long gone.
Meanwhile, I am baffled why the servers are retrying so many times to get a reply to their keep-alive packets.
TCP keep-alive behavior is typically controlled by three parameters: \
(1) Time to wait until the next "burst" or "probe" attempts;
(2) Time interval between sending each keep-alive packet during a single "probe" attempt;
(3) The maximum number of "probe" attempts before the "burst" is considered a failure (and the TCP connection is consequently considered permanently broken).
For the TCP keep-alive packets I am seeing from the three different servers, the time interval between "probe" retries is exactly 60 seconds. But, it seems like the maximum number of "probe" retries is infinite, which seems like a really bad choice for any server!
Although I am curious about how this relentless stream of keep-alives was created and sustained, I am more immediately interested in how I might use a client-side application to force the server-side endpoints to close, given that there aren't existing local TCP endpoints receiving those keep-alive packets.
My rough idea is to create an app which creates a TCP-mode socket, binds (with port-number reuse allowed) to the port number to which the incoming keep-alives are directed, and then call "open" followed by "close", hoping that the server endpoint will make the TCP state transitions to reach the closed state one way or another! Another way might be to create a raw-mode socket, and receive the TCP keep-alive packet (which is just an ACK), and then form and send an appropriate FIN packet (with proper sequence number, etc., to pick up where the long-terminated client app evidently left off), and then receive an ACK and FIN before sending the final ACK.
One final note -- and I know there will be eye-rolling and ridicule: the working environment here is Windows XP SP3 running in VirtualBox on Windows 7! So, I'd prefer code or an open-source app which could achieve the goal (closing half-open TCP connection) within Windows XP SP3. Sure, I could restart the snapshot, which might close the connections -- but I am more interested in learning how to get more information about the state of network connections, and what I can do to handle this kind of TCP state problem.
I succeeded in provoking the closing of each apparent half-open TCP connection by writing a simple program (full code appears below) which binds a local socket to the port to which the server believes it is already connected, attempts to establish a new connection, and then closes the connection.
(Note: If the connection succeeds, I make an HTTP GET request, just because the phantom TCP keep-alives in my case are apparently originating from plain HTTP servers, and I was wondering what response I might get back. I think the "send" and "recv" calls could be removed without affecting the ability of the code to achieve the desired result.)
In the following code, the src_port_num variable represents the client-side port number (currently unused) to which the server is sending "TCP keep-alive" packets, and dst_ip_cstr is the IP address of the server (e.g., an Akamai web server), and dst_port_num is the port number (which, in my situation, happens to be a plain HTTP server at port 80).
CAUTION! By sharing this code I do not mean to imply that its theory of operation can be rigorously explained by an understanding of the TCP protocol specification. I just guessed that claiming an abandoned local port to which a remote endpoint is sending TCP keep-alive packets, and attempting to establish a new connection to that very same remote endpoint, would, one way or another, prod the remote endpoint to close the stale half-open connection -- and it happened to work for me.
#define _CRT_SECURE_NO_WARNINGS
#include <stdio.h>
#include <winsock2.h>
#pragma comment(lib, "ws2_32.lib")
void main()
{
// Local IP and port number
char * src_ip_cstr = "10.0.2.15";
int src_port_num = 4805;
// Remote IP and port number
char * dst_ip_cstr = "23.215.100.98";
int dst_port_num = 80;
int res = 0;
WSADATA wsadata;
res = WSAStartup( MAKEWORD(2,2), (&(wsadata)) );
if (0 != res) { printf("WSAStartup() FAIL\n"); return; }
printf( "\nSRC IP:%-16s Port:%d\nDST IP:%-16s Port:%d\n\n",
src_ip_cstr, src_port_num, dst_ip_cstr, dst_port_num );
sockaddr_in src;
memset( (void*)&src, 0, sizeof(src) );
src.sin_family = AF_INET;
src.sin_addr.S_un.S_addr = inet_addr( src_ip_cstr );
src.sin_port = htons( src_port_num );
sockaddr_in dst;
memset( (void*)&dst, 0, sizeof(dst) );
dst.sin_family = AF_INET;
dst.sin_addr.S_un.S_addr = inet_addr( dst_ip_cstr );
dst.sin_port = htons( dst_port_num );
int s = socket( PF_INET, SOCK_STREAM, IPPROTO_TCP );
if ((-1) == s) { printf("socket() FAIL\n"); return; }
int val = 1;
res = setsockopt( s, SOL_SOCKET, SO_REUSEADDR,
(const char*)&val, sizeof(val) );
if (0 != res) { printf("setsockopt() FAIL\n"); return; }
res = bind( s, (sockaddr*)&src, sizeof(src) );
if ((-1) == res) { printf("bind() FAIL\n"); return; }
res = connect( s, (sockaddr*)&dst, sizeof(dst) );
if ((-1) == res) { printf("connect() FAIL\n"); return; }
char req[1024];
sprintf( req, "GET / HTTP/1.1\r\nHost: %s\r\nAccept: text/html\r\n"
"Accept-Language: en-us,en\r\nAccept-Charset: US-ASCII\r\n\r\n",
dst_ip_cstr );
printf("REQUEST:\n================\n%s\n================\n\n", req );
res = send( s, (char*)&req, strlen(req), 0 );
if ((-1) == res) { printf("send() FAIL\n"); return; }
const int REPLY_SIZE = 4096;
char reply[REPLY_SIZE];
memset( (void*)&reply, 0, REPLY_SIZE );
res = recv( s, (char*)&reply, REPLY_SIZE, 0 );
if ((-1) == res) { printf("recv() FAIL\n"); return; }
printf("REPLY:\n================\n%s\n================\n\n", reply );
res = shutdown( s, SD_BOTH );
res = closesocket( s );
res = WSACleanup();
}
HILARIOUS / SHAMEFUL / FASCINATING DISCLOSURES
As I mentioned in my original question, I observed these "TCP keep-alive" packets with Wireshark within VirtualBox running Windows XP SP3, where the host OS was Windows 7.
When I woke up this morning and looked at the phenomenon again with a cup of coffee and fresh eyes, with the "TCP keep-alive" packets still appearing every 60 seconds even after 24 hours, I made a hilarious discovery: These packets continued to arrive from the three different IP addresses, precisely at 60-second intervals (but staggered for the three IPs), even when I disconnected the Ethernet cable from the Internet! My mind was blown!
So, although the three IP addresses did correspond to real-world web servers to which my web browser connected long ago, the TCP keep-alive packets were clearly originating from some local software component.
This revelation, as shocking as it was, did not change my thinking about the situation: from my client-side software perspective, there were "server-side" half-open TCP connections that I wanted to provoke to closing.
Within VirtualBox, choosing "Devices" -> "Network" -> "Connect Network Adapter" toggles the virtual network adapter on or off, as if a virtual Ethernet cable were connected or disconnected. Toggling to a disconnected state caused the phantom TCP keep-alive packets to stop arriving to Wireshark. Subsequently toggling to a connected state caused the TCP keep-alive packets to resume arriving in Wireshark.
Anyway, I sometimes needed to run the code above TWICE to succeed in closing the half-open connection. When running the code a first time, Wireshark would show a packet with an annotation "[TCP ACKed unseen segment]", which is just the kind of TCP gas-lighting confusion I hoped to create, haha! Because the new client endpoint is unexpected by the remote endpoint, the call to "connect" hangs for maybe 30 seconds before failing. For a couple of the zombie/phantom half-open connections, running the program just once was enough to also cause an RST packet.
I needed to modify the program repeatedly to change the combination of local port number, remote IP, and remote port number, to match each phantom TCP keep-alive packet I observed in Wireshark. (I leave implementing user-friendly command-line parameters to the dear reader (that's you!).) After a few rounds of modifying and running the program, all zombie keep-alive packets were stopped. "Silence of the Packets", one might say.
EPILOGUE
[In tuxedo, martini glass in hand, gazing wistfully at the ocean from the deck of a yacht, in the company of fellow hackers] "I never did figure out where those zombie packets came from... Was it the 'VirtualBox Host-Only Network' virtual Ethernet adapter? Only the Oracle knows!"
There is nothing you need to do to close the remote socket, it is already built into the TCP protocol. If the system receives TCP packets which don't create a new connection (i.e. have SYN set) and don't belong to any established connection, it will reply with a RST packet. This way the peer will know that the endpoint is no longer there and abandon the connection.
I have a problem in the WSASocket programming. I want to do some trick at server side so that
it could hold the client side wait for a while in the WSAWaitForEvents() function waiting for
FD_CONNECT event.
Details are as follows:
At the client side, the socket is non-blocking mode. And it tries to connect to the server. It main code is something like:
SOCKET s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
WSAEvent hEvent = WSACreateEvent();
WSAEventSelect(s, hEvent, FD_CONNECT); //this also make socket non-blocking mode.
connect(s, &someserveraddr, sizeof(someserveraddr)); //connect to some server in non-block mode
WSAWaitForMultipleEvents(1, &hEvent, TRUE, WSA_INFINITE, FALSE); //this will block until success or faile
At the server side, one the server sees a connection from that client, it will do something special which will also take sometime, for example: calling doSomethingLengthy(), So I want to hold the client side blocking at function WSAWaitForMultipleEvents(...) until the server finishes that task. But I don't know how to achieve this. Usually, the server code would like:
SOCKET s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
bind(s, &someaddress, sizeof(someaddress));
listen(s, 5);
SOCKET acceptSocket = accept(s, &someotheraddress, sizeof(someotheraddress));
The problem is that, I don't know where/when to call doSomethingLengthy(). I know that once listen() is done, the client will be notified done and WSAWaitForMultipleEvents() will return. But I cannot call doSomethingLengthy() before listen() otherwise the client side connect() will fail.
You can't. The server end of connect happens before the server gets to see the accepted socket, via the backlog queue.
FD_CONNECT tells you when the socket has connected, but FD_WRITE tells you when you are allowed to send data over the connection. So try waiting for FD_WRITE instead (do note that you can get FD_WRITE multiple times during a connection's lifetime, but you will always get it after a successful connect() in addition to FD_CONNECT).
I want to connect clients to a server using ZeroMQ (java bindings, jzmq), but I need the TCP information badly, for example the TCP/IP address of a client request! The problem is, for being able to announce a service in the network I need to grab the TCP address of a request to be able to redirect clients to that service. The broker is a central "service registry" in that case. However, having ZeroMQ services on both sides, I do not see an option to retrieve that information.
What I do now, is to establish a dummy connection using a standard socket to the broker, after the connection is established I grab the IP address used for this connection and close the connection again. The IP address which has been retrieved is now being used for binding on it using a ZeroMQ socket on a random port.
I think this solution is the ugliest solution ever possible, so: What is a better solution to this problem?
Greetings.
0MQ doesn't provide the address of peers, for a number of reasons. It's also not that useful since what you really want is the endpoint to receive connections on, not the address the connection was made on.
What I usually do, and it's elegant enough, is pass bind a service to an ephemeral port, get a full connection endpoint ("tcp://ipaddress:port") and send that string in some way, either broadcast to peers, to a central registry, etc. along with my service name. Then, peers who want to connect back can take the service name, look up to find my endpoint, and connect back to me.
In ZMQ 4.x, you may get the string property "Peer-Address" or the "Identity" property. http://api.zeromq.org/4-2:zmq-msg-gets
The Identity is set in the other peer before connect(). http://api.zeromq.org/4-2:zmq-setsockopt#toc20
For example,
const char *identityString = "identity";
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_REQ);
socket.setsockopt(ZMQ_IDENTITY, identityString, strlen(identityString));
socket.connect("tcp://127.0.0.1:5555");
Then the other side:
while(1)
{
zmq::message_t request;
if (socket.recv(&request, ZMQ_NOBLOCK))
{
const char* identity = request.gets("Identity");
const char* peerAddress = request.gets("Peer-Address");
printf("Received from %s %s\n", peerAddress, identity);
break;
}
}
I'm using CppZmq btw, you should be able to find the relevant calls easily.
Digging deeper into the libzmq code, I discovered that the library attaches to every message instance the file descriptor that it was received on.
This worked for me
int sockfd = zmq_msg_get(&msg, ZMQ_SRCFD);
sockaddr_in addr;
socklen_t asize = sizeof(addr);
getpeername(sockfd, (sockaddr*)&addr, &asize);
std::cout << inet_ntoa(addr.sin_addr) << ":" << addr.sin_port << std::endl;
Note that the FDs can and will be reused by other connections.
I'm working with version 4.2.1 of the api using the CZMQ binding and I found a solution for my case (ZMQ_STREAM). It works by setting an id before connecting.
The relevant socket option is "ZMQ_CONNECT_RID".
ZMQ api via zmq_setsockopt()
CZMQ api via zsock_set_connect_rid()
Some codes with redacted redacted ips.
const char endpoint1[] = "tcp://1.2.3.4:12345"
const char endpoint2[] = "tcp://5.6.7.8:12345"
zsock_t *stream = zsock_new(ZMQ_STREAM);
zsock_set_connect_rid(stream, endpoint1);
zsock_connect(stream, endpoint1);
zsock_set_connect_rid(stream, endpoint2);
zsock_connect(stream, endpoint2);
Then I get those 2 messages if there is a connection. First frame is the id and second frame is empty on connect/disconnect for ZMQ_STREAM sockets.
[Message1]
[019] tcp://1.2.3.4:12345
[000]
[Message2]
[019] tcp://5.6.7.8:12345
[000]
Another option is to use the zmq_socket_monitor() or czmq zmonitor. It was one of my first solution but I was looking for something lighter. I was able the get the endpoint that way without setting the id directly on the socket.
The zmonitor zactor make it possible to subscribe to socket events and then it sends a message with 3 frames:
[009] CONNECTED
[002] 14
[021] tcp://127.0.0.1:33445
I am working with client-server programming I am referring this link and my server is successfully running.
I need to send data continuously to the server.
I don't want to connect() every time before sending each packet. So for first time I just created a socket and send the first packet, the rest of the data I just used write() function to write data to the socket.
But my problem is while sending data continuously if the server is not there or my Ethernet is disabled still it successfully write data to socket.
Is there any method by which I can create socket only at once and send data continuously with knowing server failure?.
The main reason for doing like this that, on the server side I am using GPRS modem and on each time when call connect() function for each packet the modem get hanged.
For creating socket I using below code
Gprs_sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (Gprs_sockfd < 0)
{
Display("ERROR opening socket");
return 0;
}
server = gethostbyname((const char*)ip_address);
if (server == NULL)
{
Display("ERROR, no such host");
return 0;
}
bzero((char *) &serv_addr, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
bcopy((char *)server->h_addr,(char *)&serv_addr.sin_addr.s_addr,server->h_length);
serv_addr.sin_port = htons(portno);
if (connect(Gprs_sockfd,(struct sockaddr *) &serv_addr,sizeof(serv_addr)) < 0)
{
Display("ERROR connecting");
return 0;
}
And each time I writing to the socket using the below code
n = write(Gprs_sockfd,data,length);
if(n<0)
{
Display("ERROR writing to socket");
return 0;
}
Thanks in advance.............
TCP was designed to tolerate temporary failures. It does byte sequencing, acknowledgments, and, if necessary, retransmissions. All unacknowledged data is buffered inside the kernel network stack. If I remember correctly the default is three re-transmission attempts (somebody correct me if I'm wrong) with exponential back-off timeouts. That quickly adds up to dozens of seconds, if not minutes.
My suggestion would be to design application-level acknowledgments into your protocol, meaning the server would send a short reply saying that it received that much data up to now, say every second. If the client does not receive suck ack in say 3 seconds, the client knows the connection is unusable and can close it. By the way, this is easier done with non-blocking sockets and polling functions like select(2) or poll(2).
Edit 0:
I think this would be very relevant here - "The ultimate SO_LINGER page, or: why is my tcp not reliable".
Nikolai is correct here, the behaviour you experience here is desirable as basically you could continue transfering data after network outage without any logic in your application. If your application should detect outages longer that specified amount of time, you need to add heartbeating into your protocol. This is standard way of solving the problem. It can also allow you for detect situation when network is all right, receiver is alive, but it has deadlocked (due to to a software bug).
Heartbeating could be as simple as mentioned by Nikolai -- sending a small packet every X seconds; if the server can't see the packet for N*X seconds, the connection would be dropped.