I have the situation where a service running in the background makes itself available for ASCII-based commands via a socket (SOCK_DGRAM) placed on the file system. I am able to successfully send commands to this interface but cannot receive any response generated by the background service.
As I understand it, the reason I am not receiving the service's response is because the underlying IPC is not technically between two processes, but is rather between to addresses. As such, it is necessary to bind my endpoint to a particular address location so the service knows were to send its response. However, the problem is that I do not want to pollute the directory space with too many additional socket files.
That is to say, I can make this work by simply doing something like:
struct sockaddr_un local;
int len;
s = socket(AF_UNIX, SOCK_DGRAM, 0);
local.sun_family = AF_UNIX;
strcpy(local.sun_path, "/path/to/some/dir/mySocketFile");
len = strlen(local.sun_path) + sizeof(local.sun_family);
bind(s, (struct sockaddr *)&local, len);
//Send commands to control interface of background service
And all is well, because by binding to mySocketFile the service has an address to which is will respond.
In short, is there a way to communicate to the service through its available socket interface and receive the response without binding the local endpoint such that it creates another socket-type file on the file system? i.e. some kind of a nameless socket, of sorts?
Of course, if anyone spots any misconceptions or misunderstandings in my logic please point them out.
If the client does not bind its socket to an filesystem address, it still has a notional address assigned by the system (which may exist in the filesystem in /tmp somewhere, or may not exist in the filesystem at all, depends on the OS). The server can get this address by using the recvfrom(2) call to receive the incoming packets from clients -- this call takes additional sockaddr * and socklen_t * arguments that it fills in with the client socket address. You then use sendto(2) to send the reply back to the client.
Related
Can we detect different clients(devices) from same IP on TCPConnection ?
Example :
I have a TCP Server called s1 and I have 2 PCs called p1,p2 and my PCs IP is same (e.g. 1.2.3.4)
when I connect to s1 (my TCP Server) with p1 and p2 (Pc1 and Pc2) can s1(my TCP server) detect these clients with same IP, isn't same device ?
From my understanding you are basically asking to detect different devices behind a NAT, i.e. devices sharing the same external IP address. There is no fully reliable way to do this but one can employ heuristics. Typically these are based on the ID field in the IP header and/or the TCP timestamp option, see for example A Technique for Counting NATted Hosts or Time has something to tell us about Network
Address Translation. One might also try to use passive OS fingerprinting in order to detect if different OS are used (and thus different real or virtual devices) - see Passive Fingerprinting.
None of these heuristics are fully reliable though and they also will not work if the devices are behind a proxy, since in this case the TCP/IP connections visible to the server originate from a single device - the proxy.
Yes you can. The server can ask the operating system for the connection information of client associated with the socket. In 'C' this would look like:
//Accept and incoming connection
puts("Waiting for incoming connections...");
c = sizeof(struct sockaddr_in);
new_socket = accept(socket_desc, (struct sockaddr *)&client, (socklen_t*)&c);
if (new_socket<0)
{
perror("accept failed");
return 1;
}
The client sockaddr structure will be filled with the information about the connecting client. The server can look into this to extract the IP Address as a string doing something like:
char *ip = inet_ntoa((struct sockaddr_in *)client.sin_addr);
You can now see if ip matches p1 or p2.
What I want to do is make my own transport layer protocol in C++. I can't figure out how to create a raw socket that that automatically resolves IP headers, and leaves it up to me to set the payload.
I managed to receive packets on the server using
socket(AF_PACKET, SOCK_RAW, htons(ETH_P_IP))
but didn't manage to create a client that can send data to the server. (I'm not even sure if the above socket is L2 or L3)
From what I understand from reading about raw sockets, a L3 socket would look like
socket(AF_INET, SOCK_RAW, protocol)
Thing is, I don't know what to fill in for the protocol, if my intention is to create my own and not to use existing ones. (I have tried many of the iana numbers, including the range 143-252)
So the question is: how to create a socket, server and client sided, on top of the Internet Protocol such that two computers can communicate in an arbitrary protocol (or send data to each other)? In other words, I want to specify the end IP address and a payload and have the socket take care of the IP header.
What I have now:
server.cpp: https://pastebin.com/yLMFLDmJ
client.cpp: https://pastebin.com/LWuNdqPT
For those who are searching, here is the solution I found: http://www.pdbuchan.com/rawsock/rawsock.html
In the file tcp4.c on the above mentioned page, there is a client implementation using a raw socket. The code adds both IP and TCP headers, but you can simply remove the lines where the TCP headers are added and replace them with your own protocol. You also need to change this line: iphdr.ip_p = IPPROTO_TCP to iphdr.ip_p = 200 (200 or any number in the range 143-252; see https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml) and set the name of the interface you are using, as well as change the addresses.
So here is a stripped down version with the mentioned changes that sends an IP packet only containing IP headers: https://pastebin.com/z2sGmtQd
And here is a very simple server that can receive these packets: https://pastebin.com/jJgZUv5p
I've followed the code in this answer to create a pair of programs which send and receive datagrams via a Unix socket.
What's awkward about this: On the side which creates the first socket (i.e. the "server"), I can't use calls to send, recv, read or write because there is no destination set (those calls fail with "Destination address required" error.
I've tried working around this by adding an initial call to recvfrom and using the address given back through there, but it never has the correct value (on OSX at least). It also doesn't work to use sendto since we don't know the client address.
The way which I have got it working is roughly following this process:
Start server program, which:
Calls socket and bind to create the server socket.
It waits here.
Start client program, which:
Calls socket and bind to create the client socket.
It knows the path to the server socket and calls connect.
This side is now set up correctly.
Server program:
Accepts the path to the client socket via stdin
Copies the path to a struct sockaddr_un and uses that to call connect (as in the linked answer).
This is pretty awkward! If I was doing this with SOCK_STREAM sockets, I could use listen and accept; the flow is much more straight-forward without the server needing to know the client's socket path.
Is there a more elegant way of getting these sockets connected?
SOCK_DGRAM (UDP) sockets are "Connectionless", so you cannot "connect" the two sockets. They only send packets to the designated destination address and the client simply captures it. So you'll to first decide if you are going to use a SOCK_DGRAM (UDP) or SOCK_STREAM (TCP).
If you are using UDP sockets the client side socket need not connect, you simply sendto the destination address (Server in this case) after creating and binding.
So if you need a dedicated connected connection you are better off using TCP socket. Or if you are using this over the internet the closest thing you can find for UDP is Hole punching.
One way to solve the problem:
Your messages probably have common header.
Add address information of sender to the header.
Then your server can respond to the correct client by using sendto.
Pseudo example:
void handle_my_message(const my_message_t *msg)
{
struct sockaddr_un client_address = msg->header.sender;
my_message_response_t response_msg;
... handle the message and fill the response...
// Send response message
sendto(fd, &response_msg, sizeof(response_msg), 0,
(struct sockaddr*)&client_address, sizeof(client_address));
}
This way your server programs does not need to keep book of connections.
Instead of struct sockaddr_un in the header you maybe should use something smaller and more portable format, that can be converted to struct sockaddr_un.
You should also bind the client side socket to an address. If the client socket is bound (i.e. has its own name), then you don't need an out-of-band mechanism to communicate the client's address to the server. The OS sends it along with each datagram.
Sample code for client (in python because it's quick and easy to prototype -- should be easy to translate to the equivalent C):
#!/usr/bin/env python3
import os
import socket
server_addr = "/tmp/ux_server"
client_addr = "/tmp/ux_client"
if os.path.exists(client_addr):
os.remove(client_addr)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.bind(client_addr)
for n in range(5):
data = "Hello " + str(n)
data = data.encode()
print("Sent '{}' to {}".format(data, server_addr))
sock.sendto(data, server_addr)
data, addr = sock.recvfrom(16000)
print("Got '{}' back from {}".format(data, addr))
Furthermore, you can execute a connect on the client side. Since it's a datagram socket, that doesn't actually create a connection between the two but it does fix the address of the server endpoint, relieving you of the need to provide the server address on every send (i.e. you can use simple send rather than sendto).
For completeness, here's the echo server corresponding to the above:
#!/usr/bin/env python3
import os
import socket
server_addr = "/tmp/ux_server"
if os.path.exists(server_addr):
# Bind will fail if endpoint exists
os.remove(server_addr)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.bind(server_addr)
while True:
data, addr = sock.recvfrom(16000)
print("Got '{}' from {}".format(data, addr))
sock.sendto(data, addr)
EDIT
Hmm... I see now that you say you're already binding the client socket, and then connecting to the server side. But that means you simply need to have the server use recvfrom once initially to obtain the client's address. The OS will send the address along and you don't need to use an out-of-band mechanism.
The downside to connecting the socket is that if the client goes down, the server won't know that unless it attempts to send, but the client won't be able to reconnect because the server's socket is already connected. That's why datagram servers typically use recvfrom and sendto for all messages.
Updated server with initial recvfrom followed by connect:
#!/usr/bin/env python3
import os
import socket
server_addr = "/tmp/ux_server"
if os.path.exists(server_addr):
# Bind will fail if endpoint exists
os.remove(server_addr)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.bind(server_addr)
client_addr = None
while True:
if client_addr:
data = sock.recv(16000)
else:
data, client_addr = sock.recvfrom(16000)
sock.connect(client_addr)
print("Got '{}' from {}".format(data, client_addr))
sock.send(data)
Updated client with connected socket.
#!/usr/bin/env python3
import os
import socket
server_addr = "/tmp/ux_server"
client_addr = "/tmp/ux_client"
if os.path.exists(client_addr):
os.remove(client_addr)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.bind(client_addr)
sock.connect(server_addr)
for n in range(5):
data = ("Hello " + str(n)).encode()
print("Sent '{}'".format(data))
sock.send(data)
data = sock.recv(16000)
print("Got '{}' back".format(data))
I want to connect clients to a server using ZeroMQ (java bindings, jzmq), but I need the TCP information badly, for example the TCP/IP address of a client request! The problem is, for being able to announce a service in the network I need to grab the TCP address of a request to be able to redirect clients to that service. The broker is a central "service registry" in that case. However, having ZeroMQ services on both sides, I do not see an option to retrieve that information.
What I do now, is to establish a dummy connection using a standard socket to the broker, after the connection is established I grab the IP address used for this connection and close the connection again. The IP address which has been retrieved is now being used for binding on it using a ZeroMQ socket on a random port.
I think this solution is the ugliest solution ever possible, so: What is a better solution to this problem?
Greetings.
0MQ doesn't provide the address of peers, for a number of reasons. It's also not that useful since what you really want is the endpoint to receive connections on, not the address the connection was made on.
What I usually do, and it's elegant enough, is pass bind a service to an ephemeral port, get a full connection endpoint ("tcp://ipaddress:port") and send that string in some way, either broadcast to peers, to a central registry, etc. along with my service name. Then, peers who want to connect back can take the service name, look up to find my endpoint, and connect back to me.
In ZMQ 4.x, you may get the string property "Peer-Address" or the "Identity" property. http://api.zeromq.org/4-2:zmq-msg-gets
The Identity is set in the other peer before connect(). http://api.zeromq.org/4-2:zmq-setsockopt#toc20
For example,
const char *identityString = "identity";
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_REQ);
socket.setsockopt(ZMQ_IDENTITY, identityString, strlen(identityString));
socket.connect("tcp://127.0.0.1:5555");
Then the other side:
while(1)
{
zmq::message_t request;
if (socket.recv(&request, ZMQ_NOBLOCK))
{
const char* identity = request.gets("Identity");
const char* peerAddress = request.gets("Peer-Address");
printf("Received from %s %s\n", peerAddress, identity);
break;
}
}
I'm using CppZmq btw, you should be able to find the relevant calls easily.
Digging deeper into the libzmq code, I discovered that the library attaches to every message instance the file descriptor that it was received on.
This worked for me
int sockfd = zmq_msg_get(&msg, ZMQ_SRCFD);
sockaddr_in addr;
socklen_t asize = sizeof(addr);
getpeername(sockfd, (sockaddr*)&addr, &asize);
std::cout << inet_ntoa(addr.sin_addr) << ":" << addr.sin_port << std::endl;
Note that the FDs can and will be reused by other connections.
I'm working with version 4.2.1 of the api using the CZMQ binding and I found a solution for my case (ZMQ_STREAM). It works by setting an id before connecting.
The relevant socket option is "ZMQ_CONNECT_RID".
ZMQ api via zmq_setsockopt()
CZMQ api via zsock_set_connect_rid()
Some codes with redacted redacted ips.
const char endpoint1[] = "tcp://1.2.3.4:12345"
const char endpoint2[] = "tcp://5.6.7.8:12345"
zsock_t *stream = zsock_new(ZMQ_STREAM);
zsock_set_connect_rid(stream, endpoint1);
zsock_connect(stream, endpoint1);
zsock_set_connect_rid(stream, endpoint2);
zsock_connect(stream, endpoint2);
Then I get those 2 messages if there is a connection. First frame is the id and second frame is empty on connect/disconnect for ZMQ_STREAM sockets.
[Message1]
[019] tcp://1.2.3.4:12345
[000]
[Message2]
[019] tcp://5.6.7.8:12345
[000]
Another option is to use the zmq_socket_monitor() or czmq zmonitor. It was one of my first solution but I was looking for something lighter. I was able the get the endpoint that way without setting the id directly on the socket.
The zmonitor zactor make it possible to subscribe to socket events and then it sends a message with 3 frames:
[009] CONNECTED
[002] 14
[021] tcp://127.0.0.1:33445
I have a root node(server) connected to many other nodes(clients) through TCP sockets. I want to send some data from server to client, but the data is different for each node and depends on the ip address of that node.
Thus I should have ip address of each node connected to server. How can I have that information?
When you call accept(2) you can choose to retrieve the address of the client.
int accept(int socket, struct sockaddr *restrict address,
socklen_t *restrict address_len);
You need to store those addresses and then send(2) to each what you need to send.
So the workflow should be something like this:
Keep a list of connected clients. Initially the list is empty, of course
When you accept a connection, push its details into that list (the address and the socket returned by accept(2)).
When you need to send something to every client, simply walk the list and send it (using the stored socket)
The one tricky part is that socklen_t *restrict address_len is a value-result argument, so you need to be careful with that.
This is a more nuanced question than it first appears.
If the clients are sitting behind a NAT, you may get the same IP from more than one client. This is perfectly natural and expected behavior. If you need to distinguish between multiple clients behind the same NAT, you'll need some other form of unique client id (say, IP address and port).
As long as you have access to the list of file descriptors for the connected TCP sockets, it is easy to retrieve the addresses of the remote hosts. The key is the getpeername() system call, which allows you to find out the address of the remote end of a socket. Sample C code:
// This is ugly, but simpler than the alternative
union {
struct sockaddr sa;
struct sockaddr_in sa4;
struct sockaddr_storage sas;
} address;
socklen_t size = sizeof(address);
// Assume the file descriptor is in the var 'fd':
if (getpeername(fd, &address.sa, &size) < 0) {
// Deal with error here...
}
if (address.sa.family == AF_INET) {
// IP address now in address.sa4.sin_addr, port in address.sa4.sin_port
} else {
// Some other kind of socket...
}