I'm using netty 4.x, and I finally find it calls
public boolean isActive() {
SocketChannel ch = javaChannel();
return ch.isOpen() && ch.isConnected();
}
can I use it to check whether the connection between client and server is readable/writeable?
If the answer is yes, then I believe isActive use sth. like heartbeat to check the status of the client. but when I grab the tcp packets I can't find any packets work as heartbeat through wireshark filter tcp.port==12093(it's my server port).
any suggestion is helps.
Channel.isActive() works by looking up the querying the underlying socket to see if it is connected.
So if we want to know how it works we should dig a little deeper, if you are using netty with tcp connections, you are almost sure you are using the java nio sockets.
From io.netty.channel.socket.nio.NioSocketChannel:
#Override
public boolean isActive() {
SocketChannel ch = javaChannel();
return ch.isOpen() && ch.isConnected();
}
Both of the above methods aren't that interesting, and are simply boolean flags set to the correct value by native code.
However, since isActive() just returns those boolean flags, you cannot be sure that the connection is still working at the moment you call the function, the best way would be to change your protocol to include some kind of ping message, for example IRC has a ping command, or FTP has the NOOB command.
When sending a ping message, you must make sure that it leaves enough headroom for partial network dropout, for example if your computer is connected through WIFI, there will be some packet loss. However, the native tcp layer in the operating system should handle these packet loss combined with resending for you, but it may mean that after you send a ping message, it may take up to a full minute before isActive() will return false.
If you want to add automatic ping messages to your protocol, it may be wise to look into the IdleStateHandler and the WriteTimeoutHandler.
Related
Help me track the status of a specific port: "LISTENING", "CLOSE_WAIT", "ESTABLISHED".
I have an analog solution with the netstat command:
local command = 'netstat -anp tcp | find ":1926 " '
local h = io.popen(command,"rb")
local result = h:read("*a")
h:close()
print(result)
if result:find("ESTABLISHED") then
print("Ok")
end
But I need to do the same with the Lua socket library.
Is it possible?
Like #Peter said, netstat uses the proc file system to gather network information, particularly port bindings. LuaSockets has it's own library to retrieve connection information. For example,
Listening
you can use master:listen(backlog) which specifies the socket is willing to receive connections, transforming the object into a server object. Server objects support the accept, getsockname, setoption, settimeout, and close methods. The parameter backlog specifies the number of client connections that can be queued waiting for service. If the queue is full and another client attempts connection, the connection is refused. In case of success, the method returns 1. In case of error, the method returns nil followed by an error message.
The following methods will return a string with the local IP address and a number with the port. In case of error, the method returns nil.
master:getsockname()
client:getsockname()
server:getsockname()
There also exists this method:
client:getpeername() That will return a string with the IP address of the peer, followed by the port number that peer is using for the connection. In case of error, the method returns nil.
For "CLOSE_WAIT", "ESTABLISHED", or other connection information you want to retrieve, please read the Official Documentation. It has everything you need with concise explanations of methods.
You can't query the status of a socket owned by another process using the sockets API, which is what LuaSocket uses under the covers.
In order to access information about another process, you need to query the OS instead. Assuming you are on Linux, this usually means looking at the proc filesystem.
I'm not hugely familiar with Lua, but a quick Google gives me this project: https://github.com/Wiladams/lj2procfs. I think this is probably what you need, assuming they have written a decoder for the relevant /proc/net files you need.
As for which file? If it's just the status, I think you want the tcp file as covered in http://www.onlamp.com/pub/a/linux/2000/11/16/LinuxAdmin.html
I apologize before hand if some of these questions might be obvious for expert network programmers. I have researched and read about coding in networking and it is still not clear to me how to do this.
Assume that I want to write a tcp proxy (in go) with the connection between some TCP client and some TCP server. Something like this:
First assume that these connection are semi-permanent (will be closed after a long long while) and I need the data to arrive in order.
The idea that I want to implement is the following: whenever I get a request from the client, I want to forward that request to the backend server and wait (and do nothing) until the backend server responds to me (the proxy) and then forward that response to the client (assume that both TCP connection will be maintained in the common case).
There is one main problem that I am not sure how to solve. When I forward the request from the proxy to the server, and get the response, how do I know when the server has sent me all the information that I need if I do not know beforehand the format of the data being sent from the server to the proxy (i.e. I don't know if the response from the server is of the form of type-length-value scheme nor do I know if `\r\n\ indicates the end of the message form the server). I was told that I should assume that I get all the data from the server connection whenever my read size from the tcp connection is zero or smaller than the read size that I expected. However, this does not seem correct to me. The reason it might not be correct in general is the following:
Assume that the server for some reason is only writing to its socket one byte at a time but the total length of the response to the "real" client is much much much longer. Therefore, isn't it possible that when the proxy reads the tcp socket connected to the server, that the proxy only reads one byte and if it loops fast enough (to do a read before it receives more data), then read zero and incorrectly concludes that It got all the message that the client intended to receive?
One way to fix this might be to wait after each read from the socket, so that the proxy doesn't loop faster than it gets bytes. The reason that I am worried is, assume there is a network partition and i can't talk to the server anymore. However, it is not disconnected from me long enough to timeout the TCP connection. Thus, isn't it possible that I try to read from the tcp socket to the server again (faster than I get data) and read zero and incorrectly conclude that its all the data and then send it pack to the client? (remember, the promise I want to keep is that I only send whole messages to the client when i write to the client connection. Thus, its illegal to consider correct behaviour if the proxy goes, reads the connection again at a later time after it already wrote to the client, and sends the missing chunk at a later time, maybe during the response of a different request).
The code that I have written is in go-playground.
The analogy that I like to use to explain why I think this method doesn't work is the following:
Say we have a cup and the proxy is drinking half the cup every time it does a read from the server, but the server only puts 1 teaspoon at a time. Thus, if the proxy drinks faster than it gets teaspoons it might reach zero too soon and conclude that its socket is empty and that its ok to move on! Which is wrong if we want to guarantee we are sending full messages every time. Either, this analogy is wrong and some "magic" from TCP makes it work or the algorithm that assumes until the socket is empty is just plain wrong.
A question that deals with a similar problems here suggests to read until EOF. However, I am unsure why that would be correct. Does reading EOF mean that I got the indented message? Is an EOF sent each time someone writes a chunk of bytes to a tcp socket (i.e. I am worried that if the server writes one byte at a time, that it sends 1 EOF per bytes)? However, EOF might be some of the "magic" of how a TCP connection really works? Does sending EOF's close the connection? If it does its not a method that I want to use. Also, I have no control of what the server might be doing (i.e. I do not know how often it wants to write to the socket to send data to the proxy, however, its reasonable to assume it writes to the socket with some "standard/normal writing algorithm to sockets"). I am just not convinced that reading till EOF from the socket from server is correct. Why would it? When can I even read to EOF? Are EOFs part of the data or are they in the TCP header?
Also, the idea that I wrote about putting a wait just epsilon bellow the time-out, would that work in the worst-case or only on average? I was also thinking, I realized that if the Wait() call is longer than the time-out, then if you return to the tcp connection and it doesn't have anything, then its safe to move on. However, if it doesn't have anything and we don't know what happened to the server, then we would time out. So its safe to close the connection (because the timeout would have done that anyway). Thus, I think if the Wait call is at least as long as the timeout, this procedure does work! What do people think?
I am also interested in an answer that can justify maybe why this algorithm work on some cases. For example, I was thinking, even if the server only write a byte at a time, if the scenario of deployment is a tight data centre, then on average, because delays are really small and the wait call is almost certainly enough, then wouldn't this algorithm be fine?
Also, are there any risks of the code I wrote getting into a "deadlock"?
package main
import (
"fmt"
"net"
)
type Proxy struct {
ServerConnection *net.TCPConn
ClientConnection *net.TCPConn
}
func (p *Proxy) Proxy() {
fmt.Println("Running proxy...")
for {
request := p.receiveRequestClient()
p.sendClientRequestToServer(request)
response := p.receiveResponseFromServer() //<--worried about this one.
p.sendServerResponseToClient(response)
}
}
func (p *Proxy) receiveRequestClient() (request []byte) {
//assume this function is a black box and that it works.
//maybe we know that the messages from the client always end in \r\n or they
//they are length prefixed.
return
}
func (p *Proxy) sendClientRequestToServer(request []byte) {
//do
bytesSent := 0
bytesToSend := len(request)
for bytesSent < bytesToSend {
n, _ := p.ServerConnection.Write(request)
bytesSent += n
}
return
}
// Intended behaviour: waits until ALL of the response from backend server is obtained.
// What it does though, assumes that if it reads zero, that the server has not yet
// written to the proxy and therefore waits. However, once the first byte has been read,
// keeps writting until it extracts all the data from the server and the socket is "empty".
// (Signaled by reading zero from the second loop)
func (p *Proxy) receiveResponseFromServer() (response []byte) {
bytesRead, _ := p.ServerConnection.Read(response)
for bytesRead == 0 {
bytesRead, _ = p.ServerConnection.Read(response)
}
for bytesRead != 0 {
n, _ := p.ServerConnection.Read(response)
bytesRead += n
//Wait(n) could solve it here?
}
return
}
func (p *Proxy) sendServerResponseToClient(response []byte) {
bytesSent := 0
bytesToSend := len(request)
for bytesSent < bytesToSend {
n, _ := p.ServerConnection.Write(request)
bytesSent += n
}
return
}
func main() {
proxy := &Proxy{}
proxy.Proxy()
}
Unless you're working with a specific higher-level protocol, there is no "message" to read from the client to relay to the server. TCP is a stream protocol, and all you can do is shuttle bytes back and forth.
The good news is that this is amazingly easy in go, and the core part of this proxy will be:
go io.Copy(server, client)
io.Copy(client, server)
This is obviously missing error handling, and doesn't shut down cleanly, but clearly shows how the core data transfer is handled.
Okay, I want to send a UDP packet to an IP, and then receive it.
As I can't get it to work, I have to ask for assistance.
I want it to work on 1 app, meaning client and server is on the same application.
I currently got it working with Audio sending and receiving on the same app (using sample codes and stuff).
And the thing I can´t understand is just the UDP, and probably Sockets.
It seems simple, but I can´t get it to work.
So if I can get a very simple sample, like:
"UDP Send "hello" to IPofchoice"
"UDP receive "data" from IPofchoice"
Messagebox.show("decoded(data));
Well hope you get what I mean.
EDIT:
Here is my "example" which doesn´t work at all.
void VoiceChat_KeyPress(object sender, KeyPressEventArgs e)
void VoiceChat_KeyPress(object sender, KeyPressEventArgs e)
{
string text = "Hello";
byte[] send_buffer = Encoding.ASCII.GetBytes(text);
otherPartyIP = new IPEndPoint(IPAddress.Parse(txtCallToIP.Text), 1450);
udpClientKey.Send(send_buffer, send_buffer.Length, "127.0.0.1", 1450);
byte[] byteData = udpClientKey.Receive(ref otherPartyIP);
MessageBox.Show(otherPartyIP.Address.ToString());
MessageBox.Show(System.Text.Encoding.ASCII.GetString(byteData));
}
Where txtCallToIP.tex = the ip i write, which is 127.0.0.1 currently.
This i solved.
To have a client and server in UDP it´s very easy, though a bit dangerous, as you don´t know for sure if it´s working or not unless you test it (UDP just send and don´t care about anything else).
So basically to do it.
Client.Send(buffer,0,buffer.size,IPAdress,Port);
byte[] rec = Client.Receive(ref IPAdress);
To receive you must have bind it to an IP, so you can just bind it to listen to Any ipadress,
The port however needs to be set (i think), and you want to do that, as listening to everything there is, is not a good idea.
So ref IPAdress will be that listening IP and Port.
So it´s very easy to set up. And you can use the same UDPClient for both receiving and sending if you want, though it´s probably better to have 2 separate .
I want to connect clients to a server using ZeroMQ (java bindings, jzmq), but I need the TCP information badly, for example the TCP/IP address of a client request! The problem is, for being able to announce a service in the network I need to grab the TCP address of a request to be able to redirect clients to that service. The broker is a central "service registry" in that case. However, having ZeroMQ services on both sides, I do not see an option to retrieve that information.
What I do now, is to establish a dummy connection using a standard socket to the broker, after the connection is established I grab the IP address used for this connection and close the connection again. The IP address which has been retrieved is now being used for binding on it using a ZeroMQ socket on a random port.
I think this solution is the ugliest solution ever possible, so: What is a better solution to this problem?
Greetings.
0MQ doesn't provide the address of peers, for a number of reasons. It's also not that useful since what you really want is the endpoint to receive connections on, not the address the connection was made on.
What I usually do, and it's elegant enough, is pass bind a service to an ephemeral port, get a full connection endpoint ("tcp://ipaddress:port") and send that string in some way, either broadcast to peers, to a central registry, etc. along with my service name. Then, peers who want to connect back can take the service name, look up to find my endpoint, and connect back to me.
In ZMQ 4.x, you may get the string property "Peer-Address" or the "Identity" property. http://api.zeromq.org/4-2:zmq-msg-gets
The Identity is set in the other peer before connect(). http://api.zeromq.org/4-2:zmq-setsockopt#toc20
For example,
const char *identityString = "identity";
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_REQ);
socket.setsockopt(ZMQ_IDENTITY, identityString, strlen(identityString));
socket.connect("tcp://127.0.0.1:5555");
Then the other side:
while(1)
{
zmq::message_t request;
if (socket.recv(&request, ZMQ_NOBLOCK))
{
const char* identity = request.gets("Identity");
const char* peerAddress = request.gets("Peer-Address");
printf("Received from %s %s\n", peerAddress, identity);
break;
}
}
I'm using CppZmq btw, you should be able to find the relevant calls easily.
Digging deeper into the libzmq code, I discovered that the library attaches to every message instance the file descriptor that it was received on.
This worked for me
int sockfd = zmq_msg_get(&msg, ZMQ_SRCFD);
sockaddr_in addr;
socklen_t asize = sizeof(addr);
getpeername(sockfd, (sockaddr*)&addr, &asize);
std::cout << inet_ntoa(addr.sin_addr) << ":" << addr.sin_port << std::endl;
Note that the FDs can and will be reused by other connections.
I'm working with version 4.2.1 of the api using the CZMQ binding and I found a solution for my case (ZMQ_STREAM). It works by setting an id before connecting.
The relevant socket option is "ZMQ_CONNECT_RID".
ZMQ api via zmq_setsockopt()
CZMQ api via zsock_set_connect_rid()
Some codes with redacted redacted ips.
const char endpoint1[] = "tcp://1.2.3.4:12345"
const char endpoint2[] = "tcp://5.6.7.8:12345"
zsock_t *stream = zsock_new(ZMQ_STREAM);
zsock_set_connect_rid(stream, endpoint1);
zsock_connect(stream, endpoint1);
zsock_set_connect_rid(stream, endpoint2);
zsock_connect(stream, endpoint2);
Then I get those 2 messages if there is a connection. First frame is the id and second frame is empty on connect/disconnect for ZMQ_STREAM sockets.
[Message1]
[019] tcp://1.2.3.4:12345
[000]
[Message2]
[019] tcp://5.6.7.8:12345
[000]
Another option is to use the zmq_socket_monitor() or czmq zmonitor. It was one of my first solution but I was looking for something lighter. I was able the get the endpoint that way without setting the id directly on the socket.
The zmonitor zactor make it possible to subscribe to socket events and then it sends a message with 3 frames:
[009] CONNECTED
[002] 14
[021] tcp://127.0.0.1:33445
I am writing a UDP test client/server and i want to get it through firewall. Supposedly all i need to do is have both sides send to the correct IP and server. Getting an IP is not a problem but how do i have the client pick a random free port and report it to the user? I eventually would want it to connect to a matchmaker server but right now i need a simple working prototype and i would like to cout the port number so my friend/tester can send me the # via IM so we can test.
How do i get the port number?
sorry for the long desc. I notice people tell me not to do what i am asking when i dont give a desc :(
To use the highly technical term, this is actually a pretty icky problem or even a pair of icky problems. Depending on the configuration of the firewall, it will usually allow responses from another endpoint on the IP endpoint as the request came from. So... if you friend receives the UDP datagram using something like the recvfrom() system call, the address parameter will receive the IP endpoint information to respond to. So the other end should be able to respond with a sendto() using the same addressing information. Something like:
/* initiator */
struct sockaddr_in hisaddr;
memset(&hisaddr, 0, sizeof(hisaddr));
hisaddr.sin_addr.s_addr = htonl(target_ip);
hisaddr.sin_port = htons(target_port);
sendto(sd, msg_ptr, msg_sz, 0, (struct sockaddr*)&hisaddr, sizeof(hisaddr));
/* receiver */
struct sockaddr_in peeraddr;
socklen_t peer_sz = sizeof(peeraddr);
recvfrom(sd, buf_ptr, buf_sz, 0, (struct sockaddr*)&peeraddr, &peer_sz);
/* build response */
sendto(sd, msg_ptr, msg_sz, 0, (struct sockaddr*)&peeraddr, peer_sz);
The peeraddr on the other side will be your external address or, more correctly, the IP address of your firewall and the port number that it chose to use. The port number that you specify in your code may be completely different than the port that your friend would have to send data to. Ultimately, it might not matter what port you choose to use since the firewall might be sending and receiving on an entirely different port - this is what Network Address Translation is all about. I would recommend reading RFC3235 for some tips on how to overcome that hurdle.
The best approach IMHO is to:
Let the OS choose a port by either calling bind() with a zero port number or skipping the bind altogether
Having the client receive the address information from the socket layer (e.g., the fifth and sixth arguments to recvfrom())
The client sends response to the endpoint retrieved in the previous step
Tweak the firewall configurations until the previous steps work
Of course, all of the magic is in the last step. If you can disable NAT or ensure that the firewall is never going to switch ports, then nailing down a port number and bind-ing to it will work as well. You might want to take a look at %WINDIR%\system32\drivers\etc\services (or /etc/services depending on your OS inclination) to get an idea of what port numbers are reserved or generally in use.
bind() the socket before you send your data. Specify port 0 to bind(), and the OS will pick an unused port for you. You can then use getsockname() to find out what port wsa chosen.
Generally speaking - you - as the developer - choose the port. You can set your application to read the port from a config file or user input - but no magic firewall is going to tell you what port to use...
If I'm understanding your question correctly, I'm not sure there's a way to do what you want programatically (and even if there is, I don't think it's the right approach). I think you need to find a port that isn't in use on the server machine (and perhaps a different or the same port on the client machine, if communication is bi-directional) AND that port must be able to pass through your firewall. I assume since you say "getting an IP is not a problem", you've already configured your firewall to forward some or all ports to a specific computer inside the firewall? If so, the port you seek is one of the ones you forwarded. You can just pick an arbitrary one, as long as no other service is running on that port. Ports below 1024 are reserved, so you probably want to pick a higher number than that. You can use a simple portscanning tool such as nmap to see which services are running on your computer on which ports and pick a different one. Note that nmap can be fooled by firewalls and various bind rules when sockets are created.
I think you're better off picking a fixed port rather than relying on the random port number chosen by the O/S.
If you use a random port you'd have to change your firewall settings each and every time you run the program.
If you're using WINSOCK check this link:
http://msdn.microsoft.com/en-us/library/aa280717(VS.60).aspx
Basically you have 2 choices set the port to 0 and let the system assign you one or chose a random one try to open the socket if it doesn't work try another (be sure to steer clear of reserved ports)