I'm writing a short script to query domain names from their respective whois servers - while in most cases, while the TCP connection via port 43 seems to be working for most whois servers, the queries to whois.markmonitor.com seems to be failing with an odd error that says Invalid query.
Here's the barebones of what I'm using:
#!/usr/bin/perl
#whois.pl
use strict;
use IO::Socket;
my $domain_name = "google.com";
my $query_socket = new IO::Socket::INET(
PeerAddr => 'whois.iana.org',
PeerPort => 43,
Proto => 'tcp');
print $query_socket "$domain_name ";
print $query_socket "\n\r";
while(my $this_line = <$query_socket>) {
print $this_line;
}
close($query_socket);
As seen above, the whois server used is whois.iana.org; this also works as expected with whois.internic.net as well. Only in the case of whois.markmonitor.com, the following error is seen:
$ perl whois.pl
Invalid query
Could someone help shed more light on how can I perhaps get a more verbose output to check if there are any errors in the query that is being made to the server?
As an added test, a normal connection via telnet seems to be working as expected as seen below:
$ telnet whois.markmonitor.com 43
Trying 64.124.14.21...
Connected to whois.markmonitor.com.
Escape character is '^]'.
google.com
Domain Name: google.com
Registry Domain ID: 2138514_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.markmonitor.com
Registrar URL: http://www.markmonitor.com
Updated Date: 2015-06-12T10:38:52-0700
Creation Date: 1997-09-15T00:00:00-0700
......<output truncated>......
which leads me to believe that the actual connections to the server via port 43 are being accepted on the server's side.
As artistoex notes - it's because there's a space in your domain name.
Change your print line to:
print {$query_socket} "$domain_name\n";
(Note - the curly braces are for style reasons and can be omitted - I prefer them to make clear this is a file handle)
Per RFC3912 the client is expected to communicate like that in whois protocol: "All requests are terminated with ASCII CR and then ASCII LF"
So in your code, instead of "\n\r" please use "\r\n". And remove the extra space like written in other replies.
Note however that whoisis not a well defined structured protocol: do not expect all whois servers to work in the same way nor to adhere to some kind of standards. You will find a lot of strange cases...
Related
I have a CGI script to load publications from BibBase:
#!/usr/bin/perl
use LWP::UserAgent;
my $url = 'https://bibbase.org/show?bib=http://www.example.com/pubs.bib';
my $ua = LWP::UserAgent->new;
my $can_accept = HTTP::Message::decodable;
my $response = $ua->get($url, 'Accept-Encoding' => $can_accept);
print "Content-type: text/html\n\n";
print $response->decoded_content;
(This is copied from BibBase with the exception that the URL is hard-coded.)
I have three webservers running RHEL7 and Apache 2.4 that are configured the same way by Puppet. On all three I can run the script on the command line and get the expected results:
[root#server1 cgi-bin]# ./bibbase_proxy2.cgi | head
Content-type: text/html
<img src="//bibbase.org/img/ajax-loader.gif" id="spinner" style="display: none;" alt="Loading.." />
<div id="bibbase">
<script type="text/javascript">
var bibbase = {
params: {"bib":"http://www.example.com/pubs.bib","host":"bibbase.org"},
When I try to run the script with CGI, I get three different results:
Server1
Unrecognised protocol tcp at /usr/share/perl5/LWP/Protocol/http.pm line 31.
Server2
Can't connect to bibbase.org:443 System error at /usr/share/perl5/LWP/Protocol/http.pm line 51.
Server3
No http output and the error log says AH01215: Out of memory!.
I can't find anything different between the three servers and I can't figure out why the script works fine on the command line and doesn't work when run as a CGI.
I have selinux in permissive mode and it is logging the outgoing request, so I know the script gets that far:
type=AVC msg=audit(1532465859.921:331235): avc: denied { name_connect } for pid=161178 comm="perl" dest=80 scontext=system_u:system_r:httpd_sys_script_t:s0 tcontext=system_u:object_r:http_port_t:s0 tclass=tcp_socket
For testing, I have set selinux to disabled and restarted the server.
SE-Linux denied the TCP connection.
avc: denied { name_connect }
The default access controls for networking by SELinux are based on the labels assigned to TCP and UDP ports and sockets. For instance, the TCP port 80 is labeled with http_port_t (and class tcp_socket). Access towards this port is then governed through SELinux access controls, such as name_connect and name_bind.
When an application is connecting to a port, the name_connect permission is checked. However, when an application binds to the port, the name_bind permission is checked.
Permissive mode or not, Perl is acting like it was denied a TCP connection. Unrecognised protocol tcp means getprotobyname("tcp") failed inside IO::Socket::IP. That's very, very unusual. One of the ways that can happen is via exactly that SELinux denial.
I'm no SELinux expert, but according to RedHat and Gentoo some SELinux aware applications will ignore the global permissive setting and go it alone. RHEL 7 Apache appears to be one of them. It appears to have its own domain which must be set permissive.
On all three I can run the script on the command line and get the expected results:
There's two reasons for that, and they both have to do with users.
When you run the program you're running as your own user with your own configuration, permissions, and environment variables. In fact, you ran it as root which usually bypasses restrictions. When it runs on the server it runs as a different user, probably the web server user with severe restrictions.
In order to do a realistic test, you need to run it as the same user the web server will. You can use sudo -u for this. For example, if the user is apache...
sudo -u apache ./bibbase_proxy2.cgi
BTW Do not test software as root! Not only is it not going to give you sensible results, but if there's a bug in the software there are no safeguards preventing it from wrecking your system.
The second problem is #!/usr/bin/env perl. That means to run whatever perl is in your PATH. PATH will be different for different users. Running ./bibbase_proxy2.cgi may run with one Perl on the command line and a different one via the web server.
In a server environment, use a hard coded path to Perl like #!/usr/bin/perl.
We tested by rewriting the same script in Python and PHP. Both of them showed error which pointed us in the right direction.
Python urllib2 produced the error
<class 'urllib2.URLError'>: <urlopen error [Errno 16] Device or resource busy>
args = (error(16, 'Device or resource busy'),)
errno = None
filename = None
message = ''
reason = error(16, 'Device or resource busy')
strerror = None
PHP (run as CGI) wouldn't even start:
[Wed Jul 25 15:24:52.988582 2018] [cgi:error] [pid 10369] [client 172.28.6.200:44387] AH01215: PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/curl.so' - libssh2.so.1: failed to map segment from shared object: Cannot allocate memory in Unknown on line 0
[Wed Jul 25 15:24:52.988980 2018] [cgi:error] [pid 10369] [client 172.28.6.200:44387] AH01215: PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/dba.so' - libtokyocabinet.so.9: failed to map segment from shared object: Cannot allocate memory in Unknown on line 0
---- Similar lines for all extensions. ----
It appears that RLimitMEM blocks access to shared memory and that is required for opening sockets. I can't find any documentation, but removing that line makes it work.
I've got a problem when using IO::Socket::SSL.
Everything works fine under normal operation but I had an issue where the web server (IIS) locked up and Perl got stuck, even after the web server became active again.
I'm running the Perl script as an exe under Windows and I can't actually see the program running as it is hidden - here is the code:
sub api_action
{
$api_action = $_[0];
use IO::Socket::SSL;
$EOL = "\015\012";
$BLANK = $EOL x 2;
$remote = IO::Socket::SSL->new( Proto => "tcp",
PeerAddr => "api.xxxxx",
PeerPort => "443",
SSL_verify_mode => SSL_VERIFY_NONE,
verify_hostname => 0,
Timeout => 120,
);
unless ($remote) { print "cannot connect to API\n"; return "ERROR"
}
$remote->autoflush(1);
print $remote "GET /API/?action=$api_action" . $BLANK;
return <$remote>;
close $remote;
}
My thought is that it has made the initial connection, but is still waiting for a return from the GET command. I would have thought setting the Timeout would just cause it to return the error but it doesn't look like it works.
Any ideas on where I'm going wrong or what I've missed?
You actually send this to the server inside the SSL connection:
GET /path\r\n
\r\n
This is no valid HTTP/1.0 or HTTP/1.1 request but a HTTP/0.9 request. Since HTTP/0.9 is obsolete since 20 years I would not expect to IIS still support this old protocol so it might just show unexpected behavior when confronted with such a request. A proper HTTP/1.0 request would look like this:
GET /path HTTP/1.0\r\n
Host: www.example.com\r\n
\r\n
For more information on how to send proper requests and how to properly deal with the response see the HTTP/1.0 and HTTP/1.1 standards.
Apart from that:
SSL_verify_mode => SSL_VERIFY_NONE,
verify_hostname => 0,
verify_hostname has no meaning in IO::Socket::SSL. You probably saw this with LWP::UserAgent and it is only relevant there.
Everything works fine under normal operation but I had an issue where the web server (IIS) locked up and Perl got stuck, even after the web server became active again.
Unfortunately this is not usable as an error description because it is unclear what you consider as "locked up" and "active again" but essentially if the server behaves erratically then the client might be affected by this.
Currently your are dealing with the request by reading until end of TCP connection. If the server behaves erratically and will not close this connection then you will wait forever.
Note, that the timeout you set might only be relevant for the initial connect and not for further reads. And it might not even work properly on Windows depending on the version of IO::Socket::SSL you use. You might instead add a alarm(60) or similar to make sure that the client does not wait forever on a broken server or you might try to work with non-blocking sockets to deal with the issue.
The laptops in our company go to the network either about LAN (workplace) or about WLAN (conference room). Depending on how they go to the net, they get from DHCP different IPs.
Certain Perl application on a server, copies files from the client (e.g. laptop above) with Net::FTP. The piece of code looks like this:
# don't wait for ftp-timeout if the host is not reachable
my $alive = Net::Ping::External(host => $clnt_host);
if ($alive) {
$ftp = Net::FTP->new($clnt_host, Debug => 0, Timeout => 200)
or return "Cannot connect to $clnt_host: $#\n";
....
....
}
else {
dbgout(1, "Host $clnt_host unreachable.\n");
$st = "'FTPGETFAILED'";
return ($st);
}
Sometimes the code above doesn't work: Net::Ping::External() returns "alive", but Net::FTP->new() gets a "timeout".
Obviously "FTP" and "ping" resolve the hostname differently.
On the OS ping reslove as follows:
C:\Users\converter>ping -n 1 lap314-034
Ping wird ausgeführt für lap314-034.domain.de [10.140.12.110] mit 32 Bytes Daten:
Antwort von 10.140.12.110: Bytes=32 Zeit=2ms TTL=127
However, "nslookup" returns 2 possibilities:
C:\Users\converter>nslookup lap314-034
Server: domaincontroller.domain.de
Address: 123.123.123.123
Name: lap314-034.domain.de
Addresses: 10.192.3.145
10.140.12.110
The not active IP address is delivered from nslookup at the first place back.
I suppose that Net::FTP also uses this address to connect to the client.
How can I "convince" FTP to use the active DNS entry for the connection?
=============================================================
Thanks for your answers. I followed your suggestions. The solution bases on: http://code.activestate.com/lists/perl-win32-users/32624/
#------------------------------------------------------------------
sub getActiveIP {
#------------------------------------------------------------------
my $hostname = shift;
my $host = (gethostbyname ($hostname))[0] or return undef;
my #addr = gethostbyname ($host);
# delete the first 4 array elements
splice #addr, 0, 4;
foreach (#addr) {
my $IPstr = sprintf ("%s", &inet_ntoa ($_));
my $alive = ping(host => $IPstr);
if ($alive) {
return $IPstr;
}
}
return undef;
}
Nevertheless, I believe that a widespread Perl-library should not make such surprises to the user.
How can I "convince" FTP to use the active DNS entry for the connection?
There is no good way to let Net::FTP decide it.
Instead you should determine it outside of Net::FTP and then use the usable IP address instead of the hostname in Net::FTP.
Maybe you would be able to use only Net::FTP with the new versions (Net::FTP > 3.0) which can use IO::Socket::IP instead of IO::Socket::INET as the underlying module. This module can try all the IP addresses returned for a hostname until it gets a successful connection. But these retries will be done for every connection, that is the control connection and every data transfer. Since the connection to the inactive host only fails after some timeout everything will just be horribly slow.
The solution seems obvious: Get the IP addresses, ping them, figure out which one is live, and use the IP address instead of host name in the Net::FTP constructor.
I'm currently starting a Tcl socket server like this:
socket -server Server 0
This lets the operating system pick an available port to start listening on. The question is that I don't want it to pick any port between 1025 and 64k, instead want to know if I can specify a range of ports? Something like this:
socket -server Server 40000-41000
And then the operating system would pick an available port between 40000 and 41000 for the server to listen on. Is there a way to do this? I can't find it in the Tcl API, but is there some nice API call way to do it rather than iterating through the port range until finding an available port?
The OS itself doesn't provide an API capable of doing that, and Tcl doesn't wrap one up for you as it is actually a pretty rare requirement. Conventionally, servers either listen on specific ports (so clients can know exactly what service to ask for; e.g., 21 for FTP, 22 for SSH, 25 for SMTP, 80 for HTTP, 161 for SNMP, 443 for HTTPS, 993 for secure IMAP) or the clients have some other way of being told what to ask for and genuinely any port will do (0 is the special “pick a card, any card” address). You can ask a Tcl server socket for what port it is actually using fconfigure:
set portNumber [lindex [fconfigure $socket -sockname] 2]
But to get a server socket on a port in a specific range? No API for that. We have to cook something ourselves:
for {set port 40000} {$port <= 41000} {incr port} {
if {![catch {
set sock [socket -server $yourHandler $port]
}]} then {
break
}
}
# If we failed...
if {![info exist sock]} {
error "No ports free in range 40k-41k"
}
This works because failing to bind the port will make the socket creation fail (neatly, catchably) and you can then try to bind the next port. It will take a while to scan over the port range, but it will work.
It's probably neater to wrap this up in a procedure. And Tcl 8.6's try construct will make the code a little less obscure:
proc portInRange {from to handler} {
for {set p $from} {$p <= $to} {incr p} {
try {
return [socket -server $handler $p]
} on error {} continue
}
error "No ports free in range $from-$to"
}
No, there's no API for that.
Generally servers listen on a specific port so that the clients can find the server. So such an API is not particularly useful.
Best to just write it yourself.
I need your help/advices on my very short script I wrote in Perl in order to send SMS via SMPP protocol.
I got a SMS gateway which is perfectly working (sending SMS via HTML request or via web interface works), let's say this gateway has IP 192.168.1.15.
Its SMPP service is listening to TCP 2775 (I can successfully telnet to 2775, so I guess SMPP service is working on my SMS gateway)
my $smpp = Net::SMPP->new_transmitter(192.168.1.15,
port=>2775,
system_id =>"administrator",
password =>"passwdexample") or die;
$resp_pdu = $smpp->submit_sm(destination_addr => '+400123456789',
short_message => 'test message') or die;
die "Response indicated error: " . $Resp_PDU->explain_status() if $resp_pdu->status;
When I run the script, here the error I got :
Response indicated error: Incorrect BIND Status for given command (ESME_RINVBNDS TS=0x00000004) at C:\temp\smpptest.pl line .
Unfortunately, I haven't find so much help on internet, but according to this link : SMPP Errors Codes It says :
You must bind first before any other request is handled.
However, my bind is done with new_transmitter, and I don't get any errors at this point, so I don't understand how it cannot bind the TCP session (my credentials are corrects, I tried that in a telnet session).
That's the first time I use such a plugin, so maybe I'm missing something, and maybe someone has already met this error !
Many thanks for your help :-)
Try to quote the IP address:
Net::SMPP->new_transmitter("192.168.1.15", ...);